Hi,
I managed to put together an implementation of Hartmann pipelines that are an extension to the pipes usually used in your OS of choice (see
http://en.wikipedia.org/wiki/Hartmann_pipeline for more details).
The code and some documentation (some of it already outdated) is to be found at
http://code.google.com/p/erl-pipes/*Brief description:
Pipelines are formed by pipe segments connected to each other, forming a directed graph. Data flows through the pipeline and is transformed along the way.
A pipe segment is represented by a process and messages are bearers of the data flow. This allows multi-processor systems to split the load between the CPUs.
There is a soft flow control mechanism (meaning that the control signals are not authoritative) that prevents mailboxes to become overflowed.
A few pipe segments are defined in pips_builtins. One of the more interesting ones is 'partition' that splits the flow in two, based on a user-supplied predicate.
See pipes_test.erl for some examples and a simple benchmark.
This brings up the question of performance. Well, since I'm the author, I will be nice and say there's a lot of room for improvement. I've measured worst case runs that were 30x slower than doing this in the old-fashioned way, but I've also had cases where the slowdown factor was 1.4, which is not too bad. I suppose this kind of system would benefit a lot from a shared heap runtime.
For this to be easy to use, we will need a simple way to put together the segments. A language to describe the graph that is "input friendly".
This is (I think) also a good example where channels might prove useful.
All in all, this is just a scratch on the surface. There is a lot to improve and to add.
best regards,
Vlad
_______________________________________________
erlang-questions mailing list
erlang-questions@...
http://www.erlang.org/mailman/listinfo/erlang-questions
