Streams library 2.0 development part 2 (Part 1)

[qw23]

@bh

Honestly I've never thought about two threads using the same stream, apart from the exercise about joint bank accounts.

I suppose it must have been at the back of your mind when you wrote the Streams library, v.1. If not, why has the definition of tail of stream (and some other blocks) been warp-ed at all?

@bh

But I'm pleased that you thought about it!

I’m actually baffled no one (including myself) ever noticed the effect of using streams on parallel processes - or at least no one raised an issue in the forum. I guess it only goes to show that, over the (five?) years the Streams library has been available, it hasn’t been used much. Well, never mind, with some practical application code added that might change.

@bh

And I'll take your word for the solution.

You shouldn’t. I’m as fallible as anyone. I haven’t even tried yet if it works with two (or more) processes trying to access the same stream, like the bank account case. Hadn’t even thought of that when I wrote my post. Now that I’m aware of it, I will, of course. Still, please keep challenging my ideas and results.

@bh

Are you proposing to implement all that other mechanism, looking for dead threads and restarting their computation? (If so, don't forget the possibility that the first thread stalled out for a reason, and its replacement will also stall out for the same reason.)

Yeah, I had thought of that in the meantime (probably while I was sleeping): the evaluating process may be waiting for user input, for example. So on second thought I’m probably not going to implement the time-out mechanism for now; better wait until a generally suitable solution will have come to mind, if ever. A message-based replacement for the active wait block may be easier to build, though it may not be necessary to do it anytime soon (depending on test results), plus it could be done without fundamentally changing the data structure, in a future minor release.

@bh

I still think that reporters don't need to use WARP because they're warped automatically, but I think that only because I've been told, not from firsthand observation.

IIRC only Snap! primitive blocks are atomic. So change (balance) by (-100) is guaranteed to work correctly even in a multi-process environment. But set (available) to ((balance) ≧ (100)); if (available) (change (balance) by (-100)) is not. I’m not sure about if (my function (arguments)) (change (balance) by (-100)). In all cases: warp-protection (and, by the same token, the atomic nature of primitives) will last up to 0.5 seconds. So it’s both safer and more efficient to protect a resource using a kind of semaphore, unless the processing time is very short, and warp is the easy (and uncomplicated) solution.
Better check that with jens, and add a big “caveat” to warp’s online help (I mean, hardly anyone is aware it should only be used for a very short time, to protect a critical section),