Streams library 2.0 development, part 1

I asked ChatGPT to count from 0:

  1. ala
  2. wan
  3. tu
  4. mute (or poka)
  5. ale (or mute mute)
  6. luka (or ale ale)
  7. luka wan (or ale ale ale)
  8. luka tu (or ale ale ale ale)
  9. luka mute (or ale ale ale ale ale)
  10. luka ale (or ale ale ale ale ale ale)
  11. luka luka (or ale ale ale ale ale ale ale)
  12. luka luka wan (or ale ale ale ale ale ale ale ale)
  13. luka luka tu (or ale ale ale ale ale ale ale ale ale)
  14. luka luka mute (or ale ale ale ale ale ale ale ale ale ale)
  15. luka luka ale (or ale ale ale ale ale ale ale ale ale ale ale)
  16. luka luka luka (or ale ale ale ale ale ale ale ale ale ale ale ale)
  17. luka luka luka wan (or ale ale ale ale ale ale ale ale ale ale ale ale ale)
  18. luka luka luka tu (or ale ale ale ale ale ale ale ale ale ale ale ale ale ale)
    Etc.

So, the variant in parentheses is base one, like a Turing machine; to know whether the (I guess) standard dialect is truly base five we have to see how it says twenty-five. (Do they say five five times, or do they have a word for squared?)

Binary would not just be harder to learn, but also harder to speak, since numbers require more digits in binary. Of course higher bases also have a cost, in the number of digit names you have to remember. Some people have made a mathematical argument that the most economical tradeoff between these costs would be base e, and so we should use trinary, the closest integer base to e.

How about this:

(Never mind the block color; I just like baby blue. :~) ) I think maybe it's faster this way?

MONITOR needs a mention in its help screen text that it breaks the space efficiency of streams by constructing a list at the same time.

UNIQUES: We have a sort of standard for handling the question of which version to show by using the fast version and also including the simple version as a detached script in the block's script editor. (The detached script isn't runnable unless you reattach it to the header block, to bind its formal parameters.)

In this section of the fast implementation


why don't you just use HISTORY as the second input to the recursive call? HEAD is already included in HISTORY, in this case. That change would speed up future calls to CONTAINS by shortening HISTORY.

PROCESS/PROJECT: Yeah I get confused trying to figure out why they're different. Also I really want to talk you out of the right to left processing; that makes sense, and falls out naturally, when you actually compose functions by physically inserting one block in the input slot of another, but when you have a variadic function input, I expect it to behave like PIPE, which also has the virtue of being simple to implement:

STREAM CASCADING: I like the idea a lot. I think the INCLUSIVE input makes the block significantly harder to understand, and you can always just take TAIL OF STREAM of the result if you don't want to include the starting value.

We’d have to ask a native speaker :wink:
ChatGPT: « In Toki Pona, "twenty five" would be expressed as "tu tu tu tu luka". »
(I’m sorry to say I don’t really understand the system either)

I remember when the Euro (€) was going to be introduced, one public intellectual made a case for creating 1/3/10 denominations. Which, indeed, would have made more sense than the 1/2/5 system we currently use. On the other hand, for electronic payments it doesn’t matter much anymore.

I made that:

:white_check_mark:

:white_check_mark:

I must have overlooked that. :white_check_mark:

I’ll take that into account. :white_check_mark: I made a single block that can do all that both of the older versions could do, and some more (even though I would have preferred a simpler construction, without a carrier block for operators - perhaps in a future version if “reporter didn’t report” can be caught). The order of execution is left-to-right now.

:white_check_mark: BTW changed the name to generate stream …, as it works a bit like the List utilities block cascade but that block reports only one result whereas this block reports a new result after each step.

I'm done for now. Examples I moved to the definition blocks. I added help text, and some comments on the internal workings.
Would you (@bh) take another look, and try if you can find any remaining issues?

Link to latest code version.

P.S. I added an improved (IMO, that is) version of show stream, supporting user-specified halt conditions (or would such be called a “stop condition”?), and resume from item.

Besides, I added: merge streams sorted by (valuator): < streams … >; it will merge any number of streams, sorted according to a reporter (valuator) applied on each item - provided each of the input streams was pre-sorted in a similar way.

I used to love the £/s/d system, especially before Nixon floated the dollar, when £1 was $2.40, so a penny was exactly a penny. :~)

Awesome, thanks! I'm going to switch to ztreams.

SHOW-STREAMS: I have several questions. First, I think the actual stream should be the first input, right after the title word "stream." (I guess the question is, are you okay with that?)

Second, in


is there a reason why you don't say

so that an empty input slot will work in the CALL?

And similarly, in
Streams extended script pic (2)
I think both the CALL and the REPORT should be changed:


The CALL issue is the same as above, empty slots. The REPORT issue is that the range of this function is plain lists, not streams! (Of course testing won't catch that since the empty stream = the empty list.)

And finally, shouldn't
Streams extended script pic (4)
really be
Streams extended script pic (5)
using the right variable in the empty test, and reporting a list rather than a stream.

Actually, why use a separate variable STREAM* at all? You could call it STREAM, which would have the advantage of not keeping the head of the input stream referenced as you move down into it, through the miracle of tail call elimination. ;~)

GENERATE STREAM: I bet we can find a name that's more self-documenting. The closest thing I find in Wolfram is FixedPointList, which stops when two consecutive values are equal according to some comparison function you can supply. But this function is more general than that; the sequence is not expected to converge. And anyway, our users probably mostly don't know what "fixed point" means. I'll think more about it. (I'd like it better if it were USING _ REPEATEDLY, but the block is too wide already!)

Again,
Streams extended script pic (2)
should call FINISHED with ORIGINAL as input.

I tried your demo, but I messed up and removed the # from #>7:


It gives me one number too many in the result. But if I put the # back I get seven items, not eight. This works:

(moving the "image←" up to the end test).

I can't get PROJECT to work at all. For one thing, there's this upvar that's called ORIGINAL in the prototype, but called # in the recursive call, which I don't understand. Did you change your mind about what it means? Where you used it in your demo, it seems always to be 1. One little braino in your demo is that you have
Streams extended script pic (11)
where you meant
Streams extended script pic (12)
After fixing that, and removing all references to ORIGINAL, I get



But I can see why you want ORIGINAL, because I had to use that kludgy call to SQRT to reconstruct it. So, maybe you could debug that? Thanks! (Something about scope of variables, I'm sure. Maybe Streams extended script pic (14) could fix it?)

Thx4 the elaborate review! Meanwhile I found some issues, too. They may may (partly) the same. I’m going to look into it during the next week or so.

At first glance:

Nowadays £1 is about $1.30, so if you've kept your dollars you must have gained a lot in pounds' worth. Not so, of course, if there's a hefty mortgage loan in dollars to pay off.

Great! I've implement that in TAIL OF STREAM and IN FRONT OF STREAM now. If there is going to be a relase note, it'll have to mention the change of data structure, I suppose. I'll mention it in help text anyway. (BTW I am changing the help texts' references to SICP, as the old link no longer works).

Let me tell you why I moved it to the back of the hat block. Unlike lists, one can't inspect what a stream reports by simply clicking the applicable block - you need show stream for that. Thus, when writing code with streams, I use show stream all of the time. Especially so when a more or less complicated stream is to be inspected. In that case it really helps if the stream slot is at the back of the show stream block, so I can easily drop the stream block without having to move show stream all the way to the right of my screen, and avoid the risk of dropping it into the wrong slot, like:

Streams extended script pic
(I would like to circle the empty "stream" slot, but don't know how to do that)

By the same token I try to manoeuvre variadic inputs to the back of any hat block.

In case of multiple variadic inputs I'll use $nl after the first, and even then I'll often need to temporarily move a block away from its own slot in order to get stuff inserted.

So that's why I really, really (really!) want to have the "stream" input slot to the right of any other slot(-s) in show stream and any variations thereof. Of course we can discuss self-documenting labels! Perhaps something like:

Streams extended script pic (1)

As for your (undoubtedly sensible) technical suggestions: I'll look into that later, and try implementing 'em (this may take substantial testing and debugging time, which is why I'm not doing it right away).

Well, I've done some research meanwhile. As it turns out, upvars leaking out of scope is a serious issue:

I tried to produce similar quirky behaviour with cascade from the Iteration library, which has an upvar, but didn't find anything strange here:

Streams extended script pic (8)

As a first step, I have dropped the complicated version of show stream (with upvars and all that); terminate stream is an alternative for cutting off a stream with a criterium different from its length.
Secondly, I'm going to look into it further. Perhaps upvars and lists don't mix well at all?

Hmm ... I'm not a Roman catholic.

That's not so bad (other than the hideous olive green). And in the simple version with just two inputs the order isn't so important to me anyway.

Groovy.

Oh, I don't believe in those miracles (although btw all the Abrahamic religions believe in burning bushes and parting the Red Sea and so on, and earlier religions were even worse, so you shouldn't pick on Catholics), but I do believe in technological ones. A covid-19 vaccine in the first year of the pandemic (the only good thing Trump ever did, so of course he later disowned it) [no politics on the forum --moderator] is an example. But the paradigmatic modern example, for all its myriad woes, is the smartphone, especially for those of us who grew up with million-dollar huge computers whose storage capacity and speed were tiny compared to what everyone now carries around in their pockets.

That’s just a temporary category for Under (re-)construction.

I have been thinking about an alternative for use of upvars in callback blocks.
It’s a little less convenient perhaps, but empty slots may be safer in the end:


Yes - end is reported as input by a Boolean (unevaluated) slot. So normally it would be a predicate. But I wanted to enable users to alternatively enter a positive integer as index of the last item to be reported. A number in a Boolean (unevaluated) slot is interpreted as a reporter of that number, so it must be call-ed; there’s no use for other input at calling time, as all of the required information will already be there.

Are you suggesting something like the following?

Thus the internal function becomes more iterative and less recursive, i noticed that internal procedures, assigned to a variable, are often somewhat slow when compared to external helper procedures. Is there a connection?

I know; I was (mostly) joking.

I'm not sure what you mean. I don't see any empty slots in your example.

But can't the number be provided by an enclosing function?

No, I'm suggesting deleting all the asterisks and leaving the code otherwise unchanged.

Yeah, my example was unclear. What I meant is to not use upvars but tell users instead that the block (e.g. exec, a simple example block I made up) will supply e.g. value and index as parameters to the callback function:

Streams extended script pic (10)

Knowing that, the user can decide how they will define the callback function. The list below contains 5 examples:

  • the 1st example is with empty slots;
  • the 2nd is equivalent to the first but with (optional) user-created parameter names;
  • in the 3rd example the user must use parameter names, as the callback function will otherwise use the wrong order (and the result will be like examples 1 and 2);
  • it's about the same with example 4; if no parameters are specified the result will be like ...
  • example 5.

Streams extended script pic (14)

What I wanted to achieve is that a user could simply enter a number instead of a predicate; and that the number would then be interpreted as [ _ > number ]. It was actually an input parameter slot of type Any (unevaluated) - not Boolean (unevaluated) as the latter won't accept numbers to be kbd-entered. Perhaps the next script / result pic will clarify my point:

Streams extended script pic (18)

Since I'm probably not going to be using this trick any more, it doesn't really matter.

Do you mean this then?

Streams extended script pic (20)

I also tried other options, such as an external Helper block, and an iterative version - but differences in runtime are IMO negligable (and not systematic):

Streams extended script pic (19)

I don't know of any differences of memory use - wouldn't know how to measure these.

Preparations for library distribution

I prepared three sets of blocks:

  1. a comprehensive set of library blocks to be distributed anytime you like
  2. a set of blocks that need some more testing
  3. a set of blocks under development

I put a warning message about the slight change of data structure in the help text of tail of stream, and referred to this text from the latest versions of the pre-existing blocks (= currently included in the Streams library). They may be recognized by a check mark ( :heavy_check_mark: ) as first character of their labels. The help text of tail of stream must be updated with the new version id (or distribution date) of the library, IMO.

Did you reconsider the categorization thing? Since streams are actually a separate data type (if stream blocks look almost the same as list blocks, it will be easier for users to mix them up, especially in front of, map and keep, perhaps uniques of) (and is empty? - not such a big deal), and we have plenty of blocks (24 27, added combine a.o.) which would otherwise overcrowd the already overcrowded Variables / Lists / Other palette, I really think stream blocks should be a separate category, with a distinctive colour, just like e.g. TuneScope and SciSnap!. What do you think?

Recently discovered issue (at least: new to me)

I tried: Streams extended script pic (21), and found the process wouldn't terminate.
Then I tried the current library's: untitled script pic (57), which produced the same effect.
I guess there is no solution for this, a stream must be evaluated once to find out if there are any elements in it. I suppose you must have been aware of this. Any thoughts?

I made a remark in the Help text of is stream empty?.

Another question

Do you think it would be useful to offer both value and index to all map reporters and keep predicates? Like I did with the terminate block (you won't like the layout - I'm looking for a better name of the block, where it makes sense that the stream input is at the back: once…?). It may not always be useful, but at least it would be more consistent with similar list operations. The only downside I can think of is that existing user code may have used more than one empty slot for value, like in:
untitled script pic (58).

Repairing show stream

You may not be going to believe this … I fixed show stream. What was even wrong with it? Well, it kept searching for one more item after all of the requested items were found. Unnecessary and even potentially harmful! E.g.:

will not terminate. This is caused by the fact that the input parameter stream is evaluated even if number = 0. I repaired this by making stream an unevaluated input, and testing for number = 0 before evaluating:

Thus:

(in my library version show stream is a compatibility block, effectively just calling list items from stream)

Meanwhile I discovered a downside to my improved block: monitor (log↑) stream (s) doesn’t work any more, i.e. I can’t read the upvar in a next line of code. Probably because the stream chain is now unevaluated. Well, never mind, I wasn’t too enthusiastic about upvars in the context of streams anyway. I re-engineered monitor ….

Something else I encountered (is it a bug?)

I made a copy of an existing custom block (show stream), and changed just the type of 1 inout parameter (from List to Any (unevaluated)). I could now save the new block without a trailing (2) in its label. so it was apparently recognized as different from the original. However, I could not edit both blocks at the same time: once I opened a second block in the editor, the first one disappeared, and vice versa.

Ah, I see, one of the problems I'm having is that in my idiolect, "callback" means something that happens asynchronously, later. Your FUNCTION is just a function. :~)

Huh. I would have expected the fifth example not to fill any empty slots, since the number of actual arguments provided is neither the same as the number of empty slots nor 1. I vaguely remember Jens changing some rule about filling slots for some reason; that must be it.

I don't absolutely hate your idea here, but I don't love it either, because I think something that looks like a variable is much more self-documenting than just a piece of title text. I had no idea what the "(value, index)" was trying to convey until you explained it.

We have to reach a meeting of minds about this, because I see this style is growing like rabbits in your library.

In the list HOFs, you never get (value, index) put into two empty slots. MAP and KEEP expect a one-input function and put a list item into all of its empty slots. If you want to know the index, you use explicit formal parameters in the ring. COMBINE expects a two-input function, and an associative one at that. (Your COMBINE has title text "(precursor, value)" but as a matter of pedagogy we never talk about it that way; we don't say whether it folds left or right. We think of it exactly like the newly variadic infix operators;


simply means
untitled script pic (3)
and since + is associative our students don't know or care whether the addition is done left to right or right to left. So I especially don't want that parenthesized-input-names notation in COMBINE.)

I'd like the stream HOFs to behave the same way. MAP and KEEP should substitute a stream item into all empty slots inside the ring. (The same item into all of them.) The trouble is that the list HOFs, being primitives, have this special deal of being able to say what input names it wants in that ring, instead of getting #1, #2, etc., as custom blocks get. I wonder if I can talk Jens into adding that for custom blocks somehow.

Yeah, I guess so. If you made that by renaming all the STREAM* things to STREAM. Did it work?

The problem has nothing to do with IS EMPTY?. The KEEP by itself has the same problem. It will fail if the input stream is infinite, but only finitely many items satisfy the condition; when the last one is reached, it goes into an infinite loop (because it's always looking one ahead of the one it just found). This is officially not a bug.

IS EMPTY? itself doesn't look at any items of the stream, not even the first item. Read the code! (The code, actually, should compare its input to THE EMPTY STREAM rather than calling the list IS EMPTY?, just in case we decided to use a non-list to represent the empty stream.)

So the comment is wrong. There's no danger of IS EMPTY? looping. That warning should, if anything, be in the KEEP help text. Also, the first line of the help text is kinda wrong, because it suggests that the block returns NOT of what it actually returns.

Yeah, okay, if there are going to be all these new blocks I guess it has to be a separate category. We should talk about whether it should look almost-list-color or totally different. Maybe a reddish orange? Such as crayon 46. Sorta list color is the right thing, I think, since abstractly they are lists, just implemented differently.


Could you look inside all these blocks again? SHOW STREAM had nothing in its REPORT block. STRICTLY UNIQUES fails if the last item of a finite stream is repeated:


... never mind that "strictly" is an adverb and "uniques" is a noun. And "strictly" isn't what you mean anyway; if anything it's the other one that's strict, because it catches equal items even when they're separated in the stream, whereas the allegedly strict one doesn't catch separated equal items.

But anyway I don't see why you need two kinds of UNIQUES. The not-STRICTLY one works on sorted or non-sorted, finite or infinite streams. What's the virtue of STRICTLY? Just efficiency? It's true that for finite streams UNIQUES is quadratic time while STRICTLY is linear. I'm not sure how to talk about efficiency for infinite streams. In the limit, ℵ₀²=ℵ₀. :~)

Another thing we have to talk about is blocks that work only for finite streams, such as STREAM SORT. That's an extreme case; its input isn't even a stream, let alone an infinite one! And even if it did take a (finite) stream, there wouldn't even be any efficiency benefit, because you traverse the entire stream to find its minimum element. So I want to leave that one out for sure. We can talk about ones that require finite streams but do result in an efficiency gain.

The 27 blocks (plus hidden ones?) are a weakness, not a strength, imho. For sure I should have had INTERLEAVE. But a real virtue of HOFs for lists is that there are three of them: MAP, KEEP, COMBINE. (And FIND FIRST, a variant of KEEP for efficiency reasons. Which should have been FIND FIRST INDEX.) I'm not convinced we need HEADS, ONCE, MERGE, ITEM, or LOG, but I'm willing to be convinced. If we have MERGE, its naming needs work. "ORDERED BY" suggests a comparator, such as <, to me. What you have is a selector, e.g., untitled script pic (5). I can see use cases for both possibilities, although you can use


as the comparator, whereas I don't see how to hide a comparator in the selector if that's what you take as input.

Oh speaking of ITEM, why do I say we don't need ITEM for streams? Because that's not how you use a stream. ITEM is great if you have the whole sequence in memory, as in a list. But with streams, the typical workflow is that you read an item, process it somehow, and then it's gone. You don't jump around in a stream. So HEAD and TAIL really are the only selectors you need. (There's an "on the other hand," namely that if streams are an abstraction over lists, you should be able to do anything to a stream that you could do to a list. And indeed, you (the user) can write stream ITEM, in Snap!, if you want to. But if we provide it in the library, we're encouraging a non-idiomatic way to use streams.)

Okay, I'm tired, I'm going to bed.

Yeah, I used “callback” to distinguish it from the block to which it is an argument. But indeed, the term may refer to asynchronous~.

Stream equivalents of list HOFs
I would absolutely prefer analogous APIs and workings between streams and lists here.

  • I wasn‘t aware of the current working of combine (which is different form SICP), but that difference has now been removed, by dropping the “starter” (and no more parentheses needed).
  • As for map and keep: I would prefer to use the 3 standard parameters that appear when you depress the ring’s right arrow of regular (= list~) map and keep, respectively (and of course the 3rd parameter would be stream - which IMO would have to be limited to the yet-evaluated part of the stream; and preferably with items in reverse order, what I’ve called history). But as you noted, this is not (yet) working for custom functions - let me know if Jens would be willing to change that anytime soon. For the meanwhile, in my experience upvars and streams don’t go together very well, so I think either it’s my system for now or an equivalent (any suggestions?) or only the value can be used.

It worked, but no better than with asterisks.

I’ll look into that again. Thx4 your explanation.

Categorization
OK, I’ll make a proposal.
i found colours in between Variables and Lists offering too little contrast with either one. My #1 candidate colour is crayon #23 (dark candy apple red). I found it by extrapolating the difference in RGB values of Lists and Variables beyond Lists (RGB = 191, 36, 5) and searching for the crayons at closest pythagorean distance - this colour is in the top-3 or so. It’s not yet used by any official Snap! library AFAIK (SciSnap!, MQTT). What do you think?

Ii repaired show stream. As for strictly uniques: it will select only items that occur exactly once. But I think I’ll drop this block.

True, stream sort does not operate on streams - what it does is use the concept if streams for a partial sort. So it’s a demonstration of streams’ practical use (and if Jens hadn’t pumped up the speed of [sorted] OF lately, it would actually have been useful :smirk: ).

I agree, 27 is too many. So we must make a selection. I’d drop heads in front of stream (at least from the palette), stream repeating, keep from map over, strictly uniques. I’m not sure about item of stream - it’s from SICP where it’s called stream-ref I think; show stream is to be kept for compatibility, sieve and stream sort can be moved to the demo block IMO.

So we’ll end up with about 20 blocks in all, I guess. Do you think that’s acceptable?

COMBINE knows about the identity elements of the primitive associative dyadic functions:

So you should do that too. (This is why we don't need the null case input like SICP!)

Hard no on a general feature letting users specify input names. For my next trick, I'm going to try for value/index/list as a special case, or maybe just value/index since, as you say, "list" doesn't work for streams.

I know you've explained this before, but could you give me a minimal failing example? I really like # as a convention.

Oh, I see, if you want the top 10 of something or other and don't care about the other 99,990 of them. Yeah, maybe that goes in the examples.

I still don't see why KEEP FROM MAP OVER is less useful than MAP OVER KEEP FROM. Do you have a convincing reason, beyond that some language that doesn't even understand how lambda works has a syntax for it?

Ooh, crazy idea:


or maybe even

If you leave one of the map function inputs empty, it means the identity function. Or, since efficiency is the object of the exercise, you check first thing for an empty map function, and if so, which will almost always be the case, I bet, call one of your two existing blocks.

I would leave out GENERATE STREAM, for several reasons:

  1. Just look at the block!
  2. Finite streams aren't where it's at. Look at your example; who would use streams to make a mere finite sequence of Fibonacci numbers when, with less effort, you can make all the Fibonacci numbers?
  3. I don't like the HISTORY idea because it duplicates information we already have, in a space-inefficient way.

I would also drop ONCE, another finitization block. Also, here's a heuristic: If the help screen is in portrait orientation, the block is too complicated. :~)

Interruption, before I forget: All those demo scripts with empty bodies (just an empty REPORT) should go, especially the ones with a comment that doesn't apply to the block in question! :~)

About UNIQUES: I can see that being useful, but since we're in the pedagogy business rather than the production code business, I want, as much as possible, for everything in the library to be readable and inspiring. So if we're going to have UNIQUES, I want this one:


even though it runs way slower.

INTERLEAVE: Radical proposal: Write it for exactly two inputs. If you want more, you can always


(Note that that's the List COMBINE block.)

A secondary reason (besides simplicity) that I find this pedagogically better is that I'm nervous about your BEHIND procedure. I think it's too easy to miss the (short) name and think it's IN FRONT OF STREAM. (If you need that block, call it STREAM _ FOLLOWED BY ITEM _ or something.) Anyway, have you ever needed to interleave more than three streams?

What about MERGE STREAMS? I can see how that could be useful, kinda like a special case of INTERLEAVE, but maybe only useful for finite streams? In particular, I don't think you can write mergesort for infinite streams. (Or any kind of sort, really, I guess, except maybe for special purpose ones such as bucket sort.) Again, I would consider taking just two streams as inputs. I would also consider leaving it out.

LOG TO: The block is okay, apart from the bug -- you called one of the inputs REPORT in the header but LOG in the body. I don't like your input ordering, mainly because in English you don't generally put prepositional phrases before the direct object, so it sounds really awkward. And imho easier to get the order of inputs confused. (I haven't forgotten you like the stream at the end for ease of dropping into the input slot. Consider instead putting a newline after the input slot.) Also I added a feature: annotating each entry with a text so that you can log more than one stream into the same list. (See revised demo.)


Output:

As the example shows, one reason for the two separate ADD blocks, rather than adding (list text item) each time, is that this way the whole thing is readable in Snap! table mode display, instead of showing Screen Shot 2024-03-18 at 6.03.20 PM.

Huh. Shows it's been too long since I've read Chapter 3. Okay, groovy.

Is that true? I don't so much care about STREAM SORT, because I don't teach it, but how are you going to put SIEVE in the demo block? We don't have internal definitions. I suppose you could metaprogram it, or you could put it in a variable and CALL it, but neither of those is beautiful. Opening SIEVE in a block editor is beautiful. I guess if you have one favorite amazing demo you can make it a block too, even if it's STREAM SORT, provided you rename it and give it a good help screen to make its domain and its purpose clear. :~)

I missed this part before:

Oh yes that's beautiful. Boy, all these years I saw the Lists color as very close to red, but in comparison to your color it's practically orange!

The zebrafied version of your color is okay, kind of a dark pink, but compared to the vibrancy of #23 it's really nondescript!

I’m already using a different way, not requiring any knowledge at all. :smirk:

I’ll leave an example here as soon as I have the time to create it.

See Sprite “Upvars issue”.

I happen to love crazy ideas that actually work!

The basic idea is to have a block that is really flexible in creating a stream, ideally such that this block is all you ever need to do so - in contrast with a chain of streams, consuming (much) more runtime.
Ad 1, 2: Agreed, I could drop the finished predicate, thus the block becomes shorter and less complicated.
Ad 3. However I’m not ready to drop the history part, since it facilitates creating e.g. a Fibonacci stream - searching for data in the already evaluated part of a stream may still be very time-consuming due to the underlying data structure. On the other hand, I do understand your concerns about double storage. So what I’m going to do is limit the amount of history to the length of the precursor list, e.g. 2 items in case of “Fibonacci”.

About Uniques: let’s have the “paedagogical” version in the palette; and hide the fast version, to be included as an alternative implementation in the edit window.

Great! I’ll implement that, and add the multi-stream merge (using combine) as a demo.
What I hadn’t realized was: the share of the first pair of input streams in the output stream is disproportionally low. E.g. if there are 4 streams, their respective shares will be 12.5 (2x), 25, and 50%. That might become an issue, I suppse.

Nice additions!

OK, let’s keep sieve in the palette, if only for historical reasons; and take stream sort - why change its label? it isn’t sort stream, or something! - out of the palette (so as a hidden but otherwise full-blown block), and integrate it in the demo.

Does it get the right answer if you say
COMBINE (EMPTY STREAM) USING (MAX)
?

Oops I forgot, you don't give the final answer; you give a stream of partial answers. Maybe we should change its name to ACCUMULATE or PARTIALLY COMBINE or something?

Thank you! I'll look at this.

Groovy; do you like the version with colored text?

Hmm, that's a good solution from an efficiency standpoint, but I'm afraid it may confuse them. I really want ITEM ((#)-1) OF STREAM (which could be implemented with a bounded history list internally if you like).

Fine.

Huh. You're folding right to left, which is fine. I'm accustomed to getting too much of the first stream -- which is actually quite convenient if you're trying to make the stream of disk moves to solve an infinitely tall Towers of Hanoi problem! :~)

This is one of my favorite programs! So elegant! Would be even better if INTERLEAVE delayed its inputs, so I wouldn't need the IN FRONT OF around it; the program would just say

When I gave this as an exam problem I got several entirely different good solutions! :~)

Right, I may have forgotten that, late at night, but I'd really prefer a name such as PARTIAL SORT OF LIST _ USING STREAMS or something. Maybe that's overkill. The key point is "list" in the name.

How about incrementally combine?

I constructed an even more basic example (it's at the same sprite):

Streams extended script pic (25)

Streams extended script pic (26)

Streams extended script pic (27)

with:

Streams extended script pic (28)

I'll think it over some more.

It's not just from an efficiency point of view - referring to history from the current item down (let's call it: hindsight, actually suggesting a reverse view) feels so much more natural to me. When you are e.g. at item 100 of a stream, either all of the previous items are relevant, or none, or just a few recent ones, but never just items 1/2/3 ... - or do you know of a counter example?

I copied your code proposal for interleave, didn't I? :wink: However that may be, I still don't like the unintended effect of combine (streams ...) using (interleave () () ), resulting in an uneven distribution of output items between input streams. So I'm including the variadic interleave many streams (not in palette) as a demo example in the edit window (and using your suggested (data) followed by (value), even though I find that a bit awkward).

I'm using stream of smallest items from list now.

Sure.

Thanks!

That's why I said ITEM ((#)-1)! I envision that this would only work for a small set of negative offsets from #.

I think the party line about this is that once you're interleaving, you don't care about the exact order in the output. (My Towers of Hanoi example is an exception.) You care only that every item you allege to be in the stream is reachable in finite time; or as @polymations would say, the ordinality of the stream must be ≤ ω. (This is why there's no APPEND for streams.)

Oh, that's not right; the stream has the entire list, doesn't it? So I guess PARTIAL SORT is wrong, too. SORTED STREAM FROM LIST maybe?

Appending an item to a stream is like adding 1 to the left side of omega. (1+ω)

I may not understand what you mean … What would that look like?

OK, I’m prepared to suppress my skepticism. Let’s see how it behaves with the logic programming interpreter.

The input is an unsorted list. The stream of sorted items will be constructed item-by-item. If asked for e.g. the 5 smallest items, only 5 stream items will be computed.

That's > ω, right?