Experimental First-Class Processes in v12.1-dev

[jens]

Folks, I’ve been experimenting with adding first-class processes to Snap! and I’d love for you to give them a banging and let me know any issues you encounter (here, in this thread).

In the current v.12.1-dev version there is a new library called “Processes”, along with a new selector in the “my” (attributes) reporter:

You can use these blocks to programmatically launch, suspend, resume and terminate threads, and to inspect their properties, such as their current state, their procedure-script, their current block (stackframe) etc. You can keep track of processes in variables and lists, and I’ve made some new “gears” symbols to visualize their state. Some of these are animated and can also be used elsewhere in your custom blocks.

Here’s a rough demo of how the blocks work:

First-class processes is - I think - among the last first-class things Snap! is missing, so this addresses that.

What’s the use case I have in mind for this? Some extensions (and projects) can be greatly improved by one or more helper-processes that always run “in the background”. For example, if you want your project to track your hand and fingers in your webcam’s video feed you need to constantly take single snaps of the video and compute the body / face / hand landmarks on them, before you can even do something interesting with them. If more than a single script follow different body features every script needs to make sure to keep track of the relevant landmarks all the time. This can be a pain to program and a significant strain on performance (as I’ve found out when I tried to do just that with kids last week at SAP).

Here’s an example. Nevermind the state-machine of that custom event block, even the simple script on the left that makes the sprite follow the hand is really complicated:

this is where - I think - first-class processes can help: Every custom block that relies on a certain feature can first call a block that constantly tracks them. This way, the “center of hand” reporter doesn’t need an input slot for the “hand landmarks” anymore:

That block, in turn, checks if a tracking process is already running, and only (re-) starts it if it isn’t (either because it hasn’t been started or because the user has pressed “stop” at some time).

The tracking process constantly analyzes the current webcam frame for certain body features and publishes them in a global variable (which can be hidden, but the other custom blocks query it). And that lets us build really easy-to-use custom blocks for such domains:

Here’s how that project-in-the-making looks right now:

And here’s a link if you want to give it a try yourself:

Body Language NT

Thanks!

[onu032001]

It’s ~ process ~, not ~ Prozess ~.

The number of sawtooth has been decreased, from 8 to 6.

[jens]

@onu032001

It’s ~ process ~, not ~ Prozess ~.

Thank you! Fixed.

@onu032001

The number of sawtooth has been decreased, from 8 to 6.

Yes, I’ve said so in the change log.

[petermathijssen]

This is really nice. I like the body language library. Of course I tried all your gestures With some I had to look up in the code which fingers I had to use. Some were easy.

[jens]

Thanks for the quick test, Peter! I’m not yet decided whether to leave these gesture blocks in the final version of the library. This was mainly an experiment as to what it would take to detect “meaningful” custom gestures. One idea is to let students make custom gesture-detection blocks themselves, so I had to check with some that I found every kid at SAP last week to enthusiastically try :D

[dardoro]

What about this(process) ?
Can process be an argument to ()of()? Say to get variables of the running script.
If my(processes) is available, then the name/description of user created processes can be useful.

[jens]

Ah, great minds!

I’ve been thinking about just that myself, using `this` to access the current process (not just the current context, as is the case right now), and (ab-) using the `… of …` reporter for selectors. Technically this would surely work, but I’m not convinced about the benefits and the clarity. The idea is to keep track of individual processes using variables and lists rather than giving them a name. Of course, if you want to identify a specific process that’s already running it’s a bit harder. In this case you’ll have to compare it’s script to one that matching your expectations. Also, external introspections into more interesting attributes (for the metaprogrammers amongst y’all) is going to be somewhat limited, because things like the continuation of the current context only gets captured when that context is evaluated and there is an internal block that refers to it.

[niki650]

Yo, that’s cool

[angzarr_812new]

WHAT! Animated symbols??? cool

[bh]

Wow! This is cool.

There seems to be a lot of overlap conceptually between these processes and continuations, as can be seen from the thread manager example in the manual, Section X.B, page 100 of the PDF version. So my first reaction was “can’t we already do this?” Maybe it would help me understand this feature if you could post an explicit comparison with continuations in which you explain when you’d use one instead of the other – unless you think this makes continuations obsolete? Or, alternatively, would it make sense to be able to ask for the continuation of a (running) process?

The first class data type I’ve always thought we were missing is first class environments, with an environment represented as a list of frames, each of which is basically a dictionary. It would be especially helpful to be able to pair an environment with the environmentless scripts that your approach to metaprogramming creates. I know this paragraph isn’t directly relevant to this feature announcement, but I wrote it because you said “among the last first-class things Snap! is missing.”

Of course you understand that (at least here in the US) a menu item of fuck you would instantly kill any use of Snap! in schools! Right? I do understand that it’s attractive to kids; that’s exactly why. I think even that gesture with a different name might be enough to get us banned. Sorry for being a grinch.

Wearing your AI hat (as opposed to your media computation hat) you built a gesture recognizer that can be trained by the user. I wonder if providing a primitive gesture recognizer will work against the compellingness of that (great) example project.

[angzarr_812new]

• The animated symbols pause when viewed in a table
• Pressing the pause button while a process is paused unpauses it even if it isn’t paused, but the pause button doesn’t turn into a play button. This allows for a pretty neat pause detector
  
• First-Class*

[jens]

@bh

processes and continuations

Thanks! In Snap, continuations are bounded by processes, they are not used to implement processes. This makes continuations in Snap much more safe to use than in Scheme (something that actually convinced Shriram about Snap, which makes me really proud). But you’re right that continuations are a great representation of a single process. As I’ve mentioned above, continuations can really only be captures by the current context inside a process, and not when looking at a process from the outside, i.e. from another process. This is - I believe - a benefit, because it shields processes from interfering with each other.

When would I use a continuation? Well, for the usual examples, especially flavors of non-local exit. When do I want processes? To handle parallel threads, background processes, to track things like external hardware or an api. Processes are kinda unknown territory in most programming languages, and in Snap they are really integrated into the IDE, so we didn’t really have any need to programmatically deal with them. But here we are ;)

RE the dropdown menu: Don’t worry, as I’ve mentioned I want kids to model their own gestures, including the “forbidden” ones. Isn’t it cure that people in the U.S. seem more concerned about offensive words than, say, … ah, forget it, many things, really.

You’ve got a great point about using a neural network to classify things like hand gestures. But I believe that this little script that simply identifies outstretched fingers and directly maps them to a gesture name is really quick, easy and satisfying to program yourself. Much easier than first compiling a data set of different gestures and training a neural network on it. So to me, this is a very rewarding example of how we can combine AI (in the form of computer vision) with classical programming in a way that is both fun and surprisingly impressive.

[jens]

@angzarr_812new

animated symbols pause when viewed in a table

Yes, sadly they do. Tables are highly optimized for displaying large, even giant data sets. Therefore I’ve limited some of their “liveness” for static symbols (e.g. when displaying nested lists). Inside a table view processes only display as small interlocked gears, without animation and without updating to another symbol when their state changes.

[redgeographysnap]

THIS LOOKS SO GOOD! I’m going to try it out

[redgeographysnap]

Or asshole…

[codingisfun2831t]

Fake OS creators rejoice. I should make one, for that matter, using this..

[codingisfun2831t]

Seems like that process is of course the process running the current script:

But the block of process returns a weird ring! Now I do know that the block is of course, the current bllock the script is running. in that situation, it WOULD be blockof (since its getting the current block!) However I don’t understand why the inputs are the input values themselves?

[redgeographysnap]

I don’t mean to get all JavaScripty :(, but Snap! stores it’s current block, and the inputs passed into it, into a special ring.

[codingisfun2831t]

Yeah, I get that, but why are the input names just the input values? From the gif, I can see that the inputs names should just be the inputs of the block? (e.g. #1, #2, etc for built in blocks)

[codingisfun2831t]

Is there any specfic way to set variables or such to a specfic process? Like sprite-local variables but for processes. I know inputs to the ring are there, but is there any local variables?