Hi,
Snap! looks amazing. I'm new here - does it have hygienic macros?
Kind regards,
Stephen
(for context I'm a Racket user and I've heard the core team speak highly of Snap!)
No, at the moment it doesn't have macros, period. That's the one big hole in it -- we do have continuations.
I confess that I'm thinking of just having old-fashioned FEXPR-type macros. But we'll see.
Nice to hear that they're saying good things about us!
1 Like
I’m no expert but fexpr’s seem like a bad idea.
Have you seen he HOPL paper Hygienic macro technology by Clinger and Wand?
William D. Clinger and Mitchell Wand. 2020. Hygienic macro technology. Proc. ACM Program. Lang. 4, HOPL, Article 80 (June 2020), 110 pages. DOI: https://doi.org/10.1145/3386330
I was part of the conversation on comp.lang.scheme back when they were inventing hygienic macros, so I know all the arguments.
The thing is, doing macros at all is going to be very hard in a block-based language. In Lisp, it's easy because code is automatically data -- the surface representation is a list. Our scripts are first class, but they're atomic; there's no way to look inside them. So the first thing I have to do is invent a translation between blocks and manipulable code. That's already messy: Do I treat variable references as a special case, as in Lisp, or do I just represent them as niladic functions, which is how they look as blocks? How do I represent the header (the prototype, in our documentation) of a procedure? Lisp has the procedure's name as a single atom at the beginning, but our blocks' titles are spread out all over the block. All that sort of thing has to be specified before we begin.
So once that's ready to go, I want to be able to test its usability without debugging hygienic macros at the same time. FEXPRs I can implement in my sleep. So after that's debugged, then I can think about implementing hygienic macros instead.
But, let's talk about why you think it's a good idea for us. Our users aren't writing humongous industrial software in teams of 500 programmers. They're kids. Back in the day, when they first invented the R5RS macro system, there were a bevy of comp.lang.scheme posts from famous Knights of the Lambda Calculus who couldn't figure out how to express what they wanted to do hygienically. By contrast, anyone can understand FEXPRs. They have to be taught about the danger of name capture, yes, but I think that's an easier lesson than using hygienic macros.
As an analogy, Logo is dynamically scoped because, for several reasons, that's better for learners. I can go into the reasons if you like. I wish Snap! were dynamically scoped. Jens won't allow it because it would make our interpreter slower. (I thought I had a solution to that problem, but my solution fails in cases of tail call elimination.) That's actually a pretty good example, because hygienic macros are the compile-time analog of lexical scope.
So when you say "fexpr’s seem like a bad idea," is that on general principles or do you have a teaching reason in mind?
I admit that my slogan "Snap! is Scheme disguised as Scratch" would sort of push in the direction of hygienic macros. But historically we built Snap! more with the idea of Logo disguised as Scratch. It was just when we added lambda that we started thinking more in Scheme terms.
1 Like
Just feeling the need to officially log my objection to the claim that dynamic scope has ... any ... benefits whatsoever, except being trivial to implement.
But it's not trivial at all to implement efficiently. You need shallow binding, and you need to think about what happens in a tail call elimination -- as you pointed out to me.
Sigh, okay, here's the argument:
First of all, in a language without lambda (such as Logo), a procedure's dynamic environment is a superset of its lexical environment, so it makes more of the user's variables available in the procedure. And unless you call all your variables x it's likely to be perfectly clear which you mean.
That last sentence slides into the second point of the argument, which is that unwanted name capture is a likely problem only in the situation of multiple programmers working on the code (and no module feature).
The third part of the argument has to do with wanted name capture. In particular, in a dynamically scoped language, a procedure is equivalent to its text (i.e. it has no remembered environment), so you can use EVAL as a sort of poor-man's APPLY in writing higher order functions:
to map :function :data
if emptyp :data [output []]
make "question.mark.variable first :data
output fput (run :function) (map :function butfirst :data)
end
to ?
output :question.mark.variable
end
? show map [? * ?] [4 3 2]
[16 9 4]
Not only does the ? procedure get to capture the value set in MAP, but map can capture variables from its caller:
to scale :scalar :vector
output map [:scalar * ?] :vector
end
? show scale 3 [4 5 6]
[12 15 18]
This isn't quite falling-off-a-log easy, but it's easier than the struggle with lambda that all Snap! newbies have to transcend. (Jens will say that hyperblocks help, and he's right. But they're not perfectly general.)
The remaining points have to do with debugging. For interactive debugging you want the capability to get to a repl with the environment at the time of the error intact. We sort of do that with the PAUSE ALL block, but we should really have the ability to pause rather than stop automatically when we catch an error. Once you do that, though, you need to learn a special debugging language, different from the language you're accustomed to programming in. For example, in Snap!, you have to know to right click on a suspended script to get options to open stage watchers for local variables. (I note in passing that Jens implemented this so as to allow the user to access the variables of all suspended scripts, not just the one on which the user right-clicked. So Jens does understand that you need -- absolutely need -- dynamic scope, which is what that is, to make debugging tolerable.)
But in Logo there's no issue about learning a special debugging language. Because of dynamic scope, all relevant local variables are directly accessible to Logo instructions you type at the repl prompt. Your debugging uses Logo itself. You can set as well as read variables, you can specify what value to return from the primitive that threw the error, you can use Logo's nonlocal exit features to continue from a different part of the program. The only thing you have to learn just for debugging is the CONTINUE command that continues from the point of the pause.
None of these arguments are meant to apply to professional programmers in industry. We're talking about learners, especially young learners (or "young thinkers" as they say in Germany).
I shudder by reading these examples and arguments. They're a hack offering an ugly shortcut for already proficient kids. The very idea that a function behaves differently - or even fails altogether - depending on the internals of which other function calls it - and expecting learners to appreciate such horror as in their own best interest - must be among the worst pedagogical delusions ever. I'm not buying into this "kids vs. professional programmers" dichotomy at all. Especially since we're literally representing functions als graphical building blocks the mere idea that you should not be able to use them interchangeably is ... I'm running out of strong vocabulary here.
Well, I know you do work with kids, but this doesn't sound like it! We kids often write procedures that will only be used by one or two callers, and that make all sorts of assumptions about what the caller wants and what environment the caller provides. We don't publish such procedures as libraries (although we do sometimes include such a procedure as an internal helper for the library).
Just for example, in the Colors and Crayons library I have a procedure called "initialize variables." It sets up a bunch of global -- well, global constants, really, although that's not a concept in Snap! -- after first checking to make sure they don't already exist. That procedure is absolutely useless in any context other than this library.
I picked that example because it doesn't raise any contentious issues about scope of variables; the variables it deals with are global. All it does for my purposes is make the point that your phillipic against procedures that can't be called by anybody is overblown and way overgeneralized. (The strong vocabulary doesn't exactly lend itself to the reasoned exchange of ideas. As we speak I'm trying to teach kids on the forum not to bully verbally.)
The specific case of a procedure meant to be used by only one caller could be completely solved without recourse to dynamic scope if we had internal definitions. But a procedure meant to be called by one of three other procedures is very hard to solve, if the procedure wants access to its caller's variables, in a lexically scoped language. I didn't say impossible to solve; SICP does it in the context of implementing a numeric tower, including alternate forms of a given type (think rectangular vs. polar forms of complex numbers). But it's pages and pages of code, really hard for students to read and understand, one of the rare places in SICP where the pain doesn't pay off in a huge general computer science idea.
I'm guessing that by "an ugly shortcut for already proficient kids" you mean my definition of MAP in Logo. That definition isn't necessarily something I'd teach kids, but they can use MAP, and use its ? feature (the historical origin of Snap!'s empty input slots) without having any more sophisticated understanding of MAP's function input than that it's an expression, the same as they might use in a PRINT instruction. We have a really, really beautiful representation of lambda that allows kids to start using higher order functions without having to think very hard about what the rings mean, but I point out that it took us three tries to get it right, and if we were in a text language we couldn't use it. The kind of thing I'd show a kid and expect them to understand is the definition of SCALE in that message, not the definition of MAP.
And if you still can't stand any of that, don't forget that it was just one of four arguments in favor of dynamic scope in a language for learners.
P.S. When I suggest something to you, and you calmly tell me why I'm wrong, you're usually right. But when I suggest something and you blow up and can barely get the words out and walk out of the conversation, I'm usually right. Just so you know.
Not telling you that you're wrong. Dynamic scope, after all, has been - and perhaps still is - a thing in programming languages. I do have strong objections against dynamic scope both from a programming-wise point of view (abstraction, interchangeable parts) and also from a pedagogical one (for learners, beginners even, to distinguish between different situations in which the same block with the same input behaves differently or fails is expecting a lot of them), that are not about "making our interpreter slower".
Right, I left out the part of the story in which I proposed hybrid scope, which satisfies most of your objections, and that's when you told me it couldn't be done for efficiency reasons.
About pedagogy, you are imagining a situation in which a procedure presents itself as a general-purpose procedure but has a secret gotcha hidden inside. But, again, the situations in which kids benefit from dynamic scope are the ones in which only those three specific procedures over there are ever going to call this one, and it just wouldn't make any sense regardless of scope for anyone else to call this procedure. Situations like that come up all the time, and the way our users solve them is by throwing their hands up in the air and using global variables.
It's true that in Snap! we have a better solution to the problem of higher order functions: gray rings. I'm very happy with our treatment of higher order functions. My showing the Logo code for MAP wasn't to say we should do it that way; it was to say how Logo, which as a text language didn't have the option of gray rings, was able to come up with a good enough solution to the problem of higher order functions needing access to their callers' variables without making the users say
(map (lambda (x) (* scalar x)) vector)
But since that seems to be confusing the issue, let's just talk about cases in which our users have to add a bunch of extra inputs to their helper functions whose values are exactly those of the caller.
hybrid scope makes block behavior even more unpredictable.
OK so first of all, don't forget the context of this discussion, which isn't that I'm proposing to change Snap!'s scope rules, but that I'm using dynamic scope as a metaphor for FEXPR macros, and lexical scope as a metaphor for hygienic macros, to make the argument that we may be better off with the simpler-to-explain kind of macro.
So, if you hate dynamic scope, maybe you'll hate FEXPR macros too, and we need to clarify that, but we don't need to make any decisions about scope in the context of this discussion.
Having said that, I don't see why hybrid scope is "even more unpredictable." All it does is make something useful happen instead of an error message in cases where it's absolutely clear that a dynamic reference is what the user intends. So people who hate dynamic scope will never see it.
The thing that's unpredictable and horrible to you is if the user makes what's intended as a lexical reference and surprise! they get a caller's local variable instead of the lexically available variable that thay were trying to access. And hybrid scope doesn't raise that possibility because, in particular, if there is a global variable matching the name the user uses, that's what they get, even if a caller also has a local variable with the same name. So I don't see why you're reacting so extremely.
And this is a new thing! I brought up hybrid scope long ago, in the BYOB days I think, and you had practical problems but didn't use words like "horror."
P.S. I could live with having a DYNAMIC VARIABLES block that looks like SCRIPT VARIABLES but instead of creating a variable it allows dynamic reference if there's no lexical candidate.
Thanks guys I’m amazed by snap and how it is literally light years ahead of any of the other block PL’s I’ve seen (TBH I only know blockly for the code bug and scratch).
It is truly an amazing achievement. The macros was just an question not a suggestion - it seems right that that might be a bridge too far with the intended audience. (But I’m not a CSE expert - I’m a health developer with an interesting in PL’s - so I’m really out of my depth here!)
I know at least one other language with lexical scope that retains a sort of dynamic scope with special protections so it can be used when appropriate - but beyond using the feature I don’t really know the implications for snap and it’s audience/purpose.
To be honest I thought dataflow is a more interesting direction than either scope or macros and is a more natural fit with the snap as a ‘functional programming’ paradigm language. That said I’m really out of my depth here, and haven’t though carefully about this so please don’t take me too seriously. (I have played a little with Pure Data and it is fun!)
Kind regards
Stephen
Thanks!
What's the other language with (what sounds like) hybrid scope?
I've thought about dataflow. But (maybe because of my ignorance) I don't see how to handle conditional evaluation other than using explicit continuation passing style, so I always get hung up on that. But it does fit with the visual metaphor! I'll have to look into it more.
Oh, I see, it's not an alternate binding discipline for regular variables, but a special kind of variable, that's implemented as a procedure. That's kind of like my DYNAMIC VARIABLES suggestion earlier in this thread. I wonder if Jens could be talked into that; it would completely eliminate the possibility of getting a dynamic binding instead of an error message when you mistype a variable name, which is the only possible bad result of hybrid scope.