Convert Scratch 3 projects to Snap!

Unfortunately the Advanced Topics are essentially a ghost town compared to the old days. I suspect it's because the forums are much less visible now that the Discuss has been removed from the header. You probably wouldn't be getting many new faces over there anyway.

Have you tried importing the project into Scratch 2.0, and the converting the SB2 to Snap! with Snapin8r2? Scratch 2.0 tends handle large projects better than Scratch 3.0.

If that fails, trying to extract the images from the YPR file is probably a bad idea. The legacy SB and YPR formats were rather poorly documented, and there were never many tools for working with them compared to SB2. These are my alternative suggestions, from least to most complex:

  1. If Scratch 2.0 can handle it, use it to open and save the project. Change the .sb2 file extension to .zip, and all the images should be in the zip file.
  2. Extract the images from the SB file using https://github.com/tjvr/kurt
  3. Debug and fix BYOB's export code. :slight_smile:

I've avoided working on Snapinator 3 since Scratch 3.0's file format makes me want to bang my head against a wall, but I do plan on implementing SB conversion. So when that's done, it should probably be able to convert your project. I need to do some tech support stuff for my family before I head back to school, but after I finish that up I'll get to work on Snapinator...

Yes. My projects unofortunately are too big even for Scratch 2. Often, when I can open them, Scratch 2 fails then to save them to SB2.

If that fails, trying to extract the images from the YPR file is probably a bad idea.

Extracting the images was just a joke between Brian and me :slightly_smiling_face:. As I said, I have way to many images to think to export and reimport them.

If Scratch 2.0 can handle it, use it to open and save the project. Change the .sb2 file extension to .zip, and all the images should be in the zip file.

Yes, I know. Thanks.

Debug and fix BYOB's export code. :slight_smile:

As Jens knows very well, I'm not a SmallTalk guy at all. That is why I started looking at Nathan's YPR import code two days ago. Unfortunately, Nathan's code is the cryptic Javascript style that I don't love at all. But it is very well organized, so I'm trying to see if there is a problem in the code. My impression is that BYOB and Snap have just two different ways of handling images, and some images that BYOB can show are instead "wrong" for Snap. So it seems to me that the problem is not the image itself but that it must be converted so that Snap can handle it. I don't know yet if this is true and if I can handle the conversion.

I've avoided working on Snapinator 3 since Scratch 3.0's file format makes me want to bang my head against a wall

Really? I don't know the SB3 format at all. Is it so different from SB2? Can you give me some details about the nature of the problem?

I do plan on implementing SB conversion

That will be fantastic. Unfortunately I need to handle this conversion well before mid feburary. Or, at least, I need to try to understand if I have to surrender and stick to BYOB 3 with my students even for this academic year.

Not at all. I didn't mean to do it manually in BYOB; I meant to write a program to parse just the images in a .ypr, all at once -- they're in there somewhere!

Not at all. I didn't mean to do it manually in BYOB

:slightly_smiling_face:

I meant to write a program to parse just the images in a .ypr, all at once -- they're in there somewhere!

Right. That is way I'm "debugging" Nathan's ypr importer. Having a fresh mind helping me doing it would be certainly of great help!

NOTE: you are not in California, are you?

The SB3 format is just bloated, heavily inconsistent, and seemingly poorly thought out in general. These are my main complaints:

  1. The main benefit of JSON is that it's extremely human-readable, when done right. SB2 did a great job at this, and it stored blocks as nested arrays so you could easily see their order and hierarchy. SB3, on the other hand, stores all the blocks as objects in a flat map, and it assigns every block an extremely long, gibberish ID. Each block object points to the IDs of the blocks its related to, and you end up with something like this:
{
    ...
    "(e^J!4h:Sw%:)6]c1{T6": {
        "opcode": "event_whenbroadcastreceived",
        "next": "PJb38pO:f:%v8hL@aP_p",
        "parent": null,
        "inputs": { },
        "fields": {
            "BROADCAST_OPTION": [
                "Game On",
                "broadcastMsgId-game on"
            ]
        },
        "shadow": false,
        "topLevel": true,
        "x": 59,
        "y": 1016
    },
    "PJb38pO:f:%v8hL@aP_p": {
        "opcode": "control_repeat_until",
        "next": null,
        "parent": "(e^J!4h:Sw%:)6]c1{T6",
        "inputs": {
            "CONDITION": [
                2,
                "/gYo2SR*AuVPoDQH94JZ"
            ],
            "SUBSTACK": [
                2,
                "Yc2a!.i7cs?q^[%fZ_dF"
            ]
        },
        "fields": { },
        "shadow": false,
        "topLevel": false
    },
    ...
}

This ends up making the format totally unreadable to humans without a significant amount of effort.

  1. It is bloated. There's a large amount of redundant information stored with each block. For example:
    a. See how the "BROADCAST_OPTION" field up above stores two values, a broadcast name and a broadcast ID? The broadcast ID is just a modified version of the name, and the name acts as an identifier anyway, so there's no need to store both. Variables, lists, and a few other things are also stored each way.
    b. Block IDs. They're random and rather long, and they're only repeated a couple times in each file, so there's no way that they compress well.
    c. A lot of extra information that could be computed at runtime, like each block's parent, is unnecessarily stored.
    All of these things are tiny, but they add up, making projects stored as an SB3 significantly larger than projects stored as an SB2. Those 2 blocks above would have just been a few lines and a couple dozen bytes in the old format.

  2. Magic numbers everywhere. What does that 2 in the PJb38pO:f:%v8hL@aP_p block's input arrays mean? No clue. Whenever I encounter these I have to dig through Scratch 3's source code to find the meaning.

  3. Inconsistency. Sometimes inputs' values are stored in ALL CAPS, sometimes they're not. Sometimes the inputs' values are contained in arrays marked with magic numbers, sometimes the inputs are stored as entirely separate blocks for some reason. Sometimes booleans are stored as actual booleans, sometimes they're stored as strings. This last one isn't even consistent for the same locations, and it depends on how the project was created. Trying to parse anything in an SB3 file is a crapshoot.

  4. Lazy design decisions everywhere. Some portions of the format are just XML converted to JSON in the laziest way possible, with fields for the XML tag's name and its usually non-existent children:

{
    ...
    "mutation": {
        "tagName": "mutation",
        "children": [ ],
        "proccode": "Clone Row %n",
        "argumentids": "[\"input0\"]"
    }
}

And look at that last property, "argumentids". That's stringified JSON stored in JSONified XML stored in JSON. The number of unnecessary layers you encounter is astounding.

I am indeed in California.

Do you still teach at Berkeley?

Not regularly. I retired six years ago. I taught a course after retirement, but not recently, because the Beauty and Joy of Computing project has me spending half my time in Massachusetts, not lining up with Berkeley semesters.

I see.

Hmm, Bingo can read Scratch 1.4 projects, but not BYOB projects. But none of them can read Bingo projects. Any idea on converting Bingo projects?

There is a bug in my BYOB project, which was converted from Scratch 1.4:

If you look in the editor, you will see that the stage is not how you can see in the thumbnail, and while you are viewing it.

Yeah I'm not surprised. Since Nathan grew up nobody's maintaining that code.

I never heard of him. Could you tell me who he is?

nXIII on Scratch. World's best programmer, or at least the world's fastest. He did a lot of work for us when he was a teenager -- just for example, he designed the :snap: logo. But also the code to read BYOB projects in :sn:. And, as it says in the credits, zillions of bug fixes. But now he's a super busy grownup. :~(

So won't you be doing anything? And do I have to redraw that? Thank you.

Okay, at least one of us is confused. We are not going to work on BYOB importing in the foreseeable future. Why would you have to redraw anything?

Okay.

I've implemented Scratch 1.4 project conversion! Costume conversion should also be fixed, as well as some of the missing blocks. https://djsrv.github.io/snapinator/

Yay! Thanks.