Support Scratch 1.4 costumes with text (that are imported from 2.0)

[towerofnix]

This pull request is a work-in-progress. Most of the code was written several months ago and may be more complicated than is now necessary.

Resolves

Fixes #672.

Proposed Changes

Adds support for importing Scratch 1.4 costumes that contain text from Scratch 2.0 projects. This is done by fetching both the "base" and "text" layers of the costume, which are both bitmap images, then flattening the two into a single bitmap image.

This implementation differs from Scratch 2.0, which creates a vector image including a "pseudo"-text element and a bitmap image embed. That seemed rather confusing, and certainly difficult to work with.

Reason for Changes

To implement features which Scratch 2.0 has.

Test Coverage

No test coverage.

[towerofnix]

The current Big Challenge: These costumes cannot be duplicated (or rather, sprites containing these costumes cannot deleted), potentially because their assetIds are not set (correctly). However, the costumes of modified sprites can be deleted.. so looking at how assetIds of modified (or newly-created) costumes works should help.

Edit: this is because of these lines, which make costumeAsset = null, by the way:

https://github.com/LLK/scratch-vm/blob/115184a2a39b990dee9218aa15081b334516e1a8/src/sprites/sprite.js#L92-L97

[towerofnix]

The relevant code for this is in VM.updateSvg:

https://github.com/LLK/scratch-vm/blob/115184a2a39b990dee9218aa15081b334516e1a8/src/virtual-machine.js#L432-L437

So the updated SVG is stored in a cache of sorts. Now to figure out how to store a bitmap(?)...

[thisandagain]

@cwillisf Can you please weigh in here?

[cwillisf]

I'm marking this as blocked for now, and I filed scratchfoundation/scratch-storage#30 to track the issue.

[cwillisf]

Sorry for letting this hang for so long. I think the best move is to close this, since there have been significant changes to scratch-storage as well as the costume loading pipeline, including the creation of scratch-svg-renderer. Parts of scratch-svg-renderer, such as the font replacement step, may be relevant to this work. We also plan on supporting the import of Scratch 1.4 projects (see #128) so it might make sense to hold onto this until after that work is completed.

If you'd like to update these changes to work with those new features then I'm happy to take another look; if not I certainly understand. Either way, thanks for your contributions!

[towerofnix]

Thanks for checking in on this!

We also plan on supporting the import of Scratch 1.4 projects (see #128) so it might make sense to hold onto this until after that work is completed.

That's fine with me! It should definitely be noted that this is exclusively for 1.4 projects which were then opened and saved in 2.0, although the "merge the two images together" step will probably be relevant in 1.4 import too.

Definitely agreed that most of this code is unusable, or at least not directly usable, presently. Best to leave this PR closed and as needing further discussion (then to open a new PR with fresh code).

---

Just some quick notes, for picking up work again on this whenever:

The core of what we need to do here is:

1. If there's no "text" layer, just stop and behave as normal.
2. Fetch both the base and text layers.
3. Merge the two together.

That second step is totally exclusive to 1.4 projects saved in 2.0, since in Scratch 1.4 projects, while there might indeed be a separate bitmap layer for text (I don't know for sure since I don't know the .sb project format), that would be included inside the Scratch 1.4 project binary. Assets in 2.0 projects are (typically) stored separately from the project JSON, i.e. on Scratch's servers, so we need to make sure to fetch the text layer as well as the base layer, when present.

Assuming there is a separate text bitmap layer in Scratch 1.4 projects, the third step, merging the base layer and text layer, will be relevant when importing 1.4 projects, since obviously they should be flattened the same way.

Parts of scratch-svg-renderer, such as the font replacement step, may be relevant to this work.

Maybe, but I don't really think so. The text layer in 1.4 projects (or at least ones which were later edited in 2.0) is its own bitmap image. Assuming that we would want to convert the font used in 1.4 projects to a 3.0-valid font, the same way we do with converting the fonts used in 2.0 projects, I'm not sure that would even be possible. If the font family is even stored in the SB2 (it must be in 1.4, but I'm not sure it is in 2.0), there would be no way to realistically translate it to a 3.0-valid font, because any font which is installed on the author's system may be used in a 1.4 project. The magic is that, since the text is saved as a bitmap image, that font will "work" on any computer even if it's not installed, since really Scratch (in version 1.4 and 2.0) is loading a screenshot of the text. (If you, if you do not have the font family installed on your computer, go to edit the text in 1.4, the font family will be lost; it will reset to the default Helvetica font. In 2.0 editing the text will just scrap the font altogether regardless of whether you previously had it installed, I think, because 2.0 only supports the basic sans-serif/Marker/Scratch/etc fonts.)

Anyways, importing a 1.4 costume, which is composed of two bitmap images, as a vector image would probably be a mess anyways. The resulting image would be a vector image composed of a bitmap image (the base layer) and a text object (generated according to the text). A user would probably end up converting the image to a bitmap image (the "convert to bitmap" button), in which case you'd have been better off just merging the base and text layers in the first place - which would have been the most accurate anyways.

Maybe I'm totally misinterpreting what you were thinking with using scratch-svg-renderer here..?

My proposed solution, and what I started work on here, is to take the two bitmap layers, base and text, and merge/flatten them, creating a new bitmap asset in the process (which is used for the resulting costume).