New syntax embedding feature in sublime-syntax files

[keith-hall]

Sublime Text 3153 was just released with

a change to how syntax embedding works, which makes languages embedding lots of others more practical, and can also significantly reduce memory usage.

the changelog shows:

Syntax Highlighting: Add embed action to .sublime-syntax files, which can significantly reduce memory usage

No details or documentation updates yet, but wanted to log it here because I imagine we will want to add support for it to syntect.

[trishume]

The most I can find on it is this commit: sublimehq/Packages@72bd6bd

Interesting, I wonder why it improves memory usage in Sublime. I'm not sure it'll do much with syntect's architecture other than make things more complicated.

[wbond]

Interesting, I wonder why it improves memory usage in Sublime.

Rather than creating copies of regexes with with_prototype patterns added to do escapes, we match the escapes separately. We also improved memory usage related to syntaxes in a number of other places.

There is temporary info/docs available at https://forum.sublimetext.com/t/dev-build-3153/33014/8.

[keith-hall]

The embed feature is now being used quite a lot in the syntax definitions that ship with ST. I'm thinking we could get syntect to support it by converting embed/escape to the equivalent with_prototype at sublime-syntax "YAML load" time, which should hopefully avoid making the architecture too much more complicated.

[trishume]

Yah if there's a straightforward translation that would work. I'm absurdly busy with school for the next month so I won't have time to do anything for the next month, possibly longer. But if someone else wants to do this I can do code review.