Add API to syntax-highlight code

[rock3r]

Users may want to display code, and having syntax highlighting is important for readability.

We need an API that can be used to turn a string of code into a highlighted AnnotatedString. On the bridge side, the styling must come from the EditorScheme, and on the standalone side, we should use the defaults for Light and Dark themes. On the bridge side, we expect fully semantic highlighting where supported by the current IDE, and on the standalone side, we expect basic syntax-based highlighting. On the bridge side, all the languages supported by the current IDE must be supported, and on the standalone side we must support at a minimum Kotlin, Java, XML, JSON, JS, TS, HTML, CSS, Python. Other languages support is nice to have but not required.

Thoughts:

1. Highlighting depends on the EditorScheme in bridge, and theme in standalone. If it is possible to avoid re-computing the highlighing info (e.g., determining code semantics) but limit ourselves to creating a new AnnotatedString with updated spans, this would be preferable for perf reasons.
2. The styling scheme should be stored in a composition local, and accessible via a theme-level getter, such as JewelTheme.codeStyling
3. Styling should follow the usual pattern of "definition in foundation, implementations in standalone and bridge"

[francisconoriega]

@rock3r I created PR #565 and was about to open the corresponding GH Issue for it, but I see its already here :)

That PR mostly follows what you have defined here, except for the the theme-level getter.

Thoughts on the implementation / use of the 3rd party library?

[AlexVanGogen]

Hi @francisconoriega! Hope you don't mind me jumping in, I was looking at the issue recently and gotten a couple thoughts.

Editor scheme in the IDE allows specifying different colors for the same token for different languages. For example, set orange for keywords in Java and blue in Kotlin. Implementors might also want to highlight identifiers (even more so, different identifiers with different colors), underline syntax errors and so on.

It would be nice if Jewel API could reflect all that. Which means highlighter should also be a composition local, because all this extravaganza might be a bit too much for standalone. Something like this:

public interface MarkdownCodeHighligher {
    public /* suspend? */ fun highlight(code: String, mimeType: MimeType /* , codeBlock: MarkdownBlock.Code.Fenced */): AnnotatedString
}

i.e. pretty much how it's described in the issue 😄

It gives a lot of flexibility to implementors, and leaves room for optimizations. Like, cache lexer/parser result using MarkdownBlock as a key so that parsing could be skipped if text remained the same, or something like that.

And I think foundation should be kept as simple as possible. Probably there shouldn't be any highlighting logic (and an additional dependency that comes with it) at all. A 3rd party library is worth using in standalone version, whereas in bridge we could just use IntelliJ SDK.

[rock3r]

Another note — I think syntax highlighting is not necessarily something we want to tie exclusively to the Markdown renderer. I can see the need for it elsewhere too

[francisconoriega]

hmm all fair points.

@rock3r do you want to go with a different approach (ie I should stop work on the current PR), or do feel okay going this way, if the comments in the PR are addressed?

[rock3r]

I feel like we should probably have a chat before you spend more time on this, so we are better aligned. Can jump on Slack in a few minutes?