Function to get all tokens of a line

[marcelgerber]

So in Brackets, we've a function that runs getTokenAt() for every single token to find all modes in a particular file.
While this is OK on normal-length lines, it takes ages on very long lines. So, assuming you have a line with 10000 tokens, the main while loop, and therefore readToken will run 50,005,000 times! (t / 2 * (t + 1), where t is the token count).
With the function getTokensOfLine, newly introduced in this PR, the while loop will only run 10,000 times, as it pushes all its results into an array ;)

Hope you're interested in this function as well.

[marijnh]

All token objects will end up with the same state. It seems that your use case requires getting information from this state, so it likely won't work with your function. Calling CodeMirror.copyState when storing the state should help there.

Can you find a way to express this without duplicating most of the code in getTokenAt? Maybe define a utility that both methods use.

Also, public methods need documentation.

[marcelgerber]

Renamed the function to getLineTokens, added documentation, made use of copyState. Also, using a utility function now.

[marijnh]

Thanks! Merged as da3e8f2, followed up by 71ef0cf

[marcelgerber]

Thank you :)