Idiomatic text buffers

[rjkroege]

Edwood could stand to have a new and improved text storage system. We've talked about various notions. A piece table with deltas would seem to be the most reasonable choice.

[mibk]

Not sure that would be a good fit for Edwood, but years ago I came across a vim-like editor called vis. Its text management system was based on the Project Oberon (the introduction blog post: http://www.brain-dump.org/blog/entry/144/vis_a_highly_efficient_vim_like_text_editor). I ported (and then slightly changed the API) the text management part of the editor into the package undo.

[rjkroege]

Thanks. This sounds promising. I'll take a look at your undo package.

[rjkroege]

Notes on using the undo package (or something similar)

• I need to eliminate all the places where Edwood reaches into File. In particular, seq management and the list of observing Text instances need to be excised.
• I can remove File's list of Texts if I simply get each Text to redraw if it needs to at the end of any mutation.
• Text can deduce if it needs to draw by comparing a File's seq to the "lastdrawn" seq.
• undo may need some extensions. In particular, I may want to ask the underlying text buffer implementation what changes have happened since the last seq so that they can be replayed into a Text when we redraw (if necessary.)

Concrete Steps

• de-globalize seq
• rationalize and document "dirty" tracking
• remove the observer pattern
• clarify the API for text buffers
• insert undo

[rjkroege]

Strings vs Runes

• The Undo package operates on byte buffers.
• Edwood's Buffer object is an array of rune.
• Using Undo would therefore entail a choice: switch Edwood to using utf-8 buffers (or strings) or modify Undo to use rune.

I somewhat prefer switching Edwood to utf-8 buffers if only because for most workloads it will greatly the heap space used for various text buffers by nearly a factor of 4.

This however seems like a somewhat far-reaching alteration. Many access patterns to Buffer iterate (e.g. ReadC loops in text.go or addr.go.) But sam addressing can be to a specific character.

@fhs Do you have an opinion on this?

Sequencing

It's not obvious to me how to sequence this work. I have several incomplete CLs that attempt to make progress on this task per comment above and none of them seem to necessarily move forward. I think the primary first task it to make sure that all Edwood code outside of File stops depending on its internal details around dirty tracking.

[fhs]

Switching to UTF-8 buffers does seem nicer, but the problem is sam addressing. Maybe we can use golang.org/x/exp/utf8string to efficiently index strings by rune. Given how much work is involved to switch to utf8 buffer, I think for now, we can just pick a Buffer implementation that works on runes, write some benchmarks, and call it a day. After Edwood has more test coverage and easier to refactor, we can come back to this problem.

The Buffer implementation should probably be moved into an internal package. It should also return errors for invalid index instead of panic, which will make it easier to test.

[rjkroege]

Update on this work: I want to relocate File into a separate module so that its public interface becomes obvious. This will permit writing unit tests that exercise the public interface that would (ideally) not be fragile across replacing the File implementation.

[rjkroege]

As of #430, Edwood is operating with the new-type utf8 byte buffers. Exciting! Thanks everybody who helped with the implementation. This unlocks many possible code improvements.

[rjkroege]

When should I consider this done? We now have new type buffers. But I've not finished taking advantage of them to improve Edwood operation. I'm going to declare this finished. Yay! 😀