Add IndexedJsonDocument, a JSONWrapper implementation that stores JSON documents in a prolly tree with probabilistic hashing.

[nicktobey]

tl;dr: We store a JSON document in a prolly tree, where the leaf nodes of the tree are blob nodes with each contain a fragment of the document, and the intermediate nodes are address map nodes, where the keys describe a JSONPath.

The new logic for reading and writing JSON documents is cleanly separated into the following files:

IndexedJsonDocument - The new JSONWrapper implementation. It holds the root hash of the prolly tree.

JsonChunker - A wrapper around a regular chunker. Used to write new JSON documents or apply edits to existing documents.

JsonCursor - A wrapper around a regular cursor, with added functionality allowing callers to seek to a specific location in the document.

JsonScanner - A custom JSON parser that tracks that current JSONPath.

JsonLocation - A custom representation of a JSON path suitable for use as a prolly tree key.

Each added file has additional documentation with more details about the individual components.

Throughout every iteration of this project, the core idea has always been to represent a JSON document as a mapping from JSONPath locations to the values stored at those locations, then we could store that map in a prolly tree and get all the benefits that we currently get from storing tables in prolly trees: fast diffing and merging, fast point lookups and mutations, etc.

This goal has three major challenges:

• For deeply nested JSON documents, simply listing every JSONPath requires asymptotically more space than the original document.
• We need to do this in a way that doesn't compromise performance on simply reading JSON documents from a table, which I understand is the most common use pattern.
• Ideally, users should not need to migrate their databases, or update their clients in order to read newer dbs, or have to choose between different configurations based on their use case.

This design achieves all three of these requirements:

• While it requires additional storage, this additional storage cannot exceed the size of the original document, and is in practice much smaller.
• It has indistinguishable performance for reading JSON documents from storage, while also allowing asymptotically faster diff and merge operations when the size of the changes is much smaller than the size of the document. (There is a cost: initial inserts of JSON documents are currently around 20% slower, but this is a one-time cost that does not impact subsequent reads and could potentially be optimized further.)
• Documents written by the new JSONChunker are backwards compatible with current Dolt binaries and can be read back by existing versions of Dolt. (Although they will have different hashes than equivalent documents that those versions would write.)

[reltuk]

Generally looks awesome and a really cool approach. Not too mind bending after starting at it for a while. I took a pass on some suggestions, but I may be missing some things. Please push back if I'm off base anywhere :).

[reltuk]

What is nd.Count() on one of these nodes? Why is this special case necessary?

[nicktobey]

These are the flattened leaf nodes. They contain 0 keys and 1 value. If we don't have this check, then the binary search will attempt to compare the search key to key[0] in the node, which will be out of bounds.

I added a comment.

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
f820432	ok	5937457

version	total_tests
f820432	5937457

correctness_percentage
100.0

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
443e348	ok	5937457

version	total_tests
443e348	5937457

correctness_percentage
100.0

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
1e72313	ok	5937457

version	total_tests
1e72313	5937457

correctness_percentage
100.0

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
8113baa	ok	5937457

version	total_tests
8113baa	5937457

correctness_percentage
100.0

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
c79d151	ok	5937457

version	total_tests
c79d151	5937457

correctness_percentage
100.0

[coffeegoddd]

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
87ac817	ok	5937457

version	total_tests
87ac817	5937457

correctness_percentage
100.0

[coffeegoddd]

@nicktobey DOLT

comparing_percentages
100.000000 to 100.000000

version	result	total
e4a6d6e	ok	5937457

version	total_tests
e4a6d6e	5937457

correctness_percentage
100.0