Add serialization and deserialization of numerals larger than Number.MAX_SAFE_INTEGER

[AMoo-Miki]

Description

In JavaScript, a Number is a 64-bit floating-point value which can store 16-ish digits. However, the serializer and deserializer will need to cater to numeric values generated by other languages which can have up to 19 digits. Native JSON parser and stringifier, incapable of handling the extra digits, corrupt the values, making them unusable.

To work around this limitation, with this change, the deserializer converts long sequences of digits into strings and marks them before applying the parser. During the parsing, string values that begin with the mark are converted to BigInt values. Similarly, during stringification, the serializer converts BigInt values to marked strings and when done, it replaces them with plain numerals.

Issues Resolved

Related to opensearch-project/OpenSearch-Dashboards#4536

Check List

• New functionality includes testing.
  • All tests pass
• Linter check was successfull - yarn run lint doesn't show any errors
• Commits are signed per the DCO using --signoff
• Changelog was updated.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[AMoo-Miki]

Let's hold on merging this for a little while.

[wbeckler]

We can do a patch release of the js client right now if that would help

…

On Tue, Jul 11, 2023, 12:08 PM Miki ***@***.***> wrote: Let's hold on merging this for a little while. — Reply to this email directly, view it on GitHub <#544 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5PRLSZTEDJVPC3ZB47UJDXPV3BNANCNFSM6AAAAAA2FPHO4U> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dblock]

Did you write this code from scratch? :)

• Is there really no BOM character that can never be present in the serialized data?
• Where is this serialization used? I think we should have at least one higher level test that shows it.

[dblock]

I know these are unlikely, but is there a more reliable way to have a BOM?

[AMoo-Miki]

It doesn't just use these guys. It creates a combination of 1 to Infinity of them. So even if these guys are all present, there is always a sequence that is long enough to not be present.

A future enhancement could dynamically generate a large collection based on ranges and experiment with them.

[dblock]

I'm probably slow. These are all digits. You get things like ෴18014398509481982, why not N:18014398509481982?

[AMoo-Miki]

Did you write this code from scratch? :)

Yes :)

* Is there really no BOM character that can never be present in the serialized data?

A string containing serialized JSON is still a JS string so all chars that can be used in JS strings can also be present in the serialized JSON.

PS, with the exception of U+2028 and U+2029, anything that goes into JSON, goes into JS, but then again, all of this serializing and deserializing is done in JS.

* Where is this serialization used? I think we should have at least one higher level test that shows it.

Transport.js uses it to produce the body of requests. It is also exported to be used by others. I will add an integration test.