Add Deserialization for ErasedUnions

[Booksbaum]

Currently ErasedUnionConverter doesn't implement ReadJson -> ErasedUnions cannot be deserialized from json.

But this is needed for inlayHint/resolve: input and output is InlayHint -> InlayHint must be serializable as well as deserializable.

This PR implements ReadJson for ErasedUnionConverter -> enables deserialization of Erased Unions (and as a result deserialization of InlayHint) :

[<ErasedUnion>]
type U =
| String of string
| Data of {| Name: string; Value: int |}

(JToken.Parse "\"text\"" |> deserialize<U>) = U.String "text"
(JToken.Parse """{ "name":"foo", "value":42 }""" |> deserialize<U>) = U.Data {| Name = "foo"; Value = 42 |}

But note: This only works when types in each Union case are distinguishable from each other.
For example: U<float, int> -> always deserializes to first Case because a number always matches float.

There are some additional deserialization enhancements to match correct cases:

Stricter basic type deserializing

By default Newtonsoft.Json is rather lax when deserializing -> types must not match exactly. For example: "42" deserializes to int, and 42 deserializes successful to string.
This is an issue with Erased Union like U2<int, string> -> "42" gets deserialized as U2.First 42 instead of U2.Second "42"
This PR adds extra converters for string, bool and numbers so these cases get deserialized correctly.

Proper handling of optional & required fields in Records

Currently all fields are optional in practice -> field missing in json still deserializes to object:

type R = { Name: string; Value: int }
let json = """{ value: 42 }"""
let r = JToken.Parse json |> deserialize<R>

This works, but r.Name is null.

That's especially an issue for matching Cases in Erased Unions: every json objects can be deserialized into any record. For example {} can be deserialized into R. And in Union: U2<R, {| MyValue: int |}> -> Second never gets matched, because every json object fits R. So json {"myValue": 42} gets deserialized to U2.First {R.Name=null; Value=0}.

I added a custom ContractResolver, which ensures something can only be deserialized when all required fields exists.
Option fields (: _ option) are optional and don't need to be present in json:

type S = { Name: string; Value: int option }

{}                              // error: missing name
{ "name": "foo"}                // ok: name exists, value is optional
{ "name": "foo", "value": 42 }  // ok
{ "value": 42 }                 // error: missing name
{ "name": "foo", "bar": "baz"}  // ok: name exists, value is optional, additional properties get ignored

-> U2<S, {|MyValue: int|}> can be correctly deserialized

Note: Cases in Erased Unions still must be distinguishable from each other:
U2<S, {| Name: string; Something: int |}> still gets always deserialized to U2.First (S) because Name is required in S while Value isn't -- and additional fields get ignored -> every json object with property name can be deserialized to S

Note: I do this by overwriting Required property for contracts. That means: I currently also overwrite custom required annotations specified in Attribute for a field ([<JsonProperty(Required = ...)>]) (Currently never used here in this Repo)
-> Required and Optional is solely based on option.
Update: JsonProperty.Required is now honored when specified and takes precedence over option

Note: Doesn't work with ValueOption (Supporting voption would require custom deserialize for ValueOption -- which is surprisingly complex (compared to Option (-> None=null -> handled by Newtonsoft))) ... and I didn't consider it necessary (currently not used in this repo))

Note on speed of ErasedUnionConverter:
The converter tries to deserializes the current json value to each case in order. When one case doesn't fit json, deserializing this case throws an exception, which the converter catches and tries the next case.
But exceptions are quite slow -> put most likely case first (U2<LikelyUsed, UnlikelyUsed>)
And additional there's special handling for basic types (string, bool, numbers) -> get handled without exceptions -> put case with basic type first (U2<int, MyRecord>)

---

Additional:

Add Tests
mostly targeting Converters and ContractResolver (-> Required/Optional). But not a complete test suite -- just the examples I used to implement this PR.

• Tests are not hooked into any build process.
• To run: dotnet run --project tests

Add very simple benchmarks
Extremely basic benchmarks I used to measure speed (and allocations). Currently just serialize and deserialize back again of two large_ish (in terms of property count) LSP types (InitializeParams & InlayHint).
It's questionable how representative these two actually are (InitParams is just used once in LSP lifecycle...), but I wanted to test with something with a lot of properties that cover a couple of different deserialization cases (and InlayHint is the reason for this PR).

• Currently located inside test project (just evolved naturally -- and I didn't refactor it...)
• Can be switched on with --benchmarks switch:
  dotnet run --project tests -c Release -- --benchmarks --filter "*"

---

Because of additional project:
Github Workflow files required updates to just process ./src and ignore ./tests. I think I adjusted all case -- but be sure to check again!

[Booksbaum]

Ok, found some additional issues with LSP Types & (de)serializing
-> PR switched to Draft

For now I'm looking at obj types: Can be anything -- including a type that cannot be serialized. And of course issue when deserializing: Deserialize to what?...
I think I change all objs to JToken

[Booksbaum]

Ok, done (I think...)

Note: I changed some LSP types (see commits) -> Clients might need slight adjustments

I added some Benchmarks to play around and experiment with. But they are basically (at least partially) just way-too-complex, not-really-used, distracting clutter. But they are in a separate file and don't bother -- so I kept them. Might be useful again when changing Converters.

---

One test enhancement:
All LSP types are now automatically checked if they are serializable and deserializable back again ((o |> serialize |> deserialize) = o). (Data is randomly generated (with FsCheck))
Though that cannot guarantee the produced json is according to specs, it should catch at least de/serializing exceptions (like failure to deserialize InlayHint)

There are now quite few tests -- and I think especially these new LSP de/serialization tests are worth running on every change.
Is it ok to add tests to GH workflow?

[baronfel]

Is it ok to add tests to GH workflow?

Yes, absolutely!

[Booksbaum]

Tests added

Though just as dotnet run, no dotnet test (which gives more status output by default -- but requires additional packages I didn't add (now) -- though in case of error default expecto output is nicer)

[baronfel]

Overall this looks quite good, but I'd like to change it to be dotnet test-able and add the GithubActionsTestLogger. I can do those tasks shortly.

[baronfel]

Is it ok if I squash this?

[Booksbaum]

yes, of course