[manifest license] fully implement SPDX, plus general parsing stuff

[strega-nil-ms]

This includes the following features:

• LicenseRef-[blah], for licenses which have not yet been added to SPDX
• null license, for proprietary licenses generally (this means "look at share/<port>/copyright" for license info)
• unrecognized licenses warn, instead of erroring
• existing accepted incorrect license expressions are now correctly errored out on: (a AND b) WITH c was accepted before, and isn't now
• the errors are much better:

Error Examples:

Unrecognized ASCII character

{
  "license": "\u0000"
}

=>

Errors occurred while parsing /Users/nimazzuc/projects/misc-work/test-license/vcpkg.json
    $.license (an SPDX license expression): SPDX license expression contained an invalid character (0x00 '').
    $.license: mismatched type: expected an SPDX license expression
See https://github.com/Microsoft/vcpkg/tree/master/docs/users/manifests.md for more information.

Unicode characters

{
  "license": "é"
}

=>

Errors occurred while parsing /Users/nimazzuc/projects/misc-work/test-license/vcpkg.json
    $.license (an SPDX license expression): SPDX license expression contained a unicode character (U+00e9 'é'), but these expressions are ASCII-only.
    $.license: mismatched type: expected an SPDX license expression
See https://github.com/Microsoft/vcpkg/tree/master/docs/users/manifests.md for more information.

Incorrect exceptions on multiple licenses

{
  "license": "(GPL-2.0+ AND Apache-2.0) WITH LLVM-exception"
}

=>

Errors occurred while parsing /Users/nimazzuc/projects/misc-work/test-license/vcpkg.json
    $.license (an SPDX license expression): Expected either AND or OR, found WITH (WITH is only allowed after license names, not parenthesized expressions).
    $.license: mismatched type: expected an SPDX license expression
See https://github.com/Microsoft/vcpkg/tree/master/docs/users/manifests.md for more information.

Unknown License

{
  "license": "asdf"
}

Unknown license identifier 'asdf'.
Known values are listed at https://spdx.org/licenses/

[BillyONeal]

The "request changes" is more about not enough tests for that much terrifying string manipulation code; everything else is nitpicks.

[BillyONeal]

Whoops, thanks VS Code.

[ras0219-msft]

Why not use our existing parser framework for this (Parse::Parserbase)?

I see two differences compared to the code you have here:

1. Localization - we must eventually add Localization to ParserBase for all the other parsers in the product, so this doesn't seem like a good reason to do things completely ad-hoc here. Either this new code should use the existing framework (and get localized when we localize every other parser) or we should localize ParserBase first in a separate PR and then add this code.
2. Warnings - adding warnings to the existing parsing framework seems obviously good and not long or complex.

[strega-nil-ms]

I didn't think about it until I was very close to done with this implementation. I can switch it over, I think, pretty easily tho.

[strega-nil-ms]

hmm, now that I'm looking at this, it seems like it'd be quite difficult to push this parser into the old ParserBase model; I would prefer to rewrite the model from scratch, and then switch stuff over as it comes up. I also don't want to add new un-localized code. Also, moving everything over to the new model would be prohibitively expensive.

[ras0219-msft]

LGTM with the std::string{} change.

[autoantwort]

This caused microsoft/vcpkg#23110