Add ByteString type to support any CBOR byte string #376
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows the parsing of CBOR data containing maps with byte string keys, which is commonly seen in Cardano blockchain data. The default behavior remains the legacy behavior (throw an error) for backward compatibility. Signed-off-by: Andrew Gaffney <[email protected]>
This commit makes ByteString support CBOR byte string (major type 2) without being limited to map keys. ByteString can be used when using a []byte is not possible or convenient. For example, Go doesn't allow []byte as map key, so ByteString can be used to support data formats having CBOR map with byte string keys. ByteString can also be used to encode invalid UTF-8 string as CBOR byte string. - Modified ByteString to use string type. - Implemented cbor.Marshaller and cbor.Unmarshaller for ByteString. - Simplified ByteString encoding and decoding. - Renamed MapKeyByteStringFail to MapKeyByteStringForbidden. - Renamed MapKeyByteStringWrap to MapKeyByteStringAllowed. - Added more tests for ByteString.
Also added support for: - CBOR tag with byte string tag content as map key. - nested CBOR tag with byte string tag content as map key. This new feature can be disabled by specifying decoding option MapKeyByteStringForbidden, which will return an error when attempting to decode CBOR byte string as map key into an empty Go interface value.
InvalidMapKeyTypeError is returned when attempting to decode CBOR map key to a Go type that cannot be used as map key. This allows callers to check for this error without requiring string comparison of the error text. To maintain backward compatibility, the text of InvalidMapKeyTypeError remains the same as the plain error that used to be returned.
Added support for decoding CBOR map with byte string keys to Go map with empty interface key type. This supports decoding these CBOR map key types: - byte string - tagged byte string - nested tagged byte string Previous commit already added support for decoding CBOR maps with byte string keys to empty Go interface value.
|
@agaffney Thanks again for raising this issue and opening PR #319. If you have time, please take a look and let me know if this PR works for you. This PR includes PR #319 and some of the differences are:
BTW, thanks for the link to Cardano's CDDL. That was really helpful in understanding the usefulness of adding this feature. It's one of the main reasons I enabled this feature by default. |
6 tasks
x448
approved these changes
Dec 29, 2022
|
Passed fuzzing overnight and today. Merging. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
By default, CBOR byte string is decoded as
[]byte, but Go doesn't allow[]byteas map keys. Given this, decoding a CBOR map with byte string keys produced "invalid map key type" error.Many thanks to @agaffney for his patience after raising this issue and for opening PR #319 which is included in this PR with permission.
Closes #312
Solution
This PR defines
ByteStringtype to support CBOR byte string (major type 2) without being limited to map keys.ByteStringcan be used when using a[]byteis not possible or convenient. For example, Go doesn't allow[]byteas map key, soByteStringcan be used to support data formats having CBOR map with byte string keys. Another use forByteStringis to encode invalid UTF-8 string as CBOR byte string.By default, this PR allows decoding CBOR maps with keys of type:
When decoding CBOR maps, the above CBOR key types can be decoded to:
Changes
ByteStringwith underlyingstringtype.cbor.Marshallerandcbor.UnmarshallerforByteString.MapKeyByteStringForbiddenandMapKeyByteStringAllowed.InvalidMapKeyTypeError.ByteString.Caveats
MapKeyByteStringForbidden.