JSON: 'Any' message should serialize to/from bytes when the message type is not well known.

[charlie430]

Problem

The Any message type specifies that in JSON the properties of the message serialized within an Any message should be serialized as properties.

The JSON representation of an Any value uses the regular representation of the deserialized, embedded message, with an additional field @type which contains the type URL. Example:

package google.profile;
message Person {
  string first_name = 1;
  string last_name = 2;
}

{
  "@type": "type.googleapis.com/google.profile.Person",
  "firstName": <string>,
  "lastName": <string>
}

Reference

This forces a system that serializes using JSON with unregistered types to either throw an error because it does not know how to parse an unregistered type (code ref), or to gracefully skip the type and lose information (pull request #4505). If the latter is chosen, then when serializing or deserializing the message, information is lost.

Suggestion

When serializing to and from JSON, messages within an 'Any' message should serialize to and from bytes when the message type is not well known.

Figure 1: Example of serializing an 'Any' message with not well known type to bytes

{
  "@type": "type.googleapis.com/MyCustomType",
  "value": "eyAiZm9vIjogImJhciIgfQ==" // Base64 encoded bytes
}

Use Case

I have many systems that use common message types, but do not have all message types registered in all systems. They only register the message types that are relevant to them.

If system X requests data from system Y and system Z, system X should not have to know what the message types are in order to pack them into an 'Any'.

Thoughts

From my limited experience with Protocol Buffers, it seems like all message types have to be registered in the type registry in order to serialize and deserialize fully. This does not seem scalable when an organization shares thousands of message types, and messages are being sent to and from systems that need to modify message types that only they know how to modify, or add message types that only they know about.

This does not seem to be a problem with binary serialization, as in practice, it uses the @value property. However, I don't see the @value property documented, so I don't know if this was intentional or not.

Figure 2: Binary serialization uses the @value property when deserializing an unregistered type

Thank you for taking the time to read through this issue.

[charlie430]

I found issue #658 which is relevant to this topic. I understand that it is valuable to be able to access the properties of types that are unknown. However, this precludes systems who serialize to JSON from being able to modify messages with unknown types.

[emcfarlane]

This issue is affecting a middleware gRPC proxy that translates messages from Proto to JSON. On RPC errors, details commonly contain Any and if unable to resolve the type we are forced to either: 1. drop the error and raise a resolver error, or 2. convert the Any payload into a message type we understand which then requires the upstream to decode.

Would be great to have support for Any to be encoded in JSON as {"@type": "<url>", "@value": "<bytes>"}. Then the fallback is well defined and the client can handle it gracefully.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.