fix: code-generation for Services with Struct response types

[boukeversteegh]

Should fix #403

Could you have a try? @mkmarek -- and thanks for reporting with a reproducing proto file.

You can switch to this version temporarily by changing the version in your package.json to:

"ts-proto": "https://github.com/stephenh/ts-proto/tarball/fix/grpc-struct"

[mkmarek]

Thanks for the swift response @boukeversteegh !

I tested it on the https://github.com/mkmarek/ts-proto-bug repo and it worked like a charm.

So I tried to use it on the project where I got the error originally and noticed we're using a bit different configuration there. We use it with this parameter --ts_proto_opt=outputServices=grpc-js (since we're using it with grpc-js) and that still outputs the wrong thing.

Example:

export const ServiceService = {
foo: {
        path: '/pkg.Service/Foo',
        requestStream: false,
        responseStream: false,
        requestSerialize: (value: FooRequest) =>
          Buffer.from(FooRequest.encode(value).finish()),
        requestDeserialize: (value: Buffer) => FooRequest.decode(value),

       // ↓↓↓↓↓
        responseSerialize: (value: {[key: string]: any} | undefined) =>
          Buffer.from({[key: string]: any} | undefined.encode(value).finish()),
        responseDeserialize: (value: Buffer) => {[key: string]: any} | undefined.decode(value),
      }
}

I updated https://github.com/mkmarek/ts-proto-bug with the --ts_proto_opt=outputServices=grpc-js parameter.

[stephenh]

Lgtm with @mkmarek 's caveat that we probably need another "keepValueTypes" (I assume) somewhere.

@boukeversteegh could you add an RPC that returns a struct to this file:

https://github.com/stephenh/ts-proto/blob/main/integration/grpc-js/simple.proto#L27

As part of the fix, to prevent future regressions?

[stephenh]

Ah that's subtle; thanks @boukeversteegh !

[mkmarek]

Looked at the codebase a bit and adding the same { keepValueType: true } to

ts-proto/src/encode.ts

Line 9 in 57f2473

return code`${messageToTypeName(ctx, typeName)}.encode(value).finish()`;

ts-proto/src/encode.ts

Line 47 in 57f2473

return code`${messageToTypeName(ctx, typeName)}.decode(value)`;

Seems to do the trick

[boukeversteegh]

Lgtm with @mkmarek 's caveat that we probably need another "keepValueTypes" (I assume) somewhere.

@boukeversteegh could you add an RPC that returns a struct to this file:

https://github.com/stephenh/ts-proto/blob/main/integration/grpc-js/simple.proto#L27

As part of the fix, to prevent future regressions?

Added that case, and fixed some issues exposed by it as well.

I'll also add a case for Value, because I suspect it will also break.

[stephenh]

Nice! I really like where you put the unwrap methods on each type:

Value.unwrap(Value.decode(reader, reader.uint32()));

I think let's merge this as-is; just as a curiosity, seeing ^ made me wonder, @boukeversteegh wdyt about having Value/Struct/List.decode return the already-unwrapped values? And similarly, Value/Struct/List.encode could accept the not-yet-wrapped values (or maybe if we're really fancy, encode could accept a union of either Value | ...the unwrapped values...?

Just thinking that might be neat for users b/c it would make the wrapper/value types "fade away" to be mostly serialization. Well, although I guess our message representation is already doing that, i.e. if you have a Struct inside another message, it already looks like "just JS", and it's only the encode/decode methods that want/need to know about the wrapped Struct type.

And so my musing of changing decode to return the already-unwrapped type is probably not something that users would use/notice that often anyway, b/c they're more likely interacting with a value type through some message that uses it. So maybe it's better to stay consistent and have encode/decode keep using the wrapper/message types, just like all of the other encode/decode methods do.

[boukeversteegh]

... just as a curiosity, seeing ^ made me wonder, @boukeversteegh wdyt about having Value/Struct/List.decode return the already-unwrapped values? And similarly, Value/Struct/List.encode could accept the not-yet-wrapped values (or maybe if we're really fancy, encode could accept a union of either Value | ...the unwrapped values...? Just thinking that might be neat for users b/c it would make the wrapper/value types "fade away" to be mostly serialization. Well, although I guess our message representation is already doing that, i.e. if you have a Struct inside another message, it already looks like "just JS", and it's only the encode/decode methods that want/need to know about the wrapped Struct type.

And so my musing of changing decode to return the already-unwrapped type is probably not something that users would use/notice that often anyway, b/c they're more likely interacting with a value type through some message that uses it. So maybe it's better to stay consistent and have encode/decode keep using the wrapper/message types, just like all of the other encode/decode methods do.

I had the exact same train of thought, and ended up with the same tentative conclusion, to keep the signatures of the encode/decode methods the same as the others.

Two other reasons why I felt it might be better to keep the wrapping separate from the encoding:

• Loss of ability to work with the wrapped objects, such as the ability to manually provide a wrapped object {stringValue: "hello"} and encode it. This may be useful when dealing with proto agents or generators that don't properly (un)wrap the values.
• I think it may make the plugin code less straight forward to understand. I.e. to know where wrapping/unwrapping happens, and the interfaces/method signatures will be inconsistent. During implementation, it was quite convenient to know that all methods on the interfaces always work with the message instances, and not with native types.

I think let's merge this as-is;

Great, thanks for your review and support. I will go ahead and merge!

[stephenh]

🎉 This PR is included in version 1.90.1 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀