Marshaling into already allocated proto.Buffer stopped working

[jrouviere]

Marshalling into an already allocated proto.Buffer stopped working after the merge of #560.

For instance:

var msg proto.Message

// using a proto.Buffer:
buf := proto.NewBuffer(make([]byte, 0, 1024))
buf.Marshal(msg)
dataBuf := buf.Bytes()

// using direct proto.Marshal:
dataMarshal, _ := proto.Marshal(msg)

// if the encoded message is smaller than 1024, dataBuf is invalid
bytes.Equal(dataBuf, dataMarshal) // false

I believe the issue is caused by the generated XXX_Marshal methods, which now expect the capacity of the buffer to be exactly the size of the message.

The first time a proto.Buffer object is used its capacity will be set to the size of the message, the second time it is used, and if the message to be marshalled is smaller, the capacity stay the same and is now higher than the length of the message.

The problem is that XXX_Marshal force the size of the buffer to its capacity (b = b[:cap(b)]), and then truncate it back to the actual marshalled size (return b[:n], nil), this behaviour was fine before #560.
Now that we marshal backwards, the data is written at the end of the buffer (at full capacity), and XXX_Marshal returns a slice at the beginning of the buffer containing uninitialised data.

[jrouviere]

Replacing
b = b[:cap(b)]
by
b = b[:m.Size()]
in protoc-gen-gogo/generator/generator.go seems to fix the issue, but I'm not sure about the wider impact.

[jmarais]

Thanks for the report.
@apelisse It seems like we will have to call MarshalTo instead of MarshalToSizedBuffer in the XXX_Marshal method.
This cause at least two Size() calls for marshaling any message. Do you think we can do anything better here?

[jmarais]

@jrouviere
Can you please let me know if:
https://github.com/jmarais/protobuf/tree/issue619
fixes your problem.

[jrouviere]

@jmarais, Yes your branch fixes the problem I was observing. Awesome work, thanks for your quick response!