Add compiler types to C# like CodeGeneratorRequest / CodeGeneratorResponse

[invidious9000]

What language does this apply to?
C#

Describe the problem you are trying to solve.
Write a protoc plugin in C#

Describe the solution you'd like
Types from plugin.proto made available in a proposed Google.Protobuf.Compiler namespace, specifically CodeGeneratorRequest and CodeGeneratorResponse. Java library includes these types:
https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/compiler/PluginProtos.CodeGeneratorRequest.Builder

Describe alternatives you've considered
Building my own artifact based on protobuf repo. For C# protoc will not build proto2 syntax, which plugin.proto is written in, so this means I'd have to keep a parallel hand-maintained proto3 variant of it.

[jskeet]

I'm actively working on this at the moment. My solution will involve:

• Allowing a set of FileDescriptors to be loaded dynamically
• Potentially including the plugin types within Google.Protobuf (still TBD)

The reason for the latter is that we still won't provide access to FileDescriptor itself, which is still proto2. Instead, if we locally modify CodeGeneratorRequest.proto_file to be a repeated bytes field, then generate the C# code, we can ask the support library to parse each ByteString (as we have internal support for FileDescriptor).

I already have a working proof of concept of this - I'm turning it into production code, and adding DynamicMessage support at the same time (as it's odd to be able to load descriptors dynamically but not support dynamic message handling). No ETA right now, but I'm working on it...

[invidious9000]

This is great, thank you.

Are there any plans for implementing extension registries or some equivalent alternative? They seem to fit really well with codegen, on the java side we do something like this so we can 'annotate' messages with additional proto messages and have first-class handling in plugin:

Define options (CodegenV1.proto):

extend google.protobuf.FieldOptions {
    ColumnDescriptor column = 80000;
}

message ColumnDescriptor {
    bool someValue = 1;
    string anotherThing = 2;
}

We apply it to messages we want to handle for codegen like this:

message AnnotatedMessageToGenerate {
    string id = 1 [(myPlugin.column).someValue = true];
}

And finally consume it in protoc plugin like this:

ExtensionRegistry registry = ExtensionRegistry.newInstance();
CodegenV1.registerAllExtensions(registry);
CodeGeneratorRequest request = CodeGeneratorRequest.parseFrom(stdIn, registry);

// ... get a FieldDescriptor from DescriptorProto field list

DescriptorProtos.FieldOptions options = fieldDescriptor.getOptions();
ColumnDescriptor columnDescriptor = options.getExtension(CodegenV1.column);

Is there a different or planned approach to this for C#? We can pull custom options using field number or name but the static message/field extension members are brilliant.

[jskeet]

I'm not intending to change how custom options are exposed at the moment. There's no need for an extension registry, but you do need to know the field number. That's a topic discussed in #4591.

But I've checked that the options defined in protos are available via the descriptors passed into the C# plugins. For example, here's a sample of some prototype code I wrote to use the HttpRule option

const int HttpRuleOptionNumber = 72295728;

...

foreach (var service in input.Services)
{
    Console.WriteLine($"  {service.Name}");
    foreach (var method in service.Methods)
    {
        if (method.CustomOptions.TryGetMessage<HttpRule>(HttpRuleOptionNumber, out var rule))
        {
            Console.WriteLine($"    {method.Name}: {rule}");
        }
        else
        {
            Console.WriteLine($"    {method.Name}: no HTTP binding");
        }
    }
}

[eschgi]

@invidious9000, @jskeet: Exists there a workaround how I can implement plugins in C#? Do I need to convert the plugin.proto to proto3 syntax?

Thanks.

[jskeet]

Yes, it's possible to write a plugin in C# now. We have that for the "microgenerator" for Google APIs, which is here: https://github.com/googleapis/gapic-generator-csharp.

That uses a proto3 version of plugin.proto, here: https://github.com/googleapis/gapic-generator-csharp/tree/master/Google.Api.Generator/Google.Protobuf.Compiler

With the proto2 support now available, it's possible that you could just use the existing plugin.proto, but I haven't tried that.

[eschgi]

@jskeet: Thanks for the quick response. That's exactly what I have looked for.

[eschgi]

@jskeet: When I try to compile the proto2 version of plugin.proto file then I get the following error:

plugin.proto: C# code generation only supports proto3 syntax.

I have used protoc version 3.10.0.

I have used following command line parameters:

C:\Users\xxx\Downloads\protoc-3.10.0-win64\bin\protoc.exe --proto_path=C:\Users\xxx\Downloads\protoc-3.10.0-win64\bin --csharp_out=C:\Users\xxx\Downloads\protoc-3.10.0-win64\bin --csharp_opt=file_extension=.cs C:\Users\xxx\Downloads\protoc-3.10.0-win64\bin\plugin.proto

Should that work?

[jskeet]

I'm afraid I don't know - I haven't been following all the proto2 work in detail. It may be that there's an extra option required; I haven't looked into this. The simplest approach would probably be to use a proto3 version for now.

[ObsidianMinor]

Proto2 is in master but missed the 3.10 release. It'll release in 3.11.

[Tohnmeister]

Is there some sort of package already available that has the generated plugin code from the latest plugin.proto? Or must I still generate it myself?

[jskeet]

@Tohnmeister: As far as I'm aware, plugin.proto still isn't included in anything. (We still generate from the proto for the C# Google Cloud Client Libraries for example - see https://github.com/googleapis/gapic-generator-csharp/tree/main/Google.Api.Generator/Google.Protobuf.Compiler)

[Tohnmeister]

Thx for the answer. I'll do the same then for now.