Support interface-only output

[iangregsondev]

Hi,

I forked it but i notice there is a missing directory under build called pbjs, i figured this was pbjs from npm so I replaced it but it missing some things - is this a custom build ? Can you include it in repo ?

I stumbled onto this package by accident and its almost what I need. I just want simple TS files.

I was hoping to be able to extend the

export type Options = {
  useContext: boolean;
};

To contain useObservables: boolean - as I notice you are returning a promise - but i would love to have an observable returned here - i thought adding an option would be great.. I am using NestJS - and my services return observables. (there is a link here - if you are not aware of it ( https://docs.nestjs.com/microservices/grpc )

Also another option like

minimalGeneration: boolean

or something similar, basically meaning that it just creates the interfaces and nothing else - let me show you an example...

I will show you my proto file and what I prefer (just a minimal output using only interfaces etc) and the generated code that ts-proto produces :-

syntax = "proto3";

package translation;

service TranslationService {
  rpc FindOne (HeroById) returns (Hero) {}
}

message HeroById {
  int32 id = 1;
}

message Hero {
  int32 id = 1;
  string name = 2;
}

What I would like, notice it's minimal and also I return an observable rather than a promise, I don't have any ctx etc... Just the interfaces for the types and the interface for the service

export interface HeroById {
  id: number;
}

export interface Hero {
  id: number;
  name: string;
}

export interface TranslationService {

  FindOne(request: HeroById): Observable<Hero>;

}

Here is what ts-proto produces

import { Reader, Writer } from 'protobufjs/minimal';
import * as Long from 'long';


export interface HeroById {
  id: number;
}

export interface Hero {
  id: number;
  name: string;
}

const baseHeroById: object = {
  id: 0,
};

const baseHero: object = {
  id: 0,
  name: "",
};

export interface TranslationService<Context extends DataLoaders> {

  FindOne(ctx: Context, request: HeroById): Promise<Hero>;

}

export class TranslationServiceClientImpl<Context extends DataLoaders> implements TranslationService<Context> {

  private readonly rpc: Rpc<Context>;

  constructor(rpc: Rpc<Context>) {
    this.rpc = rpc;
  }

  FindOne(ctx: Context, request: HeroById): Promise<Hero> {
    const data = HeroById.encode(request).finish();
    const promise = this.rpc.request(ctx, "translation.TranslationService", "FindOne", data);
    return promise.then(data => Hero.decode(new Reader(data)));
  }

}

interface Rpc<Context> {

  request(ctx: Context, service: string, method: string, data: Uint8Array): Promise<Uint8Array>;

}

interface DataLoaders {

  getDataLoader<T>(identifier: string, cstrFn: () => T): T;

}

function longToNumber(long: Long) {
  if (long.gt(Number.MAX_SAFE_INTEGER)) {
    throw new global.Error("Value is larger than Number.MAX_SAFE_INTEGER");
  }
  return long.toNumber();
}

export const HeroById = {
  encode(message: HeroById, writer: Writer = Writer.create()): Writer {
    writer.uint32(8).int32(message.id);
    return writer;
  },
  decode(reader: Reader, length?: number): HeroById {
    let end = length === undefined ? reader.len : reader.pos + length;
    const message = Object.create(baseHeroById) as HeroById;
    while (reader.pos < end) {
      const tag = reader.uint32();
      switch (tag >>> 3) {
        case 1:
          message.id = reader.int32();
          break;
        default:
          reader.skipType(tag & 7);
          break;
      }
    }
    return message;
  },
  fromJSON(object: any): HeroById {
    const message = Object.create(baseHeroById) as HeroById;
    if (object.id) {
      message.id = Number(object.id);
    }
    return message;
  },
  fromPartial(object: DeepPartial<HeroById>): HeroById {
    const message = Object.create(baseHeroById) as HeroById;
    if (object.id) {
      message.id = object.id;
    }
    return message;
  },
  toJSON(message: HeroById): unknown {
    const obj: any = {};
    obj.id = message.id || 0;
    return obj;
  },
};

export const Hero = {
  encode(message: Hero, writer: Writer = Writer.create()): Writer {
    writer.uint32(8).int32(message.id);
    writer.uint32(18).string(message.name);
    return writer;
  },
  decode(reader: Reader, length?: number): Hero {
    let end = length === undefined ? reader.len : reader.pos + length;
    const message = Object.create(baseHero) as Hero;
    while (reader.pos < end) {
      const tag = reader.uint32();
      switch (tag >>> 3) {
        case 1:
          message.id = reader.int32();
          break;
        case 2:
          message.name = reader.string();
          break;
        default:
          reader.skipType(tag & 7);
          break;
      }
    }
    return message;
  },
  fromJSON(object: any): Hero {
    const message = Object.create(baseHero) as Hero;
    if (object.id) {
      message.id = Number(object.id);
    }
    if (object.name) {
      message.name = String(object.name);
    }
    return message;
  },
  fromPartial(object: DeepPartial<Hero>): Hero {
    const message = Object.create(baseHero) as Hero;
    if (object.id) {
      message.id = object.id;
    }
    if (object.name) {
      message.name = object.name;
    }
    return message;
  },
  toJSON(message: Hero): unknown {
    const obj: any = {};
    obj.id = message.id || 0;
    obj.name = message.name || "";
    return obj;
  },
};

type DeepPartial<T> = {
  [P in keyof T]?: T[P] extends Array<infer U>
  ? Array<DeepPartial<U>>
  : T[P] extends ReadonlyArray<infer U>
  ? ReadonlyArray<DeepPartial<U>>
  : T[P] extends Date | Function | Uint8Array | undefined
  ? T[P]
  : T[P] extends infer U | undefined
  ? DeepPartial<U>
  : T[P] extends object
  ? DeepPartial<T[P]>
  : T[P]
};

[iangregsondev]

If you feel its something you could release then that would be great, otherwise I don't mind doing it but right now - I am not able to build it because of the missing pbjs under build.

There are some other minimal things also which I think can fixed very easily, my typescript is complaining about implicit "any".

I will paste the output here. ( i ran tsc manually but if you do an npm install, it does the same as you have a "prepare" script in the package.json)

➜  ts-proto git:(master) ✗ tsc
src/main.ts:14:24 - error TS2307: Cannot find module '../build/pbjs'.

14 import { google } from '../build/pbjs';
                          ~~~~~~~~~~~~~~~

src/main.ts:272:61 - error TS2571: Object is of type 'unknown'.

272       initialValue = initialValue.addHashEntry(snakeToCamel(field.name), defaultValue(field.type));
                                                                ~~~~~

src/main.ts:272:87 - error TS2571: Object is of type 'unknown'.

272       initialValue = initialValue.addHashEntry(snakeToCamel(field.name), defaultValue(field.type));
                                                                                          ~~~~~

src/main.ts:317:48 - error TS7006: Parameter 'field' implicitly has an 'any' type.

317   messageDesc.field.filter(isRepeated).forEach(field => {
                                                   ~~~~~

src/main.ts:329:29 - error TS7006: Parameter 'field' implicitly has an 'any' type.

329   messageDesc.field.forEach(field => {
                                ~~~~~

src/main.ts:399:29 - error TS7006: Parameter 'field' implicitly has an 'any' type.

399   messageDesc.field.forEach(field => {
                                ~~~~~

src/main.ts:486:48 - error TS7006: Parameter 'field' implicitly has an 'any' type.

486   messageDesc.field.filter(isRepeated).forEach(field => {
                                                   ~~~~~

src/main.ts:492:29 - error TS7006: Parameter 'field' implicitly has an 'any' type.

492   messageDesc.field.forEach(field => {
                                ~~~~~

src/main.ts:554:29 - error TS7006: Parameter 'field' implicitly has an 'any' type.

554   messageDesc.field.forEach(field => {
                                ~~~~~

src/main.ts:597:48 - error TS7006: Parameter 'field' implicitly has an 'any' type.

597   messageDesc.field.filter(isRepeated).forEach(field => {
                                                   ~~~~~

src/main.ts:603:29 - error TS7006: Parameter 'field' implicitly has an 'any' type.

603   messageDesc.field.forEach(field => {
                                ~~~~~

src/plugin.ts:3:24 - error TS2307: Cannot find module '../build/pbjs'.

3 import { google } from '../build/pbjs';
                         ~~~~~~~~~~~~~~~

src/plugin.ts:16:39 - error TS7006: Parameter 'file' implicitly has an 'any' type.

16   const files = request.protoFile.map(file => {
                                         ~~~~

src/types.ts:1:24 - error TS2307: Cannot find module '../build/pbjs'.

1 import { google } from '../build/pbjs';
                         ~~~~~~~~~~~~~~~

src/types.ts:303:9 - error TS7006: Parameter 't' implicitly has an 'any' type.

303         t =>
            ~

src/types.ts:307:12 - error TS7006: Parameter 'mapType' implicitly has an 'any' type.

307       .map(mapType => {
               ~~~~~~~

src/types.ts:313:13 - error TS7006: Parameter '_' implicitly has an 'any' type.

313       .find(_ => true);
                ~

src/types.ts:322:32 - error TS2571: Object is of type 'unknown'.

322       return message.oneofDecl[f.oneofIndex].name;
                                   ~


Found 18 errors.

[stephenh]

i figured this was pbjs from npm

Ah, sorry, no, build/pbjs contains a few pbjs-generated types that ts-proto uses for bootstrapping + for its test suite. I just added a section to the readme about this, but running ./pbjs.sh will generate that directory and the appropriate contents.

Just the interfaces for the types and the interface for the service

I'm not against a sort of interfacesOnly type option, that would only output the types that you're asking for. I'm curious, what are you doing with the interfaces? I.e. how are you implementing/using them from your code?

The Observable change also seems fine, as long as both your proposed interfacesOnly and the regular/as-is modes worked, i.e. if someone used useObservable: true but still wanted all of the concrete types that ts-proto usually generates, ideally that would still work (but the concrete impls would of course now return observables).

[rory-ye-nv]

+1 for interface only. Here is the scenario, we use protobuf in server side and it will return either protobuf or json result. In the client side, we want to use json query and result in type script and don’t want to manually maintain the interface. So we want to generate the interface and keep it update during build. And we want to have an undefined value type in interface

[stephenh]

Ah sure, that makes sense. I've not personally used that setup before, so I'm not 100% sure the TS types would 100% match what the server-side JSON result is, but it seems reasonable. (And if it's not 100% aligned, we could probably nudge ts-proto to make it line up.)

Is that something you're fine with prototyping on your own, and could submit a PR? That would be great if so. I think you probably just want to add an "if interfaceOnly then don't generate service/etc./etc" check to this method:

https://github.com/stephenh/ts-proto/blob/master/src/main.ts#L51

[n3rdyme]

This is implemented on #28

The following options provide for interface-only output:

--plugin=./node_modules/.bin/protoc-gen-ts_proto
--ts_proto_out=./src/proto/
--ts_proto_opt=serializers=false,toFromJson=false,serviceStub=false

Until this is merged you can use npm i @n3rdyme/[email protected]

[stephenh]

Thanks @n3rdyme ! This is merged into master and will be in the next release.