Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pbjs does not use "import * as ..." for importing js files #1839

Open
AVKurbatov opened this issue Dec 19, 2022 · 2 comments
Open

Pbjs does not use "import * as ..." for importing js files #1839

AVKurbatov opened this issue Dec 19, 2022 · 2 comments

Comments

@AVKurbatov
Copy link

Versions:
protobufjs: 7.1.2,
protobufjs-cli: 1.0.2,
pbjs: 0.0.14,
ptjs: 0.2.4.

Hello!

We have a problem using protobuf-js-cli (pbjs, pbts) and protobuf-js.
In our project we use more than 100 proto files and some of them use more than 20 imports for other proto files.
Protobuf-js-cli generates all the code for importing protofiles into one.
As a result, one of the js files we generated has a size of 60 MB and 300,000 lines of code.
The pbts plugin can't handle this. It just crashes without any error messages and no d.ts file is created.
For other cases, it would be generally convenient for us to use imports in the generated js files instead of
large js files with all the imported code in them.

Could you give some advice? Can we use protobuf-js-cli to generate files without imported code in them?

@AVKurbatov
Copy link
Author

Let me explain it with the example.
I have two files: sub_module.proto:

syntax = "proto3";
package Codegen;

message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}

and module.proto:
syntax = "proto3";
package Codegen;

import "sub_module.proto";
message SomeComposition {
HelloRequest title = 1;
HelloReply assets = 2;
}
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}

If I generate js file for this two proto files, I get file module.js with definitions for HelloRequest and HelloReply.
I use command line "pbjs -t static-module -p protoFolder -w commonjs -o module.js module.proto"
But I want to import sub_module into module.
I see it in this way.

  1. We need to add imports to the head of the generated file:
    image
  2. We need to add the namespace to names for all classes described in the sub_module:
    image
  3. And finally we need to remove definitions of all classes that are defined in the sub_module:
    image

I'm going to update pbjs and pbts code according to this ideas.
What do you think about this plan? Are there any critical flaws in it?
Do we have any easier ways to solve this problem?

khmseu pushed a commit to khmseu/protobuf.js that referenced this issue Aug 3, 2023
AVKurbatov
commented
7 months ago
Fixed two issues.

If protofile uses some Namespace (for example, Test) and it contains some Service with the same name, nested Namespace Test.Test will be generated.
This Namespace will contain classes for in and out arguments for the Service (for example Test.Test.Request_in).
They can't be used from the same Namespace becouse of names conflict.
In this fix names of nested Namespaces are updated. The argument class now will be named Test.TestService.Request_in.
To use this mode run pbjs with the flag "-u", "--unify-names".
Proto-file can contain imports like
import "submodule.proto";
Currently all code from them is included into the result of the protobuf-js generated js-file.
So, if we have a lot of imports, the result of generation can be enormous.
This can be a problem and it was deeply discussed in the issue protobufjs#1839 (comment).
With this fix only content of the requested proto file will be generated. All content from imported protofiles will be imported in
the generated js and d.ts files.
To use this mode run pbjs and pbts with the flag "--use-imports".
@Yongle-Fu
Copy link

same issue, hope support import submodule.proto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants