bug: broken vendoring in [email protected]

[nickeskov]

Description

When using vendoring (go mod vendor) and then compiling dependencies, an error appears regarding missing .s files.

I'll demonstrate the compilation error using the gowaves project as an example.
When attempting to build the executable, the following error messages appear from the go compiler.

Go compiler output

# github.com/consensys/gnark-crypto/ecc/bls12-377/fp
vendor/github.com/consensys/gnark-crypto/ecc/bls12-377/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_6w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls12-381/fp
vendor/github.com/consensys/gnark-crypto/ecc/bls12-381/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_6w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls24-315/fp
vendor/github.com/consensys/gnark-crypto/ecc/bls24-315/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_5w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls24-317/fp
vendor/github.com/consensys/gnark-crypto/ecc/bls24-317/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_5w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bn254/fp
vendor/github.com/consensys/gnark-crypto/ecc/bn254/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bw6-633/fp
vendor/github.com/consensys/gnark-crypto/ecc/bw6-633/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_10w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bw6-761/fp
vendor/github.com/consensys/gnark-crypto/ecc/bw6-761/fp/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_12w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bn254/fr
vendor/github.com/consensys/gnark-crypto/ecc/bn254/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls12-377/fr
vendor/github.com/consensys/gnark-crypto/ecc/bls12-377/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls12-381/fr
vendor/github.com/consensys/gnark-crypto/ecc/bls12-381/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls24-315/fr
vendor/github.com/consensys/gnark-crypto/ecc/bls24-315/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bls24-317/fr
vendor/github.com/consensys/gnark-crypto/ecc/bls24-317/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_4w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bw6-633/fr
vendor/github.com/consensys/gnark-crypto/ecc/bw6-633/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_5w_amd64.s: no such file or directory
# github.com/consensys/gnark-crypto/ecc/bw6-761/fr
vendor/github.com/consensys/gnark-crypto/ecc/bw6-761/fr/element_amd64.s:10: #include: open /usr/local/opt/go/field/asm/element_6w_amd64.s: no such file or directory

The issue does not occur when vendoring is not used.

Expected Behavior

Successful compilation both with and without vendoring.

Actual Behavior

Compilation works fine without vendoring.
With go mod vendor compilation fails with the error described above.

Steps to Reproduce

1. Clone gowaves
2. Run go get github.com/consensys/[email protected] in the project directory (it also updates gnark-crypto to 0.15.0)
3. Run go mod tidy && go mod vendor in the project directory
4. Run make build-node-native in the project directory which is equivalent to go build -o build/bin/native/node ./cmd/node
5. Observe the output

Context

Vendoring support is necessary for us in the gowaves project.

Your Environment

• gnark-crypto version used: 0.15.0
• go version (e.g. 1.20.6): go1.23.5 and go1.22.11
• Operating System and version: macOS 14.7.2 and ubuntu-24.04

[gbotrel]

Managed to reproduced easily on an empty project. Thanks for the report.

Possible fixes (cc @ivokub, thoughts?) :

1. We revert to generating assembly files directly per package and duplicate things. The only silly thing that bother me here, is that then github classify gnark-crypto as assembly project whereas we just have 3 or 4 methods in assembly 🙄
2. Ugly hack but should work; we use a dummy //go:embed directive to force go mod vendor to include the file. I think that would increase binary size of about 5kb for most modulus, 50kb for the fat ones with more assembly.. not too bad but could lead to unwanted side effects in some environments.
3. put the assembly files in proper go packages maybe internal, export methods, call them. most cases the compiler should inline the wrapper just fine (edit: I hope).

[nickeskov]

Thank you for the fix. Works for us. 🙂