[gmp, nettle, ] build failure x64-windows/x64-windows-static using x86-yasm-tool and x86-yasm-tool-helper

[StarGate-One]

Host Environment

• OS: [Microsoft Windows [Version 10.0.19042.572]]
• Compiler: Visual Studio Version: 16.7.7 VS Tools Version: 14.27.29110
  Visual Studio Version: 15.9.28 VS Tools Version: 14.16.27023
  Reference PR [vcpkg baseline] [vs-yasm] Build yasm instead of downloading it to work around memory corruption bugs in yasm itself. #14003 [vcpkg baseline] [vs-yasm] Build yasm instead of downloading it to work around memory corruption bugs in yasm itself
  Reference Issue [nettle:x64-windows] build failure #12240 [nettle:x64-windows] build failure
  Reference Issue [vcpkg] use x64 (64-bit) compiler/linker/utilities and tools when building x64 triplets on windows #13642 [vcpkg] use x64 (64-bit) compiler/linker/utilities and tools when building x64 triplets on windows
  See additional context below...

To Reproduce
Steps to reproduce the behavior:

bootstrap-vcpkg.bat is run with -win64 option

./vcpkg install gmp:x64-nettle nettle:x64-windows --editable --x-use-aria2

Failure logs

• Cut and past the appropriate build messages from the console output.
  x64-windows and x64-windows-static
  None - build just exits into oblivion.

• Please attach any additional failure logs mentioned in the console output.
  None

Additional context

• After updating vcpkg with latest updates (includes PR [vcpkg baseline] [vs-yasm] Build yasm instead of downloading it to work around memory corruption bugs in yasm itself. #14003) gmp and nettle build just disappears into oblivion.
• I had the similar issue in the past and it was solved when I found Issue [nettle:x64-windows] build failure #12240 where someone else was having a similar issue.
• Bottom line, once I built a x64-windows-static version of yasm, and configured vcpkg to use it instead of the downloaded version found in vcpkg_find_acquire_program.cmake, I no longer experience the problem with ports gmp and nettle (not the x86-windows/x86-windows-static).
• I spent the past few hours figuring out how to hack the vs-yasm, yasm-tool and yasm-tool-helper to use a x64 version instead of x86 version.
• Afterwards I was able to compile ports gmp and nettle without problem.
• So, can we please fix these utilities so they install/use the same bitness of yasm as the architect triplet being built as using the x86 version just cannot produce desired results?
• I am not sure why we keep using x86 binaries to build x64 binaries when there is a x64 tool/utility - it is very frustrating, especially when using the x86 binaries you get memory corruption/segmentation faults or it just plain runs out of memory allowed for a 32 bit binary and it tries to restart as a 64 bit binary but always fails.

[JackBoosY]

cc @BillyONeal

[BillyONeal]

Unfortunately we were forced to build yasm from source ourselves rather than downloading it through vcpkg_find_acquire_program due to the high number of memory corruption bugs in yasm itself, see #14003

the particular one being: yasm/yasm#153 (comment)

I spent the past few hours figuring out how to hack the vs-yasm, yasm-tool and yasm-tool-helper to use a x64 version instead of x86 version.

You don't want to do that: those tools are used only when building nettle etc. and you use the x86 version even though you want an x64-windows-static output. We wanted to use an x64 version of yasm, but unfortunately when you build yasm.exe as x64, more aforementioned memory corruption bugs happen and you get malformed .objs, resulting in linker errors like:

C:\Dev\vcpkg\buildtrees\gmp\x64-windows-static-rel\adcc6b3cc4-2b6258d695.clean\SMP\obj\Release\x64\libgmp\addaddmul_1msb0.obj : fatal error LNK1106: invalid file or disk full: cannot seek to 0x23B

(Note that vcpkg_find_acquire_program always acquired an x86 version of yasm before)

I'm apologize that this experience is so confusing :(. We have other ports that need this crosscompiling "build the x86 one first" but not for x64 :/.

If you have better suggestions or know how to put pressure on the yasm folks to make it not corrupt memory or produce malformed output so that we can go back to vcpkg_find_acquire_program that would be great!

[StarGate-One]

@BillyONeal - see inline...
cc: @JackBoosY @voskrese @Neumann-A @ras0219-msft

Unfortunately we were forced to build yasm from source ourselves rather than downloading it through `vcpkg_find_acquire_program` due to the high number of memory corruption bugs in yasm itself, see #14003
the particular one being: [yasm/yasm#153 (comment)](https://github.com/yasm/yasm/issues/153#issuecomment-706038815)

What prevents us from building from source the x64 yasm for x64 triplets and x86 yasm for x86 triplets, as I have been doing for months? I have a script to build yasm x64-windows-static and copy the exe over to tools\yasm directory

> I spent the past few hours figuring out how to hack the vs-yasm, yasm-tool and yasm-tool-helper to use a x64 version instead of x86 version.

You don't want to do that: those tools are used only when building nettle etc. and you use the x86 version even though you want an x64-windows-static output. We wanted to use an x64 version of yasm, but unfortunately when you build `yasm.exe` as x64, more aforementioned memory corruption bugs happen and you get malformed `.objs`, resulting in linker errors like:

I have not experienced this issue building the x64 yasm for x64 ports. The corruption issues have only been present for me when using an x86 yasm on x64 ports, the reason I hacked the new changes to create and use an x64 yasm

C:\Dev\vcpkg\buildtrees\gmp\x64-windows-static-rel\adcc6b3cc4-2b6258d695.clean\SMP\obj\Release\x64\libgmp\addaddmul_1msb0.obj : fatal error LNK1106: invalid file or disk full: cannot seek to 0x23B

Yes, I receive this error when building x64 ports needing yasm when I use the x86 yasm, they disappear when I use the x64 build yasm

(Note that `vcpkg_find_acquire_program` always acquired an x86 version of yasm before)

As I ran into the issue months ago, to eliminate these issues I locally customized the vcpkgTools.xml, vcpkg_acquire_msys.cmake and vcpkg_find_acquire_program.cmake functions to obtain x64 bit versions instead of the x86 versions, similar to how python3 is handle in the current vcpkg_find_acquire_program.cmake function now. All windows ports use x64 tools/utilities, whether they be x86 or x64, dynamic or static. In other words, if an x64 version of the tool/utility is available, then my version of vcpkg uses it for windows builds and I have a lot less unexplained issues with x64 ports than we see each day in the issues list,

I'm apologize that this experience is so confusing :(. We have other ports that need this crosscompiling "build the x86 one first" but not for x64 :/.

No need to apologize, it is a learning experience. Just have an objective outlook to different ideas and/or solutions/suggestions. Really listen to what someone else is saying, instead of just hearing why they say but already having your mind made up before they finish. It took me a couple decades to really understand listening and hearing are really two totally different things.

If you have better suggestions or know how to put pressure on the yasm folks to make it not corrupt memory or produce malformed output so that we can go back to `vcpkg_find_acquire_program` that would be great!

I am thinking to fork vcpkg and build and x64-windows/x64-windows-static only package manager, no Unix/linux, macOS or x86 support, would this be ok? After all, the Unix/Linux/macOS all have several package managers, and x86 is on it way out as 16 bit was in the 80's/90's

[BillyONeal]

What prevents us from building from source the x64 yasm for x64 triplets and x86 yasm for x86 triplets

If you build yasm from source as 64 bit, it produces malformed output. Like they are writing a size_t in the output that should be a uint32_t or something like that. At least, with the SHA we had to fetch to get the fix for the other memory corruption bug indicated.

I have not experienced this issue building the x64 yasm for x64 ports.

I'm not sure what's going on then; maybe the problem was committed between the version you tried and the version we're building?

Yes, I receive this error when building x64 ports needing yasm when I use the x86 yasm, they disappear when I use the x64 build yasm

I don't understand given that vcpkg has always only ever used the 32 bit yasm (vcpkg_find_acquire_program downloaded the 32 bit copy), and our CI infrastructure that builds nettle and gmp has had no such issues since. Although that change was admittedly only a couple of days ago.

I am thinking to fork vcpkg and build and x64-windows/x64-windows-static only package manager, no Unix/linux

I don't think you need to fork vcpkg over that; we would gladly just build it, like we do for protoc and friends, if the result would actually link. It did not in my testing. (To be clear, the link failure I posted was building gmp x64 using the x64 built from source yasm)

I just wish that these ports didn't depend on build tools with so many memory corruption bugs :(

[StarGate-One]

I take your word for it on the CI build infrastructure, as I do not have something as elaborate.
I just know I have eliminated many of my "memory fragmentation, corruptions and other unexplained issues" by using 64 bit tools/utilities to build my 64 bit ports, programs, etc.

I do maintain a vanilla, as delivered vcpkg instance, so when I do have issues, I can see if same issue exists in the vanilla, then I can submit issue against vanilla and not worry it was some customization I have made.

I close for now, and I will make customizations to keep my customized versions working.

Thank you for all your hard work, I appreciate it a lots.