Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: remove configure checks for win libraries we don't link against #17740

Merged
merged 1 commit into from
Jan 24, 2020

Conversation

fanquake
Copy link
Member

While cross compiling, HOST=x86_64-w64-mingw32, none of these libs actually seem to be passed to the linker. i.e tailing a build with make -j5 V=1 | rg -i 'mingwthrd|winspool|rpcrt4|crypt32'.

I'm not 100% sure about crypt32, even though the majority of our Windows cryptography usage, i.e CryptAcquireContextW or CryptGenRandom is provided by advapi32.

Note that rpcrt4 and mingwthrd are already missing from the MSVC build, so we can sync the remainder once it's clear what's actually needed. Hopefully sipsorcery can add some MSVC insight.

@DrahtBot
Copy link
Contributor

DrahtBot commented Dec 13, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17213 (gui: Add Windows taskbar progress by luke-jr)
  • #15382 (util: add runCommandParseJSON by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Contributor

@sipsorcery sipsorcery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me. My understanding is most of the mingw libraries are simple wrappers that load the matching Windows dll at runtime. If the build works without referencing a library then it seems pretty safe to remove it.

@sipsorcery
Copy link
Contributor

ACK 1778e20.

Note the appveyor build failed on the python functional tests which is nothing to do with this PR.

@laanwj
Copy link
Member

laanwj commented Dec 13, 2019

I'm not 100% sure about crypt32, even though the majority of our Windows cryptography usage, i.e CryptAcquireContextW or CryptGenRandom is provided by advapi32.

As for crypt32: we don't use any kind of weak linking of symbols, I'd say if it links, it's safe.

The only one that slightly scares me (that it can be dropped, not the change itself) is mingwthrd because in the deep past there have been issues with mingw not using the right threading runtime. But I suspect this is simply because we require using the -posix variant of the tool chain (because of using C++11 threads, instead of using win32 threads directly), so the library is bypassed.

@laanwj
Copy link
Member

laanwj commented Dec 13, 2019

Hmm this has some information about the hack that mingwthrd does: https://gcc.gnu.org/ml/java/2006-07/msg00003.html
it contains two functions which perform setup and cleanup based on loading/unloading notifications, so the symbols are not used directly.
I'm not sure about this one! It might not be something that's ever required to explicitly link against, but it feels risky somehow.

Oh, I missed the "none of these libs actually seem to be passed to the linker"—ok in that case, I don't see why it's checking for the library

So this is safe. These checks don't cause linkage. They just check whether the library exists. I suppose the explicit check for mingwthrd was added at some point to be sure a thread-supporting mingw is used. Maybe @theuni still knows.

@fanquake fanquake force-pushed the remove_windll_configure_checks branch from 1778e20 to fd33d90 Compare December 13, 2019 13:37
@fanquake
Copy link
Member Author

Added some additional changes for the MSVC build system.

@fanquake fanquake marked this pull request as ready for review December 13, 2019 13:38
@sipsorcery
Copy link
Contributor

fgep'ing for the defines below does not get any results for the src directory. As far as the msvc build goes if the define is not used in the code it's not used at all. They can be removed along with the ones already taken out.

It does actually seem a bit strange for any defines not to be used in the code no matter what the build system. I'd be surprised if the mingw makefile parses bitcoin_config.h to set it's linker command.

/* Define to 1 if you have the `advapi32' library (-ladvapi32). */
#define HAVE_LIBADVAPI32 1

/* Define to 1 if you have the `comctl32' library (-lcomctl32). */
#define HAVE_LIBCOMCTL32 1

/* Define to 1 if you have the `comdlg32' library (-lcomdlg32). */
#define HAVE_LIBCOMDLG32 1

/* Define to 1 if you have the `crypt32' library (-lcrypt32). */
#define HAVE_LIBCRYPT32 1

/* Define to 1 if you have the `gdi32' library (-lgdi32). */
#define HAVE_LIBGDI32 1

/* Define to 1 if you have the `imm32' library (-limm32). */
#define HAVE_LIBIMM32 1

/* Define to 1 if you have the `iphlpapi' library (-liphlpapi). */
#define HAVE_LIBIPHLPAPI 1

/* Define to 1 if you have the `kernel32' library (-lkernel32). */
#define HAVE_LIBKERNEL32 1

/* Define to 1 if you have the `mingwthrd' library (-lmingwthrd). */
#define HAVE_LIBMINGWTHRD 1

/* Define to 1 if you have the `mswsock' library (-lmswsock). */
#define HAVE_LIBMSWSOCK 1

/* Define to 1 if you have the `ole32' library (-lole32). */
#define HAVE_LIBOLE32 1

/* Define to 1 if you have the `oleaut32' library (-loleaut32). */
#define HAVE_LIBOLEAUT32 1

/* Define to 1 if you have the `rpcrt4' library (-lrpcrt4). */
#define HAVE_LIBRPCRT4 1

/* Define to 1 if you have the `rt' library (-lrt). */
/* #undef HAVE_LIBRT */

/* Define to 1 if you have the `shell32' library (-lshell32). */
#define HAVE_LIBSHELL32 1

/* Define to 1 if you have the `shlwapi' library (-lshlwapi). */
#define HAVE_LIBSHLWAPI 1

/* Define to 1 if you have the `ssp' library (-lssp). */
#define HAVE_LIBSSP 1

/* Define to 1 if you have the `user32' library (-luser32). */
#define HAVE_LIBUSER32 1

/* Define to 1 if you have the `uuid' library (-luuid). */
#define HAVE_LIBUUID 1

/* Define to 1 if you have the `winmm' library (-lwinmm). */
#define HAVE_LIBWINMM 1

/* Define to 1 if you have the `winspool' library (-lwinspool). */
#define HAVE_LIBWINSPOOL 1

/* Define to 1 if you have the `ws2_32' library (-lws2_32). */
#define HAVE_LIBWS2_32 1

/* Define to 1 if you have the `z ' library (-lz ). */
#define HAVE_LIBZ_ 1

@@ -115,7 +115,7 @@
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<AdditionalDependencies>crypt32.lib;Iphlpapi.lib;ws2_32.lib;Shlwapi.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalDependencies>Iphlpapi.lib;ws2_32.lib;Shlwapi.lib;kernel32.lib;user32.lib;gdi32.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can confirm removing those two libraries doesn't break the build.

@fanquake
Copy link
Member Author

Discussed with @theuni. Going to add specific symbols to each of the AC_CHECK_LIB calls in place of main.

@laanwj
Copy link
Member

laanwj commented Dec 15, 2019

Discussed with @theuni. Going to add specific symbols to each of the AC_CHECK_LIB calls in place of main.

Right.
I've started to wonder: why are these checks there in the first place? Why does it check for libraries it doesn't directly link against?
(and if it's libraries it does link against, would it be an idea to combine the check and adding the library in the same place, so it's easier to see why a certain library is there?)

@fanquake
Copy link
Member Author

Incorporated suggestions from @sipsorcery and added specific symbols to some of the AC_CHECK_LIB calls.

@sipsorcery
Copy link
Contributor

tACK (msvc and mingw on Win10) 890baac.

@bitcoin bitcoin deleted a comment from DrahtBot Jan 2, 2020
@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 3, 2020

Gitian builds

File commit 35fff5b
(master)
commit 65c7a3f
(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz 1a1b79c30ee23b1e... 6e3d069e49d0568c...
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz 4f5477e988e15339... 7f1e8fe70b2b9b5c...
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz 907d28fc95980468... 61f66840cbfeb2bb...
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz e55899ead1a7ff56... 3820fc0cd9640855...
bitcoin-0.19.99-i686-pc-linux-gnu-debug.tar.gz 95fa4bccc8b22ac0... ca541891a3c32a98...
bitcoin-0.19.99-i686-pc-linux-gnu.tar.gz 340c796226da90c7... d37ae258b32889d5...
bitcoin-0.19.99-osx-unsigned.dmg 1c5ab941efd90df4... d343969d150f639c...
bitcoin-0.19.99-osx64.tar.gz e0d2d4ef7befbf1f... ad92ec957ef6a02f...
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz 7fb3888850d15c57... 91c0c886a5e0b811...
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz c6bd3037c22bbde5... dca861193424cdd0...
bitcoin-0.19.99-win64-debug.zip a2369f394ffcb551... 9b23196ccac0dc5f...
bitcoin-0.19.99-win64-setup-unsigned.exe ffc23fa5abc3ca5f... 125ddc9039515ac1...
bitcoin-0.19.99-win64.zip 2d13e80f0011ce7d... 8ef5b7ae1a8aa1ce...
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz 2a4aa0a81c4d2591... 780d73b51acc3d67...
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz 8940a74e71b82107... 0170a14cbf1b191e...
bitcoin-0.19.99.tar.gz 7703c9c861b5b755... ff916a409a89a239...
bitcoin-core-linux-0.20-res.yml 5db46f5eca4bb180... 531a6705ffd1cb07...
bitcoin-core-osx-0.20-res.yml 9db10d1794650fb3... df8f5ee6051d739b...
bitcoin-core-win-0.20-res.yml 89078fed82c8928e... 3080c08906b966bf...
linux-build.log a3067a2425690fd1... 5729d5af45be9425...
osx-build.log a15a76ffd1535083... 92478cfb5c2b9455...
win-build.log 615dbd3684fcd0b0... 23ca0dee1ee3fd70...
bitcoin-core-linux-0.20-res.yml.diff 752f63b68fe13c99...
bitcoin-core-osx-0.20-res.yml.diff 8b1ae7821af4ca09...
bitcoin-core-win-0.20-res.yml.diff 64b1ae0e9f8f9cdd...
linux-build.log.diff 9b0f122c4c7126af...
osx-build.log.diff d10aef9c012c9b44...
win-build.log.diff 881623b0ad491f2e...

@practicalswift
Copy link
Contributor

Concept ACK

Thanks for removing all this old cruft.

@DrahtBot
Copy link
Contributor

Needs rebase

While cross compiling, HOST=x86_64-w64-mingw32, none of these
libs actually seem to be passed to the linker.
@fanquake fanquake force-pushed the remove_windll_configure_checks branch from 890baac to 2525c09 Compare January 23, 2020 00:01
@fanquake fanquake requested a review from sipsorcery January 23, 2020 00:02
@sipsorcery
Copy link
Contributor

ACK 2525c09.

@practicalswift
Copy link
Contributor

ACK 2525c09 -- diff looks correct

fanquake added a commit that referenced this pull request Jan 24, 2020
…'t link against

2525c09 build: remove configure checks for win libraries we don't link against (fanquake)

Pull request description:

  While cross compiling, `HOST=x86_64-w64-mingw32`, none of these libs actually seem to be passed to the linker. i.e tailing a build with `make -j5 V=1 | rg -i 'mingwthrd|winspool|rpcrt4|crypt32'`.

  I'm not 100% sure about `crypt32`, even though the majority of our Windows cryptography usage, i.e [`CryptAcquireContextW`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw) or [`CryptGenRandom`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom) is provided by `advapi32`.

  Note that `rpcrt4` and `mingwthrd` are already missing from the MSVC build, so we can sync the remainder once it's clear what's actually needed. Hopefully sipsorcery can add some MSVC insight.

ACKs for top commit:
  practicalswift:
    ACK 2525c09 -- diff looks correct
  sipsorcery:
    ACK 2525c09.

Tree-SHA512: c756618f85ce2ab1e14e5514dbdc490d94c1c6dfd7a3e3d3b16344ae302fb789585dd10b5c2d784f961f3115bec1d914615051b3184bea00dfbcc3c23884ab4a
@fanquake fanquake merged commit 2525c09 into bitcoin:master Jan 24, 2020
@fanquake fanquake deleted the remove_windll_configure_checks branch January 24, 2020 07:36
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jan 24, 2020
… we don't link against

2525c09 build: remove configure checks for win libraries we don't link against (fanquake)

Pull request description:

  While cross compiling, `HOST=x86_64-w64-mingw32`, none of these libs actually seem to be passed to the linker. i.e tailing a build with `make -j5 V=1 | rg -i 'mingwthrd|winspool|rpcrt4|crypt32'`.

  I'm not 100% sure about `crypt32`, even though the majority of our Windows cryptography usage, i.e [`CryptAcquireContextW`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw) or [`CryptGenRandom`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom) is provided by `advapi32`.

  Note that `rpcrt4` and `mingwthrd` are already missing from the MSVC build, so we can sync the remainder once it's clear what's actually needed. Hopefully sipsorcery can add some MSVC insight.

ACKs for top commit:
  practicalswift:
    ACK 2525c09 -- diff looks correct
  sipsorcery:
    ACK 2525c09.

Tree-SHA512: c756618f85ce2ab1e14e5514dbdc490d94c1c6dfd7a3e3d3b16344ae302fb789585dd10b5c2d784f961f3115bec1d914615051b3184bea00dfbcc3c23884ab4a
@luke-jr
Copy link
Member

luke-jr commented Mar 25, 2020

These checks don't cause linkage. They just check whether the library exists.

This is incorrect. AC_CHECK_LIB does cause linkage...

I don't see how this didn't just reintroduce CVE-2012-1910.

@Yuthana826
Copy link

btcsuite/btcutil#161

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Jun 9, 2020
…n libraries we don't link against"

This partially reverts commit 2525c09.
sidhujag pushed a commit to syscoin-core/syscoin that referenced this pull request Nov 10, 2020
… we don't link against

2525c09 build: remove configure checks for win libraries we don't link against (fanquake)

Pull request description:

  While cross compiling, `HOST=x86_64-w64-mingw32`, none of these libs actually seem to be passed to the linker. i.e tailing a build with `make -j5 V=1 | rg -i 'mingwthrd|winspool|rpcrt4|crypt32'`.

  I'm not 100% sure about `crypt32`, even though the majority of our Windows cryptography usage, i.e [`CryptAcquireContextW`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw) or [`CryptGenRandom`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom) is provided by `advapi32`.

  Note that `rpcrt4` and `mingwthrd` are already missing from the MSVC build, so we can sync the remainder once it's clear what's actually needed. Hopefully sipsorcery can add some MSVC insight.

ACKs for top commit:
  practicalswift:
    ACK 2525c09 -- diff looks correct
  sipsorcery:
    ACK 2525c09.

Tree-SHA512: c756618f85ce2ab1e14e5514dbdc490d94c1c6dfd7a3e3d3b16344ae302fb789585dd10b5c2d784f961f3115bec1d914615051b3184bea00dfbcc3c23884ab4a
luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 17, 2020
…n libraries we don't link against"

This partially reverts commit 2525c09.
luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 25, 2020
…n libraries we don't link against"

This partially reverts commit 2525c09.
luke-jr added a commit to luke-jr/bitcoin that referenced this pull request Oct 6, 2021
…n libraries we don't link against"

This partially reverts commit 2525c09.
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants