build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat #22287

hebasto · 2021-06-20T00:04:50Z

This PR is a partial fix for #21454 (see #22281 for the complete fix for the bitcoind compatibility).

Gitian builds:

Generating report
5d2b1cf6c0a59796e0e087877faff290a252f79bae1a61aeceeda15e5c316186  bitcoin-530bc278b9fa-aarch64-linux-gnu-debug.tar.gz
a67a2a8a160d289fabe786e00715eeccf800e418a305d35a774721a74f738a4a  bitcoin-530bc278b9fa-aarch64-linux-gnu.tar.gz
315f787b8f77b55a37bf56201c747f77c5641c6eab1779049fdc65ef6e3aa162  bitcoin-530bc278b9fa-arm-linux-gnueabihf-debug.tar.gz
58d1b886f10b9a6011645bb65795ed0d418433c44b83fe8359ed4b31f501c167  bitcoin-530bc278b9fa-arm-linux-gnueabihf.tar.gz
31eedc8531fca0b4bd76c5b0cbc8799f6621a758e5fe9abda3aef3f264832c89  bitcoin-530bc278b9fa-powerpc64-linux-gnu-debug.tar.gz
79b8ca4bee6fc24a5c3216fed95f341510f15cdaaf04e2b6d4db991a3d077450  bitcoin-530bc278b9fa-powerpc64-linux-gnu.tar.gz
0fcd01009aa80b9c4b567a241d0d56b579756d9f70a1a78e0fa8c015b99d9bfa  bitcoin-530bc278b9fa-powerpc64le-linux-gnu-debug.tar.gz
af63f25b3de98cf2f25b6592dec9c3b4f95021fcc11396fb345c592979b9e4b7  bitcoin-530bc278b9fa-powerpc64le-linux-gnu.tar.gz
539d6317a7c2cfe6cbecb07eb82ef9b039010ef544cc9a74984a7cb23c03b58a  bitcoin-530bc278b9fa-riscv64-linux-gnu-debug.tar.gz
75ce0a0eabb4555589e632676b2e44df8365f320dea28d9edcd54ec78b4038d1  bitcoin-530bc278b9fa-riscv64-linux-gnu.tar.gz
9dc543d3b0c9ea39e153616132b0c41205a639d0129b752e7151712b8eda54b7  bitcoin-530bc278b9fa-x86_64-linux-gnu-debug.tar.gz
ba5452615ebd1c1398916837ee1a36233721819c08f48ea12ac7c03ac67526c7  bitcoin-530bc278b9fa-x86_64-linux-gnu.tar.gz
2b618a7b6e99bffc15522ca9f4bc0bfb1187a9bb0b005cc32d23127289644cc1  src/bitcoin-530bc278b9fa.tar.gz
9db429d8b661e2d7a48be8b8f9bcbb3df4e8a6b8cce0bd0660d54c1422c0d696  bitcoin-core-linux-22-res.yml
Done.

I tested only:

bitcoin-530bc278b9fa-arm-linux-gnueabihf.tar.gz
bitcoin-530bc278b9fa-x86_64-linux-gnu.tar.gz

sipa · 2021-06-20T00:29:11Z

Perhaps you want to make it print the value of cmd, so we can figure out which F_ constants are used but not handled by the wrapper. The partial approach is pretty uncomfortable though.

sipa · 2021-06-20T00:35:01Z

src/compat/glibc_compat.cpp

+    case F_SET_FILE_RW_HINT: goto takes_uint64_t_ptr;
+
+    default:
+        fprintf(stderr, "fcntl64 workaround got unknown F_XXX constant");


Don't remove the exit here. This is sort of an assert; if you continue it'll invoke fcntl with who knows what behavior.

hebasto · 2021-06-20T05:11:12Z

Updated 639bfc4 -> f0ca8bc (pr22287.01 -> pr22287.03, diff):

addressed @sipa's comments
fixed CI builds

@sipa

The partial approach is pretty uncomfortable though.

Here are some possible alternatives:

bump minimum glibc version for the 22.0 release binaries
revert ca85449 (gitian: Bump descriptors to Focal for 22.0 #21036) for Linux builds
something else?

sipa · 2021-06-20T05:18:52Z

How did you fix the failures?

hebasto · 2021-06-20T05:25:44Z

How did you fix the failures?

ARM hosts are excluded from this solution
#if !defined(__arm__) && defined(__GLIBC__) && (__GLIBC__ == 2) && (__GLIBC_MINOR__ >= 28)

hebasto · 2021-06-20T05:42:08Z

Closing. Binaries are broken (

sipa · 2021-06-20T05:58:45Z

How are they broken?

hebasto · 2021-06-20T06:00:30Z

How are they broken?

On x86_64 machine:

$ bin/bitcoind -signet
2021-06-20T05:45:29Z Bitcoin Core version v21.99.0-f0ca8bc08c0d (release build)
2021-06-20T05:45:29Z Signet derived magic (message start): 0a03cf40
2021-06-20T05:45:29Z Assuming ancestors of block 0000002a1de0f46379358c1fd09906f7ac59adf3712323ed90eb59e4c183c020 have valid signatures.
2021-06-20T05:45:29Z Setting nMinimumChainWork=00000000000000000000000000000000000000000000000000000019fd16269a
2021-06-20T05:45:29Z Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
2021-06-20T05:45:29Z Using RdSeed as additional entropy source
2021-06-20T05:45:29Z Using RdRand as an additional entropy source
2021-06-20T05:45:29Z Default data directory /home/hebasto/.bitcoin
2021-06-20T05:45:29Z Using data directory /home/hebasto/.bitcoin/signet
2021-06-20T05:45:29Z Config file: /home/hebasto/.bitcoin/bitcoin.conf
2021-06-20T05:45:29Z Config file arg: dbcache="8192"
2021-06-20T05:45:29Z Config file arg: logips="1"
2021-06-20T05:45:29Z Config file arg: logthreadnames="1"
2021-06-20T05:45:29Z Config file arg: maxmempool="500"
2021-06-20T05:45:29Z Config file arg: server="1"
2021-06-20T05:45:29Z Config file arg: [main] addnode="ejxefzf5fpst4mg2rib7grksvscl7p6fvjp6agzgfc2yglxnjtxc3aid.onion"
2021-06-20T05:45:29Z Config file arg: [main] addnode="rp7k2go3s5lyj3fnj6zn62ktarlrsft2ohlsxkyd7v3e3idqyptvread.onion"
2021-06-20T05:45:29Z Config file arg: [main] addnode="mwmfluek4au6mxxpw6fy7sjhkm65bdfc7izc7lpz3trewfdghyrzsbid.onion"
2021-06-20T05:45:29Z Config file arg: [main] blocksdir="/home/hebasto/Blockchain"
2021-06-20T05:45:29Z Config file arg: [main] dnsseed="0"
2021-06-20T05:45:29Z Config file arg: [main] listenonion="1"
2021-06-20T05:45:29Z Config file arg: [main] onion="127.0.0.1:9050"
2021-06-20T05:45:29Z Config file arg: [main] uacomment="h42"
2021-06-20T05:45:29Z Config file arg: [main] wallet=false
2021-06-20T05:45:29Z Config file arg: [regtest] blocksdir="/home/hebasto/VMs/bitcoin"
2021-06-20T05:45:29Z Config file arg: [signet] blocksdir="/home/hebasto/VMs/bitcoin"
2021-06-20T05:45:29Z Config file arg: [test] blocksdir="/home/hebasto/VMs/bitcoin"
2021-06-20T05:45:29Z Config file arg: [test] listen="1"
2021-06-20T05:45:29Z Config file arg: [test] onlynet="ipv4"
2021-06-20T05:45:29Z Config file arg: [test] server="1"
2021-06-20T05:45:29Z Setting file arg: wallet = ["210528d","210528-bdb"]
2021-06-20T05:45:29Z Command-line arg: signet=""
2021-06-20T05:45:29Z Using at most 125 automatic connections (1024 file descriptors available)
2021-06-20T05:45:29Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2021-06-20T05:45:29Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2021-06-20T05:45:29Z Script verification uses 7 additional threads
2021-06-20T05:45:29Z scheduler thread start
2021-06-20T05:45:29Z HTTP: creating work queue of depth 16
2021-06-20T05:45:29Z Using random cookie authentication.
2021-06-20T05:45:29Z Generated RPC authentication cookie /home/hebasto/.bitcoin/signet/.cookie
2021-06-20T05:45:29Z HTTP: starting 4 worker threads
2021-06-20T05:45:29Z Using wallet directory /home/hebasto/.bitcoin/signet/wallets
2021-06-20T05:45:29Z init message: Verifying wallet(s)…
2021-06-20T05:45:29Z Using SQLite Version 3.32.1
2021-06-20T05:45:29Z Using wallet /home/hebasto/.bitcoin/signet/wallets/210528d
fcntl64 hack can't use glibc flock directlyterminate called without an active exception
Aborted (core dumped)

sipa · 2021-06-20T06:01:45Z

You should compile with the change that prints the cmd value, so the missing cmd can be added. This is entirely expected - the wrapper needs to know all the used cmd values, and the code you copied only has a few.

hebasto · 2021-06-20T06:05:27Z

You should compile with the change that prints the cmd value, so the missing cmd can be added. This is entirely expected - the wrapper needs to know all the used cmd values, and the code you copied only has a few.

Ok. I'll do.

FWIW, it is not a missed cmd value, rather this stub for file locking features:

takes_flock_ptr_INCOMPATIBLE:
    //
    // !!! This is the breaking case: the size of the flock
    // structure changed to accommodate larger files.  If you
    // need this, you'll have to define a compatibility struct
    // with the older glibc and make your own entry point using it,
    // then call fcntl64() with it directly (bear in mind that has
    // been remapped to the old fcntl())
    //
    fprintf(stderr, "fcntl64 hack can't use glibc flock directly");
    exit(1);

sipa · 2021-06-20T06:06:21Z

Oh, you're right. That's a bigger problem.

sipa · 2021-06-20T06:16:22Z

Reverting #21036 isn't an option for guix, I assume.

hebasto · 2021-06-20T06:34:52Z

Oh, you're right. That's a bigger problem.

Btw, the offensive cmd is F_SETLK 6 /* Set record locking info (non-blocking). */

sipa · 2021-06-20T06:36:29Z

Yes, so we'll need to implement a wrapper for that. None of the call sites do anything complicated (and in particular, none pass file ranges, which could lead to actual imcompatibilities).

hebasto · 2021-06-20T12:50:37Z

@sipa

Yes, so we'll need to implement a wrapper for that. None of the call sites do anything complicated (and in particular, none pass file ranges, which could lead to actual imcompatibilities).

Done. OP updated.

sipa · 2021-06-20T13:44:08Z

I don't think that's correct. The old comment for the incompatible case explained that the flock data structure had changed in old glibc vs new one, so you'd need to convert between the two in the wrapper.

hebasto · 2021-06-21T05:33:28Z

Btw, if we stop release ARM-32bit binaries, the fcntl64@GLIBC_2.28 symbol could be completely eliminated with $(package)_cppflags_linux = -DSQLITE_DISABLE_LFS in depends/packages/sqlite.mk.

hebasto · 2021-06-21T09:32:15Z

@sipa

I don't think that's correct. The old comment for the incompatible case explained that the flock data structure had changed in old glibc vs new one, so you'd need to convert between the two in the wrapper.

I've compared struct flock and struct flock64 in glibc 2.31 (new) to struct flock in glibc 2.27 (old) -- all member sizes are the same for all tested archs. It seems with our (default) glibc feature set no need to remap struct flock.

laanwj · 2021-06-21T09:32:45Z

Btw, if we stop release ARM-32bit binaries, the fcntl64@GLIBC_2.28

NACK on so suddenly stopping support for ARM 32 bit before the release. If there is a reason to phase it out we should at least announce it a release in advance. But not sure this is a good idea. For the record I'm still running a node on 32-bit ARM hardware myself, and remember that 64-bit RPi's used to ship with 32 bit software stack until very recently.

hebasto · 2021-06-21T09:36:35Z

For the record I'm still running a node on 32-bit ARM hardware myself...

So am I :)

laanwj · 2021-06-21T14:40:33Z

could be completely eliminated with $(package)_cppflags_linux = -DSQLITE_DISABLE_LFS in depends/packages/sqlite.mk.

Having thought about it a bit I think for other platforms this is a good angle though. The only use of this function in our distributed binary is from sqlite, after all. Not using the symbol would be more comfortable to me than providing our own (potentially buggy) implementation. For ARM32 we could bump the minimum GLIBC version.

DrahtBot · 2021-06-21T18:37:16Z

Guix builds

File	commit `6a67366` (master)	commit `b5b94f1` (master and this pull)
SKIPATTEST.TAG	`e3b0c44298fc1c14...`	`e3b0c44298fc1c14...`
*.tar.gz	`ed3096aa949f1b56...`	`4a313dce87f47a84...`
guix_build.log	`7920a375a22476b1...`	`7d5cc645c0a46904...`
guix_build.log.diff		`ec96553de4e951ec...`

hebasto · 2021-06-22T01:26:09Z

could be completely eliminated with $(package)_cppflags_linux = -DSQLITE_DISABLE_LFS in depends/packages/sqlite.mk.

Having thought about it a bit I think for other platforms this is a good angle though. The only use of this function in our distributed binary is from sqlite, after all. Not using the symbol would be more comfortable to me than providing our own (potentially buggy) implementation. For ARM32 we could bump the minimum GLIBC version.

#22305 is an alternative approach.

DrahtBot · 2021-06-22T03:54:04Z

Gitian builds

File	commit `398dd67` (master)	commit `af38111` (master and this pull)
*-osx-unsigned.dmg	`92e2d116b58d4214...`	`84609ca486e37151...`
*-osx64.tar.gz	`02578406a266bd45...`	`edf43cc82a17ce60...`
*.tar.gz	`b16fc2a40cba5b8c...`	`a154737ea6fb8a77...`
bitcoin-core-osx-22-res.yml	`f2972b31a41adb54...`	`09a3a0c97a691317...`
linux-build.log	`6c8e74cad31e39a3...`	`ed443242cb7063fd...`
osx-build.log	`bb8f3e7c16328172...`	`b83096dfc1d68087...`
win-build.log	`2eea226b75083143...`	`c97c905ce7e176db...`
bitcoin-core-osx-22-res.yml.diff		`65aeb4d4d464eee3...`
linux-build.log.diff		`b140ad14e0f0c7b5...`
osx-build.log.diff		`1d12c97443fcdbee...`
win-build.log.diff		`82e48d93bb7e9584...`

hebasto · 2021-07-01T08:59:28Z

Closing in favor of #22365.

hebasto added the Build system label Jun 20, 2021

This was referenced Jun 20, 2021

build: Avoid @GLIBC_2.29 libm symbols when --enable-glibc-back-compat #22281

Closed

gitian: GLIBC_2.29 not found error at bitcoind startup on bionic #21454

Closed

hebasto force-pushed the 210619-fcntl branch from 2ff4bdd to 639bfc4 Compare June 20, 2021 00:17

hebasto marked this pull request as draft June 20, 2021 00:19

sipa reviewed Jun 20, 2021

View reviewed changes

hebasto mentioned this pull request Jun 20, 2021

devtools: Correctly extract symbol versions in symbol-check #22244

Merged

hebasto force-pushed the 210619-fcntl branch 3 times, most recently from ef22154 to f0ca8bc Compare June 20, 2021 04:24

hebasto marked this pull request as ready for review June 20, 2021 05:02

hebasto closed this Jun 20, 2021

hebasto reopened this Jun 20, 2021

hebasto marked this pull request as draft June 20, 2021 06:06

build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat

530bc27

hebasto force-pushed the 210619-fcntl branch from f0ca8bc to 530bc27 Compare June 20, 2021 11:19

hebasto marked this pull request as ready for review June 20, 2021 12:49

maflcko added DrahtBot Guix build requested Needs gitian build labels Jun 21, 2021

laanwj added this to the 22.0 milestone Jun 21, 2021

DrahtBot removed the DrahtBot Guix build requested label Jun 21, 2021

hebasto mentioned this pull request Jun 22, 2021

build: Avoid fcntl64@GLIBC_2.28 in libsqlite3.a #22305

Closed

DrahtBot removed the Needs gitian build label Jun 22, 2021

hebasto closed this Jul 1, 2021

hebasto mentioned this pull request Jul 8, 2021

release: Remove gitian #22418

Merged

hebasto deleted the 210619-fcntl branch October 5, 2021 11:38

bitcoin locked and limited conversation to collaborators Oct 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat #22287

build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat #22287

hebasto commented Jun 20, 2021 •

edited

Loading

sipa commented Jun 20, 2021

sipa Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021 •

edited

Loading

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 21, 2021

hebasto commented Jun 21, 2021

laanwj commented Jun 21, 2021 •

edited

Loading

hebasto commented Jun 21, 2021

laanwj commented Jun 21, 2021 •

edited

Loading

DrahtBot commented Jun 21, 2021

hebasto commented Jun 22, 2021

DrahtBot commented Jun 22, 2021

hebasto commented Jul 1, 2021

build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat #22287

build: Avoid fcntl64@GLIBC_2.28 symbol when --enable-glibc-back-compat #22287

Conversation

hebasto commented Jun 20, 2021 • edited Loading

Gitian builds:

sipa commented Jun 20, 2021

sipa Jun 20, 2021

Choose a reason for hiding this comment

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021 • edited Loading

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 20, 2021

sipa commented Jun 20, 2021

hebasto commented Jun 21, 2021

hebasto commented Jun 21, 2021

laanwj commented Jun 21, 2021 • edited Loading

hebasto commented Jun 21, 2021

laanwj commented Jun 21, 2021 • edited Loading

DrahtBot commented Jun 21, 2021

Guix builds

hebasto commented Jun 22, 2021

DrahtBot commented Jun 22, 2021

Gitian builds

hebasto commented Jul 1, 2021

hebasto commented Jun 20, 2021 •

edited

Loading

sipa commented Jun 20, 2021 •

edited

Loading

laanwj commented Jun 21, 2021 •

edited

Loading

laanwj commented Jun 21, 2021 •

edited

Loading