Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compilation failure #21

Closed
planetA opened this issue Nov 15, 2021 · 12 comments · Fixed by #41
Closed

Compilation failure #21

planetA opened this issue Nov 15, 2021 · 12 comments · Fixed by #41

Comments

@planetA
Copy link
Contributor

planetA commented Nov 15, 2021

Hello,

when I try to compile rust-ibverbs, I get an error with the following error message during compilation of the rdma-core package.

Scanning dependencies of target sminfo
Scanning dependencies of target saquery
Scanning dependencies of target ibtracert
Scanning dependencies of target ibcacheedit
Scanning dependencies of target testleaks
In file included from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/minmax.h:7,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/cma.h:48,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c:43:
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c: In function 'ucma_ib_init':
Scanning dependencies of target perfquery
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/build_assert.h:23:26: error: size of unnamed array is negative
   23 |  do { (void) sizeof(char [1 - 2*!(cond)]); } while(0)
      |                          ^
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c:105:3: note: in expansion of macro 'BUILD_ASSERT'
  105 |   BUILD_ASSERT(sizeof(IBACM_SERVER_PATH) <=
      |   ^~~~~~~~~~~~
Scanning dependencies of target ibsysstat
Scanning dependencies of target ibccquery
Scanning dependencies of target ibqueryerrors
Scanning dependencies of target dump_fts
Scanning dependencies of target smpquery
Scanning dependencies of target ibportstate
Scanning dependencies of target ibstat
Scanning dependencies of target smpdump
Scanning dependencies of target ibnetdiscover
Scanning dependencies of target ibping
Scanning dependencies of target ibccconfig
Scanning dependencies of target ibroute
Scanning dependencies of target iblinkinfo
Scanning dependencies of target ibaddr
In file included from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/minmax.h:7,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/linux/osd.h:48,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/acm.c:38:
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/acm.c: In function 'acm_listen':
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/build_assert.h:23:26: error: size of unnamed array is negative
   23 |  do { (void) sizeof(char [1 - 2*!(cond)]); } while(0)
      |                          ^
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/acm.c:629:3: note: in expansion of macro 'BUILD_ASSERT'
  629 |   BUILD_ASSERT(sizeof(IBACM_IBACME_SERVER_PATH) <=
      |   ^~~~~~~~~~~~
In file included from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/minmax.h:7,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/linux/osd.h:48,
                 from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/libacm.c:33:
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/libacm.c: In function 'ib_acm_connect_unix':
Scanning dependencies of target vendstat
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/build_assert.h:23:26: error: size of unnamed array is negative
   23 |  do { (void) sizeof(char [1 - 2*!(cond)]); } while(0)
      |                          ^
/home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/ibacm/src/libacm.c:107:3: note: in expansion of macro 'BUILD_ASSERT'
  107 |   BUILD_ASSERT(sizeof(IBACM_IBACME_SERVER_PATH) <=
      |   ^~~~~~~~~~~~
make[2]: *** [librdmacm/CMakeFiles/rdmacm.dir/build.make:82: librdmacm/CMakeFiles/rdmacm.dir/acm.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2512: librdmacm/CMakeFiles/rdmacm.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[2]: *** [ibacm/CMakeFiles/ib_acme.dir/build.make:95: ibacm/CMakeFiles/ib_acme.dir/src/libacm.c.o] Error 1

The issue happens it this line:

  BUILD_ASSERT(sizeof(IBACM_SERVER_PATH) <=
                             sizeof(addr.unx.sun_path));

inside ~/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c

The reason is that IBACM_SERVER_PATH generated during configure step is too long (>108 chars). And it is too long, because it is
<build directory>/var/run/ibacm.sock.

@planetA
Copy link
Contributor Author

planetA commented Nov 15, 2021

I am not sure that I pinpointed the actual problem correctly here is the error from cargo build:

  [ 26%] Built target ccan
  [ 28%] Built target rdma_util_pic
  [ 29%] Built target rdma_util
  [ 29%] Built target rdma_rename
  [ 33%] Built target kern-abi
  [ 34%] Built target ibumad
  [ 42%] Built target ibverbs
  [ 42%] Building C object librdmacm/CMakeFiles/rdmacm.dir/acm.c.o

  --- stderr
  In file included from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/minmax.h:7,
                   from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/cma.h:48,
                   from /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c:43:
  /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c: In function 'ucma_ib_init':
  /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/build/include/ccan/build_assert.h:23:26: error: size of unnamed array is negative
     23 |  do { (void) sizeof(char [1 - 2*!(cond)]); } while(0)
        |                          ^
  /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/vendor/rdma-core/librdmacm/acm.c:105:3: note: in expansion of macro 'BUILD_ASSERT'
    105 |   BUILD_ASSERT(sizeof(IBACM_SERVER_PATH) <=
        |   ^~~~~~~~~~~~
  make[2]: *** [librdmacm/CMakeFiles/rdmacm.dir/build.make:82: librdmacm/CMakeFiles/rdmacm.dir/acm.c.o] Error 1
  make[1]: *** [CMakeFiles/Makefile2:2512: librdmacm/CMakeFiles/rdmacm.dir/all] Error 2
  make: *** [Makefile:149: all] Error 2
  /usr/include/sched.h:29:10: fatal error: 'stddef.h' file not found
  /usr/include/sched.h:29:10: fatal error: 'stddef.h' file not found, err: true
  thread 'main' panicked at 'Unable to generate bindings: ()', /home/mplaneta/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/ibverbs-sys-0.1.0/build.rs:57:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

It seems that the build fails, because stddef.h is not found.

I have a successful build at another system, but when I try to build rdma-core downloaded by ibverbs-sys manually, I see the same error.

@jonhoo
Copy link
Owner

jonhoo commented Nov 20, 2021

Hmm, this seems more like an issue with the host system missing a -dev package with C header files somewhere than a bug in this crate's build logic 🤔

@zhucan
Copy link

zhucan commented Jan 17, 2022

[root@k8s-1 rust-ibverbs]# cargo build
Compiling ibverbs-sys v0.1.0 (/home/deeproute/workspace/rust-ibverbs/ibverbs-sys)
error: failed to run custom build command for ibverbs-sys v0.1.0 (/home/deeproute/workspace/rust-ibverbs/ibverbs-sys)

Caused by:
process didn't exit successfully: /home/deeproute/workspace/rust-ibverbs/target/debug/build/ibverbs-sys-5284e52423cd7f69/build-script-build (exit status: 101)
--- stdout
cargo:include=vendor/rdma-core/build/include
cargo:rustc-link-search=native=vendor/rdma-core/build/lib
cargo:rustc-link-lib=ibverbs
-- Could NOT find cython (missing: CYTHON_EXECUTABLE CYTHON_VERSION_STRING)
-- Could NOT find pandoc (missing: PANDOC_EXECUTABLE PANDOC_VERSION_STRING)
-- Could NOT find rst2man (missing: RST2MAN_EXECUTABLE RST2MAN_VERSION_STRING)
-- Checking for modules 'libnl-3.0;libnl-route-3.0'
-- No package 'libnl-3.0' found
-- No package 'libnl-route-3.0' found
-- Configuring incomplete, errors occurred!
See also "/home/deeproute/workspace/rust-ibverbs/ibverbs-sys/vendor/rdma-core/build/CMakeFiles/CMakeOutput.log".
See also "/home/deeproute/workspace/rust-ibverbs/ibverbs-sys/vendor/rdma-core/build/CMakeFiles/CMakeError.log".

--- stderr
CMake Deprecation Warning at CMakeLists.txt:56 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.

CMake Error at /usr/local/share/cmake-3.22/Modules/FindPkgConfig.cmake:603 (message):
A required package was not found
Call Stack (most recent call first):
/usr/local/share/cmake-3.22/Modules/FindPkgConfig.cmake:825 (_pkg_check_modules_internal)
CMakeLists.txt:468 (pkg_check_modules)

vendor/rdma-core/libibverbs/verbs.h:600:32: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_FIRST'
vendor/rdma-core/libibverbs/verbs.h:2506:35: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_RANGE'
vendor/rdma-core/libibverbs/verbs.h:2529:35: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_RANGE'
vendor/rdma-core/libibverbs/verbs.h:600:32: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_FIRST', err: true
vendor/rdma-core/libibverbs/verbs.h:2506:35: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_RANGE', err: true
vendor/rdma-core/libibverbs/verbs.h:2529:35: error: use of undeclared identifier 'IBV_ACCESS_OPTIONAL_RANGE', err: true
thread 'main' panicked at 'Unable to generate bindings: ()', ibverbs-sys/build.rs:57:10
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

@jonhoo Please help me take a look at this error.

@jonhoo
Copy link
Owner

jonhoo commented Jan 17, 2022

The error output indicates the problem:

-- Could NOT find cython (missing: CYTHON_EXECUTABLE CYTHON_VERSION_STRING)
-- Could NOT find pandoc (missing: PANDOC_EXECUTABLE PANDOC_VERSION_STRING)
-- Could NOT find rst2man (missing: RST2MAN_EXECUTABLE RST2MAN_VERSION_STRING)
-- Checking for modules 'libnl-3.0;libnl-route-3.0'
-- No package 'libnl-3.0' found
-- No package 'libnl-route-3.0' found
-- Configuring incomplete, errors occurred!

I'm going to go ahead and close this issue, since none of these errors appear related to this crate.

@jonhoo jonhoo closed this as completed Jan 17, 2022
@zhucan
Copy link

zhucan commented Jan 18, 2022

yes, and the version of the clang must >= 3.9, is it needed to update the readme? @jonhoo

@zhucan
Copy link

zhucan commented Jan 18, 2022

I have other question, how to run the example?

@jonhoo
Copy link
Owner

jonhoo commented Jan 18, 2022

I don't think clang >= 3.9 is needed?

You run the example the same way other Rust examples are run cargo run --example loopback :)

@zhucan
Copy link

zhucan commented Jan 18, 2022

exec "cargo run --example loopback" with error like this.
image

@microyahoo
Copy link

Are these libraries currently used in production? @jonhoo

@jonhoo
Copy link
Owner

jonhoo commented Jan 22, 2022

@zhucan This library links against libibverbs, so yes, you need to have that installed.

@microyahoo I don't know of any production users, but could be.

@jonhoo
Copy link
Owner

jonhoo commented Dec 27, 2024

I came across this again recently, and I think this actually happens if you try to build RDMA too "deeply" in your file system. That is, literally if your path is too many characters. You hit this assert, which is there to guard against overflowing the struct that holds UNIX socket paths. The solution, as far as I can tell, is to build at a shorter path 😅

jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness[2][3].

Specifically, rdma-core has an assert[2] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[3]. The assert uses `IBACM_SERVER_PATH`, which is set[4] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[5] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: #21 (comment).
[3]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[4]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[5]: https://man7.org/linux/man-pages/man7/unix.7.html). It's not
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[7]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness[2][3].

Specifically, rdma-core has an assert[2] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[3]. The assert uses `IBACM_SERVER_PATH`, which is set[4] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[5] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: #21 (comment)
[3]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[4]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[5]: https://man7.org/linux/man-pages/man7/unix.7.html
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[7]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness[2][3].

Specifically, rdma-core has an assert[4] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[5]. The assert uses `IBACM_SERVER_PATH`, which is set[6] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[7] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: #21 (comment)
[3]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[4]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[5]: https://man7.org/linux/man-pages/man7/unix.7.html
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[7]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness ([2], [3]).

Specifically, rdma-core has an assert[4] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[5]. The assert uses `IBACM_SERVER_PATH`, which is set[6] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[7] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: #21 (comment)
[3]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[4]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[5]: https://man7.org/linux/man-pages/man7/unix.7.html
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[7]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
@jonhoo
Copy link
Owner

jonhoo commented Dec 27, 2024

Finally fixing this now that I know what's causing it: #41

jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness (#21, [2]).

Specifically, rdma-core has an assert[3] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[4]. The assert uses `IBACM_SERVER_PATH`, which is set[5] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[6] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

Fixes #21 and (hopefully) docs.rs builds.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[3]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[4]: https://man7.org/linux/man-pages/man7/unix.7.html
[5]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
jonhoo added a commit that referenced this issue Dec 27, 2024
When using `IN_PLACE=1`, the build assumes that the build directory will
_also_ be the run directory[1], which doesn't hold for our case.
Furthermore, it's actively problematic as it means we'll end up fairly
long paths, which can cause sadness (#21, [2]).

Specifically, rdma-core has an assert[3] that triggers if the path to
one of the build dependencies is longer than what's allowed for UNIX
sockets[4]. The assert uses `IBACM_SERVER_PATH`, which is set[5] to
`@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[6] to

    ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR}

(when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the
case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the
project directory of rdma-core when `IN_PLACE=1`, which may be quite
deep, and thus the whole thing is likely to exceed the 108 bytes allowed
for `sun_path`.

It's worth noting that since we only build in order to get the various
header files so we can then generate bindings from them, the setting of
the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_
have anything to say for us.

Fixes #21 and (hopefully) docs.rs builds.

[1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100
[2]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723
[3]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106
[4]: https://man7.org/linux/man-pages/man7/unix.7.html
[5]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35
[6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants