Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'configure' fails on ZFS 0.8.3 with kernel builtin option #9887

Closed
morian opened this issue Jan 25, 2020 · 7 comments
Closed

'configure' fails on ZFS 0.8.3 with kernel builtin option #9887

morian opened this issue Jan 25, 2020 · 7 comments
Labels
Type: Building Indicates an issue related to building binaries

Comments

@morian
Copy link

morian commented Jan 25, 2020

System information

Type Version/Name
Distribution Name Gentoo
Linux Kernel 5.2.21
Architecture amd64
ZFS Version 0.8.3

Describe the problem you're observing

Configure step ends with the following error:

checking kernel source version... 5.2.21
checking kernel file name for module symbols... NONE
checking whether modules can be built... no
configure: error: 
	*** Unable to build an empty module.
	*** Please run 'make scripts' inside the kernel source tree.

Describe how to reproduce the problem

It used to work very well on ZFS 0.8.2 with the following steps:

  • clone a linux-stable.git repository, move on a release tag (here v5.2.21)
  • In the working directory perform the following commands:
make olddefconfig
make prepare
make scripts
  • clone zfs on 'zfs-0.8-release' branch
  • Perform the following commands:
bash autogen.sh
./configure --with-linux="${LINUX_DIR}" --enable-linux-builtin

I have used a custom script for years now to create my own monolithic Linux kernel with builtin ZFS support, but found out that it is now broken.

Include any warning/errors/backtraces from the system logs

This is the interesting part of config.log:

configure:51628: checking kernel source version
configure:51680: result: 5.2.21
configure:51693: checking kernel file name for module symbols
configure:51723: result: NONE
configure:51824: checking whether modules can be built
configure:51883: 
      KBUILD_MODPOST_NOFINAL= KBUILD_MODPOST_WARN=
      make modules -k -j8 -C /home/redacted/repos/linux-stable.git/ 
      M=/home/redacted/repos/zfs.git/build/conftest &>build/conftest/build.log
configure:51886: $? = 2
configure:51889: test -f build/conftest/conftest.ko
configure:51892: $? = 1
configure:51900: result: no
configure:51907: error: 
  *** Unable to build an empty module.
  *** Please run 'make scripts' inside the kernel source tree.

Here is the content of build.log:

make: Entering directory '/home/redacted/repos/linux-stable.git'

  WARNING: Symbol version dump ./Module.symvers
           is missing; modules will have no dependencies and modversions.

  CC [M]  /home/redacted/repos/zfs.git/build/conftest/conftest.o
  Building modules, stage 2.
  MODPOST 1 modules
WARNING: modpost: missing MODULE_LICENSE() in /home/redacted/repos/zfs.git/build/conftest/conftest.o
see include/linux/module.h for more information
  CC      /home/redacted/repos/zfs.git/build/conftest/conftest.mod.o
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:12:1: error: variable ‘__this_module’ has initializer but incomplete type
 __attribute__((section(".gnu.linkonce.this_module"))) = {
 ^~~~~~~~~~~~~
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:13:3: error: ‘struct module’ has no member named ‘name’
  .name = KBUILD_MODNAME,
   ^~~~
<command-line>: error: excess elements in struct initializer [-Werror]
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:13:10: note: in expansion of macro ‘KBUILD_MODNAME’
  .name = KBUILD_MODNAME,
          ^~~~~~~~~~~~~~
<command-line>: note: (near initialization for ‘__this_module’)
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:13:10: note: in expansion of macro ‘KBUILD_MODNAME’
  .name = KBUILD_MODNAME,
          ^~~~~~~~~~~~~~
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:14:3: error: ‘struct module’ has no member named ‘arch’
  .arch = MODULE_ARCH_INIT,
   ^~~~
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:14:10: error: ‘MODULE_ARCH_INIT’ undeclared here (not in a function); did you mean ‘MODULE_LICENSE’?
  .arch = MODULE_ARCH_INIT,
          ^~~~~~~~~~~~~~~~
          MODULE_LICENSE
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:14:10: error: excess elements in struct initializer [-Werror]
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:14:10: note: (near initialization for ‘__this_module’)
/home/redacted/repos/zfs.git/build/conftest/conftest.mod.c:11:25: error: storage size of ‘__this_module’ isn’t known
 __visible struct module __this_module
                         ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [scripts/Makefile.modpost:113: /home/redacted/repos/zfs.git/build/conftest/conftest.mod.o] Error 1
make[1]: Target '_modpost' not remade because of errors.
make: *** [Makefile:1601: modules] Error 2
make: Leaving directory '/home/redacted/repos/linux-stable.git'

Identified cause of the issue (maybe?):

On 0.8.3, in config/kernel.m4:

  • ZFS_AC_KERNEL_TEST_MODULE uses ZFS_LINUX_TRY_COMPILE
  • ZFS_LINUX_TRY_COMPILE uses ZFS_LINUX_TEST_PROGRAM which is not a valid kernel module.
  • ZFS_LINUX_TRY_COMPILE checks for generated conftest.ko file (only .o file exists)

On 0.8.2:

  • ZFS_AC_TEST_MODULE uses ZFS_LINUX_TRY_COMPILE as well
  • ZFS_LINUX_TRY_COMPILE uses AC_LANG_SOURCE([ZFS_LANG_PROGRAM([[$1]], [[$2]])]) which is not a valid kernel module.
  • ZFS_LINUX_TRY_COMPILE checks for generated build/conftest.o file (and it works...)
@behlendorf behlendorf added the Type: Building Indicates an issue related to building binaries label Jan 27, 2020
@behlendorf
Copy link
Contributor

The 0.8.3 tag did include changes to significantly speed up the ./configure checks. However, we didn't encounter any issues when testing them with the built-in kernel configs. Here's a link to the exact steps which were run.

Are you able to consistently reproduce this when starting with a clean source tree? Would it be possible for you to run a test build with the master source to see if you have the same issue there?

@gcs-github
Copy link

gcs-github commented Jan 31, 2020

I'm reproducing with a vanilla, non-patched 4.14.169 kernel and a vanilla, non-patched zfs 0.8.3 tarball.

On my end, the issue seems to be related to CONFIG_MODULES being enabled or not in the kernel's config. With ZFS 0.8.2, you could run ./configure --enable-linux-builtin successfully even when you had CONFIG_MODULES=n in the kernel config. ZFS 0.8.3 broke that.

@gcs-github
Copy link

gcs-github commented Feb 1, 2020

Looking into @morian's initial diagnosis: Interestingly, there is code in kernel.m4 to artificially generate a corresponding .ko file when --enable-linux-builtin is used, so that tests would pass. Somehow, it doesn't seem to be used/working in this particular instance.

@gcs-github
Copy link

Replacing test -f build/conftest/conftest.ko with test -f build/conftest/conftest.o in ZFS_LINUX_TRY_COMPILE seems to work around the problem. I've tried replacing the test with an AS_IF conditional on test "x$enable_linux_builtin" != "xyes" in order to test for the presence of a .ko or a .o selectively, but without success: the configure script ends up failing with an error about about a premature EOF if I introduce a conditional structure there. I'm not sure how to properly fix this.

@morian
Copy link
Author

morian commented Feb 2, 2020

Sorry for the response delay!
Indeed I forgot to set it clear in my report that I have no support for modules in the target kernel .config file (that step is supposed to be right before make olddefconfig).
Replacing test -f build/conftest/conftest.ko with test -f build/conftest/conftest.o did work and it seems like that is what was performed in 0.8.2.
Thanks for the findings @gcs-github, I share your doubts on it being a proper fix...

@tonyhutter
Copy link
Contributor

@morian thanks for the detailed instructions. I was able to reproduce the error on a Fedora 31 VM with both the zfs-0.8-release and master branches. This seems to fix it:

diff --git a/config/kernel.m4 b/config/kernel.m4
index ec8f6a5c1..401389e8a 100644
--- a/config/kernel.m4
+++ b/config/kernel.m4
@@ -461,15 +461,13 @@ AC_DEFUN([ZFS_AC_KERNEL_TEST_MODULE], [
        ZFS_LINUX_TRY_COMPILE([], [], [
                AC_MSG_RESULT([yes])
        ],[
-               AC_MSG_RESULT([no])
                if test "x$enable_linux_builtin" != xyes; then
+                       AC_MSG_RESULT([no])
                        AC_MSG_ERROR([
        *** Unable to build an empty module.
                        ])
                else
-                       AC_MSG_ERROR([
-       *** Unable to build an empty module.
-       *** Please run 'make scripts' inside the kernel source tree.])
+                       AC_MSG_RESULT([unneeded, using --enable-linux-builtin])
                fi
        ])
 ])
checking kernel file name for module symbols... NONE
checking whether modules can be built... unneeded, using --enable-linux-builtin
checking for kernel config option compatibility... done
checking whether kernel was built with 16K or larger stacks... yes

I'm going to put together a PR.

tonyhutter added a commit to tonyhutter/zfs that referenced this issue Feb 4, 2020
This allows --enable-linux-builtin to work when CONFIG_MODULES is
not set in the kernel.

Fixes: openzfs#9887

Signed-off-by: Tony Hutter <[email protected]>
@tonyhutter
Copy link
Contributor

Proposed fix: #9940

behlendorf added a commit to behlendorf/zfs that referenced this issue Feb 26, 2020
When configuring as builtin (--enable-linux-builtin) for kernels
without loadable module support (CONFIG_MODULES=n) only the object
file is created.  Never a loadable kmod.

Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to
the ZFS_LINUX_TEST_COMPILE_ALL macro.

Signed-off-by: Brian Behlendorf <[email protected]>
Issue openzfs#9887
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Apr 22, 2020
When configuring as builtin (--enable-linux-builtin) for kernels
without loadable module support (CONFIG_MODULES=n) only the object
file is created.  Never a loadable kmod.

Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to
the ZFS_LINUX_TEST_COMPILE_ALL macro.

Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes openzfs#9887
Closes openzfs#10063
tonyhutter pushed a commit to tonyhutter/zfs that referenced this issue Apr 22, 2020
When configuring as builtin (--enable-linux-builtin) for kernels
without loadable module support (CONFIG_MODULES=n) only the object
file is created.  Never a loadable kmod.

Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to
the ZFS_LINUX_TEST_COMPILE_ALL macro.

Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes openzfs#9887
Closes openzfs#10063
tonyhutter pushed a commit that referenced this issue May 12, 2020
When configuring as builtin (--enable-linux-builtin) for kernels
without loadable module support (CONFIG_MODULES=n) only the object
file is created.  Never a loadable kmod.

Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to
the ZFS_LINUX_TEST_COMPILE_ALL macro.

Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes #9887
Closes #10063
jsai20 pushed a commit to jsai20/zfs that referenced this issue Mar 30, 2021
When configuring as builtin (--enable-linux-builtin) for kernels
without loadable module support (CONFIG_MODULES=n) only the object
file is created.  Never a loadable kmod.

Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to
the ZFS_LINUX_TEST_COMPILE_ALL macro.

Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes openzfs#9887
Closes openzfs#10063
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Building Indicates an issue related to building binaries
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants