lispPackages_new: fixed a bunch of packages #200573

Uthar · 2022-11-10T18:52:27Z

Description of changes

This should fix another ~50 builds on hydra.

Fixed the lisp packages:

magicl
cl-cairo2
cl-ana
gsll
cl-libyaml
cl-libxml2
cl-readline

magicl, cl-cairo2, cl-ana, gsll, cl-libyaml, cl-libxml2, cl-readline

Uthar · 2022-11-11T00:12:15Z

@vcunat sounds like a good deal? let me know if I should stop bugging you

divanorama · 2022-11-12T18:04:11Z

gsll builds fine for me on x86_64-linux and aarch64-linux so makes sense to also revert #200285

divanorama · 2022-11-12T18:06:59Z

on x86_64-darwin and aarch64-darwin lispPackages_new.sbclPackages.gsll gets an error though (cffi-libffi dependency fails)

; wrote /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-tmpGHU3ALSV.fasl
; compilation finished in 0:00:00.001
; pkg-config libffi --cflags
; ERROR: Couldn't execute "pkg-config": No such file or directory


; Attempting to continue anyway.
; clang -o /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel-tmpAAURSO1.o -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -g -Wall -fdollars-in-identifiers -mmacosx-version-min=10.6 -D_DARWIN_USE_64_BIT_INODE -arch x86_64 -fno-omit-frame-pointer -I/opt/local/include/ -fPIC -I/nix/store/lacv9ayg6kvyghrh7kh3vm4rgys2nhzc-cffi-grovel-0.24.1/ /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel.c
/private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel.c:6:10: fatal error: 'ffi/ffi.h' file not found
#include <ffi/ffi.h>
         ^~~~~~~~~~~

divanorama · 2022-11-12T18:29:21Z

with following patch

diff --git a/pkgs/development/lisp-modules-new/ql.nix b/pkgs/development/lisp-modules-new/ql.nix
index 8e281f6b2c5..36490990e55 100644
--- a/pkgs/development/lisp-modules-new/ql.nix
+++ b/pkgs/development/lisp-modules-new/ql.nix
@@ -53,7 +53,7 @@ let
       nativeLibs = [ openblas ];
     };
     cffi-libffi = pkg: {
-      nativeBuildInputs = [ libffi ];
+      nativeBuildInputs = [ libffi pkg-config ];
       nativeLibs = [ libffi ];
     };
     cl-rabbit = pkg: {

it gets a little further on darwin, but still fails

building
; compiling file "/nix/store/ghz8dmxx4hahlzzq5kpxd18alm1mzigh-source/libffi/libffi.lisp" (written 01 JAN 1970 12:00:01 AM):

; wrote /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-tmpGHU3ALSV.fasl
; compilation finished in 0:00:00.001
; pkg-config libffi --cflags
-I/nix/store/hnpbq34ak4r9xa4zaxn8pmzdsmbs2xkf-libffi-3.4.4-dev/include
; clang -o /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel-tmpAAURSO1.o -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -g -Wall -fdollars-in-identifiers -mmacosx-version-min=10.6 -D_DARWIN_USE_64_BIT_INODE -arch x86_64 -fno-omit-frame-pointer -I/opt/local/include/ -I/nix/store/hnpbq34ak4r9xa4zaxn8pmzdsmbs2xkf-libffi-3.4.4-dev/include -fPIC -I/nix/store/lacv9ayg6kvyghrh7kh3vm4rgys2nhzc-cffi-grovel-0.24.1/ /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel.c
/private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source/libffi/libffi-types__grovel.c:6:10: fatal error: 'ffi/ffi.h' file not found
#include <ffi/ffi.h>
         ^~~~~~~~~~~
1 error generated.

ffi.h exists directly under /nix/store/hnpbq34ak4r9xa4zaxn8pmzdsmbs2xkf-libffi-3.4.4-dev/include so not as ffi/ffi.h

Uthar · 2022-11-12T21:29:49Z

I'll check it out, thanks for testing.

Uthar · 2022-11-12T23:23:52Z

I think we should patch cffi-libffi to make this work:

--- /home/kasper/repos/github/nix-cl/result/libffi/libffi-types.lisp
+++ #<buffer libffi-types.lisp>
@@ -43,9 +43,6 @@
 
 (pkg-config-cflags "libffi" :optional t)
 
-#+darwin
-(include "ffi/ffi.h")
-#-darwin
 (include "ffi.h")
 
 (cenum status

Diff finished.  Sun Nov 13 00:23:10 2022

Uthar · 2022-11-12T23:24:21Z

Do you want to test @divanorama ?

Add patch for cffi-libffi on Darwin

divanorama · 2022-11-13T12:56:31Z

building
; compiling file "/nix/store/ghz8dmxx4hahlzzq5kpxd18alm1mzigh-source/libffi/libffi.lisp" (written 01 JAN 1970 12:00:01 AM):

; wrote /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source-patched/libffi/libffi-tmpGHU3ALSV.fasl
; compilation finished in 0:00:00.001
; pkg-config libffi --cflags
; ERROR: Couldn't execute "pkg-config": No such file or directory


; Attempting to continue anyway.
; clang -o /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source-patched/libffi/libffi-types__grovel-tmpAAURSO1.o -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -g -Wall -fdollars-in-identifiers -fno-omit-frame-pointer -I/opt/local/include/ -fPIC -I/nix/store/mfb0x7w99xyxq82h7zp3i1y8r5258bbq-cffi-grovel-0.24.1/ /private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source-patched/libffi/libffi-types__grovel.c
/private/tmp/nix-build-cffi-libffi-0.24.1.drv-0/source-patched/libffi/libffi-types__grovel.c:6:10: fatal error: 'ffi/ffi.h' file not found
#include <ffi/ffi.h>
         ^~~~~~~~~~~
1 error generated.

maybe that's not the way to patch it 🤔

divanorama · 2022-11-13T13:02:08Z

@ofborg build lispPackages_new.sbclPackages.gsll

divanorama · 2022-11-13T13:04:34Z

patch seems to be applied

$ nix show-derivation  /nix/store/5b8rhw698my30k9v1sa93nvii2yim4nb-cffi-libffi-0.24.1.drv | grep '"src"'
      "src": "/nix/store/p1b6ijcwkw5swifzy1agd5zcv0ydrsck-source-patched",
$ grep ffi.h /nix/store/p1b6ijcwkw5swifzy1agd5zcv0ydrsck-source-patched/libffi/libffi-types.lisp 
(include "ffi.h")

but maybe some other version of libffi-types gets pulled in too

divanorama · 2022-11-13T13:24:32Z

hmm, inside the build

$ grep ffi.h libffi/libffi-types.lisp 
(include "ffi.h")
$ grep -B 5 ffi.h libffi/libffi-types__grovel.c 
/*
 * This file has been automatically generated by cffi-grovel.
 * Do not edit it by hand.
 */

#include <ffi/ffi.h>

so probably cffi-grovel needs to be patched too

Uthar · 2022-11-13T13:50:24Z

very weird

Uthar · 2022-11-13T14:05:41Z

unfortunately fixing cffi-libffi like here will probably cause much more rebuilds on hydra

divanorama · 2022-11-13T14:12:25Z

I think it comes from /nix/store/kfzj91blvyiy966z7p1ghs633fyjnyl0-cffi-0.24.1//libffi/libffi-types.lisp
in build script (nix-shell --pure -A lispPackages_new.sbclPackages.cffi-libffi on linux too):

echo $CL_SOURCE_REGISTRY | tr : "\n"    
/nix/store/lvijpsh5ism8gfkvixp4wibh4hcwwwiz-alexandria-20220707-git//
/nix/store/22bmr1lsrqxfxz0whyxwn5419206sl4m-babel-20200925-git//
/nix/store/kfzj91blvyiy966z7p1ghs633fyjnyl0-cffi-0.24.1//
/nix/store/k0ypdlfyzx0ab1a4sin3ry19r2sac06k-cffi-grovel-0.24.1//
/nix/store/ykhys5nhrzr0ng8xy466brawdi2lkjns-cffi-toolchain-0.24.1//
/nix/store/sxxn64hsf0dnfikp7zjzgfil68gqkipf-trivial-features-20211209-git//
/nix/store/ghz8dmxx4hahlzzq5kpxd18alm1mzigh-source//

and

grep ffi.h /nix/store/kfzj91blvyiy966z7p1ghs633fyjnyl0-cffi-0.24.1//libffi/*
/nix/store/kfzj91blvyiy966z7p1ghs633fyjnyl0-cffi-0.24.1//libffi/libffi-types.lisp:(include "ffi/ffi.h")
/nix/store/kfzj91blvyiy966z7p1ghs633fyjnyl0-cffi-0.24.1//libffi/libffi-types.lisp:(include "ffi.h")

Uthar · 2022-11-13T14:14:31Z

Can you try with this commit? f9e9336
The issue maybe was in the $buildPhase setting the wrong CL_SOURCE_REGISTRY

divanorama · 2022-11-13T14:21:35Z

Can you try with this commit? f9e9336
The issue maybe was in the $buildPhase setting the wrong CL_SOURCE_REGISTRY

Nice! Works for me on linux&darwin aarch64&x86_64 for gsll package

Uthar · 2022-11-13T14:22:48Z

yeah, borg added 5000+ rebuilds label )-:

Uthar · 2022-11-13T15:06:37Z

I don't think it's needed to merge it right now, unless you want the fixes of course, because I'm thinking to do some major changes anyway which will cause another round of rebuilds. See the docs for what I mean rougly to do https://github.com/Uthar/nix-cl/blob/master/doc/api.md

vcunat · 2022-11-13T15:51:54Z

OK, sounds like definite improvement overall, though lispPackages_new.sbclPackages.magicl still won't build for me on x86_64-linux.

Uthar · 2022-11-13T16:01:02Z

Thanks for checking. Yes, it crashes on sbcl due to too small default heap. The api breaking changes I mentioned will remedy this by allowing to set heap flags. In the meantime you could use ccl, which does not have this problem

Currently broken, should be fixed a bit later see last comments in NixOS#200573 ZHF: NixOS#199919

Uthar changed the title ~~fixed a bunch of packages~~ lispPackages_new: fixed a bunch of packages Nov 10, 2022

Uthar force-pushed the lisp-packages-multiple-fixes branch from 58b6875 to 2327eb1 Compare November 10, 2022 18:53

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Nov 10, 2022

lispPackages_new: fixed a bunch of packages

ae9cb7b

magicl, cl-cairo2, cl-ana, gsll, cl-libyaml, cl-libxml2, cl-readline

Uthar force-pushed the lisp-packages-multiple-fixes branch from 2327eb1 to ae9cb7b Compare November 10, 2022 23:44

ofborg bot added 10.rebuild-linux: 101-500 and removed 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Nov 10, 2022

vcunat mentioned this pull request Nov 12, 2022

sbclPackages.gsll: mark as broken #200285

Merged

13 tasks

lispPackages_new: Fix patching without build-with-compile-into-pwd

2f103f2

Add patch for cffi-libffi on Darwin

lispPackages_new: fix for patched sources not being picked up

f9e9336

ofborg bot added 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Nov 13, 2022

ofborg bot removed the 10.rebuild-linux: 101-500 label Nov 13, 2022

vcunat changed the base branch from master to staging-next November 13, 2022 15:50

vcunat merged commit f3da70c into NixOS:staging-next Nov 13, 2022

ofborg bot added 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux and removed 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Nov 13, 2022

divanorama mentioned this pull request Nov 13, 2022

sbclPackages.magicl: mark as broken #201057

Merged

13 tasks

divanorama added a commit to divanorama/nixpkgs that referenced this pull request Nov 13, 2022

sbclPackages.magicl: mark as broken

4742b0b

Currently broken, should be fixed a bit later see last comments in NixOS#200573 ZHF: NixOS#199919

Uthar mentioned this pull request Nov 14, 2022

sbclPackages: mark some failing packages as broken #201101

Merged

rtimush pushed a commit to rtimush/nixpkgs that referenced this pull request Sep 21, 2023

sbclPackages.magicl: mark as broken

d782847

Currently broken, should be fixed a bit later see last comments in NixOS#200573 ZHF: NixOS#199919

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lispPackages_new: fixed a bunch of packages #200573

lispPackages_new: fixed a bunch of packages #200573

Uthar commented Nov 10, 2022 •

edited

Loading

Uthar commented Nov 11, 2022

divanorama commented Nov 12, 2022 •

edited

Loading

divanorama commented Nov 12, 2022 •

edited

Loading

divanorama commented Nov 12, 2022

Uthar commented Nov 12, 2022

Uthar commented Nov 12, 2022

Uthar commented Nov 12, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

Uthar commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

Uthar commented Nov 13, 2022

vcunat commented Nov 13, 2022

Uthar commented Nov 13, 2022

lispPackages_new: fixed a bunch of packages #200573

lispPackages_new: fixed a bunch of packages #200573

Conversation

Uthar commented Nov 10, 2022 • edited Loading

Description of changes

Uthar commented Nov 11, 2022

divanorama commented Nov 12, 2022 • edited Loading

divanorama commented Nov 12, 2022 • edited Loading

divanorama commented Nov 12, 2022

Uthar commented Nov 12, 2022

Uthar commented Nov 12, 2022

Uthar commented Nov 12, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

Uthar commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

divanorama commented Nov 13, 2022

Uthar commented Nov 13, 2022

Uthar commented Nov 13, 2022

vcunat commented Nov 13, 2022

Uthar commented Nov 13, 2022

Uthar commented Nov 10, 2022 •

edited

Loading

divanorama commented Nov 12, 2022 •

edited

Loading

divanorama commented Nov 12, 2022 •

edited

Loading