Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 176 additions & 0 deletions docs/apple-silicon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
## Notes: CIEL on Apple Silicon

We need to adapt requirements to our environment, and since Apple Silicon is
different from other Unix-like OSes, also in terms of where libraries
and their header files are located, i.e. installed via `homebrew`, we
need to perform a few additional steps to have CIEL working as it should.

First and foremost, make sure that you are running up to date versions of
Common Lisp implementation (only tested with SBCL), and `asdf`. Also
make sure to have added Ultralisp to Quicklisp dists.

### SBCL (homebrew)

```example
$ brew install sbcl
$ curl -o /tmp/quicklisp.lisp https://beta.quicklisp.org/quicklisp.lisp
$ sbcl --load /tmp/quicklisp.lisp
```

```commonlisp
(quicklisp-quickstart:install)
(sb-ext:exit)
```

```example
$ rm /tmp/quicklisp.lisp
```

### SBCL via Roswell (homebrew)

```example
$ brew update && brew install roswell
$ ros # installs sbcl by default
$ ros run # verify by starting REPL
```

### ASDF
```example
$ mkdir -p ~/common-lisp && cd ~/common-lisp
$ git clone https://gitlab.common-lisp.net/asdf/asdf.git
Comment on lines +37 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to install the latest asdf to sbcl. Because the sbcl which installed through homebrew has the asdf version 3.3.1 (I don't know why sbcl don't update it for years). So the make ql-deps will return the error that the asdf version is too old (it needs at least 3.3.4).

I reinstall asdf to sbcl every time the sbcl updated on homebrew.

cd ~/common-lisp/asdf
make
./tools/asdf-tools install-asdf sbcl

then the sbcl --script check-asdf-version.lisp in ciel folder will return the newest version (3.3.7.1)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I did not encounter that problem. The only thing I did with the homebrew sbcl install is to make sure quicklisp is installed and initialised via .sbclrc, then clone asdf into ~/common-lisp, or whatever other location one has chosen to use. I must be missing something, because I end up with (asdf:asdf-version) ; 3.3.7.1. Am I going about it the wrong way?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reinstalled SBCL through Homebrew, and the ASDF version was reverted to 3.3.1. I had studied the issue before, I believe the problem lies in this code, which includes the contrib/ folder (specifically located at /opt/homebrew/Cellar/sbcl/2.4.8/lib/sbcl/sbclrc). The SBCL version of ASDF has been stuck at version 3.3.1 for years.

I maybe miss something. I would love to sbcl use newest asdf automatically rather than update every time by myself.

Copy link
Author

@dotinspace dotinspace Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see! You make a fair point. However, it seems there's been multiple discussions about SBCL and ASDF, e.g.: https://bugs.launchpad.net/sbcl/+bug/1988892. Also:

You can update your own ASDF at any time, if your .asd depend on something new, insert a version assertion in them.

The goal of SBCL is to provide a CL implementation, and supplying third party software with it is out of such scope. The current ASDF works fine for SBCL's own needs, loading and building contribs, anything else is on the third party. Would've been nice if ASDF behaved differently, but it doesn't, so that's the current state. And I'm not the bigger person to accept that, so there's also that, but why should I?

Not sure what would solve the problem for you, to be honest. Apart from maybe adding an issue/pr to the Homebrew formula. Anyway, we can add a note about this in the doc?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea.. I think I read this issue in sbcl maillist too. I am thinking about give some contributions to homebrew. But I know nothing about ruby 🙃.

```

### Update Quicklisp

Needed if SBCL is installed via Roswell:

```commonlisp
(ql:update-dist "quicklisp")
```

### Ultralisp

```commonlisp
(ql-dist:install-dist "http://dist.ultralisp.org/"
:prompt nil)
(sb-ext:exit)
```

### Packages and Dependencies

#### Install dependencies via homebrew
```example
$ brew update && brew install \
zstd \
gcc \
sdl2 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised. Why sdl2?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I am not sure, I can try without it again, but need some time - incapacitated at the moment. The reason I added it, was I encountered some errors related to SDL, but it might have been dated install via homebrew or me being sort of new to the CL ecosystem, residue from brute force testing. . I will try without it. :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sdl2 is listed in the dependencies for nodgui? https://www.autistici.org/interzona/nodgui.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sdl2 is listed in the dependencies for nodgui? https://www.autistici.org/interzona/nodgui.html

indeed! thanks.

libmagic \
jpeg-turbo
```

#### Set environment variables

Then set up some environment variables before endavouring to build
everything (change llvm version according to what you have installed,
but do note that `c2ffi` requires matching tags, see c2ffi's
[README](https://github.com/rpav/c2ffi)).

```example
export CPPFLAGS=-I/opt/homebrew/opt/llvm@18/include
export LDFLAGS=-L/opt/homebrew/opt/llvm@18/lib
export LLVM_DIR=/opt/homebrew/opt/llvm@18/lib/cmake/llvm
export Clang_DIR=/opt/homebrew/opt/llvm@18
export CC=/opt/homebrew/opt/llvm@18/bin/clang
export CXX=/opt/homebrew/opt/llvm@18/bin/clang++
export CPATH=/opt/homebrew/include
```

#### c2ffi

We start by compiling `c2ffi`, which is not available as a homebrew formula:

```example
$ git clone https://github.com/rpav/c2ffi.git
$ cd c2ffi && mkdir build && cd build
```
Since resulting binary is generated as `./bin/c2ffi`, we need to
make sure is available on `PATH`, so we could do, e.g.:

```example
$ cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local ..
$ make
$ make install
[100%] Built target c2ffi
Install the project...
-- Install configuration: "Release"
-- Installing: /Users/<USERNAME>/.local/bin/c2ffi
```

#### SDL2

Specs are missing for SDL2 on MacOS, thus we need up-to-date packages, and change the
`cl-sdl2` included `autowrap.lisp` source file. Furthermore, we can download an already
generated spec, as shown below:

```example
$ cd ~/quicklisp/local-projects
$ git clone https://github.com/cffi/cffi.git
$ git clone https://github.com/rpav/cl-autowrap.git
$ git clone https://github.com/lispgames/cl-sdl2.git
$ cd cl-sdl2/src
$ curl -O https://raw.githubusercontent.com/ellisvelo/cl-sdl2/main/src/autowrap.lisp
$ cd spec
$ curl -O https://raw.githubusercontent.com/ellisvelo/cl-sdl2/main/src/spec/SDL2.aarch64-apple-darwin9.spec
```
Ref: <https://github.com/lispgames/cl-sdl2/issues/154#issuecomment-1280030566>


#### CIEL

```commonlisp
(ql:quickload :ciel)
```

**NB!** Sometimes there are issues with exhausted heap stack. Try starting
`sbcl` with `sbcl --dynamic-space-size 2024`. Or rerun the loading of CIEL
in REPL.

#### Building CIEL binary and core image (NOT WORKING YET)

**NB!** The build process generates an error based on not being able to
find foreign library on MacOS Sonoma, specifically:

- `/System/Library/Frameworks/GLUT.framework/GLUT`

which, supposedly, is now found here:

- `/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/GLUT.framework`

Note, however, that there is no `GLUT` folder in there.

```example
$ git clone https://github.com/ciel-lang/CIEL ~/quicklisp/local-projects/CIEL
$ cd ~/quicklisp/local-projects/CIEL
$ make build
````

Error snip:

```example
Unhandled FILE-DOES-NOT-EXIST in thread #<SB-THREAD:THREAD tid=259 "main thread" RUNNING
{7004F805E3}>:
The file #P"/System/Library/Frameworks/GLUT.framework/GLUT" does not exist:
No such file or directory

Backtrace for: #<SB-THREAD:THREAD tid=259 "main thread" RUNNING {7004F805E3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<FILE-DOES-NOT-EXIST {7006625A83}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<FILE-DOES-NOT-EXIST {7006625A83}>)
2: (INVOKE-DEBUGGER #<FILE-DOES-NOT-EXIST {7006625A83}>)
3: (ERROR FILE-DOES-NOT-EXIST :PATHNAME #P"/System/Library/Frameworks/GLUT.framework/GLUT" :MESSAGE "No such file or directory")
4: (SB-IMPL::FILE-PERROR #P"/System/Library/Frameworks/GLUT.framework/GLUT" 2 FILE-DOES-NOT-EXIST)
5: (SB-IMPL::%OPEN-ERROR #P"/System/Library/Frameworks/GLUT.framework/GLUT" 2 :IGNORE-THIS-ARG :ERROR)
6: (OPEN #P"/System/Library/Frameworks/GLUT.framework/GLUT" :DIRECTION :INPUT :ELEMENT-TYPE (UNSIGNED-BYTE 8) :IF-EXISTS NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT :CLASS SB-SYS:FD-STREAM)
7: (UIOP/STREAM:CONCATENATE-FILES (#P"/System/Library/Frameworks/GLUT.framework/GLUT") #P"/Users/username/quicklisp/local-projects/CIEL/bin/GLUT")
8: ((FLET DEPLOY::FOREIGN-LIBRARIES :IN "/Users/username/quicklisp/dists/ultralisp/software/Shinmera-deploy-20240730212609/deploy.lisp") :DIRECTORY #P"/Users/username/quicklisp/local-projects/CIEL/bin/")
9: (DEPLOY:RUN-HOOKS :DEPLOY :DIRECTORY #P"/Users/username/quicklisp/local-projects/CIEL/bin/" :SYSTEM #<ASDF/SYSTEM:SYSTEM "ciel/repl"> :OP #<DEPLOY:DEPLOY-OP >)
```