Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

fix(metal): patch ggml-metal.m with shader code #328

Merged
merged 2 commits into from
Jun 23, 2023

Conversation

philpax
Copy link
Collaborator

@philpax philpax commented Jun 22, 2023

Alternate fix to #324 (thanks for the inspiration!)

The Rust build script has the ability to create new files, and use them as fodder for the build process. We can use this to create an alternate ggml-metal.m that embeds the Metal shader source code and sidestep the issue of runtime loading entirely.

Let me know if you're OK with this @pixelspark @nightscape !

@philpax philpax force-pushed the metal-patch-with-shader-code branch from 594b329 to 9d39ff8 Compare June 22, 2023 22:50
@philpax
Copy link
Collaborator Author

philpax commented Jun 22, 2023

Also took the liberty of patching the calls to printf so that they wouldn't spam output in normal release mode. Happy to revert that commit if we think it's too much.

@philpax philpax added topic:backend-support Support for alternate non-GGML backends, or for particular GGML backend features os:macos Related to macOS labels Jun 22, 2023
@nightscape
Copy link
Contributor

Looks great!!

@pixelspark
Copy link
Contributor

Nice trick, maybe a bit brittle (although I suspect the line you replace is not going to change any time soon).

A cleaner way would be to make a trivial PR to llama.cpp that would make the code accept a GGML_METAL_SHADER_CODE #define and use that when set, or else loads from ggml-metal.m? (Assuming the length of #define contents is not limited. Otherwise we could add a #define that chooses between load-at-runtime or a #include).

I propose to merge this now but also initiate steps for a ‘clean’ version.

@philpax philpax merged commit 47a41c9 into main Jun 23, 2023
@philpax philpax deleted the metal-patch-with-shader-code branch July 16, 2023 19:27
@hhamud hhamud mentioned this pull request Aug 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
os:macos Related to macOS topic:backend-support Support for alternate non-GGML backends, or for particular GGML backend features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants