`build`: generate hex dump of server assets during build #6661

ochafik · 2024-04-13T15:39:43Z

Currently one needs to manually regenerate asset files in examples/server, which:

is easy to forget
~~may cause a security risk~~ (reviewers are unlikely to unescape the code - don't wanna be the next target after xz) (edit: already mitigated by ci: server: verify deps are coherent with the commit #6409 as pointed out below)

od is used in Makefile (unlike xxd, it should be available on ~all systems, incl. on Windows w/ w64devkit - thanks to busybox), while cmake and zig builds cast their own hex for portability.

phymbert · 2024-04-13T15:47:39Z

It has been addressed the xz weekend here:

ci: server: verify deps are coherent with the commit #6409

But I am happy that finally the code is removed, so looks good

ngxson · 2024-04-13T16:03:40Z

Please pay attention that the idea may look simple on the surface, but it's get messy when taking into account that we're not compiling for single target.

Some linux distro (docker/CI) may not have xxd by default. You may need to add something like apt install xxd to every dockerfile and compile scripts in the project. This will also create a new dependency in compilation phase. In fact, a while ago, I success in making a script that does the same thing with xxd but using od instead. The reason was because od is included by default in most linux distros. See this article for more details.
Things get messy when talking about Windows and MacOS. In fact, I have 0 knowledge how to compile on these targets, so I gave up.

Edit: sorry I deleted the od script that I mentioned above..

ochafik · 2024-04-13T16:14:32Z

@phymbert great to read you've already mitigated the issue, I'd missed that commit in the flow.

Things get messy when talking about Windows and MacOS. In fact, I have 0 knowledge how to compile on these targets, so I gave up.

@ngxson actually I've just hit a snag by using -n that my BSD xxd likes (on MacOS), and GNU xxd has no clue about (on CI) - fixed.

Re/ Windows, I realize I've assumed people build from within WSL (or cross-build from Linux), but would need confirmation.

In the absolute worst case we could write our own minimalistic xxd, I suppose.

phymbert · 2024-04-13T16:22:52Z

Re/ Windows, I realize I've assumed people build from within WSL (or cross-build from Linux), but would need confirmation.

We cannot assume unfortunately, I have the feeling most windows developper are using Visual Studio.

In the absolute worst case we could write our own minimalistic xxd, I suppose.

I would prefer to have a classical npm build then. Also what about:

https://stackoverflow.com/questions/4158900/embedding-resources-in-executable-using-gcc

ochafik · 2024-04-13T16:41:19Z

I would prefer to have a classical npm build then.

@phymbert possibly inevitable in the long run, would unlock new horizons...

Also what about: https://stackoverflow.com/questions/11813271/embed-resources-eg-shader-code-images-into-executable-library-with-cmake

Hah, nifty! Might need some contortions to get something similar to work w/ MSVC, but alternatively... looks like we could do it in cmake: https://stackoverflow.com/questions/11813271/embed-resources-eg-shader-code-images-into-executable-library-with-cmake

Which means out of the 3 documented ways to build on Windows:

(MS) make + w64devkit: the latter ships w/ busybox, which seems to have xxd
cmake: can do the HEX in CMakeLists.txt
zig: same (TBC, I'm not sure how much I like the language so far :-D)

(so xxd in Makefile for all platforms, and adhoc hexing in the other build scripts 🤞)

ngxson · 2024-04-14T00:57:38Z

Good job! The read file as HEX trick in cmake seem to resolve the problem on both mac & windows.

For make, there're users still using make on linux, and xxd may not be available by default, at least it's not installed on my fedora installation and my ubuntu distrobox. Of course I can simply apt install, but I'm a bit doubt that this change may break compilation for someone else.

ggerganov · 2024-04-15T10:43:41Z

In the absolute worst case we could write our own minimalistic xxd, I suppose.

This does not sound really awful - we can give it a try if we encounter issues with the proposed PR

One idea to make things even more user friendly is to have a minimal index.html committed and loaded when the hex conversion has not been applied (for whatever reason). That minimal index.html can just show information about the necessary steps to generate the UI

github-actions · 2024-04-15T18:10:38Z

📈 llama.cpp server for bench-server-baseline on Standard_NC4as_T4_v3 for phi-2-q4_0: 460 iterations 🚀

Expand details for performance related PR only

Concurrent users: 8, duration: 10m
HTTP request : avg=10214.07ms p(95)=26164.51ms fails=, finish reason: stop=412 truncated=48
Prompt processing (pp): avg=112.42tk/s p(95)=549.33tk/s
Token generation (tg): avg=24.22tk/s p(95)=36.95tk/s
ggml-org/models/phi-2/ggml-model-q4_0.gguf parallel=8 ctx-size=16384 ngl=33 batch-size=2048 ubatch-size=256 pp=1024 pp+tg=2048 branch=generate-assets commit=b9286a4d7b929b077f1d05395e2bc9ecadf5a99a

More

---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 460 iterations"
    y-axis "llamacpp:prompt_tokens_seconds"
    x-axis "llamacpp:prompt_tokens_seconds" 1713214386 --> 1713215020
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 488.34, 488.34, 488.34, 488.34, 488.34, 751.63, 751.63, 751.63, 751.63, 751.63, 475.07, 475.07, 475.07, 475.07, 475.07, 505.87, 505.87, 505.87, 505.87, 505.87, 536.01, 536.01, 536.01, 536.01, 536.01, 581.01, 581.01, 581.01, 581.01, 581.01, 583.14, 583.14, 583.14, 583.14, 583.14, 584.69, 584.69, 584.69, 584.69, 584.69, 595.22, 595.22, 595.22, 595.22, 595.22, 599.72, 599.72, 599.72, 599.72, 599.72, 601.64, 601.64, 601.64, 601.64, 601.64, 610.24, 610.24, 610.24, 610.24, 610.24, 612.16, 612.16, 612.16, 612.16, 612.16, 633.3, 633.3, 633.3, 633.3, 633.3, 623.27, 623.27, 623.27, 623.27, 623.27, 631.07, 631.07, 631.07, 631.07, 631.07, 636.81, 636.81, 636.81, 636.81, 636.81, 554.47, 554.47, 554.47, 554.47, 554.47, 558.73, 558.73, 558.73, 558.73, 558.73, 559.12, 559.12, 559.12, 559.12, 559.12, 566.66, 566.66, 566.66, 566.66, 566.66, 578.9, 578.9, 578.9, 578.9, 578.9, 583.16, 583.16, 583.16, 583.16, 583.16, 584.08, 584.08, 584.08, 584.08, 584.08, 590.17, 590.17, 590.17, 590.17, 590.17, 591.54, 591.54, 591.54, 591.54, 591.54, 595.02, 595.02, 595.02, 595.02, 595.02, 608.11, 608.11, 608.11, 608.11, 608.11, 605.23, 605.23, 605.23, 605.23, 605.23, 610.26, 610.26, 610.26, 610.26, 610.26, 612.08, 612.08, 612.08, 612.08, 612.08, 621.9, 621.9, 621.9, 621.9, 621.9, 619.29, 619.29, 619.29, 619.29, 619.29, 620.5, 620.5, 620.5, 620.5, 620.5, 621.21, 621.21, 621.21, 621.21, 621.21, 624.94, 624.94, 624.94, 624.94, 624.94, 629.13, 629.13, 629.13, 629.13, 629.13, 628.32, 628.32, 628.32, 628.32, 628.32, 630.71, 630.71, 630.71, 630.71, 630.71, 637.29, 637.29, 637.29, 637.29, 637.29, 643.9, 643.9, 643.9, 643.9, 643.9, 647.81, 647.81, 647.81, 647.81, 647.81, 654.42, 654.42, 654.42, 654.42, 654.42, 654.71, 654.71, 654.71, 654.71, 654.71, 654.36, 654.36, 654.36, 654.36, 654.36, 657.47, 657.47, 657.47, 657.47, 657.47, 661.36, 661.36, 661.36, 661.36, 661.36, 669.77, 669.77, 669.77, 669.77, 669.77, 664.06, 664.06, 664.06, 664.06, 664.06, 660.33, 660.33, 660.33, 660.33, 660.33, 658.81, 658.81, 658.81, 658.81, 658.81, 658.35, 658.35, 658.35, 658.35, 658.35, 657.23, 657.23, 657.23, 657.23, 657.23, 655.95, 655.95, 655.95, 655.95, 655.95, 654.2, 654.2, 654.2, 654.2, 654.2, 657.69, 657.69, 657.69, 657.69, 657.69, 660.03, 660.03, 660.03, 660.03, 660.03, 659.97, 659.97, 659.97, 659.97, 659.97, 661.12, 661.12, 661.12, 661.12, 661.12, 663.23, 663.23, 663.23, 663.23, 663.23, 667.18, 667.18, 667.18, 667.18, 667.18, 667.18, 667.18, 667.18]

More

---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 460 iterations"
    y-axis "llamacpp:predicted_tokens_seconds"
    x-axis "llamacpp:predicted_tokens_seconds" 1713214386 --> 1713215020
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 35.6, 35.6, 35.6, 35.6, 35.6, 35.6, 35.6, 35.6, 35.6, 35.6, 22.94, 22.94, 22.94, 22.94, 22.94, 24.33, 24.33, 24.33, 24.33, 24.33, 25.31, 25.31, 25.31, 25.31, 25.31, 25.37, 25.37, 25.37, 25.37, 25.37, 25.46, 25.46, 25.46, 25.46, 25.46, 26.09, 26.09, 26.09, 26.09, 26.09, 26.26, 26.26, 26.26, 26.26, 26.26, 26.15, 26.15, 26.15, 26.15, 26.15, 26.09, 26.09, 26.09, 26.09, 26.09, 25.26, 25.26, 25.26, 25.26, 25.26, 25.14, 25.14, 25.14, 25.14, 25.14, 25.23, 25.23, 25.23, 25.23, 25.23, 24.45, 24.45, 24.45, 24.45, 24.45, 24.45, 24.45, 24.45, 24.45, 24.45, 23.95, 23.95, 23.95, 23.95, 23.95, 23.74, 23.74, 23.74, 23.74, 23.74, 23.74, 23.74, 23.74, 23.74, 23.74, 23.81, 23.81, 23.81, 23.81, 23.81, 23.74, 23.74, 23.74, 23.74, 23.74, 23.46, 23.46, 23.46, 23.46, 23.46, 23.22, 23.22, 23.22, 23.22, 23.22, 23.2, 23.2, 23.2, 23.2, 23.2, 22.99, 22.99, 22.99, 22.99, 22.99, 23.03, 23.03, 23.03, 23.03, 23.03, 23.18, 23.18, 23.18, 23.18, 23.18, 22.92, 22.92, 22.92, 22.92, 22.92, 23.0, 23.0, 23.0, 23.0, 23.0, 23.07, 23.07, 23.07, 23.07, 23.07, 23.17, 23.17, 23.17, 23.17, 23.17, 22.96, 22.96, 22.96, 22.96, 22.96, 22.74, 22.74, 22.74, 22.74, 22.74, 22.8, 22.8, 22.8, 22.8, 22.8, 23.05, 23.05, 23.05, 23.05, 23.05, 23.15, 23.15, 23.15, 23.15, 23.15, 23.16, 23.16, 23.16, 23.16, 23.16, 23.33, 23.33, 23.33, 23.33, 23.33, 23.42, 23.42, 23.42, 23.42, 23.42, 23.32, 23.32, 23.32, 23.32, 23.32, 23.29, 23.29, 23.29, 23.29, 23.29, 23.17, 23.17, 23.17, 23.17, 23.17, 23.08, 23.08, 23.08, 23.08, 23.08, 23.11, 23.11, 23.11, 23.11, 23.11, 23.24, 23.24, 23.24, 23.24, 23.24, 23.39, 23.39, 23.39, 23.39, 23.39, 23.43, 23.43, 23.43, 23.43, 23.43, 23.36, 23.36, 23.36, 23.36, 23.36, 23.12, 23.12, 23.12, 23.12, 23.12, 22.87, 22.87, 22.87, 22.87, 22.87, 22.7, 22.7, 22.7, 22.7, 22.7, 22.61, 22.61, 22.61, 22.61, 22.61, 22.17, 22.17, 22.17, 22.17, 22.17, 21.16, 21.16, 21.16, 21.16, 21.16, 21.14, 21.14, 21.14, 21.14, 21.14, 21.14, 21.14, 21.14, 21.14, 21.14, 21.17, 21.17, 21.17, 21.17, 21.17, 21.29, 21.29, 21.29, 21.29, 21.29, 21.28, 21.28, 21.28, 21.28, 21.28, 21.46, 21.46, 21.46, 21.46, 21.46, 21.53, 21.53, 21.53]

Details

More

---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 460 iterations"
    y-axis "llamacpp:kv_cache_usage_ratio"
    x-axis "llamacpp:kv_cache_usage_ratio" 1713214386 --> 1713215020
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.04, 0.04, 0.04, 0.04, 0.04, 0.3, 0.3, 0.3, 0.3, 0.3, 0.29, 0.29, 0.29, 0.29, 0.29, 0.12, 0.12, 0.12, 0.12, 0.12, 0.17, 0.17, 0.17, 0.17, 0.17, 0.14, 0.14, 0.14, 0.14, 0.14, 0.16, 0.16, 0.16, 0.16, 0.16, 0.1, 0.1, 0.1, 0.1, 0.1, 0.14, 0.14, 0.14, 0.14, 0.14, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.26, 0.26, 0.26, 0.26, 0.26, 0.14, 0.14, 0.14, 0.14, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.27, 0.27, 0.27, 0.27, 0.27, 0.14, 0.14, 0.14, 0.14, 0.14, 0.26, 0.26, 0.26, 0.26, 0.26, 0.25, 0.25, 0.25, 0.25, 0.25, 0.16, 0.16, 0.16, 0.16, 0.16, 0.2, 0.2, 0.2, 0.2, 0.2, 0.15, 0.15, 0.15, 0.15, 0.15, 0.23, 0.23, 0.23, 0.23, 0.23, 0.23, 0.23, 0.23, 0.23, 0.23, 0.21, 0.21, 0.21, 0.21, 0.21, 0.23, 0.23, 0.23, 0.23, 0.23, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.35, 0.35, 0.35, 0.35, 0.35, 0.14, 0.14, 0.14, 0.14, 0.14, 0.09, 0.09, 0.09, 0.09, 0.09, 0.15, 0.15, 0.15, 0.15, 0.15, 0.29, 0.29, 0.29, 0.29, 0.29, 0.23, 0.23, 0.23, 0.23, 0.23, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.15, 0.15, 0.15, 0.15, 0.15, 0.11, 0.11, 0.11, 0.11, 0.11, 0.16, 0.16, 0.16, 0.16, 0.16, 0.17, 0.17, 0.17, 0.17, 0.17, 0.1, 0.1, 0.1, 0.1, 0.1, 0.18, 0.18, 0.18, 0.18, 0.18, 0.24, 0.24, 0.24, 0.24, 0.24, 0.09, 0.09, 0.09, 0.09, 0.09, 0.15, 0.15, 0.15, 0.15, 0.15, 0.12, 0.12, 0.12, 0.12, 0.12, 0.08, 0.08, 0.08, 0.08, 0.08, 0.23, 0.23, 0.23, 0.23, 0.23, 0.44, 0.44, 0.44, 0.44, 0.44, 0.6, 0.6, 0.6, 0.6, 0.6, 0.58, 0.58, 0.58, 0.58, 0.58, 0.52, 0.52, 0.52, 0.52, 0.52, 0.55, 0.55, 0.55, 0.55, 0.55, 0.45, 0.45, 0.45, 0.45, 0.45, 0.21, 0.21, 0.21, 0.21, 0.21, 0.15, 0.15, 0.15, 0.15, 0.15, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.06, 0.06, 0.06, 0.06, 0.06, 0.17, 0.17, 0.17, 0.17, 0.17, 0.2, 0.2, 0.2]

More

---
config:
    xyChart:
        titleFontSize: 12
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "#000000"
---
xychart-beta
    title "llama.cpp bench-server-baseline on Standard_NC4as_T4_v3
 duration=10m 460 iterations"
    y-axis "llamacpp:requests_processing"
    x-axis "llamacpp:requests_processing" 1713214386 --> 1713215020
    line [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 8.0, 8.0, 8.0, 8.0, 8.0, 3.0, 3.0, 3.0, 3.0, 3.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 8.0, 8.0, 8.0, 8.0, 8.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 5.0, 5.0, 5.0, 5.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 8.0, 8.0, 8.0, 8.0, 8.0, 4.0, 4.0, 4.0, 4.0, 4.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 4.0, 4.0, 4.0, 4.0, 4.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 6.0, 6.0, 6.0, 6.0, 6.0, 7.0, 7.0, 7.0, 7.0, 7.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 3.0, 3.0, 3.0, 3.0, 3.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 3.0, 3.0, 3.0, 3.0, 3.0, 5.0, 5.0, 5.0, 5.0, 5.0, 4.0, 4.0, 4.0, 4.0, 4.0, 3.0, 3.0, 3.0, 3.0, 3.0, 8.0, 8.0, 8.0, 8.0, 8.0, 4.0, 4.0, 4.0, 4.0, 4.0, 6.0, 6.0, 6.0, 6.0, 6.0, 5.0, 5.0, 5.0, 5.0, 5.0, 7.0, 7.0, 7.0, 7.0, 7.0, 5.0, 5.0, 5.0, 5.0, 5.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 5.0, 5.0, 5.0, 5.0, 5.0, 1.0, 1.0, 1.0, 1.0, 1.0, 6.0, 6.0, 6.0, 6.0, 6.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 7.0, 7.0, 7.0, 7.0, 7.0, 5.0, 5.0, 5.0, 5.0, 5.0, 3.0, 3.0, 3.0, 3.0, 3.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 2.0, 2.0, 2.0, 2.0, 2.0, 7.0, 7.0, 7.0, 7.0, 7.0, 3.0, 3.0, 3.0, 3.0, 3.0, 1.0, 1.0, 1.0]

cebtenzzre · 2024-04-15T18:41:55Z

examples/server/CMakeLists.txt

+foreach(asset ${SERVER_ASSETS})
+    # Portable equivalent of `cd public && xxd -i ${asset} ../${asset}.hpp`
+    set(input "${CMAKE_CURRENT_SOURCE_DIR}/public/${asset}")
+    set(output "${CMAKE_CURRENT_BINARY_DIR}/${asset}.hpp")
+
+    string(REGEX REPLACE "\\.| |-" "_" name "${asset}")
+    file(READ "${input}" data HEX)
+    string(LENGTH ${data} hex_len)
+    math(EXPR len "${hex_len} / 2")
+    string(REGEX REPLACE "([0-9a-f][0-9a-f])" "0x\\1," data ${data})
+    file(WRITE "${output}" "unsigned char ${name}[] = {${data}};\nunsigned int ${name}_len = ${len};\n")
+endforeach()


Doing this at configure time seems wrong - if the files change between running cmake (which shouldn't need to be done frequently) and running make or cmake --build, they will not be updated.

This could be turned into a separate .cmake script that is run at build time, similar to how we generate build info.

Thanks, now using an add_custom_command forking out to scripts/xxd.cmake

cebtenzzre · 2024-04-15T18:44:53Z

Nomic's Kompute fork builds xxd from source (it's a single .c source file) unconditionally: https://github.com/nomic-ai/kompute/blob/d1e3b0953cf66acc94b2e29693e221427b2c1f3f/CMakeLists.txt#L187

ochafik · 2024-04-15T20:34:28Z

Nomic's Kompute fork builds xxd from source (it's a single .c source file) unconditionally: https://github.com/nomic-ai/kompute/blob/d1e3b0953cf66acc94b2e29693e221427b2c1f3f/CMakeLists.txt#L187

@cebtenzzre hah great, so we kinda always have xxd after all... (assuming one does git clone --recursive, which I didn't)

In the absolute worst case we could write our own minimalistic xxd, I suppose.

This does not sound really awful - we can give it a try if we encounter issues with the proposed PR

@ggerganov @ngxson I've switched the Makefile to using od & wc (should work on Windows w/ w64devkit through busybox 🤞). None of the 3 hex-dumping methods are exactly pretty, but they're relatively short (building xxd would probably take the same amount of build script lines, or more b/c of two-phased builds).

ggerganov · 2024-04-21T12:45:37Z

Feel free to merge when ready

sorasoras · 2024-04-22T11:50:29Z

@ochafik
I think this kind of break rocm for cmake in windows.

debugbuild.txt

ochafik · 2024-04-22T15:43:05Z

I think this kind of break rocm for cmake in windows.
debugbuild.txt

@sorasoras glancing through it, think it might related to #6797 @jart FYI

* `build`: generate hex dumps of server assets on the fly * build: workaround lack of -n on gnu xxd * build: don't use xxd in cmake * build: don't call xxd from build.zig * build: more idiomatic hexing * build: don't use xxd in Makefile (od hackery instead) * build: avoid exceeding max cmd line limit in makefile hex dump * build: hex dump assets at cmake build time (not config time)

build: generate hex dumps of server assets on the fly

b5f9c70

ochafik changed the title ~~build: generate hex dump of server assets on the fly~~ build: generate hex dump of server assets during build Apr 13, 2024

build: workaround lack of -n on gnu xxd

13a36ef

ochafik force-pushed the generate-assets branch from 14677f1 to 13a36ef Compare April 13, 2024 16:20

ochafik changed the title ~~build: generate hex dump of server assets during build~~ (WIP) build: generate hex dump of server assets during build Apr 13, 2024

ochafik added 3 commits April 13, 2024 18:12

build: don't use xxd in cmake

35df587

build: don't call xxd from build.zig

1be0389

build: more idiomatic hexing

17a86b3

ochafik changed the title ~~(WIP) build: generate hex dump of server assets during build~~ build: generate hex dump of server assets during build Apr 13, 2024

ochafik marked this pull request as ready for review April 13, 2024 18:10

ngxson mentioned this pull request Apr 14, 2024

Clean up server code #5762

Closed

ochafik added 2 commits April 15, 2024 18:30

build: don't use xxd in Makefile (od hackery instead)

b2b45db

Merge remote-tracking branch 'origin/master' into generate-assets

4a33423

build: avoid exceeding max cmd line limit in makefile hex dump

5ab711e

cebtenzzre reviewed Apr 15, 2024

View reviewed changes

build: nits

b9286a4

ochafik added 2 commits April 19, 2024 20:51

build: hex dump assets at cmake build time (not config time)

5c4a948

Merge remote-tracking branch 'origin/master' into generate-assets

649b730

ggerganov approved these changes Apr 21, 2024

View reviewed changes

ngxson approved these changes Apr 21, 2024

View reviewed changes

ochafik merged commit 5cf5e7d into ggml-org:master Apr 21, 2024
52 of 59 checks passed

xatier mentioned this pull request Apr 25, 2024

server: public: move default parameters to its own file #6858

Closed

ochafik mentioned this pull request Apr 26, 2024

grammars: x{min,max} repetition operator #6640

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`build`: generate hex dump of server assets during build #6661

`build`: generate hex dump of server assets during build #6661

ochafik commented Apr 13, 2024 •

edited

Loading

phymbert commented Apr 13, 2024

ngxson commented Apr 13, 2024 •

edited

Loading

ochafik commented Apr 13, 2024

phymbert commented Apr 13, 2024

ochafik commented Apr 13, 2024

ngxson commented Apr 14, 2024

ggerganov commented Apr 15, 2024

github-actions bot commented Apr 15, 2024 •

edited

Loading

cebtenzzre Apr 15, 2024 •

edited

Loading

ochafik Apr 19, 2024 •

edited

Loading

cebtenzzre commented Apr 15, 2024 •

edited

Loading

ochafik commented Apr 15, 2024

ggerganov commented Apr 21, 2024

sorasoras commented Apr 22, 2024

ochafik commented Apr 22, 2024

build: generate hex dump of server assets during build #6661

build: generate hex dump of server assets during build #6661

Conversation

ochafik commented Apr 13, 2024 • edited Loading

phymbert commented Apr 13, 2024

ngxson commented Apr 13, 2024 • edited Loading

ochafik commented Apr 13, 2024

phymbert commented Apr 13, 2024

ochafik commented Apr 13, 2024

ngxson commented Apr 14, 2024

ggerganov commented Apr 15, 2024

github-actions bot commented Apr 15, 2024 • edited Loading

cebtenzzre Apr 15, 2024 • edited Loading

Choose a reason for hiding this comment

ochafik Apr 19, 2024 • edited Loading

Choose a reason for hiding this comment

cebtenzzre commented Apr 15, 2024 • edited Loading

ochafik commented Apr 15, 2024

ggerganov commented Apr 21, 2024

sorasoras commented Apr 22, 2024

ochafik commented Apr 22, 2024

`build`: generate hex dump of server assets during build #6661

`build`: generate hex dump of server assets during build #6661

ochafik commented Apr 13, 2024 •

edited

Loading

ngxson commented Apr 13, 2024 •

edited

Loading

github-actions bot commented Apr 15, 2024 •

edited

Loading

cebtenzzre Apr 15, 2024 •

edited

Loading

ochafik Apr 19, 2024 •

edited

Loading

cebtenzzre commented Apr 15, 2024 •

edited

Loading