Bug: No longer makes with w64devkit

[gjnave]

What happened?

I've installed this a million times, and just today went to fresh install and I cant even do a basic "make" withouth errors:

Name and Version

9022c33

What operating system are you seeing the problem on?

Windows

Relevant log output

D:/llama.cpp $ make
I ccache not found. Consider installing it for faster compilation.
I llama.cpp build info:
I UNAME_S:   Windows_NT
I UNAME_P:   unknown
I UNAME_M:   x86_64
I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -DNDEBUG -D_WIN32_WINNT=0x602 -DGGML_USE_LLAMAFILE  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -march=native -mtune=native -Xassembler -muse-unaligned-vector-move -Wdouble-promotion
I CXXFLAGS:  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -Xassembler -muse-unaligned-vector-move  -march=native -mtune=native -Wno-array-bounds -Wno-format-truncation -Wextra-semi -I. -Icommon -D_XOPEN_SOURCE=600 -DNDEBUG -D_WIN32_WINNT=0x602 -DGGML_USE_LLAMAFILE
I NVCCFLAGS: -std=c++11 -O3
I LDFLAGS:
I CC:        cc (GCC) 13.2.0
I CXX:       c++ (GCC) 13.2.0

cc  -I. -Icommon -D_XOPEN_SOURCE=600 -DNDEBUG -D_WIN32_WINNT=0x602 -DGGML_USE_LLAMAFILE  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -march=native -mtune=native -Xassembler -muse-unaligned-vector-move -Wdouble-promotion    -c ggml.c -o ggml.o
ggml.c:79:8: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
   79 | static atomic_bool atomic_flag_test_and_set(atomic_flag * ptr) {
      |        ^~~~~~~~~~~
In file included from ggml.c:4:
ggml.c: In function 'ggml_fp32_to_bf16_row':
ggml.c:420:19: error: incompatible type for argument 2 of '_mm512_storeu_si512'
  420 |             m512i(_mm512_cvtne2ps_pbh(_mm512_loadu_ps(x + i + 16),
      |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                   |
      |                   __m512bh
  421 |                                 _mm512_loadu_ps(x + i))));
      |                                 ~~~~~~~~~~~~~~~~~~~~~~~
ggml-impl.h:23:18: note: in definition of macro 'm512i'
   23 | #define m512i(p) p
      |                  ^
In file included from C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/immintrin.h:53,
                 from C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/x86intrin.h:32,
                 from C:/w64devkit/x86_64-w64-mingw32/include/intrin.h:69,
                 from ggml-impl.h:447:
C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/avx512fintrin.h:6532:41: note: expected '__m512i' {aka '__vector(8) long long int'} but argument is of type '__m512bh'
 6532 | _mm512_storeu_si512 (void *__P, __m512i __A)
      |                                 ~~~~~~~~^~~
ggml.c: In function 'ggml_vec_dot_bf16':
ggml.c:1859:42: error: incompatible type for argument 2 of '_mm512_dpbf16_ps'
 1859 |         c1 = _mm512_dpbf16_ps(c1, m512bh(_mm512_loadu_si512((x + i))),
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                          |
      |                                          __m512i {aka __vector(8) long long int}
ggml-impl.h:22:19: note: in definition of macro 'm512bh'
   22 | #define m512bh(p) p
      |                   ^
In file included from C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/immintrin.h:124:
C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/avx512bf16intrin.h:102:40: note: expected '__m512bh' but argument is of type '__m512i' {aka '__vector(8) long long int'}
  102 | _mm512_dpbf16_ps (__m512 __A, __m512bh __B, __m512bh __C)
      |                               ~~~~~~~~~^~~
ggml.c:1860:37: error: incompatible type for argument 3 of '_mm512_dpbf16_ps'
 1860 |                              m512bh(_mm512_loadu_si512((y + i))));
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                     |
      |                                     __m512i {aka __vector(8) long long int}
ggml-impl.h:22:19: note: in definition of macro 'm512bh'
   22 | #define m512bh(p) p
      |                   ^
C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/avx512bf16intrin.h:102:54: note: expected '__m512bh' but argument is of type '__m512i' {aka '__vector(8) long long int'}
  102 | _mm512_dpbf16_ps (__m512 __A, __m512bh __B, __m512bh __C)
      |                                             ~~~~~~~~~^~~
ggml.c:1861:42: error: incompatible type for argument 2 of '_mm512_dpbf16_ps'
 1861 |         c2 = _mm512_dpbf16_ps(c2, m512bh(_mm512_loadu_si512((x + i + 32))),
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                          |
      |                                          __m512i {aka __vector(8) long long int}
ggml-impl.h:22:19: note: in definition of macro 'm512bh'
   22 | #define m512bh(p) p
      |                   ^
C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/avx512bf16intrin.h:102:40: note: expected '__m512bh' but argument is of type '__m512i' {aka '__vector(8) long long int'}
  102 | _mm512_dpbf16_ps (__m512 __A, __m512bh __B, __m512bh __C)
      |                               ~~~~~~~~~^~~
ggml.c:1862:37: error: incompatible type for argument 3 of '_mm512_dpbf16_ps'
 1862 |                              m512bh(_mm512_loadu_si512((y + i + 32))));
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                     |
      |                                     __m512i {aka __vector(8) long long int}
ggml-impl.h:22:19: note: in definition of macro 'm512bh'
   22 | #define m512bh(p) p
      |                   ^
C:/w64devkit/lib/gcc/x86_64-w64-mingw32/13.2.0/include/avx512bf16intrin.h:102:54: note: expected '__m512bh' but argument is of type '__m512i' {aka '__vector(8) long long int'}
  102 | _mm512_dpbf16_ps (__m512 __A, __m512bh __B, __m512bh __C)
      |                                             ~~~~~~~~~^~~
make: *** [Makefile:696: ggml.o] Error 1

[ggerganov]

I guess it is missing some explicit casts. @chriselrod PTAL

[chriselrod]

I do not believe so. These errors are related to bf16 operations, e.g.
https://github.com/ggerganov/llama.cpp/blob/549279d8049d78620a2b081e26edb654f83c3bbd/ggml.c#L1856-L1866
It looks like these operations are deliberately missing casts on WIN32, causing the error on Windows.
https://github.com/ggerganov/llama.cpp/blob/549279d8049d78620a2b081e26edb654f83c3bbd/ggml-impl.h#L20-L30
Blame points to: #7258

[gjnave]

So what do you think is the solution here?

[chriselrod]

@Srihari-mcw how should we change the #if defined(_WIN32) check?
It is true in @gjnave's example even though it needs to be false.

[gjnave]

The biggest thing is i dont understand why this started... I have cuda 11.8... Windows 11 Pro... i dont believe my base environment is polluted... everything was working and then I deleted it and tried to remake it (something ive done dozens of times) and it stopped working.

What is the WIN32 check for?

[chriselrod]

The biggest thing is i dont understand why this started... I have cuda 11.8... Windows 11 Pro... i dont believe my base environment is polluted... everything was working and then I deleted it and tried to remake it (something ive done dozens of times) and it stopped working.

What is the WIN32 check for?

If the check is true, it won't cast types.
You're getting an error because it isn't casting types.

[Srihari-mcw]

Hi @chriselrod , @gjnave The change in PR #7258 was intended to fix the issues faced while compiling with MSVC Compiler. In my machine with cmake in windows, MSVC was taken as the default compiler in windows. Following the issue reported here, the code was updated to do type casts based on the compiler used in PR #7851. Explicit type casts are required in GCC and Clang, whereas its not required in MSVC. Thanks

[gjnave]

@Srihari-mcw @chriselrod i apologize for being obtuse.. What can I do to fix it? What can I do to be certain the type casts are being used?

[chriselrod]

@Srihari-mcw @chriselrod i apologize for being obtuse.. What can I do to fix it? What can I do to be certain the type casts are being used?

@gjnave checkout #7851

[gjnave]

Ok that change on line 20 made the difference. Appreciate your time guys. Thank you