feat: add vulkan support #42

thewh1teagle · 2024-08-23T17:22:40Z

Update whisper.cpp to commit and added Vulkan support. Tested on Linux.
Resolve #41

absadiki

Thanks @thewh1teagle for the contribution.
Please go over my comments and let me know what do you think!

absadiki · 2024-08-24T04:05:40Z

README.md

+export CMAKE_ARGS="-DGGML_VULKAN=ON"
+python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
+pip install dist/<generated>.whl


Are you sure this is working ?
You are using a flag that isn't in the Cmakelists file!
A successful compilation does not always mean it's working!

Did you run any tests, did you check in the output that your GPU is being used ?

I checked it on Linux and Windows and it worked.
The GPU being used in the output.
I suggest you test it as well at least compile since the submodule didn't updated a while.

There's another issue that I found now in Windows:

Generating code Finished generating code _pywhispercpp.vcxproj -> D:\pywhispercpp\build\lib.win-amd64-cpython-39\_pywhispercpp.pyd Building Custom Rule D:/pywhispercpp/CMakeLists.txt error: can't copy 'build\lib.win-amd64-cpython-39\_pywhispercpp.cp39-win_amd64.pyd': doesn't exist or not a regular file

Instead it generated build\lib.win-amd64-cpython-39\_pywhispercpp.pyd
So if I rename it to _pywhispercpp.cp39-win_amd64.pyd and run again it works.

Also then it failed to import the pyd file since it's installed into site-packages instead of site-packages/pywhispercpp

Related: #43
It also doesn't build on Windows on main branch.

So, it worked, interesting!
In that case can you please try to compile similar to what I did with CUDA,

git clone --recursive https://github.com/abdeladim-s/pywhispercpp cd pywhispercpp GGML_VULKAN=ON pip install .

If it worked, then update the instructions accordingly. I wanted to update the MAC instructions as well, but I needed someone to confirm that it works.

The project https://github.com/abetlen/llama-cpp-python works and it's very similar to this one.
Maybe we can change the build system to be just like there

No, I don't think it's similar! different approach of creating the bindings, and not just the build system.
But the question is why? This one works as well! If it didn't work on your machine it does not mean it's not working and we need to change everything!
Similarly, if llama-cpp-python is working for you, it does not mean it's working for everyone, check the issues there and see!

Issues and bugs are part of the game, you are a programmer and you know that!

I agree. You know the feel :)

I found the root cause and I'll check how to fix correctly.

The failed to copy build error
Fixed by upgrading python version / upgrading pip

The failed to load DLL error
Python didn't helped much with the error details... and ldd can't analyze pyd.
BUT - Dependencies was able to analyze it and turns out it depends on whisper.dll. Once I placed it in the same folder the DLL loaded and main.py didn't failed.

We can copy the DLL to the package when building or link it statically by default. but cuda / vulkan needs DLLs anyway so copy is better.

Fixed in #46 and this PR works with that

Yes, that was the idea of yesterday's patch! I located the whisper.dll file and tried copying it to the source directory. Just like what I did with Linux.
I'll check #46 now.

README.md

absadiki · 2024-08-25T06:14:26Z

README.md

+export CMAKE_ARGS="-DGGML_VULKAN=ON"
+python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
+pip install dist/<generated>.whl


So, it worked, interesting!
In that case can you please try to compile similar to what I did with CUDA,

git clone --recursive https://github.com/abdeladim-s/pywhispercpp cd pywhispercpp GGML_VULKAN=ON pip install .

If it worked, then update the instructions accordingly. I wanted to update the MAC instructions as well, but I needed someone to confirm that it works.

absadiki · 2024-08-25T06:19:51Z

README.md

+export CMAKE_ARGS="-DGGML_VULKAN=ON"
+python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
+pip install dist/<generated>.whl


I will test it on my end as well, I just believe that it's safer to usually follow the official whisper.cpp releases, rather than compiling the bleeding edge. But it's Okey.

absadiki · 2024-08-25T06:21:44Z

README.md

+export CMAKE_ARGS="-DGGML_VULKAN=ON"
+python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
+pip install dist/<generated>.whl


There's another issue that I found now in Windows:

Generating code Finished generating code _pywhispercpp.vcxproj -> D:\pywhispercpp\build\lib.win-amd64-cpython-39\_pywhispercpp.pyd Building Custom Rule D:/pywhispercpp/CMakeLists.txt error: can't copy 'build\lib.win-amd64-cpython-39\_pywhispercpp.cp39-win_amd64.pyd': doesn't exist or not a regular file

Instead it generated build\lib.win-amd64-cpython-39\_pywhispercpp.pyd So if I rename it to _pywhispercpp.cp39-win_amd64.pyd and run again it works.

Also then it failed to import the pyd file since it's installed into site-packages instead of site-packages/pywhispercpp

Let's keep track of this issue in #43.
Let's merge this first.

README.md

absadiki · 2024-08-25T06:31:04Z

README.md

@@ -180,7 +190,7 @@ usage: pwcpp [-h] [-m MODEL] [--version] [--processors PROCESSORS] [-otxt] [-ovt
             [--translate TRANSLATE] [--no_context NO_CONTEXT] [--single_segment SINGLE_SEGMENT] [--print_special PRINT_SPECIAL]
             [--print_progress PRINT_PROGRESS] [--print_realtime PRINT_REALTIME] [--print_timestamps PRINT_TIMESTAMPS]
             [--token_timestamps TOKEN_TIMESTAMPS] [--thold_pt THOLD_PT] [--thold_ptsum THOLD_PTSUM] [--max_len MAX_LEN]
-             [--split_on_word SPLIT_ON_WORD] [--max_tokens MAX_TOKENS] [--speed_up SPEED_UP] [--audio_ctx AUDIO_CTX]


BTW, could you please make this change in a separate commit and the submodule update in its own commit as well? This will help keep the change history clear in case we ever need to revert back.

I've always wanted to remove the speed_up but didn't find time.
Thanks a lot!

How should I do it in separate commit?
I think that once I'll revert this commit it will close the PR

In that case, I think just close this PR, pull the latest changes, test if everything is working on Windows with the small fix I added yesterday,
Then submit a PR to remove the speed_up, then update whisper.cpp and submit another PR.

Thanks a lot!

absadiki · 2024-08-29T02:30:39Z

README.md

+export CMAKE_ARGS="-DGGML_VULKAN=1"
+python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
+pip install dist/<generated>.whl


I think this is better, you don't need to export the flag and wait the wheel to build

GGML_VULKAN=1 pip install .

Or is it not working ?

thewh1teagle · 2024-08-29T02:31:12Z

@abdeladim-s

I rebased the commits here. now it's clean.

Regarding the insturctions in the reamde I think they can simplified from

export CMAKE_ARGS="-DGGML_VULKAN=1"
python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
pip install dist/<generated>.whl

to

export CMAKE_ARGS="-DGGML_VULKAN=1"
pip install .

Both for CoreML and Vulkan.

absadiki · 2024-08-29T02:37:15Z

@abdeladim-s

I rebased the commits here. now it's clean.

Regarding the insturctions in the reamde I think they can simplified from
export CMAKE_ARGS="-DGGML_VULKAN=1"
python -m build --wheel # in this repository to build the wheel. Assumes you have installed build with pip install build
pip install dist/<generated>.whl
to
export CMAKE_ARGS="-DGGML_VULKAN=1"
pip install .
Both for CoreML and Vulkan.

Yes, it looks clean now. Thanks!

absadiki · 2024-08-29T02:39:43Z

Thanks a lot for the contribution 👍

thewh1teagle · 2024-08-29T03:13:25Z

Yes, it looks clean now. Thanks!

Even just CMAKE_ARGS="-DGGML_VULKAN=1" pip install git+https://github.com/abdeladim-s/pywhispercpp
That way there's no need to clone it manually :)

absadiki · 2024-08-29T03:19:31Z

I think someone reported that using the link in pip install does not work!
Cloning it manually did! That's why I updated the readme accordingly.

absadiki requested changes Aug 24, 2024

View reviewed changes

absadiki requested changes Aug 25, 2024

View reviewed changes

thewh1teagle force-pushed the feat/vulkan branch 2 times, most recently from 2258be9 to 10bc609 Compare August 29, 2024 02:06

feat: update whisper.cpp

1915c83

thewh1teagle force-pushed the feat/vulkan branch from 10bc609 to 1915c83 Compare August 29, 2024 02:11

thewh1teagle added 3 commits August 29, 2024 05:14

fix: remove speedup option

c14978b

fix: remove whisper_pcm_to_mel

b66fc5f

chore: add instructions and credits

b67f020

absadiki requested changes Aug 29, 2024

View reviewed changes

absadiki merged commit 4ddfa8c into absadiki:main Aug 29, 2024
7 of 10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add vulkan support #42

feat: add vulkan support #42

thewh1teagle commented Aug 23, 2024 •

edited

Loading

absadiki left a comment

absadiki Aug 24, 2024

thewh1teagle Aug 24, 2024

thewh1teagle Aug 24, 2024 •

edited

Loading

thewh1teagle Aug 24, 2024

absadiki Aug 25, 2024

thewh1teagle Aug 28, 2024

absadiki Aug 28, 2024

thewh1teagle Aug 29, 2024 •

edited

Loading

thewh1teagle Aug 29, 2024

absadiki Aug 29, 2024

absadiki Aug 25, 2024

absadiki Aug 25, 2024

absadiki Aug 25, 2024

absadiki Aug 25, 2024

thewh1teagle Aug 28, 2024

absadiki Aug 28, 2024

absadiki Aug 29, 2024

thewh1teagle commented Aug 29, 2024 •

edited

Loading

absadiki commented Aug 29, 2024

absadiki commented Aug 29, 2024

thewh1teagle commented Aug 29, 2024

absadiki commented Aug 29, 2024

feat: add vulkan support #42

feat: add vulkan support #42

Conversation

thewh1teagle commented Aug 23, 2024 • edited Loading

absadiki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thewh1teagle Aug 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thewh1teagle Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thewh1teagle commented Aug 29, 2024 • edited Loading

absadiki commented Aug 29, 2024

absadiki commented Aug 29, 2024

thewh1teagle commented Aug 29, 2024

absadiki commented Aug 29, 2024

thewh1teagle commented Aug 23, 2024 •

edited

Loading

thewh1teagle Aug 24, 2024 •

edited

Loading

thewh1teagle Aug 29, 2024 •

edited

Loading

thewh1teagle commented Aug 29, 2024 •

edited

Loading