basic lib doesn't work with Ampere ARM processors

[myNicole98]

As the title suggest, if I try to set lib='basic' I get the following error:

/home/ubuntu/.local/lib/python3.10/site-packages/ctransformers/lib/basic/libctransformers.so: cannot open shared object file: No such file or directory

If I try to directly run libctransformers.so I get this error:

cannot execute binary file: Exec format error

[marella]

Hi, for ARM processors it has to be built from source as precompiled libs are not available #9 (comment):

git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
./scripts/build.sh

The compiled binary will be located at build/lib/libctransformers.so which can be used as:

llm = AutoModelForCausalLM.from_pretrained(..., lib='/path/to/ctransformers/build/lib/libctransformers.so')

[myNicole98]

It worked! Thanks for the help ^^

[NeonBohdan]

Thanks it's working
Is it possible to provide aarch64 pypi package with prebuild libctransformers.so for convenience?

[marella]

I'm using GitHub Actions to build binaries but it doesn't support ARM processors currently.
I made it simpler to install from source now. You can just run the following command:

pip install ctransformers --no-binary ctransformers

[NeonBohdan]

@marella thanks
As I saw it's for example possible with qemu here https://github.com/OpenNMT/CTranslate2/blob/50ce6a76920e8eef15a47997353101348871ec3d/.github/workflows/ci.yml#L163

[rvandernoort]

Hi, just stumbled on this thread trying to get ctransformers to work on an RPi4. Quick notes:

• Firstly, a deprecation warning

pip install ctransformers --no-binary ctransformers
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453

• Secondly, this installation method results in a unicode error for me:

print(llm(formatted_prompt))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 49: ordinal not in range(256)

• Lastly, manual installation from source work perfectly, however might it be a good idea to put this in the installation part of the readme, since I had to find it in a closed issue.

Overall love the library! 👍