Skip to content

Commit

Permalink
Merge pull request #823 from jxmai/feature/#789
Browse files Browse the repository at this point in the history
Fix #789: Update README with instructions for running the quantized L…
  • Loading branch information
PromtEngineer authored Sep 20, 2024
2 parents a1dea3b + b4322d4 commit b654a59
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ pip install -r requirements.txt

LocalGPT uses [LlamaCpp-Python](https://github.com/abetlen/llama-cpp-python) for GGML (you will need llama-cpp-python <=0.1.76) and GGUF (llama-cpp-python >=0.1.83) models.

To run the quantized Llama3 model, ensure you have llama-cpp-python version 0.2.62 or higher installed.

If you want to use BLAS or Metal with [llama-cpp](https://github.com/abetlen/llama-cpp-python#installation-with-openblas--cublas--clblast--metal) you can set appropriate flags:

Expand Down

0 comments on commit b654a59

Please sign in to comment.