Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compilade/fix mpt pretok #215

Merged
merged 18 commits into from
Jul 8, 2024
Merged

Compilade/fix mpt pretok #215

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
db2ffd5
llama : fix mpt and olmo pre-tokenizer
compilade Jun 30, 2024
cb4d86c
server: Retrieve prompt template in /props (#8337)
bviksoe Jul 7, 2024
210eb9e
finetune: Rename an old command name in finetune.sh (#8344)
standby24x7 Jul 7, 2024
b81ba1f
finetune: Rename command name in README.md (#8343)
standby24x7 Jul 7, 2024
d39130a
py : use cpu-only torch in requirements.txt (#8335)
compilade Jul 7, 2024
b504008
llama : fix n_rot default (#8348)
ggerganov Jul 7, 2024
905942a
llama : support glm3 and glm4 (#8031)
youth123 Jul 7, 2024
f7cab35
gguf-hash: model wide and per tensor hashing using xxhash and sha1 (#…
mofosyne Jul 7, 2024
f1948f1
readme : update bindings list (#8222)
andy-tai Jul 7, 2024
4090ea5
ci : add checks for cmake,make and ctest in ci/run.sh (#8200)
AlexsCode Jul 7, 2024
a8db2a9
Update llama-cli documentation (#8315)
dspasyuk Jul 7, 2024
ac0f33c
Merge branch 'master' into compilade/fix-mpt-pretok
compilade Jul 7, 2024
3fd62a6
py : type-check all Python scripts with Pyright (#8341)
compilade Jul 7, 2024
d5d30b2
llama : pre-tokenize non-special user-defined tokens first
compilade Jul 7, 2024
6b961e3
Merge branch 'master' into compilade/fix-mpt-pretok
compilade Jul 7, 2024
56df1fc
llama : fix detection of control-like user-defined tokens
compilade Jul 7, 2024
6e351e0
convert_hf : identify which user-defined tokens are control tokens
compilade Jul 7, 2024
f9d42c5
convert_hf : identify more added control tokens for SPM tokenziers
compilade Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
readme : update bindings list (ggml-org#8222) 
* adding guile_llama_cpp  to binding list

* fix formatting

* fix formatting
  • Loading branch information
@andy-tai
andy-tai authored Jul 7, 2024
commit f1948f1e108157d5e7eb60fc8f24a10412e5d4ff
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ Typically finetunes of the base models below are supported as well.
- Zig: [deins/llama.cpp.zig](https://github.com/Deins/llama.cpp.zig)
- Flutter/Dart: [netdur/llama_cpp_dart](https://github.com/netdur/llama_cpp_dart)
- PHP (API bindings and features built on top of llama.cpp): [distantmagic/resonance](https://github.com/distantmagic/resonance) [(more info)](https://github.com/ggerganov/llama.cpp/pull/6326)
- Guile Scheme: [guile_llama_cpp](https://savannah.nongnu.org/projects/guile-llama-cpp)

**UI:**

Expand Down