Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main : add Self-Extend support #4815

Merged
merged 9 commits into from
Jan 8, 2024
Merged

main : add Self-Extend support #4815

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
21196da
examples : add passkey test
ggerganov Oct 30, 2023
fbb999f
passkey : better prints
ggerganov Oct 30, 2023
bda3f2c
passkey : select pass key pos from CLI
ggerganov Jan 7, 2024
f2c9800
passkey : simplify n_past logic
ggerganov Jan 7, 2024
2f40c9f
llama : "self-extend"-like context extension
ggerganov Jan 7, 2024
f64cddc
passkey : add comment
ggerganov Jan 7, 2024
ea12921
main : add Self-Extend support
ggerganov Jan 7, 2024
66ad819
Merge remote-tracking branch 'origin/master' into gg/self-extend-part-2
ggerganov Jan 8, 2024
82048d4
llama : add comment about llama_kv_cache_seq_div
ggerganov Jan 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
llama : add comment about llama_kv_cache_seq_div
  • Loading branch information
@ggerganov
ggerganov committed Jan 8, 2024
commit 82048d4750781dbbe657f2036f5b55dd21581165
4 changes: 4 additions & 0 deletions llama.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,10 @@ extern "C" {
llama_pos p1,
llama_pos delta);

// Integer division of the positions by factor of `d > 1`
// If the KV cache is RoPEd, the KV data is updated accordingly
// p0 < 0 : [0, p1]
// p1 < 0 : [p0, inf)
LLAMA_API void llama_kv_cache_seq_div(
struct llama_context * ctx,
llama_seq_id seq_id,
Expand Down