Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: passkey challenge / self-extend with context shift demo #5832

Merged
merged 28 commits into from
Mar 2, 2024
Merged

server: passkey challenge / self-extend with context shift demo #5832

Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
73a7e42
server: tests: add models endpoint scenario
phymbert Mar 2, 2024
0f774a8
server: /v1/models add some metadata
phymbert Mar 2, 2024
1780d96
server: tests: add debug field in context before scenario
phymbert Mar 2, 2024
319ded7
server: tests: download model from HF, add batch size
phymbert Mar 2, 2024
18e739d
server: tests: add passkey test
phymbert Mar 2, 2024
ab5b06b
server: logs: do not truncate log values
phymbert Mar 2, 2024
60113da
server: tests: add group attention params
phymbert Mar 2, 2024
616d7e9
server: do not truncate prompt tokens if self-extend through group at…
phymbert Mar 2, 2024
2495f72
server: logs: do not truncate log values
phymbert Mar 2, 2024
af82fb4
server: revert change on slot n_ctx
phymbert Mar 2, 2024
3b8242a
server: tests - missing EOL at EOF
phymbert Mar 2, 2024
ed60b97
server: tests - fix passkey not using pre/suffix
phymbert Mar 2, 2024
cf4c86e
server: tests - passkey - first good working value of nga
phymbert Mar 2, 2024
f8773f7
server: tests - passkey - limit the number of max tokens to predix
phymbert Mar 2, 2024
a80533e
server: tests - passkey - limit the number of max tokens to predix
phymbert Mar 2, 2024
8abf8d3
server: tests: fix server timeout
phymbert Mar 2, 2024
407cc60
server: tests: fix passkey, add doc, fix regex content matching, fix …
phymbert Mar 2, 2024
178b0c6
server: tests: fix regex content matching
phymbert Mar 2, 2024
9ab72d7
server: tests: schedule slow tests on master
phymbert Mar 2, 2024
9fcfa63
server: tests: schedule slow tests on master
phymbert Mar 2, 2024
61b9791
server: metrics: fix when no prompt processed
phymbert Mar 2, 2024
763ae0a
Merge remote-tracking branch 'origin/tests/server/passkey' into tests…
phymbert Mar 2, 2024
830d0ef
server: tests: CI workflow failed on first scenario failed
phymbert Mar 2, 2024
1aa5ad9
server: tests: fix re content
phymbert Mar 2, 2024
c1f66f0
server: tests: self-extend add llama-2-7B and Mixtral-8x7B-v0.1
phymbert Mar 2, 2024
2cdd21e
server: tests: increase timeout for completion
phymbert Mar 2, 2024
a6ea725
server: tests: keep only the PHI-2 test
phymbert Mar 2, 2024
0c7f5b2
server: tests: passkey add a negative test
phymbert Mar 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions examples/server/tests/features/passkey.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,8 @@ Feature: Passkey / Self-extend with context shift
Then <n_predicted> tokens are predicted matching <re_content>

Examples:
| hf_repo | hf_file | n_ctx_train | ngl | n_ctx | n_batch | n_ga | n_ga_w | n_junk | i_pos | passkey | n_predicted | re_content |
| TheBloke/phi-2-GGUF | phi-2.Q4_K_M.gguf | 2048 | 5 | 16384 | 512 | 16 | 512 | 250 | 50 | 42 | 1 | 42 |
| hf_repo | hf_file | n_ctx_train | ngl | n_ctx | n_batch | n_ga | n_ga_w | n_junk | i_pos | passkey | n_predicted | re_content |
| TheBloke/phi-2-GGUF | phi-2.Q4_K_M.gguf | 2048 | 5 | 8192 | 512 | 16 | 512 | 250 | 50 | 42 | 1 | 42 |
| TheBloke/Llama-2-7B-GGUF | llama-2-7b.Q2_K.gguf | 4096 | 3 | 16384 | 512 | 4 | 512 | 500 | 300 | 1234 | 5 | 1234 |
| TheBloke/Mixtral-8x7B-v0.1-GGUF | mixtral-8x7b-v0.1.Q2_K.gguf | 4096 | 2 | 16384 | 512 | 4 | 512 | 500 | 100 | 0987 | 5 | 0987 |

Loading