Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: add EOT when user hit Ctrl+C #8296

Merged
merged 2 commits into from
Jul 4, 2024

Conversation

ngxson
Copy link
Collaborator

@ngxson ngxson commented Jul 4, 2024

Currently, when user in conversation mode, if user hits Ctrl+C to interrupt model's response, end-of-turn token (EOT) will not be added:

> who are you
I am LLaMA, an AI assistant developed by [[User interrupts Ctrl+C ==> no EOT is inserted here]]
> please continue

Raw tokens:

<|start_header_id|>user<|end_header_id|>

who are you<|eot_id|><|start_header_id|>assistant<|end_header_id|>

I am LLaMA, an AI assistant developed by<|start_header_id|>user<|end_header_id|>

please continue<|eot_id|>

Expected result

EOT is added after model's response

I am LLaMA, an AI assistant developed by<|eot_id|><|start_header_id|>user<|end_header_id|>

@ngxson ngxson requested a review from ggerganov July 4, 2024 12:30
@ngxson ngxson merged commit a38b884 into ggerganov:master Jul 4, 2024
53 checks passed
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 4, 2024
* main: add need_insert_eot

* do not format system prompt if it is empty
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 5, 2024
* main: add need_insert_eot

* do not format system prompt if it is empty
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 6, 2024
* main: add need_insert_eot

* do not format system prompt if it is empty
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 7, 2024
* main: add need_insert_eot

* do not format system prompt if it is empty
Nexesenex pushed a commit to Nexesenex/croco.cpp that referenced this pull request Jul 11, 2024
* main: add need_insert_eot

* do not format system prompt if it is empty
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants