[bug] chatglm3-6b No corresponding template chat-template

[lostmaniac]

https://github.com/vllm-project/vllm/blob/main/examples/template_chatml.jinja

This template is suitable for chatglm2, but not suitable for chatglm3. The new template address document is. https://github.com/THUDM/ChatGLM3/blob/main/PROMPT.md

[jrsperry]

you can specify the chat template on start up with --chat-template formatted as a single line string, or as a path to a .jinja file. The chat templates come from the tokenizer used by the model, see here for explanation. I've noticed a frequent mismatch as a result, although it's much improved compared to pre-v0.2.3.

[lostmaniac]

I am using the latest version 0.2.5, but chatglm3 has changed the chat template https://github.com/THUDM/ChatGLM3/blob/main/PROMPT.md

[lostmaniac]

I've been looking at this template for a long time, but I don't understand how to write it. Could you please help me write a template that supports chatglm3?

[ZhiWenMo]

I've been looking at this template for a long time, but I don't understand how to write it. Could you please help me write a template that supports chatglm3?

you can modify the .jinjia file like this

{% for message in messages %}
{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '\n' }}
{% endfor %}
{{ '<|assistant|>' }}

and start up with --chat-template

Remember to set the stop token <|user|>

This works fine for my api serve.

[zzhdbw]

这个模板我看了很久了，但我不明白怎么写。你能帮我写一个支持chatglm3的模板吗？

您可以像这样修改 .jinjia 文件

{% for message in messages %}
{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '\n' }}
{% endfor %}
{{ '<|assistant|>' }}

并使用 --chat-template 启动

请记住设置停止令牌<|user|>

这适用于我的 api 服务。

thank you! ti is useful for me, and could you please provide a chatglm2 template ?, cause the jinjia is hard to understand and use. thank you!

[DarkLight1337]

thank you! ti is useful for me, and could you please provide a chatglm2 template ?, cause the jinjia is hard to understand and use. thank you!

As mentioned in the original post, this one should work for chatglm2:

https://github.com/vllm-project/vllm/blob/main/examples/template_chatml.jinja

I'm closing this issue since @ZhiWenMo has already provided the template for chatglm3, which answers the original question.