Truncation support for recent Mistrals to prevent AsyncEngineDeadError on input exceeding max_model_len w/ chunked prefill#13741

Open

nightflight-dk wants to merge 2 commits intovllm-project:mainfrom nightflight-dk:mistral-chat-truncation

+8-5

Commits on Feb 24, 2025

truncation support for chat completion w/ mistral tokenizer
Damian Kowalczyk
committed
Merge branch 'vllm-project:main' into mistral-chat-truncation
nightflight-dk
authored