support for CohereForAI/c4ai-command-r-v01

[xusenlinzy]

CohereForAI/c4ai-command-r-v01 is a large language model with open weights optimized for a variety of use cases including reasoning, summarization, and question answering. Command-R has the capability for multilingual generation evaluated in 10 languages and highly performant RAG capabilities.

[simon-mo]

Contribution welcomed! https://docs.vllm.ai/en/latest/models/adding_model.html

[sungkim11]

I have read it on Twitter that Cohere will support this effort. Are they involved?

[dekelcohen]

👍
Is there someone already working on this ?
I saw recent activity in transformers repo to add Cohere Command-R.

[dekelcohen]

Command-R arch is very close to Lllama2. almost identical code in HF
LayerNorm is different compared to Lllama RMSNorm

[Playerrrrr]

So does vLLM support it now or not?

[simon-mo]

It's supported in main branch, which you can use when building from source. It will be in next release (ETA tomorrow)

[0x77dev]

Looks like it was released in v0.4.0 / #3433