[Model Request] Mistral-7b

[scottorly]

• Link to an existing implementation (e.g. Hugging Face/Github): https://huggingface.co/mistralai/Mistral-7B-v0.1
• Is this model architecture supported by MLC-LLM? (the list of supported models) No

[masahi]

#986

[masahi]

It turned out #986 is not complete for Mistral 7b support, see #986 (comment)

@scottorly I suggest reopening this issue to make it more visible and avoid misunderstanding.

[coder543]

My primary interest in this is running it on my phone. Testing it on my laptop with ollama, it seemed better than the vanilla llama2-chat 7B model.

Will MLC be uploading the built model to huggingface? I’ve never successfully gotten the app to download a model that wasn’t provided by MLC

[digisomni]

If Mistral is now supported, we should probably update the README to reflect that.

[tusing]

What is the URL for the MLC version of Mistral-7b?

[knd775]

Can this please be re-opened? I'm worried it may prevent this from actually being implemented.

[junrushao]

This is already implemented by @CharlieFRuan @davidpissarra et al a few weeks ago.

[coder543]

@junrushao what is the link to download the model to the MLCChat app on my phone?

[CharlieFRuan]

@coder543 Mistral's support on iOS is a current WIP, please stay tuned!

[junrushao]

so to summarize, for Mistral, we already got:

• Desktop (CUDA, ROCm, Metal)
• Web (WebGPU & WASM)

but the following(s) are not ready:

• iOS
• Android

Is that accurate? @CharlieFRuan

[CharlieFRuan]

@junrushao Yep that is correct! I will upload the prebuilts and update the doc sometime this week.

[philippgille]

@junrushao Yep that is correct! I will upload the prebuilts and update the doc sometime this week.

Is there any update on being able to run Mistral on Android?

[CharlieFRuan]

@philippgille I believe Mistral is supported in Android now; here is the prebuilt model library: https://github.com/mlc-ai/binary-mlc-llm-libs/blob/main/Mistral-7B-Instruct-v0.2/Mistral-7B-Instruct-v0.2-q4f16_1-android.tar.