[Tracking] Mistral/OpenHermes follow-ups

[CharlieFRuan]

Overview

We added Mistral a while back but there are some leftover follow-ups to complete the support. This tracker compiles such remaining tasks.

Action Items

• Upload prebuilt models and weights for Mistral: https://llm.mlc.ai/docs/prebuilt_models.html
• Upload prebuilt weights for OpenHermes, NeuralHermes: https://llm.mlc.ai/docs/prebuilt_models.html
• Document for both Mistral and OpenHermes
• Support OpenHermes in web-llm
  • Included in demo: https://webllm.mlc.ai/
• Support NeuralHermes-2.5-Mistral-7B

Some not-so-related items:

• Make tokenizer.json change in web-llm similar to [Tokenizer] Prioritize huggingface tokenizer.json, generate one if not included #1352
• Populate build-from-source tutorial in https://llm.mlc.ai/docs/install/mlc_llm.html#option-2-build-from-source

Left for future:

• Remove <|im_start|> in chatml-like conversation template (mentioned in comments under [OpenHermes] Add conversation template for OpenHermes Mistral #1354)

[scottorly]

Trying to load OpenHermesMistral on iOS produces the following error: Unknown conversation template: open_hermes_mistral. Adding the template to the factory method in conv_templates.cc fixes the issue.

[marekk1717]

Trying to load OpenHermesMistral on iOS produces the following error: Unknown conversation template: open_hermes_mistral. Adding the template to the factory method in conv_templates.cc fixes the issue.

I've got the same issue on mac osx x86_64. How to recompile mlc_chat ?

[CharlieFRuan]

Hi @scottorly @marekk1717 the conversation template should now be included: #1385

[marekk1717]

@CharlieFRuan Thank you, it works now.

[CharlieFRuan]

All prebuilts are documented here: https://llm.mlc.ai/docs/prebuilt_models.html