inference services, servers and endpoints for elle-elle-aime

[monperrus]

A Triton inference server might be useful for the open-source models

https://github.com/triton-inference-server

[monperrus]

inference server by huggingface https://github.com/huggingface/text-generation-inference

we may get an instance soon with StarCoder

[monperrus]

Huggingface inference with zeto-gpus

@GGmorello has set up RepairLLama over HuggingFace Spaces thanks to our zero-gpus account

[monperrus]

Groq
https://www.groq.com

@FredBonux is able to use Mixtral and LLama over groq for free

[monperrus]

OpenRouter: A unified interface for LLMs
https://openrouter.ai/

Used in repairbench

[monperrus]

Replicate
Run and fine-tune open-source models. Deploy custom models at scale. All with one line of code.
https://replicate.com/

[monperrus]

Together.ai
Train, fine-tune, and run inference on AI models blazing fast, at low cost, and at production scale.
https://www.together.ai/

[monperrus]

Github has a new inference service called Github Models with all the good models.