Serverless Inference API OpenAI /v1/chat/completions route broken

[pelikhan]

System Info

Trying to access the serverless inference endpoints using the OpenAI compatible route leads to status 400.

Invalid URL: missing field `name`

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

Here is a curl command to run inference, requires to setup your HF_TOKEN.

curl https://api-inference.huggingface.co/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $HF_TOKEN" \
-d '{
  "model": "meta-llama/Llama-3.3-70B-Instruct",
  "messages": [
    {
      "role": "user",
      "content": "Write a short poem."
    }
  ]
}'

Expected behavior

This endpoint should be "openai" compatible.

[mzyil]

+1
The same request works in this URL format though:

https://api-inference.huggingface.co/models/meta-llama/Llama-3.3-70B-Instruct/v1/chat/completions