feat(llm-proxy): Add support for O1 & other LLMs

[HamadaSalhab]

PR Type

Enhancement

---

Description

• Added support for new LLM models in litellm-config.yaml.

• Introduced configurations for OpenAI's o1-mini model.

• Added multiple OpenRouter models, including eva-llama-3.33-70b and hermes-3-llama-3.1-70b.

• Updated the model_list with detailed parameters and tags for new models.

---

Changes walkthrough 📝

	Relevant files
Enhancement	litellm-config.yaml Added new LLM models to configuration                                        llm-proxy/litellm-config.yaml Added configuration for OpenAI's o1-mini model. Introduced multiple OpenRouter models with detailed parameters. Updated model_list with tags and API key references. Enhanced support for paid LLM models. +42/-0

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

---

Important

Add support for O1 and several OpenRouter LLMs in litellm-config.yaml.

• New Models Added:
  • Added o1 model with openai/o1-mini parameters.
  • Added eva-llama-3.33-70b, l3.3-euryale-70b, magnum-v4-72b, eva-qwen-2.5-72b, hermes-3-llama-3.1-70b, and deepseek-chat models with respective OpenRouter parameters.
• Configuration:
  • Updated litellm-config.yaml to include new models under the model_list section with appropriate api_key and tags.

^{This description was created by for 003cc0b. It will automatically update as commits are pushed.}

[qodo-merge-pro-for-open-source]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 Security concerns API Key Management: The configuration references sensitive API keys through environment variables. While using environment variables is a good practice, ensure that the OPENAI_API_KEY and OPENROUTER_API_KEY environment variables are properly secured and not exposed in logs or error messages.

⚡ Recommended focus areas for review Configuration Validation Verify that the model identifiers for the new OpenRouter models are correct and active, as using incorrect model IDs could cause runtime failures - model_name: "eva-llama-3.33-70b" litellm_params: model: "openrouter/eva-unit-01/eva-llama-3.33-70b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] - model_name: "l3.3-euryale-70b" litellm_params: model: "openrouter/sao10k/l3.3-euryale-70b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] - model_name: "magnum-v4-72b" litellm_params: model: "openrouter/anthracite-org/magnum-v4-72b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] - model_name: "eva-qwen-2.5-72b" litellm_params: model: "openrouter/eva-unit-01/eva-qwen-2.5-72b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] - model_name: "hermes-3-llama-3.1-70b" litellm_params: model: "openrouter/nousresearch/hermes-3-llama-3.1-70b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] - model_name: "deepseek-chat" litellm_params: model: "openrouter/deepseek/deepseek-chat" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"]

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 003cc0b in 13 seconds

More details

• Looked at 60 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. llm-proxy/litellm-config.yaml:49

• Draft comment:
  The model 'o1' and 'o1-mini' have the same 'litellm_params'. Consider verifying if this is intentional or if 'o1' should have different parameters.
• Reason this comment was not posted:
  Comment did not seem useful.

Workflow ID: wflow_DV4ZLFMPyia8UDDN

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[qodo-merge-pro-for-open-source]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Possible issue	Add context window size limits to prevent potential token overflow issues during runtime Add model context window size limits as a parameter for each model to prevent potential runtime errors when processing large inputs. llm-proxy/litellm-config.yaml [116-120] - model_name: "eva-llama-3.33-70b" litellm_params: model: "openrouter/eva-unit-01/eva-llama-3.33-70b" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] + context_window: 8192 Apply this suggestion Suggestion importance[1-10]: 8 Why: Adding context window limits is crucial for preventing runtime errors and ensuring reliable model operation. This is a critical parameter that affects both functionality and error handling.	8
General	Include model pricing information to enable cost monitoring and budget control Add model pricing information in the configuration to enable cost tracking and budget management features. llm-proxy/litellm-config.yaml [146-150] - model_name: "deepseek-chat" litellm_params: model: "openrouter/deepseek/deepseek-chat" api_key: os.environ/OPENROUTER_API_KEY tags: ["paid"] + price_per_token: 0.000002 Apply this suggestion Suggestion importance[1-10]: 7 Why: Including pricing information is valuable for cost management and monitoring, especially for paid models. This enables better budget control and usage tracking.	7