Add Tool Choice parameter to model provider configs for tool control

[marklysze]

Why are these changes needed?

Model providers typically provide a means, through their parameters, to force tool calls (and force no tool calls).

This PR adds this parameter to the model provider LLMConfigs we have and documentation to demonstrate how to use it.

Thanks to @alecsolder for inspiration on their reliable function calling (#927).

Related issue number

N/A

Checks

• I've included any doc changes needed for https://docs.ag2.ai/. See https://docs.ag2.ai/docs/contributor-guide/documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[alecsolder]

One thing of note, doing this actually forces models to not output any other tokens besides the tool call, so a model can no longer do reasoning for tool calls, the assistant's reply is pre-filled with the beginning of the tool call basically.

link

Note that when you have tool_choice as any or tool, we will prefill the assistant message to force a tool to be used. This means that the models will not emit a chain-of-thought text content block before tool_use content blocks, even if explicitly asked to do so.
Our testing has shown that this should not reduce performance. If you would like to keep chain-of-thought (particularly with Opus) while still requesting that the model use a specific tool, you can use {"type": "auto"} for tool_choice (the default) and add explicit instructions in a user message. For example: What's the weather like in London? Use the get_weather tool in your response.

[marklysze]

One thing of note, doing this actually forces models to not output any other tokens besides the tool call, so a model can no longer do reasoning for tool calls, the assistant's reply is pre-filled with the beginning of the tool call basically.

link

Note that when you have tool_choice as any or tool, we will prefill the assistant message to force a tool to be used. This means that the models will not emit a chain-of-thought text content block before tool_use content blocks, even if explicitly asked to do so.
Our testing has shown that this should not reduce performance. If you would like to keep chain-of-thought (particularly with Opus) while still requesting that the model use a specific tool, you can use {"type": "auto"} for tool_choice (the default) and add explicit instructions in a user message. For example: What's the weather like in London? Use the get_weather tool in your response.

Thanks @alecsolder, I'll add this to the documentation, definitely a trade-off.

[codecov]

Codecov Report

Attention: Patch coverage is 51.06383% with 46 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
autogen/oai/gemini_types.py	50.72%	34 Missing ⚠️
autogen/oai/anthropic.py	25.00%	3 Missing ⚠️
autogen/oai/cohere.py	33.33%	1 Missing and 1 partial ⚠️
autogen/oai/groq.py	50.00%	1 Missing and 1 partial ⚠️
autogen/oai/together.py	50.00%	2 Missing ⚠️
autogen/oai/cerebras.py	50.00%	1 Missing ⚠️
autogen/oai/gemini.py	66.66%	1 Missing ⚠️
autogen/oai/mistral.py	50.00%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (fe622b4) and HEAD (4d2afbb). Click for more details.

HEAD has 2532 uploads less than BASE

Flag	BASE (fe622b4)	HEAD (4d2afbb)
3.10	191	0
ubuntu-latest	292	1
commsagent-discord	18	0
optional-deps	282	0
3.13	169	0
core-without-llm	28	1
macos-latest	208	0
commsagent-slack	18	0
3.9	165	0
3.11	128	1
browser-use	14	0
3.12	72	0
windows-latest	225	0
commsagent-telegram	18	0
retrievechat-qdrant	28	0
graph-rag-falkor-db	12	0
retrievechat-mongodb	20	0
jupyter-executor	18	0
retrievechat-pgvector	20	0
twilio	18	0
retrievechat	30	0
interop	26	0
interop-pydantic-ai	18	0
interop-crewai	18	0
interop-langchain	18	0
crawl4ai	26	0
websockets	18	0
docs	12	0
cerebras	29	0
teachable	8	0
agent-eval	2	0
gpt-assistant-agent	6	0
together	28	0
anthropic	32	0
long-context	6	0
retrievechat-couchbase	6	0
lmm	8	0
websurfer	30	0
gemini	30	0
mistral	28	0
llama-index-agent	6	0
swarm	28	0
groq	28	0
bedrock	30	0
cohere	29	0
ollama	29	0
core-llm	18	0
integration	24	0
openai-realtime	2	0
falkordb	4	0
gemini-realtime	2	0
captainagent	2	0
autobuild	2	0
neo4j	4	0
deepseek	2	0
openai	2	0

Files with missing lines	Coverage Δ
autogen/oai/client.py	53.37% <100.00%> (-24.25%)	⬇️
autogen/oai/cerebras.py	32.03% <50.00%> (-14.80%)	⬇️
autogen/oai/gemini.py	15.56% <66.66%> (-46.24%)	⬇️
autogen/oai/mistral.py	40.70% <50.00%> (+0.16%)	⬆️
autogen/oai/cohere.py	32.72% <33.33%> (+<0.01%)	⬆️
autogen/oai/groq.py	46.87% <50.00%> (-0.33%)	⬇️
autogen/oai/together.py	33.58% <50.00%> (-18.76%)	⬇️
autogen/oai/anthropic.py	20.94% <25.00%> (-56.08%)	⬇️
autogen/oai/gemini_types.py	50.72% <50.72%> (ø)

... and 65 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Hellisotherpeople]

Big thank you for this! This is hugely important and combined with vllm getting support for this in its next release, we are in for a renaissance in local AI agents.

[marklysze]

Big thank you for this! This is hugely important and combined with vllm getting support for this in its next release, we are in for a renaissance in local AI agents.

Thanks @Hellisotherpeople! Love local models and, therefore, local agents :)