Skip to content

Disable default cache when both cache and cache_seed are not set #1641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 16, 2025

Conversation

kumaranvpl
Copy link
Collaborator

Why are these changes needed?

Related issue number

Closes #1640

Checks

@kumaranvpl kumaranvpl requested review from marklysze and Copilot April 15, 2025 13:36
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

@kumaranvpl kumaranvpl marked this pull request as draft April 15, 2025 13:42
@kumaranvpl kumaranvpl marked this pull request as ready for review April 15, 2025 14:22
@marklysze
Copy link
Collaborator

marklysze commented Apr 16, 2025

Hey @kumaranvpl, thanks for putting this together.

This is working for me now:

# NOT CACHED - CORRECT
llm_config = LLMConfig(config_list=[{"i_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}])

# NOT CACHED - CORRECT
llm_config = LLMConfig(config_list=[{"api_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}], cache_seed=None)

# CACHED - CORRECT
llm_config = LLMConfig(config_list=[{"api_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}], cache_seed=49)
# CACHED CHANGE SEED - CORRECT
llm_config = LLMConfig(config_list=[{"api_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}], cache_seed=48)

When using the Cache context manager, I tried the below and it didn't work...

from autogen.cache import Cache

with Cache.disk(cache_seed=39, cache_path_root=".cache"):

    llm_config = LLMConfig(config_list=[{"api_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}])

    my_agent = ConversableAgent(
        name="test_agent",
        llm_config=llm_config,
    )

    my_other_agent = ConversableAgent(
        name="other_agent",
        llm_config=llm_config,
    )

    result = my_agent.initiate_chat(
        recipient=my_other_agent,
        message="Why is the sun round?",
        max_turns=2
    )

    print(result.cost)

Second run output:

{
'usage_including_cached_inference': {'total_cost': 0, 'gpt-4.1-mini-2025-04-14': {'cost': 0, 'prompt_tokens': 343, 'completion_tokens': 244, 'total_tokens': 587}},
'usage_excluding_cached_inference': {'total_cost': 0, 'gpt-4.1-mini-2025-04-14': {'cost': 0, 'prompt_tokens': 343, 'completion_tokens': 244, 'total_tokens': 587}}
}

... then I realised I needed to pass that cache into the initiate_chat as well, like the below. Just wondering why this is necessary?

from autogen.cache import Cache

with Cache.disk(cache_seed=39, cache_path_root=".cache") as dc:

    llm_config = LLMConfig(config_list=[{"api_type": "openai", "model": "gpt-4.1-mini-2025-04-14"}])

    my_agent = ConversableAgent(
        name="test_agent",
        llm_config=llm_config,
    )

    my_other_agent = ConversableAgent(
        name="other_agent",
        llm_config=llm_config,
    )

    result = my_agent.initiate_chat(
        recipient=my_other_agent,
        message="Why is the sun round?",
        max_turns=2,
        cache=dc
    )

    print(result.cost)

@marklysze
Copy link
Collaborator

It's working as planned now, thank you! I'll approve the changes.

I think we need some specific documentation on caching, I'll add an Issue.

@marklysze marklysze enabled auto-merge April 16, 2025 00:18
@marklysze marklysze added this pull request to the merge queue Apr 16, 2025
Merged via the queue into main with commit d0e0f2d Apr 16, 2025
17 checks passed
@marklysze marklysze deleted the disable-default-cache branch April 16, 2025 00:31
Copy link

codecov bot commented Apr 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ There is a different number of reports uploaded between BASE (f7a935e) and HEAD (4c077ea). Click for more details.

HEAD has 1453 uploads less than BASE
Flag BASE (f7a935e) HEAD (4c077ea)
3.13 95 0
macos-latest 115 0
commsagent-discord 9 0
optional-deps 175 0
core-without-llm 14 1
3.9 95 0
ubuntu-latest 170 1
3.10 108 0
3.12 36 0
3.11 76 1
browser-use 7 0
commsagent-slack 9 0
windows-latest 125 0
commsagent-telegram 9 0
jupyter-executor 9 0
retrievechat-pgvector 10 0
rag 7 0
retrievechat-qdrant 14 0
retrievechat 15 0
retrievechat-mongodb 10 0
graph-rag-falkor-db 6 0
docs 6 0
interop 13 0
crawl4ai 13 0
wikipedia-api 13 0
google-api 13 0
twilio 9 0
interop-pydantic-ai 9 0
websockets 9 0
interop-langchain 9 0
mcp 13 0
interop-crewai 9 0
mistral 14 0
agent-eval 1 0
gpt-assistant-agent 3 0
lmm 4 0
long-context 3 0
cohere 15 0
teachable 4 0
gemini 15 0
retrievechat-couchbase 3 0
websurfer 15 0
ollama 15 0
swarm 14 0
cerebras 15 0
groq 14 0
llama-index-agent 3 0
bedrock 15 0
anthropic 16 0
together 14 0
integration 24 0
falkordb 2 0
core-llm 9 0
neo4j 2 0
gemini-realtime 1 0
captainagent 1 0
autobuild 1 0
openai-realtime 1 0
deepseek 1 0
openai 1 0
Files with missing lines Coverage Δ
autogen/oai/client.py 53.53% <100.00%> (-23.88%) ⬇️

... and 79 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@kumaranvpl
Copy link
Collaborator Author

@marklysze I agree with you regarding context manager. If we are using context manager, then initiate_chat should automatically use cache provided by context manager.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Provide a way to disable cache
2 participants