context_length_exceeded error

[infinitivebyte]

I think the message history keeps building up when running the agent over a large loop like:

for keyword in keywords:
    result = lead_generation_agent.run_sync(user_promt, deps=keyword)

because the first iteration runs just fine but after dozen I get this error, why and how can I fix it?

File "/Users/UA/.venv/lib/python3.13/site-packages/openai/_base_client.py", line 1638, in _request
raise self._make_status_error_from_response(err.response) from None
openai.OpenAIError: Error code: 400 - {'error': {'message': "This model's maximum context length is 128000 tokens. However, your messages resulted in 136707 tokens (136367 in the messages, 340 in the functions). Please reduce the length of the messages or functions.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

How Agent class works, why does it pass the history messages to the next iteration of the for loop?

[izzyacademy]

@infinitivebyte you need to find a mechanism to trim/reduce the size of the message history you are passing to the run/run_sync method. You can chose to use a subset of the list or create a helper function to summarize the messages and update the history with a summarize version of the history that is less than the maximum context length for your language model.

[infinitivebyte]

@izzyacademy my wonder is why the history for processing the first keyword on the list passed to the second keyword?

when i do result = lead_generation_agent.run_sync(user_promt, deps=keyword) this does not instantiate new lead_generetation_agent with 0 message history, if not how do just do that?

[grassxyz]

Interesting way to use the agent. but why do you put the tool add inside the for loop?

…

On Mon, Feb 3, 2025 at 4:48 AM infinitivebyte ***@***.***> wrote: After some research, I have learnt that the Agent is stateful and preserves chat history between .run() calls which caused the message history to ads up in each iteration. here is what I did to fix it: def create_agent() -> Agent: return Agent( "openai:gpt-4o-mini", system_prompt=system_prompt), ) fresh_agent = create_agent() for keywork in keywords: fresh_agent.tool(get_city_position) fresh_agent.tool(retries=8)(google_search) fresh_agent.tool(retries=5)(fetch_url) result = fresh_agent.run_sync("what is contact info", deps=keyword, infer_name=False) — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GV7CTP4LUKBTOC4ODD2N5QR7AVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZQHA2TIOJZGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[infinitivebyte]

@grassxyz sad new is that approach didnt work neither. I am still cant figure out why the messages keeps accumulating in each iteration. my brain kind melted around this.

right now switching to pure openai api using function calling until maybe maintainers jumps in and explain

[grassxyz]

That is the design of messages history. The good news is that you can manage the message history manually if your use cases need that. That's what I do and it has been working well so far.

…

On Mon, Feb 3, 2025 at 12:04 PM infinitivebyte ***@***.***> wrote: @grassxyz <https://github.com/grassxyz> sad new is that approach didnt work neither. I am still cant figure out why the messages keeps accumulating in each iteration. my brain kind melted around this. right now switching to pure openai api using function calling until maybe maintainers jumps in and explain — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GSPMS4GKQCK4FVZAG32N7DULAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZRHE2TKNJZGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[infinitivebyte]

result = fresh_agent.run_sync("what is contact info", deps=keyword, message_history=[]) does not seems to work neither. am I doing something wrong?

[grassxyz]

Can you please elaborate your use cases? is it just a ai crawl tool to get the contact info? need more details before i can suggest a way to implement this

…

On Mon, Feb 3, 2025 at 12:16 PM infinitivebyte ***@***.***> wrote: result = fresh_agent.run_sync("what is contact info", deps=keyword, message_history=[]) does not seems to work neither. am I doing something wrong? — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GXFOC4RIDBDPKKTWVL2N7FDHAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZRHE4DCOBTGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[infinitivebyte]

@grassxyz here is my full code, so after dozen iteration i start to get context_length_exceeded error. any thoughts? thanks

from typing import Optional, List

import logfire
from pydantic_ai import Agent, RunContext
from pydantic import BaseModel, EmailStr, Field
from loguru import logger as log

from googlesearch.spider import GoogleSearch
from googlesearch.utils.fetch_search_result_url import get_markdown


logfire.configure()


class SearchDependency(BaseModel):
    city: str
    title: str


class GoogleSearchResult(BaseModel):
    url: Optional[str] = Field("", description="URL of the search result")
    snippet: Optional[str] = Field("", description="Snippet of the search result")


class Lead(BaseModel):
    """
    Lead model to store the contact information.
    """

    first_name: str
    last_name: str
    email: Optional[EmailStr] = Field(None, description="Email address of the lead")


lead_generation_agent = Agent(
    "openai:gpt-4o-mini",
    system_prompt=system_prompt,
    result_type=Lead,
)


@lead_generation_agent.tool
def get_city_position(ctx: RunContext[SearchDependency]) -> str:
    city = ctx.deps.city
    title = ctx.deps.title
    return f"The city is {city} and the position is {title}"


@lead_generation_agent.tool(retries=8)
async def google_search(
    ctx: RunContext[None], location: str, title: str
) -> List[GoogleSearchResult]:
    data = []
    keyword = f"{location} {title}"
    search = GoogleSearch(keyword)
    items = search.run()
    for item in items:
        data.append(GoogleSearchResult(**item))
    return data


@lead_generation_agent.tool(retries=5)
async def fetch_url(ctx: RunContext[None], url: str) -> str:
    mark = get_markdown(url)
    return mark

for keyword in keywords:
    keyword = SearchDependency(**keyword)
    result = lead_generation_agent.run_sync("what is contact info", deps=keyword, message_history=[], infer_name=False)
    log.success(result.data)

[grassxyz]

on a quick glance: 3 key areas you need to fix first: 1) what's your system prompts? you need a good one so the ai agent knows what to do with the run 2) I would not set the retries so high (5, 8). 2 is max for me.. if it keeps failing, meaning your prompt is not good, work on the system prompt 3) you need description for your tool, otherwise your ai agent do not know which tool to call, what parameters to put in and what to expect from the tool

…

On Mon, Feb 3, 2025 at 12:40 PM infinitivebyte ***@***.***> wrote: @grassxyz <https://github.com/grassxyz> here is my full code, so after dozen iteration i start to get context_length_exceeded error. any thoughts? thanks from typing import Optional, List import logfire from pydantic_ai import Agent, RunContext from pydantic import BaseModel, EmailStr, Field from loguru import logger as log from googlesearch.spider import GoogleSearch from googlesearch.utils.fetch_search_result_url import get_markdown logfire.configure() class SearchDependency(BaseModel): city: str title: str class GoogleSearchResult(BaseModel): url: Optional[str] = Field("", description="URL of the search result") snippet: Optional[str] = Field("", description="Snippet of the search result") class Lead(BaseModel): """ Lead model to store the contact information. """ first_name: str last_name: str email: Optional[EmailStr] = Field(None, description="Email address of the lead") lead_generation_agent = Agent( "openai:gpt-4o-mini", system_prompt=system_prompt, result_type=Lead, ) @lead_generation_agent.tool def get_city_position(ctx: RunContext[SearchDependency]) -> str: city = ctx.deps.city title = ctx.deps.title return f"The city is {city} and the position is {title}" @lead_generation_agent.tool(retries=8) async def google_search( ctx: RunContext[None], location: str, title: str ) -> List[GoogleSearchResult]: data = [] keyword = f"{location} {title}" search = GoogleSearch(keyword) items = search.run() for item in items: data.append(GoogleSearchResult(**item)) return data @lead_generation_agent.tool(retries=5) async def fetch_url(ctx: RunContext[None], url: str) -> str: mark = get_markdown(url) return mark for keyword in keywords: keyword = SearchDependency(**keyword) result = lead_generation_agent.run_sync("what is contact info", deps=keyword, message_history=[], infer_name=False) log.success(result.data) — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GRAULIJBVYVC7A7HBD2N7H4HAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZSGAZDMOJVGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[infinitivebyte]

    system_prompt=(
        """
    You are an experienced lead generation specialist.
    You are asked to find and compile the contact information of the specific position/title for a given city:
    - first name
    - last name
    - email
    - title
    - oficial city website
    - phone
    - address
    - state
    - zip code
    - direct contact form link
    - source from where the lead was collected.
    
    user google_search tool to search for the contact information of the specific position/title for a given city.
    """
    )

retries set to 1

added description/docsctring to each function, but still getting that context_limit error after dozen of iteration.

[grassxyz]

you need to work on your prompt ... this is not a good prompt to start with. keep system prompts simple and concise at high level (move all the details into the tools or BaseModel description). I can't see the docstring/description of the tools and errors. so I can't comment on those.

…

On Mon, Feb 3, 2025 at 2:14 PM infinitivebyte ***@***.***> wrote: system_prompt=( """ You are an experienced lead generation specialist. You are asked to find and compile the contact information of the specific position/title for a given city: - first name - last name - email - title - oficial city website - phone - address - state - zip code - direct contact form link - source from where the lead was collected. user google_search tool to search for the contact information of the specific position/title for a given city. """ ) retries set to 1 added description/docsctring to each function, but still getting that context_limit error after dozen of iteration. — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GR47IM7T77O5M3HHAD2N7S4LAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZSGI2DSOBZHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grassxyz]

btw, changing it to gpt4 will help but you need to fix your prompts first. otherwise you will have the same problems

…

On Mon, Feb 3, 2025 at 2:35 PM Grass Hopper ***@***.***> wrote: you need to work on your prompt ... this is not a good prompt to start with. keep system prompts simple and concise at high level (move all the details into the tools or BaseModel description). I can't see the docstring/description of the tools and errors. so I can't comment on those. On Mon, Feb 3, 2025 at 2:14 PM infinitivebyte ***@***.***> wrote: > system_prompt=( > """ > You are an experienced lead generation specialist. > You are asked to find and compile the contact information of the specific position/title for a given city: > - first name > - last name > - email > - title > - oficial city website > - phone > - address > - state > - zip code > - direct contact form link > - source from where the lead was collected. > > user google_search tool to search for the contact information of the specific position/title for a given city. > """ > ) > > retries set to 1 > > added description/docsctring to each function, but still getting that > context_limit error after dozen of iteration. > > — > Reply to this email directly, view it on GitHub > <#837 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADZK7GR47IM7T77O5M3HHAD2N7S4LAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZSGI2DSOBZHA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[infinitivebyte]

I highly suspect that this is a prompt issue. Regardless of prompt the message history accumulation within the loop, defeats the stateless claim of an Agent in the documentation.

[grassxyz]

It is stateless per thread. Your code has it run in the same thread.

…

On Tue, Feb 4, 2025 at 5:50 AM infinitivebyte ***@***.***> wrote: I highly suspect that this is a prompt issue. Regardless of prompt the message history accumulation within the loop, defeats the stateless claim of an Agent in the documentation. — Reply to this email directly, view it on GitHub <#837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZK7GQ7CI4QNEO2BHPTP2L2ODASVAVCNFSM6AAAAABWJHQRRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZUGAZDQMRVGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[github-actions]

This issue is stale, and will be closed in 3 days if no reply is received.