Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic 2 compatibility change has broken Playwright's NavigateBackTool, CurrentWebPageTool and ExtractTextTool #26758

Closed
5 tasks done
richjames0 opened this issue Sep 22, 2024 · 3 comments · Fixed by #27050
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate Flagged for investigation.

Comments

@richjames0
Copy link

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code, with an OpenAI key specified at the TODO

from langchain_community.tools.playwright.utils import create_sync_playwright_browser
from langchain_community.agent_toolkits import PlayWrightBrowserToolkit
from langchain.agents import create_openai_tools_agent
from langchain_openai import ChatOpenAI
import os
from langchain.schema import SystemMessage
from langchain_core.prompts import (
    HumanMessagePromptTemplate,
    MessagesPlaceholder,
    ChatPromptTemplate,
)

HEADLESS = True

sync_browser = create_sync_playwright_browser(headless=HEADLESS)
toolkit = PlayWrightBrowserToolkit.from_browser(sync_browser=sync_browser)
tools = toolkit.get_tools()
human_prompt = ''
system_prompt = ''
MODEL = "gpt-4o-mini"
SEED = 1337
TEMPERATURE = 0.0

prompt_messages = [
    SystemMessage(content=system_prompt),
    HumanMessagePromptTemplate.from_template(
        template=[  # type:ignore
            {
                "type": "text",
                "text": human_prompt,
            },
            {
                "type": "image_url",
                "image_url": {"url": "data:image/jpeg;base64, {image_data}"},
            },
        ]
    ),
    MessagesPlaceholder("agent_scratchpad"),
]
prompt = ChatPromptTemplate(messages=prompt_messages)
OPEN_AI_KEY = #TODO
os.environ["OPENAI_API_KEY"] = OPEN_AI_KEY
llm = ChatOpenAI(model=MODEL, temperature=TEMPERATURE, seed=SEED)
agent = create_openai_tools_agent(llm, tools, prompt)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain/agents/openai_tools/base.py", line 94, in create_openai_tools_agent
    tools=[convert_to_openai_tool(tool, strict=strict) for tool in tools]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 441, in convert_to_openai_tool
    oai_function = convert_to_openai_function(tool, strict=strict)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 388, in convert_to_openai_function
    oai_function = cast(dict, format_tool_to_openai_function(function))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/_api/deprecation.py", line 179, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 292, in format_tool_to_openai_function
    if tool.tool_call_schema and not is_simple_oai_tool:
       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/tools/base.py", line 457, in tool_call_schema
    return _create_subset_model(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/pydantic.py", line 326, in _create_subset_model
    return _create_subset_model_v2(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/pydantic.py", line 270, in _create_subset_model_v2
    field = model.model_fields[field_name]  # type: ignore
            ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'data'

Description

Note: The following is written for NavigateBackTool (where I first discovered the problem) but is the same for ExtractTextTool and CurrentWebPageTool.

The final line of the code above fails with the error message given. It fails in some sense because the args_schema of the NavigateBackTool claims to support a field 'data' which it actually does not (briefly, the code seems to get a list of properties which can be queried and then query them and there's a mismatch here). My guess is that this is a version mismatch between v1 and v2 of pydantic but I could be wrong (I am not a pydantic expert). I believe this was introduced in PR [0].

Looking at navigate_back.py I can see that the import of BaseModel has changed e.g. from [1] to [2], where [1] was explicitly including the v1 version of BaseModel (albeit from langchain_core) and 2 pulls from pydantic.BaseModel.

Other tools e.g. ClickTool don't have this problem, because they override BaseModel to provide and explicit "Input" class. I have confirmed that creating such a class is a workaround for this issue. FWIW I think there's a strong argument that making this empty set of params explicit and not implicit is better anyway.

Apologies that this isn't the most complete explanation but I hope it's enough for someone in the know to understand what's going in detail much more quickly than it would take me to understand exactly why e.g. 'data' is listed as a field when not actually present :) I have found [3] to be a great place to set a breakpoint to understand this.

[0] #26443
[1] https://api.python.langchain.com/en/latest/_modules/langchain_community/tools/playwright/navigate_back.html#NavigateBackTool
[2] https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/tools/playwright/navigate_back.py
[3] Property method tool_call_schema in VENV_PATH/lib/python3.12/site-packages/langchain_core/tools/base.py

System Info

python -m langchain_core.sys_info

System Information
------------------
> OS:  Linux
> OS Version:  #13-Ubuntu SMP Mon Jul 15 13:40:27 UTC 2024
> Python Version:  3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]

Package Information
-------------------
> langchain_core: 0.3.5
> langchain: 0.3.0
> langchain_community: 0.3.0
> langsmith: 0.1.125
> langchain_openai: 0.2.0
> langchain_text_splitters: 0.3.0

Optional packages not installed
-------------------------------
> langgraph
> langserve

Other Dependencies
------------------
> aiohttp: 3.10.5
> async-timeout: Installed. No version info available.
> dataclasses-json: 0.6.7
> httpx: 0.27.2
> jsonpatch: 1.33
> numpy: 1.26.4
> openai: 1.47.0
> orjson: 3.10.7
> packaging: 24.1
> pydantic: 2.9.2
> pydantic-settings: 2.5.2
> PyYAML: 6.0.2
> requests: 2.32.3
> SQLAlchemy: 2.0.35
> tenacity: 8.5.0
> tiktoken: 0.7.0
> typing-extensions: 4.12.2
@langcarl langcarl bot added the investigate Flagged for investigation. label Sep 22, 2024
@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Sep 22, 2024
@Omniwot
Copy link

Omniwot commented Oct 24, 2024

As a temporary fix for anyone searching for it, I fixed my code by downgrading the langchain, langchain-core, langchain-community and langchain-openai packages to the following versions: 
langchain 0.2.16 
langchain-community 0.2.17 
langchain-core 0.2.41 
langchain-openai 0.1.25
playwright 1.48.0 
pydantic 2.9.2 
pydantic-core 2.23.4 
pydantic-settings 2.6.0 

I ran the following commands in order: 
pip install langchain-community<0.3 langchain<0.3
pip install langchain-core<0.3 
pip install langchain-openai<0.2

@zhaoying9105
Copy link

meto

@zhaoying9105
Copy link

There are too many problems with this tool, it's better to write one yourself

efriis added a commit that referenced this issue Oct 30, 2024
)

- Add tests for Playwright tools schema serialization
- Introduce base empty args Input class for BaseBrowserTool

Test Plan: `poetry run pytest
tests/unit_tests/tools/playwright/test_all.py`

Fixes #26758

---------

Co-authored-by: Erick Friis <[email protected]>
Co-authored-by: Eugene Yurtsev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate Flagged for investigation.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants