Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Clients]: Add support for OpenAI's Responses API #1363

Open
marklysze opened this issue Mar 16, 2025 · 1 comment
Open

[Clients]: Add support for OpenAI's Responses API #1363

marklysze opened this issue Mar 16, 2025 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@marklysze
Copy link
Collaborator

marklysze commented Mar 16, 2025

Is your feature request related to a problem? Please describe.

OpenAI has released their responses API.

Feature Chat Completions API Responses API
Input Format Messages array with roles (user, assistant, system) Input parameter accepting multiple text, images, files
Conversation State Manual management by sending previous messages Built-in state management with previous_response_id
Tools & Functions Uses older functions parameter (deprecated) Support with built-in tools (Web Search/File Search/Computer Use)
Output Structure Returns message with content Returns structured output with additional metadata
Store for later retrieval Need to set store: true Stores responses by default
Reasoning Models Less explicit support New reasoning parameter with effort
Truncation No explicit strategy parameter Can automatically truncate outputs exceeding context window, instead of throwing 400 error
Message ID No message IDs Each message has its own ID (msg_...)

For the tools:

Output:

  • The content in a response in the Responses API can be an array of different outputs and types (with a type parameter):
"content": [
  {
    "type": "output_text",
    "text": "In a peaceful grove beneath a silver moon...",
    "annotations": []
  }

Responses vs Chat Completion API notes:

  • instructions is a parameter that essentially replaces the system/developer role messages. When using this parameter with previous_response_id, the Responses API will not carry forward the system/developer messages meaning that we can replace the agent's system message at each turn (which we need to!)
  • Use an input parameter (instead of messages) that supports an array of different type objects. This includes the ability to put in a PDF file (which will use their vision model to). Uses their Files API.
  • The input parameter can still take a set of messages as we do with the Chat Completion API.
  • Simpler way of getting the outputs (no longer have to use choices[0] and they also have an output_text attribute)
  • Responses can contain an array of different types (as noted above)
  • Ability to only send the latest message and use a previous_response_id
  • Reasoning summary available in the output. Including the ability to set the reasoning model effort
  • Can force the model to always call at least one tool (tool_choice = required) - could be very useful to ensure that certain agents always call a tool.
  • For structured outputs you can now get the Pydantic model directly from the response (response.output[0].content[0].parsed)

Considerations:

  • Could have an OpenAI Web Search tool that uses this Responses API's web_search tool (or could add as an option to an existing search tool)
  • Similarly, could have an OpenAI File Search tool
  • Computer use agent based on sample app.
@marklysze marklysze added the enhancement New feature or request label Mar 16, 2025
@marklysze marklysze added this to ag2 Mar 16, 2025
@marklysze
Copy link
Collaborator Author

@davorrunje, @sonichi - the Responses API has some useful features and could be used to make some tools and agents (noted in description). See full description of issue with differences.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

2 participants