Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support anthropic structured outputs #142

Merged
merged 7 commits into from
Feb 7, 2025

Conversation

sachinpad
Copy link
Contributor

No description provided.

@sachinpad sachinpad requested a review from ankrgyl February 4, 2025 19:03
Copy link

vercel bot commented Feb 4, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
ai-proxy ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 6, 2025 10:04pm

Copy link
Contributor

@ankrgyl ankrgyl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we apply this technique to OpenAI flavored LLMs that do not have structured output (e.g. llama hosted on together)

@sachinpad
Copy link
Contributor Author

Can we apply this technique to OpenAI flavored LLMs that do not have structured output (e.g. llama hosted on together)

Done

Copy link
Contributor

@ankrgyl ankrgyl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code mostly lgtm. will test it

package.json Outdated
Comment on lines 24 to 27
"packageManager": "[email protected]",
"resolutions": {
"zod": "3.22.4"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm I believe we've moved away from doing this (@manugoyal @ibolmo have context)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this PR's case, the version of zod was different between the proxy and SDK, which was causing some strange type errors. I guess we could manually ensure they are the same?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way we accomplished this in the past was by pinning it in pnpm-lock.yaml instead. @manugoyal or @ibolmo can help you do this the right way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will defer to @ibolmo, as I was only familiar with the resolutions approach, but now see we are using overrides in other parts of the codebase, such as our internal repo. A quick google suggests that pnpm.overrides is preferred for pnpm projects.

Copy link

@ibolmo ibolmo Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I did an override: https://github.com/braintrustdata/braintrust-sdk/pull/469/files.

I forget the exact reason (something something peer dependency), but if the override doesn't unblock you I can try to hop on and sort it out with you.

@sachinpad sachinpad merged commit 44a04ac into main Feb 7, 2025
4 checks passed
@sachinpad sachinpad deleted the sachin-anthropic-structured-out branch February 7, 2025 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants