Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(anthropic): add instrumentation for Anthropic tool calling (alternative to #1372) #2150

Merged
merged 3 commits into from
Oct 16, 2024

Conversation

dinmukhamedm
Copy link
Contributor

This commit addresses [#879] by implementing instrumentation of Anthropic tools request args and tool_use response arguments.

  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly.

This commit addresses [traceloop#879] by implementing instrumentation
of Anthropic tools request args and tool_use response arguments
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. python Pull requests that update Python code testing labels Oct 16, 2024
Comment on lines +182 to +183
if content_block_type == "text":
text += content.text
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nirga looking for your input on this. I tried to stay as close to OpenAI as possible. For OpenAI the index, is the ordinal number of the output in the choices array. I believe, there is no such thing in Anthropic, so index is always 0.

In theory, Anthropic could respond with something like [text, tool_use, text, text, tool_use]. I've never seen it do this in practice, but just in case, instead of hard-coding, I decided to concatenate everything in the text block.

An alternative to my approach would be to set these text blocks as gen_ai.completions.i.content, but I don't think this is consistent with the choices in OpenAI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense @dinmukhamedm! Long term we plan to move these to otel events so there will be a "cleaner" solution

@nirga nirga changed the title feat(anthropic): add instrumentation for Anthropic tool calling (alternative to #1372) fix(anthropic): add instrumentation for Anthropic tool calling (alternative to #1372) Oct 16, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 16, 2024
@CLAassistant
Copy link

CLAassistant commented Oct 16, 2024

CLA assistant check
All committers have signed the CLA.

@dinmukhamedm
Copy link
Contributor Author

Sorry about the back and forth, had some trouble running nx lint on my machine. Perhaps a good sign to add an instruction note to the contribution guideline? :)

@nirga nirga merged commit 8606235 into traceloop:main Oct 16, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer python Pull requests that update Python code size:L This PR changes 100-499 lines, ignoring generated files. testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants