bug(anthropic): tool use returns error

[faassen]

• I have looked for existing issues (including closed) about this

Bug Report

I'm trying to use with Anthropic. I can't get it to work, I get 400 as soon as I include any tools.

Reproduction

Any anthropic call with tools should replicate this behavior.

Expected behavior

Anthropic should use tools

Additional context

I did some debugging and there are a few things that seem to hinder tool use being initiated at all:

• For some reason there's a 'cache_control' in ToolDefinition for Anthropic. That's not documented as far as I can see, and disabling seemed to improve the situation.

• tool_choice: {"type": "auto"} seems to be required in the request JSON to get tool usage at all.

When I make both modifications to rig, things seem to go further; Anthropic thinks for a while, but finally I get the following error:

Failed to prompt Claude Sonnet: CompletionError(HttpError(reqwest::Error { kind: Decode, source: Error("data did not match any variant of untagged enum Content", line: 0, column: 0) }))

So it appears Anthropic returns something that isn't properly handled by rig?

Testing?

I like Rig's API so far, and I know it's alpha, but while the Anthropic functionality is in the API my impression (also from #65) is that not really ready yet. It strikes me that some kind of integration testing is in order in order to determine a client really works for the various works cases, and doesn't get broken.

Automated testing would be nice, but testing with an external service is always tricky to do automatically. Right now I'm testing it. :)

[0xMochan]

Appreciate the testing related to tool testing, I'll be taking a look at this closely to fix this up quickly!

Testing

100% agree. After talking with the team, we discussed moving up our plans for integration testing and perhaps some sort of mocking service to test the endpoints would be very much up the alley soon!

[cvauclair]

@0xMochan looking at the issue rn, where does ToolDefinition::cache_control (in the Anthropic integration) come from? I don't see it in the API docs 🤔