-
Notifications
You must be signed in to change notification settings - Fork 295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Bedrock Boto3 Client Timeout #1431
Fix Bedrock Boto3 Client Timeout #1431
Conversation
…meout error when setting LLMConfig timeout higher than 60 seconds
Head branch was pushed to by a user without write access
23af12d
to
4ca8281
Compare
@davorrunje my git user.email was wrong. I've forced push the same commits with my updated email now. The CLA action detected correctly that I've signed it as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ronaldpereira!
Codecov ReportAttention: Patch coverage is
... and 69 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Fix bedrock boto3 read_timeout error when setting LLMConfig timeout higher than 60 seconds.
Why are these changes needed?
When calling bedrock models, botocore gives a read_timeout exception almost every time when the model call takes longer than 60 seconds. For faster models this is ok, but for newer and heavier models (e.g., claude-3.7), it isn't enough. So we already have a
timeout
llm_config attribute that we can set directly into the boto3 config instance.Related issue number
No issue related.
Checks