fix token bucket not being set for both standard and adaptive retry modes #3964
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
Velfi
approved these changes
Jan 10, 2025
Comment on lines
+182
to
+185
| let default_retry_partition = match config.region() { | ||
| Some(region) => #{Cow}::from(format!("{default_retry_partition}-{}", region)), | ||
| None => #{Cow}::from(default_retry_partition), | ||
| }; |
There was a problem hiding this comment.
Suggested change
| let default_retry_partition = match config.region() { | |
| Some(region) => #{Cow}::from(format!("{default_retry_partition}-{}", region)), | |
| None => #{Cow}::from(default_retry_partition), | |
| }; | |
| let default_retry_partition = match config.region() { | |
| Some(region) => #{Cow}::from(format!("{default_retry_partition}-{region}")), | |
| None => #{Cow}::from(default_retry_partition), | |
| }; |
Comment on lines
+226
to
+229
| fun usesRegion(codegenContext: ClientCodegenContext) = | ||
| codegenContext.getBuiltIn(AwsBuiltIns.REGION) != null || | ||
| ServiceIndex.of(codegenContext.model) | ||
| .getEffectiveAuthSchemes(codegenContext.serviceShape).containsKey(SigV4Trait.ID) |
There was a problem hiding this comment.
Do we ever have to worry about models supporting sigv4a but not sigv4? How do retry partitions interact with Sigv4a regions (if at all)?
There was a problem hiding this comment.
I had this same thought when reading/extracting this function. I think the possibility is certainly coming but wasn't sure.
| .expect("success"); | ||
|
|
||
| let log_contents = logs_rx.contents(); | ||
| assert!(log_contents.contains("token bucket for RetryPartition { name: \"dontcare-us-west-2\" } added to config bag")); |
There was a problem hiding this comment.
We should consider updating the orchestrator to note the retry partition used for each retry
There was a problem hiding this comment.
We could, is there any advantage over logging it here?
landonxjames
approved these changes
Jan 10, 2025
ysaito1001
approved these changes
Jan 13, 2025
landonxjames
pushed a commit
that referenced
this pull request
Jan 14, 2025
…odes (#3964) <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here --> awslabs/aws-sdk-rust#1234 <!--- Describe your changes in detail --> PR adds a new interceptor registered as part of the default retry plugin components that ensures a token bucket is _always_ present and available to the retry strategy. The buckets are partitioned off the retry partition (which defaults to the service name and is already set by the default plugin). We use a `static` variable in the runtime for this which means that token buckets can and will apply to every single client that uses the same retry partition. The implementation tries to avoid contention on this new global lock by only consulting it if the retry partition is overridden after client creation. For AWS SDK clients I've updated the default retry partition clients are created with to include the region when set. Now the default partition for a client will be `{service}-{region}` (e.g. `sts-us-west-2`) rather than just the service name (e.g. `sts`). This partitioning is a little more granular and closer to what we want/expect as failures in one region should not cause throttling to another (and vice versa for success in one should not increase available quota in another). I also updated the implementation to follow the SEP a little more literally/closely as far as structure which fixes some subtle bugs. State is updated in one place and we ensure that the token bucket is always consulted (before the token bucket could be skipped in the case of adaptive retries returning a delay and the adaptive rate limit was updated in multiple branches). <!--- Please describe in detail how you tested your changes --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [x ] For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "client," "server," or both in the `applies_to` key. - [ x] For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the `.changelog` directory, specifying "aws-sdk-rust" in the `applies_to` key. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
awslabs/aws-sdk-rust#1234
Description
PR adds a new interceptor registered as part of the default retry plugin components that ensures a token bucket is always present and available to the retry strategy. The buckets are partitioned off the retry partition (which defaults to the service name and is already set by the default plugin). We use a
staticvariable in the runtime for this which means that token buckets can and will apply to every single client that uses the same retry partition. The implementation tries to avoid contention on this new global lock by only consulting it if the retry partition is overridden after client creation.For AWS SDK clients I've updated the default retry partition clients are created with to include the region when set. Now the default partition for a client will be
{service}-{region}(e.g.sts-us-west-2) rather than just the service name (e.g.sts). This partitioning is a little more granular and closer to what we want/expect as failures in one region should not cause throttling to another (and vice versa for success in one should not increase available quota in another).I also updated the implementation to follow the SEP a little more literally/closely as far as structure which fixes some subtle bugs. State is updated in one place and we ensure that the token bucket is always consulted (before the token bucket could be skipped in the case of adaptive retries returning a delay and the adaptive rate limit was updated in multiple branches).
Testing
Checklist
.changelogdirectory, specifying "client," "server," or both in theapplies_tokey..changelogdirectory, specifying "aws-sdk-rust" in theapplies_tokey.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.