Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add operation retry for exceeded quota group OperationReadGroup #3077

Conversation

modular-magician
Copy link
Collaborator

Fixes hashicorp/terraform-provider-google#8655

The error we need to retry on:

{
  "error": {
    "code": 403,
    "message": "Quota exceeded for quota group 'OperationReadGroup' and limit 'Operation read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:<>'.",
    "errors": [
      {
        "message": "Quota exceeded for quota group 'OperationReadGroup' and limit 'Operation read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:<>'.",
        "domain": "usageLimits",
        "reason": "rateLimitExceeded"
      }
    ],
    "status": "PERMISSION_DENIED"
  }

Currently only GCE has a quota group called OperationReadGroup, but the compute operations use CommonRefreshFunc. Rather than rewrite compute operations to use a non-common refresh function, I just added another retry predicate to the existing one.

Tested by spamming operation reads. Sorry GCE SRE's...

Release Note Template for Downstream PRs (will be copied)

compute: fixed an issue where exceeding the operation rate limit would fail without retrying

Derived from GoogleCloudPlatform/magic-modules#4599

@ghost ghost added size/xs labels Mar 22, 2021
@modular-magician modular-magician merged commit f953079 into hashicorp:master Mar 22, 2021
@modular-magician modular-magician deleted the downstream-pr-4d52cee4652b23b7f8c58d837267ca06e8d99cde branch November 16, 2024 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Terraform state leaks when the GCE Operation API(s) rate limit is exceeded
1 participant