Add opt-in behaviour to allow google_storage_bucket to be used without requiring Compute API permissions

[SarahFrench]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Assuming the identity Terraform is using is a service account that has Storage API permissions but no Compute API permissions, when you apply this config:

resource "google_storage_bucket" "bucket" {
  name                        = "my-bucket-20240202"
  location                    = "us-central1"
  force_destroy               = true
  public_access_prevention    = "enforced"
  uniform_bucket_level_access = true

}

data "google_storage_bucket" "bucket" {
  name = google_storage_bucket.bucket.name
}

You will get this error:

"error": {
    "code": 403,
    "message": "Required 'compute.projects.get' permission for 'projects/1234567890'",

This is because the data source receives a project number from the Storage API but uses the Compute API to convert it to a project id using an API call like GET /compute/v1/projects/1234567890

If you set a provider default project then the API call and the permission error won't occur, but then you'll potentially run into this bug: #17165

The current behaviour of the datasource to convert the project number to an id makes sense, as most Google APIs require inputs to be project ids now. This means the current behaviour helps users pass data between the data source and other parts of their configuration and know that the data is compatible with those destinations.

I propose an opt-in behaviour to avoid use of the Compute API, for users who have restrictions on granting IAM permissions to Terraform identities and are willing to accept the trade-off of the project attribute being different.

New or Affected Resource(s)

• data.google_storage_bucket

Potential Terraform Configuration

Will need input on a better name than avoid_compute_api !

# No provider default project set
provider "google" {}

resource "google_storage_bucket" "my-bucket" {
  project       = "sarahs-super-duper-project"
  name          = "sarahs-super-duper-bucket"
  location      = "US"
  force_destroy = true
  public_access_prevention = "enforced"
}

data "google_storage_bucket" "my-bucket" {
  avoid_compute_api = true # opting into the new behaviour
  name = google_storage_bucket.sarahs-bucket.name
}

# The data source above has received project number 1234567890 from the Storage API, uses it unchanged
output "project_from_datasource" {
  value = data.google_storage_bucket.my-bucket.project # value will be a project number e.g. "1234567890"
}

References

• This bug is closely related to this feature request: Incorrect project attribute value on google_storage_bucket data source when bucket belongs to a different project than provider default #17165

b/324044284

[SarahFrench]

I've assigned this to myself as it's opened on behalf of a customer.

[SarahFrench]

Notes from triage: instead of having the project attribute be either a string of a project number or project id, have two separate fields. New field would be project number, avoiding breaking project field on the resource.

Could add a project field to the data source - take in input from the user and assume it's the correct value. This mirrors how import for the resource behaves - whatever project is provided as input to the import command is taken as true.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.