Provider produced inconsistent result in bigquery_dataset_access

[farnoy]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.24

• provider.google v3.18.0

Affected Resource(s)

• google_bigquery_dataset_access

Terraform Configuration Files

resource "google_bigquery_dataset" "private_dataset" {
  project = local.project
  dataset_id = "private"
}

resource "google_bigquery_dataset_access" "private_dataset_access" {
  project = local.project
  dataset_id = google_bigquery_dataset.private_dataset.dataset_id
  role = "roles/bigquery.dataOwner"
  for_each = local.global_editors
  user_by_email = each.value
}

Debug Output

I don't feel comfortable pasting the whole thing, here's a snippet:

2020/04/23 14:20:11 [WARN] Provider "registry.terraform.io/-/google" produced an invalid plan for google_bigquery_dataset.private_dataset, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .timeouts: planned for existence but config wants absense
2020/04/23 14:20:11 [DEBUG] Resource instance state not found for node "google_bigquery_dataset_access.private_dataset_access[\"jakub.okonski\"]", instance google_bigquery_dataset_access.private_dataset_access["jakub.okonski"]

Panic Output

No panic

Expected Behavior

Permissions were created, terraform state was saved.

Actual Behavior

Permissions were created and are visible in the Cloud Console, but terraform fails to save the new state and will try to create them again on every apply.

Steps to Reproduce

1. create the dataset named private in Cloud Console
2. $ terraform import google_bigquery_dataset.private_dataset projects/xxx/datasets/private
3. $ terraform apply

Important Factoids

Authenticating as a user through gcloud SDK, not sure if creating a dataset from the UI has any effect compared to creating the same dataset with terraform.

References

[venkykuberan]

If you see default roles are trying to be removed every time you apply, add the following block should help. Please close the issue if it works

lifecycle {
        ignore_changes = [access]
    } ```

[farnoy]

I'm not sure what you mean @venkykuberan, the role bindings are being added with apply, but it does not change the terraform state, which means that the next time I run terraform, it's going to attempt (and fail) doing the same thing.

1. I run terraform apply, it executes the plan:

   Terraform will perform the following actions:
   
    # google_bigquery_dataset_access.private_dataset_access["jakub.okonski"] will be created
     + resource "google_bigquery_dataset_access" "private_dataset_access" {
         + dataset_id    = "private"
         + id            = (known after apply)
         + project       = "xxxx"
         + role          = "roles/bigquery.dataOwner"
         + user_by_email = "jakub.okonski@..."
       }
   

2. It fails with:

   Error: Provider produced inconsistent result after apply
   When applying changes to
   google_bigquery_dataset_access.private_dataset_access["jakub.okonski"],
   provider "registry.terraform.io/-/google" produced an unexpected new value for
   was present, but now absent.
   This is a bug in the provider, which should be reported in the provider's own
   issue tracker.

3. I refresh the Cloud Console and I can see the new permissions were added

4. $ terraform state list doesn't have this resource tracked

5. $ terraform apply tries to create it again and fails for the 2nd time.

The ignore_changes fails to run due to:

Error: Unsupported attribute

  on main.tf line 44, in resource "google_bigquery_dataset_access" "private_dataset_access":
  44:     ignore_changes = [access]

This object has no argument, nested block, or exported attribute named
"access".

[venkykuberan]

I see the issue, role format being used is crashing the provider. Can you use the following format as per the doc

resource "google_bigquery_dataset_access" "access" {
  dataset_id = google_bigquery_dataset.dataset.dataset_id
  #   role          = "roles/bigquery.dataOwner"
  role          = "OWNER"
  user_by_email = "[email protected]"
}

[farnoy]

It worked @venkykuberan, I assume this is a legitimate bug in the tf provider, correct? Google's API docs mention that they accept both (and these uppercase, single-word variants are deprecated).

[venkykuberan]

Agreed, provider should handle it. I will keep this issue open and move it to code change bucket. Meantime using the Upper case role would get you going. Thank you for filing the issue

[emilymye]

From docs for google_bigquery_dataset_access.role:

Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts, and will show a diff post-create.

This isn't quite accurate, since it's showing a provider error instead of a diff - we created the object and attempt to read it from a list of access objects on the dataset. Since we have to identify access objects by its fields (i.e. (user, role, ...), when the API changes one of the identifier fields (roles/datasetOwner --> OWNER), Terraform no longer can recognize it.

I'm not sure how we want to resolve this - I think that when this resource was implemented, we wanted to avoid having hardcoded (predefined role X => primative Role Y) relationships in Terraform since that depends on the upstream BigQuery API behavior not changing (which it has as they add new roles). On the other hand, this appears as an error, which can't be overridden and definitely should be fixed.

@farnoy I think the best workaround right now is to not use the predefined role (roles/*) while we figure out a fix. My inclination is to reject predefined roles that we know will cause this issue. The API allows both but it isn't behaving declaratively: it doesn't expect the user to care whether the user-provided resource and final API state are the same, but Terraform very much does. The closer we stay to the actual API response, the better. @danawillow thoughts?

[danawillow]

If we reject predefined roles from here and not from the bigquery_dataset resource, then we have a weird inconsistency between the two. But if we reject predefined roles from there, it's a breaking change. I'd lean towards just hardcoding the mapping so we can have a consistent fix now instead of having to wait for 4.0.0, and maybe also safeguarding against the possibility of a new primitive role being added by checking for all-caps (though that seems like a pretty weird scenario)

It's going to be a bit tricky to figure out what to write to state if we want to be mindful of #4328, but if it's not super obvious we can probably just wait until we actually need to fix that before we bother.

[27Bslash6]

Seeing the same behaviour with 'OWNER' role:

* provider.google: version = "~> 3.19"
...
# google_bigquery_dataset_access.owner will be created
   + resource "google_bigquery_dataset_access" "owner" {
       + dataset_id    = "example_dev_reporting"
       + id            = (known after apply)
       + project       = (known after apply)
       + role          = "OWNER"
       + user_by_email = "[email protected]"
     }
 Plan: 1 to add, 0 to change, 0 to destroy.
   + google_bigquery_dataset_access.owner

 google_bigquery_dataset_access.owner: Creating...
 Error: Provider produced inconsistent result after apply
 When applying changes to google_bigquery_dataset_access.owner, provider
 "google" produced an unexpected new value for was present, but now absent.
 This is a bug in the provider, which should be reported in the provider's own
 issue tracker.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!