Destination BigQuery: Check should fail if we have insufficient permissions

[edgao]

from https://github.com/airbytehq/oncall/issues/640

destinatoin-bigquery failed with:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST https://www.googleapis.com/bigquery/v2/projects/<redacted>
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Access Denied: Dataset <redacted>: Permission bigquery.tables.create denied on dataset <redacted> (or it may not exist).",
    "reason" : "accessDenied"
  } ],
  "message" : "Access Denied: Dataset <redacted>: Permission bigquery.tables.create denied on dataset <redacted> (or it may not exist).",
  "status" : "PERMISSION_DENIED"
}
        at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
        at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
        at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
        at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.create(HttpBigQueryRpc.java:185)
        ... 16 more

Normalization also failed:

Database Error
  Access Denied: Project <redacted>: User does not have bigquery.datasets.create permission in project <redacted>.,retryable=<null>,timestamp=1663764411243

[grishick]

Hey team! Please add your planning poker estimate with Zenhub @edgao @ryankfu @suhomud @etsybaev

[edgao]

related: check should probably fail if we don't have permissions to create datasets https://github.com/airbytehq/oncall/issues/911, or if the user has gcs staging configured but we can't read from the bucket #17327 (comment)

would also be good to check that check checks for being able to write to the GCS bucket, but I'm not aware of this causing false positives.

[killthekitten]

@edgao not sure if in case of #17327 it was write or read permissions. I would assume read, because the exception happens during copyIntoTmpTableFromStage in my case:

2022-10-27 20:27:13 �[43mdestination�[0m > 	at io.airbyte.integrations.destination.bigquery.BigQueryGcsOperations.copyIntoTmpTableFromStage(BigQueryGcsOperations.java:124) ~[io.airbyte.airbyte-integrations.connectors-destination-bigquery-0.40.3.jar:?]

[edgao]

whoops, good callout - I edited my comment to reflect that.

[etsybaev]

Blocked. Need to wait for this PR first #18560

[akashkulk]

@etsybaev I broke the PR into smaller PRs to unblock. I merged this PR : #18989
Which should unblock you

[killthekitten]

Hey @etsybaev, thanks for merging the #18554! I think you shouldn't have closed this issue though, as your PR doesn't fully resolve the issue with read/write permissions on the GCS bucket as mentioned by @edgao and me in #17327 (check this comment).

Do you both think we could re-open this one?