Surely delete temporary credentials files of Google Cloud

[yu-iskw]

Is your feature request related to a problem? Please describe.
Take the BigQuery connector, for instance. The set_google_credentials function can be called multiple times during ingesting BigQuery metadata, if we pass raw service account key information to a database source. In reality, the close method of BigQuerySource tries to delete the temporary credentials file. However, temp_credentials property isn’t set anywhere, if I am correct. Aside from that, the get_connection function creates a temporary credentials file every time. I am doubting created credentials files remains. If I am correct, I would like to know how to completely delete created temporary files.

Specifically, the issue happens with the configuration.

1. We pass raw credentials to the BigQuery connector,
2. We pass multiple project IDs

In my opinion, we can have the similar issue on other source connectors, as we don't have an abstract feature or function to surely delete used temporary credentials file. To avoid the situation, we can also improve functions to deal with connections.

Describe the solution you'd like
There are a couple of ideas to surely delete temporary credentials files:

• Plan A: We authenticat only one time, when initializing an object of BigQuerySource.
• Plan B: We improve functions around connections like get_connection by authenticating and removing credentials on the connection side. By doing so, we may have to redesign the way to deal with connections.

Describe alternatives you've considered
N/A

Additional context
N/A

[yu-iskw]

I havne't completely design how to enhance the functions around Google Cloud authorization. But, the official python SDK offers a couple of functions to safely refresh or revoke credentials. We can probably take advantage of them.

from google.oauth2.credentials import Credentials
from google.auth.transport.requests import Request
from google.oauth2 import service_account

# Create a credentials object from a service account key file
creds = service_account.Credentials.from_service_account_file('path/to/keyfile.json')

# Make a request to revoke the credentials
creds.revoke(Request())

[pmbrull]

let's clean this up in the close call for the workflow

[vanshika18]

I will take care of this issue.

[TeddyCr]

This needs to be properly handled for multi project ingestion