Plugin framework version of provider configuration code doesn't handle unknown values the same way as the SDK

[SarahFrench]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

N/A

v4.60.2 of TPG/TPGB onwards affected (versions that are muxed)

Affected Resource(s)

N/A ; problem is with provider configuration. google_*

Terraform Configuration Files

Example of a provider configured with a value that can be unknown at apply time:

provider "google-beta" {
  project     = "foo"
  credentials = base64decode(google_service_account_key.terraform_service_account.private_key)
}

See example config (originally shared here) : files.zip

For more info see:

• Initial comment describing the issue: Provider configuration values supplied as empty strings/zero-values are no longer treated like null values (muxed provider, v4.60.2+) #14255 (comment)
• Follow up message with config for reproducing the issue: Provider configuration values supplied as empty strings/zero-values are no longer treated like null values (muxed provider, v4.60.2+) #14255 (comment)

Expected Behavior

In one apply you can provision a new GCP project containing a service account using TPG and also provision Firebase resources using TPGB, which is configured using a service account key from the service account being made in the same step.

Actual Behavior

The plan step fails due to an error raised from the TPGB provider:

╷
│ Error: unable to parse credentials
│ 
│   with provider["registry.terraform.io/hashicorp/google-beta"],
│   on main.tf line 21, in provider "google-beta":
│   21: provider "google-beta" {
│ 
│ unexpected end of JSON input
╵

Cause of issue

The above fails because the value passed in for credentials in the beta provider's configuration is unknown at plan time; the service account doesn't exist yet, let alone a key to use it.

The SDK handles unknown values differently to the plugin framework. You can use the .GetOk function to get a value and check if it's a set value vs a zero value. If a value is unknown in an SDK schema then GetOk returns the zero value and false, to indicate it's not gettable. In the SDK provider config code we only set the credentials in the Config struct if ok = true, and otherwise the Config struct will contain the zero value "". As a result, the code do comparisons like != "". Here is the code relevant to the error in the reported issue in the SDK version of the provider config code. Note it is controlled by if c.Credentials != "" {

terraform-provider-google/google/transport/config.go

Lines 1775 to 1798 in e520fc3

	if c.Credentials != "" {
	contents, _, err := PathOrContents(c.Credentials)
	if err != nil {
	return googleoauth.Credentials{}, fmt.Errorf("error loading credentials: %s", err)
	}
	if c.ImpersonateServiceAccount != "" && !initialCredentialsOnly {
	opts := []option.ClientOption{option.WithCredentialsJSON([]byte(contents)), option.ImpersonateCredentials(c.ImpersonateServiceAccount, c.ImpersonateServiceAccountDelegates...), option.WithScopes(clientScopes...)}
	creds, err := transport.Creds(context.TODO(), opts...)
	if err != nil {
	return googleoauth.Credentials{}, err
	}
	return *creds, nil
	}
	creds, err := googleoauth.CredentialsFromJSON(c.Context, []byte(contents), clientScopes...)
	if err != nil {
	return googleoauth.Credentials{}, fmt.Errorf("unable to parse credentials from '%s': %s", contents, err)
	}
	log.Printf("[INFO] Authenticating using configured Google JSON 'credentials'...")
	log.Printf("[INFO] -- Scopes: %s", clientScopes)
	return *creds, nil
	}

In the plugin framework version of this code the equivalent if condition is if !data.Credentials.IsNull(). This handles if the value is Null but doesn't handle if it's Unknown.

In the scenario that raised this issue the value is not Null (it is set in the config) but it is Unknown.

The solution to this issue is to update this line:

https://github.com/hashicorp/terraform-provider-google/blob/e520fc30aaa4141dc037ad345a444c607f009770/google/framework_config.go#LL1344C2-L1344C2

With

if !data.Credentials.IsNull() && !data.Credentials.IsUnknown() {

[SarahFrench]

In this case there's an issue with unknown values of credentials, but we should investigate handling of unknown values for other arguments in the provider configuration. I think in most cases we would use them together like

if !data.Foobar.IsNull() && !data.Foobar.IsUnknown() {
   //do thing that would fail on an unknown value
}

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.