Validate/plan throws "missing provider" errors when using aliases in module calls

[fatbasstard]

Terraform Version

Terraform 0.15.1

Terraform Configuration Files

Simple setup, create the following setup:

Create the following 3 files

modules/module/module1/main.tf
modules/module/module2/main.tf
main.tf

main.tf:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

module "test" {
  source = "./modules/module2"
}

modules/module1/main.tf

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      configuration_aliases = [aws.my_alias]
    }
  }
}

#provider "aws" {
#  alias = "my_alias"
#}

data "aws_caller_identity" "current" {}

data "aws_caller_identity" "current_alias" {
  provider = aws.my_alias
}

modules/module2/main.tf

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

# provider "aws" {}

module "test" {
  providers = {
    aws.my_alias = aws
  }

  source = "../module1"
}

Pay attention to the commented (empty) provider configuration.

Expected Behavior

Expect the terraform validate all modules (root, module1 and module2) to return a: Success! The configuration is valid. message

Since the AWS provider is specified in the required_providers, the empty provider configuration should be obsolete and the aliases are added to the configuration_aliases. (https://www.terraform.io/upgrade-guides/0-15.html#alternative-provider-configurations-within-modules)

Actual Behavior

Terraform validate (but also the plan) in module1 returns a:

│ Error: Provider configuration not present
│ 
│ To work with data.aws_caller_identity.current_alias its original provider configuration at provider["registry.terraform.io/hashicorp/aws"].my_alias is required, but it has been removed. This occurs when a provider configuration
│ is removed while objects created by that provider still exist in the state. Re-add the provider configuration to destroy data.aws_caller_identity.current_alias, after which you can remove the provider configuration again.

Terraform validate (but also the plan) in module2 returns a:

Error: missing provider provider["registry.terraform.io/hashicorp/aws"]

The Terraform validate in the root returns:

Error: missing provider module.test.provider["registry.terraform.io/hashicorp/aws"]

This error can be solved by adding (in this case by uncommenting) the empty provider configurations in module1 and module2.

But then the following warning is thrown from module2:

│ Warning: Empty provider configuration blocks are not required
│ 
│   on ../module1/main.tf line 10:
│   10: provider "aws" {
│ 
│ Remove the aws.my_alias provider block from module.test.

And the following warning is thrown from the root:

│ Warning: Empty provider configuration blocks are not required
│ 
│   on modules/module2/main.tf line 9:
│    9: provider "aws" {}
│ 
│ Remove the aws provider block from module.test.

Steps to Reproduce

Construct the setup above. In short, in all modules run:

1. terraform init
2. terraform validate / terraform plan

Additional Context

References

Somewhat related to the following issues:

• configuration_aliases in child module terraform validate fails: Provider configuration not present #28490
• TF 0.15 module with multiple openstack providers #28407
• Issue running terraform validate against a module using required_providers.[provider].configuration_aliases  #28567

[jbardin]

Hi @fatbasstard,

Thanks for filing the issue. There are a couple slightly different problems here. Running validate in the child module is not expected to work, as it does not declare its own providers (which is issue #28490).

The problem when running the commands from the root module here is that module2 does not have a local aws provider config, because the parent module did not provide it via the providers map in the module call. What happens when you add the empty provider block without passing in a provider configuration, is that a default config is created hidden within the module which is almost always incorrect (preventing this is the primary purpose of these new validations and configuration_aliases).

The resolution here is to be more explicit and pass in the desired provider to the first module as well

module "test" {
  source = "./modules/module2"
  providers = {
    aws = aws
  }
}

Just to be clear, the proposed configuration would not have worked consistently or correctly in older terraform releases, which is what these changes aimed to prevent. That unexpected default provider within the module was often a frequent cause of problems for users, but often worked silently until modules or provider configuration were changed.

I think we may be able to improve the validation here a bit to point out the missing provider. I also want to see if we could allow the original config you proposed to do what (I think) was intended, which is to pass the default aws provider into the 3rd module as an alias even if that was never truly possible previously.

[fatbasstard]

Hi @jbardin ,

Thanks for the quick reply. This sample code is a simplified version of the way we use a lot of "shared modules" to encapsulate logic (an example: https://github.com/schubergphilis/terraform-aws-mcaf-s3). So if I understand it correctly:

Validate does not work for standalone modules
This would be a shame because this is the mechanism we use to "check" our shared modules. Since you cannot do a plan on a standalone module this is the perfect way to make sure the Terraform code itself is correct. If this does not work it's a step back.

Always specify all the providers when calling a module
We were under the assumption that it was only necessary to specify a provider explicitly when an alias is used, e.g.:

  providers = {
    aws = aws.alias
  }

or

  providers = {
    aws.alias = aws
  }

And that the default provider configuration is always passed along. So you say that it's better to ALWAYS pass all provider configurations explicitly?

[jbardin]

You are correct that it is only necessary to explicitly pass in a configuration for aliased providers, but the configuration shown here is somewhat of an edge case, where the first module needs an explicit configuration in order to pass it along to the second module. In other words, because the nested module requires a configuration, the first module does as well.

I'm going to take a look at making this work internally, as I don't see anything technically wrong with allowing it, because it is not ambiguous that the default "aws" provider name within that module must be coming from the root module.

[aardvarq]

Same issue here. My modules simply won't work now with tf15.

Problem: I've upgraded from TF 0.14.10 to TF 0.15.1 and now my provider aliases don't seem to work. All of the providers are at the same version, including the AWS one I'm having an issue with.

Terraform and provider versions:

$ terraform -v
Terraform v0.15.1
on linux_amd64
+ provider registry.terraform.io/hashicorp/archive v2.1.0
+ provider registry.terraform.io/hashicorp/aws v3.38.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

I did apply the list of configuration_aliases to the terraform stanza, as was suggested in the upgrade documentation. It does not seem to help. Here is my configuration:

terraform {
  required_version = ">= 0.15"
  required_providers {
      aws  = { 
        source  = "hashicorp/aws"
        version = "~> 3.33"
        configuration_aliases = [ aws.root ]
    }   
  }
}

provider "aws" {
  alias               = "root"
  region              = "us-east-1"
  allowed_account_ids = ["xxxxxxxxxxxx"]
}

module "my_bucket" {
  providers = { 
    aws = aws.root
  }
  source = "./modules/s3"
  bucket_name = "my_bucket"
}

Result of a terraform init:

│ Warning: Provider aws is undefined
│ 
│   on stack.tf line 25, in module "my_bucket":
│   25:     aws = aws.root
│ 
│ Module module.my_bucket does not declare a provider named aws.
│ If you wish to specify a provider configuration for the module, add an entry
│ for aws in the required_providers block within the module.

What am I missing?

[jbardin]

Hi @aardvarq,

What you see here is only a warning, so that should still work as it did before. The warning states that the my_bucket module does not declare the aws provider name within the module, so adding a required_providers entry for aws within the module will remove the warning. Each module should have its own required_providers section to ensure that as modules are composed in different combinations, we can always correctly determine which providers should be used.

[aardvarq]

I'm still getting the warning when I add the required_providers

[aardvarq]

$ cat stack.tf
terraform {
  required_version = ">= 0.15"
  required_providers {
      aws  = {
      source  = "hashicorp/aws"
      version = "~> 3.33"
      configuration_aliases = [ aws.root ]
    }
  }
}

provider "aws" {
  alias               = "root"
  region              = "us-east-1"
}

module "my_bucket" {
  providers = {
    aws = aws.root
  }
  required_providers {
      aws  = {
      source  = "hashicorp/aws"
      version = "~> 3.33"
    }
  }
  source = "./modules/s3"
  bucket_name = "my_bucket"
}

[jbardin]

@aardvarq, the require_providers needs to be within the terraform block in your module, just like you have it for the root module. What you have here should produce the error Blocks of type "required_providers" are not expected here. when validating the configuration.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.