ephemeral/password: Introduce a new ephemeral password resource

[austinvalle]

This PR is currently in draft as an ephemeral random password is not particularly useful until managed resources are updated to allow interaction with ephemeral data (via write-only attributes, earliest that will be available is Terraform v1.11.0)

Closes #639

This PR introduces a new random_password ephemeral resource which is functionally similar to the managed resource, minus the keepers and id attributes. Once write-only attribute are introduced in Terraform v1.11, this resource can be used to generate a random password while avoiding storing that password in state.

Notes

• keepers are not relevant because ephemeral resources do not produce a plan and are never stored in state. Triggers for producing an ephemeral password will eventually be the responsibility of the module author.
• id is not relevant because the new testing framework doesn't require this information and the practitioner doesn't benefit from the duplication with result.

[ktham]

With this, the password string would no longer be stored in Terraform state (which is exactly what we'd like to see), but would each subsequent Terraform plan result in the ephemeral random_password generating new values? Or would it be such that on subsequent plans, the result attribute will be null when being passed into another resource's write-only attribute so that we can avoid setting a new password on each Terraform plan/apply run?

[austinvalle]

Hey @ktham 👋🏻 !

So this ephemeral resource (random_password) will always generate a different value for every plan/apply it is included in the configuration. There's no real context for the ephemeral resource itself to know if the password has already been generated. (since it's not stored in state!)

Ultimately, it will be up to the consuming managed resource implementation (for example, an aws_db_instance) to define when the write-only attribute will be consumed, which likely will need to be a separate non-write-only attribute to allow the practitioner to explicitly indicate they want the provider to use the write-only value.

A hypothetical example:

ephemeral "random_password" "password" {
  length           = 16
  special          = true
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

resource "aws_db_instance" "example" {
  instance_class    = "db.t3.micro"
  allocated_storage = 64
  engine            = "mysql"
  username          = "user_one"

  # These attributes don't currently exist on the aws_db_instance resource
  password_version   = "v1" # This attribute could be changed to "trigger" the usage of writeonly_password
  writeonly_password = ephemeral.random_password.password.result
}

Another way a consuming managed resource could implement that "trigger" would be similar to the keepers pattern that some of the random provider resources use: https://registry.terraform.io/providers/hashicorp/random/latest/docs#resource-keepers

---

Additionally, write-only attribute cannot produce a diff (since their prior state is always null and their planned state will always be null), so the default planning behavior for a write-only attribute where the configuration doesn't change should be a no-op, unless the provider explicitly implements alternative planning logic.

Hopefully that helps!

[ktham]

Hi, sorry, I was thinking about it more and I have another followup question. I understand that it would be up to the provider to define how a write-only attribute is consumed, but in the situation where we are trying to have multiple resources consume the same ephemeral value, I worry that there is a possibility for the values to get into an inconsistent state.

I would expect all resources consuming the ephemeral value into its write-only attribute to all consume that ephemeral value, or for none of them to consume the value.

For example, say I also have a aws_secretsmanager_secret_version resource where I'd want to store the random_password value so that my application can then successfully authenticate to the database. I want to ensure that in all situations, that after a Terraform plan, both aws_db_instance.writeonly_password and aws_secretsmanager_secret_version.writeonly_secret_string would have identical values.

I worry that if a Terraform plan fails for whatever reason and we attempt to re-run the plan/apply, that one of the resources would consume a new random_password value, and one might not.

Here might be that example configuration:

ephemeral "random_password" "password" {
  length           = 16
  special          = true
  override_special = "!#$%&*()-_=+[]{}<>:?"
}

resource "aws_db_instance" "example" {
  # ...
  # These attributes don't currently exist on the aws_db_instance resource
  password_version   = "v1" # This attribute could be changed to "trigger" the usage of writeonly_password
  writeonly_password = ephemeral.random_password.password.result
}

resource "aws_secretsmanager_secret" "example" {
  name = "example"
}

resource "aws_secretsmanager_secret_version" "example" {
  secret_id = aws_secretsmanager_secret.example.id
  
  secret_string_version = "v1" # This attribute could be changed to "trigger" the usage of writeonly_secret_string
  writeonly_secret_string = ephemeral.random_password.password.result
}

The resolution would be to increment both password_version and secret_string_version to a new string, however, there doesn't seem to be a guardrail to prevent a Terraform plan to ensure we don't apply a terraform plan where only one of the two write only attributes gets updated. Just wondering, what are your thoughts?

[austinvalle]

No worries, I don't think I'm the foremost authority on this topic but I can certainly give you some thoughts!

The problem you're describing is valid and also an unfortunate side effect of trying to fit an imperative problem (rotating a password) into a declarative solution (defining the desired state of infrastructure as code). In an ideal world, Terraform would provider an alternative mechanism for you to declaratively describe an action you can execute, like rotating a password. So you can define the "steps" needed to accomplish that (A: generate new password, B: store in AWS secret manager, C: change db instance, etc). No such solution exists for that ATM.

In this case, ephemeral resources and write-only attributes are only solving a portion of the problem (not storing your password in the state file), but the latter half of the problem isn't solved in Terraform (the imperative part, defining steps, etc.)

I think with what will be available in Terraform at 1.11 you can still find a solution, but as you noted, it's always going to depend on what the resources you're using and what they support. For example, a managed resource may not even support a write-only attribute, making the usage of an epehemeral block invalid.

---

Back to your example, here are some thoughts:

I worry that if a Terraform plan fails for whatever reason and we attempt to re-run the plan/apply, that one of the resources would consume a new random_password value, and one might not.

I would say that rather than consuming the random_password ephemeral value, you should consume the "source of truth" of your password, whether that be Vault, AWS secret manager, Azure Key Vault, etc. So in your configuration, the rotation of your password would explicitly be between: random_password => <secret managed resource, write-only attribute>. Everything else in your configuration would then consume a different ephemeral resource provided by wherever your secret is stored.

---

The resolution would be to increment both password_version and secret_string_version to a new string, however, there doesn't seem to be a guardrail to prevent a Terraform plan to ensure we don't apply a terraform plan where only one of the two write only attributes gets updated. Just wondering, what are your thoughts?

Without a way to define a list of imperative actions in Terraform, the only options you really have are to connect the attributes via configuration. How you do that would depend on how each provider's write-only attribute is triggered, in your case, they are all versioned with a string so you could share a variable (such as a local) for that.

[ktham]

Hi @austinvalle , thank you for sharing your thoughts!! Indeed, I agree that Terraform hasn't been the ideal tool for secrets management, which is why excited and hoping that with the implementation of write-only attributes here, that TF could potentially be acceptable and become "good enough" for this use-case. (Longer term, we're hoping to reduce the amount of static credentials that we have in our system.)

I would say that rather than consuming the random_password ephemeral value, you should consume the "source of truth" of your password, whether that be Vault, AWS secret manager, Azure Key Vault, etc. So in your configuration, the rotation of your password would explicitly be between: random_password => <secret managed resource, write-only attribute>. Everything else in your configuration would then consume a different ephemeral resource provided by wherever your secret is stored.

I see, given that write-only values cannot be read, we would then need to consume the TF-managed secret using a separate ephemeral resource for it. In which case, it sounds like that cannot co-exist within the same Terraform configuration, so in other words, in your suggestion, does it mean that the random_password => <secret managed resource, write-only attribute> resources need to live in a separate Terraform configuration from the resources that would then consume it, such as aws_db_instance?

[ktham]

Ultimately, it will be up to the consuming managed resource implementation (for example, an aws_db_instance) to define when the write-only attribute will be consumed, which likely will need to be a separate non-write-only attribute to allow the practitioner to explicitly indicate they want the provider to use the write-only value.

Got it, thank you for explaining :) - aws_db_instance is precisely one of the resources that I'm waiting for a potential implementation of write-only attribute for the password field, and regarding tracking the change of a non-writeonly value to consume the write-only makes sense, thank you for the example, we'll wait for the final version of the implementation and will be eager to start using it. Thanks!

(edit: woops, sorry, I meant to reply in the thread above)