New File Hashing Errors In Terraform 0.12

[bflad]

Terraform Version

terraform 0.12-alpha4

Terraform Configuration Files

# https://www.terraform.io/docs/providers/aws/r/lambda_function.html
resource "aws_lambda_function" "example" {
  # ... other configuration ...

  filename         = "somefile.zip"
  source_code_hash = "${base64sha256(file("somefile.zip"))}"
}

# https://www.terraform.io/docs/providers/aws/r/s3_bucket_object.html
resource "aws_s3_bucket_object" "example" {
  # ... other configuration ...

  source = "somefile.zip"
  etag   = "${md5(file("somefile.zip"))}"
}

Expected Behavior

No differences between upgrading Terraform 0.11 and Terraform 0.12 or the breaking changes are documented in the Terraform 0.12 upgrade guide with a recommended alternative to achieve the equivalent file hash value. If new file hashing functions are introduced, it would be nice if this was handled automatically by terraform 0.12upgrade or if these new functions could be backported to Terraform 0.11 to lower the Terraform 0.12 barrier to entry.

Actual Behavior

When the files have binary content:

config is invalid: Error in function call: Call to function "file" failed: contents of somefile.zip are not valid UTF-8; to read arbitrary bytes, use the filebase64 function instead.

Using filebase64() would not generate an equivalent file hash.

References

• https://www.terraform.io/docs/providers/aws/r/api_gateway_authorizer.html
• https://www.terraform.io/docs/providers/aws/r/api_gateway_integration.html
• https://www.terraform.io/docs/providers/aws/r/lambda_function.html
• https://www.terraform.io/docs/providers/aws/r/lambda_layer_version.html
• https://www.terraform.io/docs/providers/aws/r/s3_bucket_object.html

[apparentlymart]

After weighing a few different options here, I decided to add some specialized functions for hashing the contents of files, in #20115. That PR also includes a new rule for the 0.12upgrade tool which should fix use of the documented idiom in existing configurations:

base64sha256(file("somefile.zip")) -> filebase64sha256("somefile.zip")

(I wrote a little about the other options I considered in the PR description, if you're interested.)

This of course doesn't fully address the problem because the existing examples in the provider docs (and, I assume, also in some provider tests) will need to be updated to reflect the new pattern.

I'm sensitive to the fact that this makes it awkward to write documentation that works for both 0.12 and 0.11. In principle we could backport these functions into a 0.11 release, but since functions work very differently in prior versions that would amount to repeating all of the work from that PR again in a different package and against a different API, so I'd like to first see if we can get away with a documentation-only solution where we update the examples to show the new usage but then put a ~> callout under the example noting that versions prior to 0.12 require a different approach.

(Part of my rationale for that is that even if we did backport, we'd still need that callout explaining that versions of Terraform prior to 0.11.x need a different usage -- since not everyone will be on the latest minor -- and so it's perhaps simpler just to make the crease for that change be on the major release which is hopefully easier for users to remember.)

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.