WAFv2: Add support for custom request and responses to rules

[andyalm]

This adds support for custom_request_handling on allow and count actions, and custom_response on block actions.

Notes:

• I'm a first time contributor new to golang, so am learning as I go. Will accept suggestions/feedback.
• I have added two acceptance tests to cover these changes. One for custom_request_handling and one for custom_response.

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #18754

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAwsWafv2WebACL_.+'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAwsWafv2WebACL_.+ -timeout 180m
=== RUN   TestAccAwsWafv2WebACL_basic
=== PAUSE TestAccAwsWafv2WebACL_basic
=== RUN   TestAccAwsWafv2WebACL_updateRule
=== PAUSE TestAccAwsWafv2WebACL_updateRule
=== RUN   TestAccAwsWafv2WebACL_UpdateRuleProperties
=== PAUSE TestAccAwsWafv2WebACL_UpdateRuleProperties
=== RUN   TestAccAwsWafv2WebACL_ChangeNameForceNew
=== PAUSE TestAccAwsWafv2WebACL_ChangeNameForceNew
=== RUN   TestAccAwsWafv2WebACL_Disappears
=== PAUSE TestAccAwsWafv2WebACL_Disappears
=== RUN   TestAccAwsWafv2WebACL_ManagedRuleGroupStatement
=== PAUSE TestAccAwsWafv2WebACL_ManagedRuleGroupStatement
=== RUN   TestAccAwsWafv2WebACL_Minimal
=== PAUSE TestAccAwsWafv2WebACL_Minimal
=== RUN   TestAccAwsWafv2WebACL_RateBasedStatement
=== PAUSE TestAccAwsWafv2WebACL_RateBasedStatement
=== RUN   TestAccAwsWafv2WebACL_GeoMatchStatement
=== PAUSE TestAccAwsWafv2WebACL_GeoMatchStatement
=== RUN   TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig
=== PAUSE TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig
=== RUN   TestAccAwsWafv2WebACL_IPSetReferenceStatement
=== PAUSE TestAccAwsWafv2WebACL_IPSetReferenceStatement
=== RUN   TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig
=== PAUSE TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig
=== RUN   TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig
=== PAUSE TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig
=== RUN   TestAccAwsWafv2WebACL_RuleGroupReferenceStatement
=== PAUSE TestAccAwsWafv2WebACL_RuleGroupReferenceStatement
=== RUN   TestAccAwsWafv2WebACL_CustomRequestHandling
=== PAUSE TestAccAwsWafv2WebACL_CustomRequestHandling
=== RUN   TestAccAwsWafv2WebACL_CustomResponse
=== PAUSE TestAccAwsWafv2WebACL_CustomResponse
=== RUN   TestAccAwsWafv2WebACL_Tags
=== PAUSE TestAccAwsWafv2WebACL_Tags
=== RUN   TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements
=== PAUSE TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements
=== RUN   TestAccAwsWafv2WebACL_MaxNestedOperatorStatements
=== PAUSE TestAccAwsWafv2WebACL_MaxNestedOperatorStatements
=== CONT  TestAccAwsWafv2WebACL_basic
=== CONT  TestAccAwsWafv2WebACL_IPSetReferenceStatement
=== CONT  TestAccAwsWafv2WebACL_MaxNestedOperatorStatements
=== CONT  TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements
=== CONT  TestAccAwsWafv2WebACL_Tags
=== CONT  TestAccAwsWafv2WebACL_CustomResponse
=== CONT  TestAccAwsWafv2WebACL_CustomRequestHandling
=== CONT  TestAccAwsWafv2WebACL_UpdateRuleProperties
=== CONT  TestAccAwsWafv2WebACL_ManagedRuleGroupStatement
=== CONT  TestAccAwsWafv2WebACL_updateRule
=== CONT  TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig
=== CONT  TestAccAwsWafv2WebACL_GeoMatchStatement
=== CONT  TestAccAwsWafv2WebACL_RateBasedStatement
=== CONT  TestAccAwsWafv2WebACL_Minimal
=== CONT  TestAccAwsWafv2WebACL_ChangeNameForceNew
=== CONT  TestAccAwsWafv2WebACL_Disappears
=== CONT  TestAccAwsWafv2WebACL_RuleGroupReferenceStatement
=== CONT  TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig
=== CONT  TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig
--- PASS: TestAccAwsWafv2WebACL_Minimal (68.69s)
--- PASS: TestAccAwsWafv2WebACL_Disappears (76.45s)
--- PASS: TestAccAwsWafv2WebACL_ChangeNameForceNew (92.42s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements (94.38s)
--- PASS: TestAccAwsWafv2WebACL_RuleGroupReferenceStatement (97.20s)
--- PASS: TestAccAwsWafv2WebACL_ManagedRuleGroupStatement (108.29s)
--- PASS: TestAccAwsWafv2WebACL_CustomResponse (109.22s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedOperatorStatements (114.64s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig (115.85s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement (132.15s)
--- PASS: TestAccAwsWafv2WebACL_Tags (133.90s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement (141.10s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement (142.37s)
--- PASS: TestAccAwsWafv2WebACL_CustomRequestHandling (143.13s)
--- PASS: TestAccAwsWafv2WebACL_basic (144.91s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig (145.64s)
--- PASS: TestAccAwsWafv2WebACL_updateRule (154.38s)
--- PASS: TestAccAwsWafv2WebACL_UpdateRuleProperties (163.60s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig (258.36s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       259.649s

[github-actions]

Welcome @andyalm 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

[andyalm]

I've removed the WIP as I believe I have gone through all of the PR checklist items. This is ready for a full review.

[andyalm]

@anGie44 I could use some help understanding these linting errors.

[anGie44]

Hi @andyalm , sure thing!

The linter check is using terrafmt which expects certain alignment in configurations, similar to what occurs when you run terraform fmt on a .tf file. To get the configuration files fixed up, you can locally run make tools from the root of the terraform-provider-aws repo, and run terrafmt fmt ./aws/resource_aws_wafv2_web_acl_test.go --check --fmtcompat . This should catch the second of the 2 errors. The first error is related to the use of %[2]s within the action {} block. Since the string doesn't get escaped, I'd recommend creating 2 separate configs (while a bit redundant) for the 2 different actions (allow {} and count {}).

One quick note on this PR: since the custom request/response attributes being added here are also supported in the RuleGroup resource, I would just update the title to reflect the affected resource and if interested, an additional PR can build on this one (to re-use the shared schemas) to add support w/in the RuleGroup.

[andyalm]

@anGie44 I had not noticed that RuleGroup also supports this, but of course, it makes sense. I would be happy to add that to this PR if you prefer (with added tests). Or would you prefer that be handled as a separate PR?

[anGie44]

@andyalm , yep let's add that to this PR . I was just skimming over the changes, and since the expand* methods are updated in wafv2_helper.go, which are then shared by the RuleGroup resource, my guess is that if we run those tests they'd fail as the schema of that resource needs to be updated as well.

[andyalm]

Great. I'll get working on that and will expand my acceptance test run to include all Wafv2 tests, not just the WebACL tests.

[anGie44]

Awesome thank you @andyalm ! I would definitely add tests similar to those you've added in the webACL resource to the aws_wafv2_rule_group_test.go file to make sure the feature is supported. in my experience, the API docs sometimes say features are shared by both resources but in practice, this may not be entirely true. I realized the tests in the RuleGroup resource would actually run without a problem, just the added support would go unused 😆

[anGie44]

Hi @andyalm , overall this PR looks great! Just some updates needed around the conventions around the flatten/expand methods and a reminder to update the .changelog/19415.txt file with a reference to the wafv2_rule_group resource and an update to the resource documentation files to reflect the new definitions for the actions updated in the schema.

Output of acceptance tests:

--- PASS: TestAccAwsWafv2RuleGroup_RegexPatternSetReferenceStatement (53.01s)
--- PASS: TestAccAwsWafv2RuleGroup_Minimal (58.29s)
--- PASS: TestAccAwsWafv2RuleGroup_IpSetReferenceStatement (60.71s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction_CustomResponse (77.46s)
--- PASS: TestAccAwsWafv2RuleGroup_basic (84.12s)
--- PASS: TestAccAwsWafv2RuleGroup_Disappears (84.10s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeNameForceNew (118.01s)
--- PASS: TestAccAwsWafv2RuleGroup_SizeConstraintStatement (128.95s)
--- PASS: TestAccAwsWafv2RuleGroup_GeoMatchStatement (132.14s)
--- PASS: TestAccAwsWafv2RuleGroup_ByteMatchStatement (136.60s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeCapacityForceNew (144.82s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeMetricNameForceNew (150.11s)
--- PASS: TestAccAwsWafv2RuleGroup_updateRule (158.37s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction (165.79s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction_CustomRequestHandling (171.87s)
--- PASS: TestAccAwsWafv2RuleGroup_updateRuleProperties (181.21s)
--- PASS: TestAccAwsWafv2RuleGroup_SqliMatchStatement (130.96s)
--- PASS: TestAccAwsWafv2RuleGroup_XssMatchStatement (128.51s)
--- PASS: TestAccAwsWafv2RuleGroup_GeoMatchStatement_ForwardedIPConfig (196.08s)
--- PASS: TestAccAwsWafv2RuleGroup_LogicalRuleStatements (212.66s)
--- PASS: TestAccAwsWafv2WebACLAssociation_basic (137.51s)
--- PASS: TestAccAwsWafv2RuleGroup_Tags (168.95s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_basic (157.12s)
--- PASS: TestAccAwsWafv2WebACL_basic (75.85s)
--- PASS: TestAccAwsWafv2WebACLAssociation_Disappears (191.03s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_disappears (154.17s)
--- PASS: TestAccAwsWafv2WebACL_Disappears (82.90s)
--- PASS: TestAccAwsWafv2RuleGroup_IpSetReferenceStatement_IPSetForwardedIPConfig (328.72s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateMethodRedactedField (226.23s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_emptyRedactedFields (184.19s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_disappears_WebAcl (175.16s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateUriPathRedactedField (222.89s)
--- PASS: TestAccAwsWafv2WebACL_Minimal (83.65s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateQueryStringRedactedField (247.68s)
--- PASS: TestAccAwsWafv2WebACL_updateRule (167.65s)
--- PASS: TestAccAwsWafv2WebACL_ChangeNameForceNew (165.19s)
--- PASS: TestAccAwsWafv2RuleGroup_ByteMatchStatement_FieldToMatch (418.92s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateSingleHeaderRedactedField (310.45s)
--- PASS: TestAccAwsWafv2WebACL_ManagedRuleGroupStatement (174.56s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateMultipleRedactedFields (306.95s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateEmptyRedactedFields (275.65s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement (101.33s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_changeLogDestinationConfigsForceNew (315.02s)
--- PASS: TestAccAwsWafv2WebACL_UpdateRuleProperties (264.79s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement (163.90s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement (161.47s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements (93.44s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig (161.96s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_changeResourceARNForceNew (344.33s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedOperatorStatements (80.04s)
--- PASS: TestAccAwsWafv2WebACL_CustomRequestHandling (143.87s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig (147.28s)
--- PASS: TestAccAwsWafv2WebACL_RuleGroupReferenceStatement (149.40s)
--- PASS: TestAccAwsWafv2WebACL_CustomResponse (130.19s)
--- PASS: TestAccAwsWafv2WebACL_Tags (141.11s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig (175.28s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_loggingFilter (347.75s)

[andyalm]

@anGie44 I have made all of the requested changes, with the exception of the exandWafv2 methods for allow, block, and count, which actually resulted in breaking functionality. I added an explanation of why in a separate comment. What are the next steps?

[andyalm]

@anGie44 that test failure doesn't seem related to my changes. Any idea how to clear that up?

[anGie44]

Changes looks great @andyalm 👍 Just a small suggestion to the changelog entries as the WebACL resource has both a default_action and rule actions. Looks like the go test CI failure might be related to a transient error, i'll trigger a re-run but it shouldn't hold up merging this in. Update: CI checks ✅

Output of acceptance tests:

--- PASS: TestAccAwsWafv2RuleGroup_RegexPatternSetReferenceStatement (54.72s)
--- PASS: TestAccAwsWafv2RuleGroup_Disappears (57.48s)
--- PASS: TestAccAwsWafv2RuleGroup_IpSetReferenceStatement (65.18s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction_CustomResponse (76.12s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeMetricNameForceNew (98.84s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeNameForceNew (99.15s)
--- PASS: TestAccAwsWafv2RuleGroup_Minimal (100.53s)
--- PASS: TestAccAwsWafv2RuleGroup_basic (107.20s)
--- PASS: TestAccAwsWafv2RuleGroup_updateRule (121.84s)
--- PASS: TestAccAwsWafv2RuleGroup_GeoMatchStatement (136.85s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction_CustomRequestHandling (137.20s)
--- PASS: TestAccAwsWafv2RuleGroup_ChangeCapacityForceNew (145.23s)
--- PASS: TestAccAwsWafv2RuleGroup_updateRuleProperties (165.01s)
--- PASS: TestAccAwsWafv2RuleGroup_SizeConstraintStatement (168.60s)
--- PASS: TestAccAwsWafv2RuleGroup_SqliMatchStatement (121.14s)
--- PASS: TestAccAwsWafv2RuleGroup_GeoMatchStatement_ForwardedIPConfig (177.87s)
--- PASS: TestAccAwsWafv2RuleGroup_ByteMatchStatement (181.21s)
--- PASS: TestAccAwsWafv2RuleGroup_XssMatchStatement (122.03s)
--- PASS: TestAccAwsWafv2RuleGroup_RuleAction (201.70s)
--- PASS: TestAccAwsWafv2WebACLAssociation_basic (129.42s)
--- PASS: TestAccAwsWafv2RuleGroup_Tags (161.55s)
--- PASS: TestAccAwsWafv2RuleGroup_LogicalRuleStatements (224.04s)
--- PASS: TestAccAwsWafv2RuleGroup_IpSetReferenceStatement_IPSetForwardedIPConfig (237.55s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_basic (147.04s)
--- PASS: TestAccAwsWafv2WebACL_basic (91.68s)
--- PASS: TestAccAwsWafv2WebACLAssociation_Disappears (202.23s)
--- PASS: TestAccAwsWafv2WebACL_Disappears (81.10s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_disappears (174.67s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_disappears_WebAcl (163.66s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateUriPathRedactedField (212.68s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_emptyRedactedFields (173.43s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateMethodRedactedField (247.38s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateQueryStringRedactedField (247.42s)
--- PASS: TestAccAwsWafv2WebACL_Minimal (75.97s)
--- PASS: TestAccAwsWafv2WebACL_updateRule (168.06s)
--- PASS: TestAccAwsWafv2WebACL_ChangeNameForceNew (154.95s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateSingleHeaderRedactedField (298.01s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateMultipleRedactedFields (271.81s)
--- PASS: TestAccAwsWafv2WebACL_ManagedRuleGroupStatement (170.59s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_updateEmptyRedactedFields (245.80s)
--- PASS: TestAccAwsWafv2RuleGroup_ByteMatchStatement_FieldToMatch (431.13s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement (93.64s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement (155.82s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedRateBasedStatements (91.03s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement (152.83s)
--- PASS: TestAccAwsWafv2WebACL_UpdateRuleProperties (255.10s)
--- PASS: TestAccAwsWafv2WebACL_MaxNestedOperatorStatements (81.98s)
--- PASS: TestAccAwsWafv2WebACL_GeoMatchStatement_ForwardedIPConfig (144.05s)
--- PASS: TestAccAwsWafv2WebACL_RateBasedStatement_ForwardedIPConfig (139.06s)
--- PASS: TestAccAwsWafv2WebACL_CustomResponse (124.67s)
--- PASS: TestAccAwsWafv2WebACL_RuleGroupReferenceStatement (141.35s)
--- PASS: TestAccAwsWafv2WebACL_CustomRequestHandling (126.72s)
--- PASS: TestAccAwsWafv2WebACL_Tags (133.40s)
--- PASS: TestAccAwsWafv2WebACL_IPSetReferenceStatement_IPSetForwardedIPConfig (168.53s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_changeResourceARNForceNew (389.20s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_changeLogDestinationConfigsForceNew (376.71s)
--- PASS: TestAccAwsWafv2WebACLLoggingConfiguration_loggingFilter (373.59s)

[andyalm]

Thanks for walking me through the process @anGie44 ! That was a really great experience.

[anGie44]

Happy to hear it @andyalm ! Thanks again for your contributions, hope to see more soon 😄

[bermannoah]

@andyalm thanks so much for doing this!

[ghost]

This has been released in version 3.43.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.