Dynamic Port Allocation silently fails to disable on Cloud NAT Gateways

[bobdanek]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

terraform -v
Terraform v1.1.7
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v4.39.0
+ provider registry.terraform.io/hashicorp/google-beta v4.39.0
+ provider registry.terraform.io/ns1-terraform/ns1 v1.12.0

Your version of Terraform is out of date! The latest version
is 1.3.6. You can update by downloading from https://www.terraform.io/downloads.html

Affected Resource(s)

• google_compute_router_nat

Terraform Configuration Files

Apply 1, to create test resources

resource "google_compute_network" "bobdanek-test" {
  name                    = "bobdanek-test"
  description             = "parent issue: https://github.com/redacted/redacted/issues/25357"
  auto_create_subnetworks = false
  routing_mode            = "REGIONAL"
}

resource "google_compute_subnetwork" "bobdanek-test" {
  name          = "bobdanek-test"
  description   = "parent issue: https://github.com/redacted/redacted/issues/25357"
  ip_cidr_range = "192.168.42.0/24"
  network       = google_compute_network.bobdanek-test.self_link
  region        = google_compute_router.bobdanek-test.region
}

resource "google_compute_router" "bobdanek-test" {
  name        = "bobdanek-test"
  description = "parent issue: https://github.com/redacted/redacted/issues/25357"
  network     = google_compute_network.bobdanek-test.self_link
  region      = "us-west1"
}

resource "google_compute_router_nat" "bobdanek-test" {
  name                                = "bobdanek-test"
  router                              = google_compute_router.bobdanek-test.name
  region                              = google_compute_router.bobdanek-test.region
  min_ports_per_vm                    = 1024
  enable_dynamic_port_allocation      = false
  enable_endpoint_independent_mapping = false
  source_subnetwork_ip_ranges_to_nat  = "ALL_SUBNETWORKS_ALL_IP_RANGES"
  nat_ip_allocate_option              = "AUTO_ONLY"
}

Apply 2, to enable dynamic port allocation

resource "google_compute_network" "bobdanek-test" {
  name                    = "bobdanek-test"
  description             = "parent issue: https://github.com/redacted/redacted/issues/25357"
  auto_create_subnetworks = false
  routing_mode            = "REGIONAL"
}

resource "google_compute_subnetwork" "bobdanek-test" {
  name          = "bobdanek-test"
  description   = "parent issue: https://github.com/redacted/redacted/issues/25357"
  ip_cidr_range = "192.168.42.0/24"
  network       = google_compute_network.bobdanek-test.self_link
  region        = google_compute_router.bobdanek-test.region
}

resource "google_compute_router" "bobdanek-test" {
  name        = "bobdanek-test"
  description = "parent issue: https://github.com/redacted/redacted/issues/25357"
  network     = google_compute_network.bobdanek-test.self_link
  region      = "us-west1"
}

resource "google_compute_router_nat" "bobdanek-test" {
  name                                = "bobdanek-test"
  router                              = google_compute_router.bobdanek-test.name
  region                              = google_compute_router.bobdanek-test.region
  min_ports_per_vm                    = 1024
  enable_dynamic_port_allocation      = true
  enable_endpoint_independent_mapping = false
  source_subnetwork_ip_ranges_to_nat  = "ALL_SUBNETWORKS_ALL_IP_RANGES"
  nat_ip_allocate_option              = "AUTO_ONLY"

Apply 3, to disable dynamic port allocation

resource "google_compute_network" "bobdanek-test" {
  name                    = "bobdanek-test"
  description             = "parent issue: https://github.com/redacted/redacted/issues/25357"
  auto_create_subnetworks = false
  routing_mode            = "REGIONAL"
}

resource "google_compute_subnetwork" "bobdanek-test" {
  name          = "bobdanek-test"
  description   = "parent issue: https://github.com/redacted/redacted/issues/25357"
  ip_cidr_range = "192.168.42.0/24"
  network       = google_compute_network.bobdanek-test.self_link
  region        = google_compute_router.bobdanek-test.region
}

resource "google_compute_router" "bobdanek-test" {
  name        = "bobdanek-test"
  description = "parent issue: https://github.com/redacted/redacted/issues/25357"
  network     = google_compute_network.bobdanek-test.self_link
  region      = "us-west1"
}

resource "google_compute_router_nat" "bobdanek-test" {
  name                                = "bobdanek-test"
  router                              = google_compute_router.bobdanek-test.name
  region                              = google_compute_router.bobdanek-test.region
  min_ports_per_vm                    = 1024
  enable_dynamic_port_allocation      = true
  enable_endpoint_independent_mapping = false
  source_subnetwork_ip_ranges_to_nat  = "ALL_SUBNETWORKS_ALL_IP_RANGES"
  nat_ip_allocate_option              = "AUTO_ONLY"

Debug Output

3 files attached here, all encrypted with Hashicorp's public GPG key:

encrypted-apply-1.log

encrypted-apply-2.log

encrypted-apply-3.log

Panic Output

None.

Expected Behavior

Applying after changing enable_dynamic_port_allocation from true to false disables dynamic port allocation on the cloud NAT gateway

Actual Behavior

Terraform reports successfully applying the change from true to false, but dynamic port allocation is still enabled

Steps to Reproduce

1. terraform apply the "Apply 1" code above to create the test resources, including the cloud NAT gateway with DPA (dynamic port allocation) disabled
2. terraform apply the "Apply 2" code above to switch DPA to enabled on the cloud NAT gateway
3. terraform apply the "Apply 3" code above to switch DPA back to disabled on the cloud NAT gateway
4. Observe that DPA is still enabled, despite Terraform reporting success setting it to false

Important Factoids

Nothing out of the ordinary as far as I can tell.

Skipping the first apply and creating the NAT gateway with DPA enabled initially may produce the same results, but I took the extra step in case there's anything useful in the debug logs when initially having it disabled.

References

• At least tangentially related: Cloud NAT Dynamic Port Allocation / Endpoint Independent Mapping Failure #11909

[bobdanek]

I suspect this PATCH request from the debug outputs is wrong. I'd expect to see enableDynamicPortAllocation": false here but instead I see enableDynamicPortAllocation": true

Lines 18470-18500 in encrypted-apply-3.log:

---[ REQUEST ]---------------------------------------
PATCH /compute/v1/projects/redacted/regions/us-west1/routers/bobdanek-test?alt=json HTTP/1.1
Host: compute.googleapis.com
User-Agent: Terraform/1.1.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
Content-Length: 447
Content-Type: application/json
Accept-Encoding: gzip

{
 "nats": [
  {
   "drainNatIps": [],
   "enableDynamicPortAllocation": true,
   "enableEndpointIndependentMapping": false,
   "endpointTypes": [
    "ENDPOINT_TYPE_VM"
   ],
   "icmpIdleTimeoutSec": 30,
   "logConfig": null,
   "minPortsPerVm": 1024,
   "name": "bobdanek-test",
   "natIpAllocateOption": "AUTO_ONLY",
   "natIps": [],
   "sourceSubnetworkIpRangesToNat": "ALL_SUBNETWORKS_ALL_IP_RANGES",
   "subnetworks": [],
   "tcpEstablishedIdleTimeoutSec": 1200,
   "tcpTransitoryIdleTimeoutSec": 30,
   "udpIdleTimeoutSec": 30
  }
 ]
}

I also see at the end of encrypted-apply-3.log (lines 18603-18605) the following:

2022-12-08T21:42:15.500Z [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/google\"]" produced an unexpected new value for google_compute_router_nat.bobdanek-test, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .enable_dynamic_port_allocation: was cty.False, but now cty.True

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.