Cannot delete google_compute_address when it's a dependency of google_compute_router_nat

[gmauleon]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.26

• provider.google v3.31.0

Affected Resource(s)

• google_compute_router_nat
• google_compute_address

Terraform Configuration Files

locals {
  router = ""
  region = ""
  subnet = ""
}


resource "google_compute_address" "nat" {
  count = 2
  name = "test-${count.index}"
}

resource "google_compute_router_nat" "nat" {
  name = "test"
  router = local.router
  region = local.region

  nat_ips = google_compute_address.nat.*.self_link

  nat_ip_allocate_option = "MANUAL_ONLY"
  source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"

  subnetwork {
      name = local.subnet
      source_ip_ranges_to_nat = ["ALL_IP_RANGES"]
  }

  min_ports_per_vm = 1024

  log_config {
    enable = true
    filter = "ERRORS_ONLY"
  }
}

Expected Behavior

When lowering the count in google_compute_address, the resource google_compute_router_nat should be updated first then the old google_compute_address resources must be destroyed

Actual Behavior

Since google_compute_address is a dependency of google_compute_router_nat, Terraform try to destroy the addresses first but failed since they are still in use by the NAT resource.

Error: Error when reading or editing Address: googleapi: Error 400: External address used for NAT cannot be deleted., badRequest

Steps to Reproduce

Note that the .tf has been adapted with fake "locals" for convenience.

1. Change count in google_compute_address to 1
2. terraform plan/apply

Important Factoids

I don't see an easy way out of that, but we could have an intermediate google_compute_router_nat_ip_attachment so that we could destroy the ip_attachment, then the address without causing dependency cycles.
However, the google_compute_router_nat must have at least one IP when first created, so it still need to come with a nat_ips parameter.

References

It's similar in nature to #6376

[edwardmedia]

I can repro this issue

  # google_compute_address.nat[1] will be destroyed
  - resource "google_compute_address" "nat" {
      - address            = "35.194.xxx.xxx" -> null
      - address_type       = "EXTERNAL" -> null
      - creation_timestamp = "2020-07-20T16:54:14.214-07:00" -> null
      - id                 = "projects/myproject/regions/us-central1/addresses/issue6812-1" -> null
      - name               = "issue6812-1" -> null
      - network_tier       = "PREMIUM" -> null
      - project            = "sunedward-1-autotest" -> null
      - region             = "us-central1" -> null
      - self_link          = "https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/addresses/issue6812-1" -> null
      - users              = [
          - "https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/routers/issue6812-router",
        ] -> null
    }
2020/07/20 23:59:53 [DEBUG] command: asking for input: "Do you want to perform these actions?"
  # google_compute_router_nat.nat will be updated in-place
  ~ resource "google_compute_router_nat" "nat" {
        drain_nat_ips                      = []
        icmp_idle_timeout_sec              = 30
        id                                 = "myproject/us-central1/issue6812-router/issue6812-nat"
        min_ports_per_vm                   = 1024
        name                               = "issue6812-nat"
        nat_ip_allocate_option             = "MANUAL_ONLY"
      ~ nat_ips                            = [
            "https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/addresses/issue6812-0",
          - "https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/addresses/issue6812-1",
        ]
        project                            = "myproject"
        region                             = "us-central1"
        router                             = "issue6812-router"
        source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
        tcp_established_idle_timeout_sec   = 1200
        tcp_transitory_idle_timeout_sec    = 30
        udp_idle_timeout_sec               = 30
        log_config {
            enable = true
            filter = "ERRORS_ONLY"
        }
        subnetwork {
            name                     = "https://www.googleapis.com/compute/v1/projects/myproject/regions/us-central1/subnetworks/default"
            secondary_ip_range_names = []
            source_ip_ranges_to_nat  = [
                "ALL_IP_RANGES",
            ]
        }
}

[edwardmedia]

2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: DELETE /compute/v1/projects/myproject/regions/us-central1/addresses/issue6812-1?alt=json HTTP/1.1
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Host: www.googleapis.com
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: User-Agent: HashiCorp Terraform/0.12.24 (+https://www.terraform.io) Terraform Plugin SDK/1.11.0 terraform-provider-google
/3.31.0
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Content-Type: application/json
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Accept-Encoding: gzip
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 
2020-07-20T23:59:57.217Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: -----------------------------------------------------
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 2020/07/20 23:59:57 [DEBUG] Google API Response Details:
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: ---[ RESPONSE ]--------------------------------------
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: HTTP/1.1 400 Bad Request
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Connection: close
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Transfer-Encoding: chunked
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Cache-Control: private
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Content-Type: application/json; charset=UTF-8
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Date: Mon, 20 Jul 2020 23:59:57 GMT
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Server: ESF
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Vary: Origin
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Vary: X-Origin
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: Vary: Referer
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: X-Content-Type-Options: nosniff
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: X-Frame-Options: SAMEORIGIN
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: X-Xss-Protection: 0
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 111
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: {
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:   "error": {
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:     "code": 400,
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:     "message": "External address used for NAT cannot be deleted.",
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:     "errors": [
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:       {
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:         "message": "External address used for NAT cannot be deleted.",
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:         "domain": "global",
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:         "reason": "badRequest"
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:       }
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:     ]
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5:   }
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: }
2020-07-20T23:59:57.538Z [DEBUG] plugin.terraform-provider-google_v3.31.0_x5: 

[slevenick]

Looks like this is the same basic problem as #6376

Unfortunately there is not a great option for resolving this permanently. We could split out a fine-grained resource similar to the suggestion in that other issue, allowing us to update nat_ips separately from the google_compute_router_nat resource.

Alternatively we could mark nat_ips as ForceNew which would force a recreation of the resource whenever this field is updated. This could have unintended consequences as it would cause destruction and recreation even if it is unnecessary.

Given that this resource doesn't lend itself to a fine-grained resource (due to the requirement of a nat_ip at create time that you noted) I'd lean towards making this field ForceNew. Would updates to this field causing recreation cause issues for your workflow @gmauleon ?

[gmauleon]

Hum unfortunately recreating the whole NAT resource would render a somewhat "seamless change" (minus a drain), disruptive, if we're talking real live operation.

Which bring another concept on the table actually, we could have an initial_nat_ip which is mandatory and ForceNew and then a fine grained resource to add more IPs, with this we could actually bake an automatic synchronous drain on destroyed IPs if a flag is set (i.e. : drain = yes or whatever).

[luis-guimaraes-exoawk]

Hi, has there been any development regarding this issue?

[matheusaleixo-cit]

Hello everyone! I am currently working on the creation of a new fine grained resource for this issue, but one problem I am facing is that since the new resource is only really separating the nat_ips field, which just references the addresses directly, the result is that the new resource just faces the same dependency problem when removing the addresses.

While I work on this, is the usage of a random_id for the addresses (as described here) a viable workaround for the time being? Example:

resource "random_id" "group-manager-suffix" {
  byte_length = 4
}

resource "google_compute_address" "address" {
  count = 3
  name = "test-address-${count.index}-${random_id.group-manager-suffix.hex}"

  lifecycle {
    create_before_destroy = true
  }
}

resource "google_compute_router_nat" "router-nat" {
  name                               = "test-router-nat"
[...]
  nat_ip_allocate_option             = "MANUAL_ONLY"
  nat_ips = google_compute_address.address.*.self_link
}

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.