Support force destruction of DataStream private connections

[laksh-bhat]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

1.3.4

Affected Resource(s)

• google_datastream_private_connection

Terraform Configuration Files

resource "google_datastream_private_connection" "datastream_conn" {
  # This resource must have an IP range with a minimum CIDR block of /29
  # https://cloud.google.com/datastream/docs/create-a-private-connectivity-configuration
  display_name          = "datastream private connection"
  project                     = var.project
  location                   = var.region

  private_connection_id = "datastream-conn"

  vpc_peering_config {
    vpc    = var.private_network_id
    subnet = var.datastream_subnet_cidr
  }
}

Debug Output

---[ REQUEST ]---------------------------------------
DELETE /v1/projects/relyance-internal/locations/us-central1/privateConnections/datastream-conn?alt=json HTTP/1.1
Host: datastream.googleapis.com
User-Agent: Terraform/1.3.4 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
Content-Type: application/json
Accept-Encoding: gzip

2022-11-16T11:05:26.577-0800 [INFO]  provider.terraform-provider-google_v4.43.0_x5: 2022/11/16 11:05:26 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Wed, 16 Nov 2022 19:05:26 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "Resource '\"projects/<internal>/locations/us-central1/privateConnections/datastream-conn\"' has nested resources. If the API supports cascading delete, set 'force' to true to delete it and its nested resources.",
    "status": "FAILED_PRECONDITION"
  }
}

Panic Output

Expected Behavior

terraform apply -var-file="$TFVARS_FILE" -destroy is successful

Actual Behavior

│ Error: Error when reading or editing PrivateConnection: googleapi: Error 400: Resource '"projects/<internal>/locations/us-central1/privateConnections/datastream-conn"' has nested resources. If the API supports cascading delete, set 'force' to true to delete it and its nested resources.

Steps to Reproduce

1. terraform apply -var-file="$TFVARS_FILE" -destroy

Important Factoids

References

• Add support for datastream resources #10810

b/336319220

[edwardmedia]

@laksh-bhat are you able to see if other tools (like gcloud) work as you expected?

[laksh-bhat]

Yes, gcloud works as expected.

gcloud datastream private-connections delete 'datastream-conn' --location=us-central1 --force --quiet

Deleted private_connection [datastream-conn].

[edwardmedia]

@laksh-bhat I do see force is available in below api but I am not sure if that should be implemented in the Terraform. 1) For use case, how do you manage those child routes? If they are created via Terraform, you better continue let Terraform to manage deletion. 2) Implementing force seems against the pattern, as Terraform is more declarative.

https://cloud.google.com/datastream/docs/reference/rest/v1/projects.locations.privateConnections/delete

If you want this feature, you may file an enhancement and then it will be triaged. This is not a bug and I am closing it accordingly.

[laksh-bhat]

I understand this may seem like an anti-pattern for terraform but the private connections API doesn't allow us to manage the routes explicitly (you set a subnet and private VPC through a peering config).

resource "google_datastream_private_connection" "private_vpc_datastream_conn" {
  # This resource must have an IP range with a minimum CIDR block of /29
  # https://cloud.google.com/datastream/docs/create-a-private-connectivity-configuration
  display_name          = "datastream private connection"
  project               = var.project
  location              = var.region
  private_connection_id = "${var.region}-${var.cell}-${var.env}-datastream-conn"

  vpc_peering_config {
    vpc    = var.private_network_id
    subnet = var.datastream_subnet_cidr
  }
}

The routes created by this, hence, aren't in terraform state. I strongly urge you to reconsider this decision to not implement the force flag @edwardmedia. Without this flag, the resource is pretty much useless.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

[melinath]

Note from triage: Reopening to track the request to add a field for configuring force to send to the API at deletion time. This should use a deletion_policy field - see https://googlecloudplatform.github.io/magic-modules/best-practices/#deletion-policy

[manitgupta]

I ran into this too. Is there an update on how to use terraform to delete google_datastream_private_connection resource? Without the ability to delete, the resource is pretty much useless.