Attempting to create a google_container_node_pool causes a 500 error response from google API #3510

quinn · 2019-04-27T22:34:03Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.13
+ provider.google v2.5.1

Affected Resource(s)

google_container_node_pool

Terraform Configuration Files

resource "google_container_cluster" "primary" {
  name = "tftest"
  project = "myproject"
  location = "us-east4"
  initial_node_count = 1
  remove_default_node_pool = true

  master_auth {
    username = ""
    password = ""
  }
}

resource "google_container_node_pool" "primary_nodes" {
  cluster = "tftest"
  zone = "us-east4-a"
}

output "client_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.client_certificate}"
}

output "client_key" {
  value = "${google_container_cluster.primary.master_auth.0.client_key}"
}

output "cluster_ca_certificate" {
  value = "${google_container_cluster.primary.master_auth.0.cluster_ca_certificate}"
}

output "host" {
  value = "${google_container_cluster.primary.endpoint}"
}

Debug Output

https://gist.github.com/quinn/962b83ce8eefd9ce8f519e67dce9ccf9

Panic Output

N/A

Expected Behavior

The node pool should have been created.

Actual Behavior

google_container_node_pool.primary_nodes: error creating NodePool: googleapi: Error 500: Internal error encountered., backendError

Steps to Reproduce

terraform apply

Important Factoids

References

#0000

The text was updated successfully, but these errors were encountered:

rileykarson · 2019-04-29T16:52:36Z

Has this been happening consistently? Have you been able to reproduce it over a period of a few days?

quinn · 2019-04-29T16:59:51Z

@rileykarson just tried it again now, same error

quinn · 2019-04-29T19:24:27Z

@rileykarson are you unable to repro?

RafaAguilar · 2019-05-19T22:50:57Z

@rileykarson @quinn with an even simple configuration we are experiencing the same issue:ç

resource "google_container_cluster" "gke-cluster" {
  name               = "main-cluster"
  network            = "default"
  zone               = "us-central1-a"
  initial_node_count = 2
}

When running apply with TF_LOG=TRACE the 500 error is raised using terraform v0.11.14:

2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: 2019/05/19 19:48:59 [DEBUG] Google API Request Details:
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: ---[ REQUEST ]---------------------------------------
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: POST /v1beta1/projects/gke-tf-jeezzETL/locations/us-central1-a/clusters?alt=json&prettyPrint=false HTTP/1.1
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Host: container.googleapis.com
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: User-Agent: google-api-go-client/0.5 Terraform/0.12.0 (+https://www.terraform.io) terraform-provider-google/2.5.1
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Content-Length: 486
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Content-Type: application/json
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Accept-Encoding: gzip
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: {
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:  "cluster": {
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "initialNodeCount": 2,
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "legacyAbac": {
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:    "enabled": false
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   },
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "name": "main-cluster",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "network": "projects/gke-tf-jeezzETL/global/networks/default",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "nodeConfig": {
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:    "oauthScopes": [
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/devstorage.read_only",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/logging.write",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/monitoring",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/service.management.readonly",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/servicecontrol",
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "https://www.googleapis.com/auth/trace.append"
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:    ]
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   }
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:  }
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: }
2019-05-19T19:48:59.138-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:
2019-05-19T19:48:59.139-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: -----------------------------------------------------
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: 2019/05/19 19:48:59 [DEBUG] Google API Response Details:
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: ---[ RESPONSE ]--------------------------------------
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: HTTP/2.0 500 Internal Server Error
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Cache-Control: private
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Content-Type: application/json; charset=UTF-8
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Date: Sun, 19 May 2019 22:48:59 GMT
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Server: ESF
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Vary: Origin
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Vary: X-Origin
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: Vary: Referer
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: X-Content-Type-Options: nosniff
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: X-Frame-Options: SAMEORIGIN
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: X-Xss-Protection: 0
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: {
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   "error": {
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "code": 500,
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "message": "Internal error encountered.",
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "errors": [
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:       {
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:         "message": "Internal error encountered.",
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:         "domain": "global",
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:         "reason": "backendError"
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:       }
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     ],
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:     "status": "INTERNAL"
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:   }
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: }
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4:
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: -----------------------------------------------------
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: 2019/05/19 19:48:59 [DEBUG] Dismissed an error as retryable based on error code: googleapi: Error 500: Internal error encountered., backendError
2019-05-19T19:48:59.383-0300 [DEBUG] plugin.terraform-provider-google_v2.5.1_x4: 2019/05/19 19:48:59 [TRACE] Waiting 1s before next try

RafaAguilar · 2019-05-20T00:40:27Z

Disregard my previous comment. The Project ID was wrong, however API message is quite misleading. Maybe we could add a "project" lookup first as part of the plugin.

rileykarson · 2019-05-20T15:57:26Z

Interesting! How was it wrong, was the id invalid?

daniellavoie · 2019-06-01T20:10:34Z

Smashed my head on the keyboard for a while with this one. In GCP, the Project name and project id aren't the same (watch out for case sensitivity).

rileykarson · 2019-06-11T21:34:22Z

There isn't really anything actionable for the provider when an API is returning useless error messages like this. I'd recommend filing against the public GKE issue tracker (https://issuetracker.google.com/issues/new?component=187077&template=0), especially if this can be repro'ed in gcloud as well.

ghost · 2019-07-12T13:50:43Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!

ghost added the bug label Apr 27, 2019

danawillow added the waiting-response label May 20, 2019

ghost removed the waiting-response label Jun 1, 2019

rileykarson added the upstream label Jun 11, 2019

rileykarson closed this as completed Jun 11, 2019

ghost locked and limited conversation to collaborators Jul 12, 2019

github-actions bot added service/container forward/review In review; remove label to forward labels Jan 15, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Attempting to create a google_container_node_pool causes a 500 error response from google API #3510

Attempting to create a google_container_node_pool causes a 500 error response from google API #3510

quinn commented Apr 27, 2019

rileykarson commented Apr 29, 2019

quinn commented Apr 29, 2019

quinn commented Apr 29, 2019

RafaAguilar commented May 19, 2019 •

edited

Loading

RafaAguilar commented May 20, 2019

rileykarson commented May 20, 2019

daniellavoie commented Jun 1, 2019

rileykarson commented Jun 11, 2019

ghost commented Jul 12, 2019

Attempting to create a google_container_node_pool causes a 500 error response from google API #3510

Attempting to create a google_container_node_pool causes a 500 error response from google API #3510

Comments

quinn commented Apr 27, 2019

Community Note

Terraform Version

Affected Resource(s)

Terraform Configuration Files

Debug Output

Panic Output

Expected Behavior

Actual Behavior

Steps to Reproduce

Important Factoids

References

rileykarson commented Apr 29, 2019

quinn commented Apr 29, 2019

quinn commented Apr 29, 2019

RafaAguilar commented May 19, 2019 • edited Loading

RafaAguilar commented May 20, 2019

rileykarson commented May 20, 2019

daniellavoie commented Jun 1, 2019

rileykarson commented Jun 11, 2019

ghost commented Jul 12, 2019

RafaAguilar commented May 19, 2019 •

edited

Loading