add ability to impersonate for kubectl-wrapper module #91

yashbhutwala · 2021-04-24T19:34:37Z

Fixes #90 here for usage in fixing terraform-google-modules/terraform-google-kubernetes-engine#874 upstream

comment-bot-dev · 2021-04-24T19:36:45Z

Thanks for the PR! 🚀
✅ Lint checks have passed.

yashbhutwala · 2021-04-24T19:49:16Z

@bharathkkb I wasn't sure how to add to the main gcloud module: https://github.com/terraform-google-modules/terraform-google-gcloud/blob/master/main.tf ... but it'd be nice to add impersonation there as well 😃

bharathkkb

Thanks for the PR
Regarding gcloud module, users can generally use this approach https://github.com/terraform-google-modules/terraform-example-foundation/blob/e30fe8cbd267b9f2e5dad9bc8fdb4360880b1cd4/1-org/envs/shared/scc_notification.tf#L45

modules/kubectl-wrapper/main.tf

yashbhutwala · 2021-04-25T13:02:30Z

I have no visiblity into why the CI fails 🙃. This is what I see:

bharathkkb

@yashbhutwala CI output
You can also run them locally. I believe the issue is var.impersonate_service_account is string with default "" so conditional should be var.impersonate_service_account == "" ?

 Running command `terraform apply -lock=true -lock-timeout=0s -input=false -auto-approve=true -no-color -parallelism=10 -refresh=true  ` in directory /workspace/test/fixtures/kubectl_wrapper_example
       module.gke.data.google_client_config.default: Refreshing state...
       module.gke.module.gke.data.google_client_config.default: Refreshing state...
       module.gke.module.kubectl-local-yaml.module.gcloud_kubectl.data.external.env_override[0]: Refreshing state...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.data.external.env_override[0]: Refreshing state...
       
       Error: Incorrect condition type
       
         on ../../../modules/kubectl-wrapper/main.tf line 32, in module "gcloud_kubectl":
         32:   create_cmd_body        = var.impersonate_service_account ? "${local.base_cmd} ${var.impersonate_service_account} ${var.kubectl_create_command}" : "${local.base_cmd} ${var.kubectl_create_command}"
           |----------------
           | var.impersonate_service_account is ""
       
       The condition expression must be of type bool.
       
       
       Error: Incorrect condition type
       
         on ../../../modules/kubectl-wrapper/main.tf line 32, in module "gcloud_kubectl":
         32:   create_cmd_body        = var.impersonate_service_account ? "${local.base_cmd} ${var.impersonate_service_account} ${var.kubectl_create_command}" : "${local.base_cmd} ${var.kubectl_create_command}"
           |----------------
           | var.impersonate_service_account is ""
       
       The condition expression must be of type bool.
       
       
       Error: Incorrect condition type
       
         on ../../../modules/kubectl-wrapper/main.tf line 35, in module "gcloud_kubectl":
         35:   destroy_cmd_body       = var.impersonate_service_account ? "${local.base_cmd} ${var.impersonate_service_account} ${var.kubectl_destroy_command}" : "${local.base_cmd} ${var.kubectl_destroy_command}"
           |----------------
           | var.impersonate_service_account is ""
       
       The condition expression must be of type bool.
       
       
       Error: Incorrect condition type
       
         on ../../../modules/kubectl-wrapper/main.tf line 35, in module "gcloud_kubectl":
         35:   destroy_cmd_body       = var.impersonate_service_account ? "${local.base_cmd} ${var.impersonate_service_account} ${var.kubectl_destroy_command}" : "${local.base_cmd} ${var.kubectl_destroy_command}"
           |----------------
           | var.impersonate_service_account is ""
       
       The condition expression must be of type bool.
       
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <kubectl-wrapper-example-default>.  Please see .kitchen/logs/kubectl-wrapper-example-default.log for more details
>>>>>> ----------------------

yashbhutwala · 2021-04-30T17:12:15Z

@bharathkkb I don't have a GCP account where I can run this or bandwidth to figure out setting it up. I tried to fix the conditional. Can you please communicate what the latest failure is? Is it possible to automate dumping the logs to a GitHub comment here?

yashbhutwala · 2021-05-07T20:12:16Z

@bharathkkb I understand your time is valuable. If you're busy, can one of the other maintainers of this repo help please?

bharathkkb

@yashbhutwala apologies, missed this. Here is the CI error. It looks like the command is malformed gcloud container clusters get-credentials gke-on-vpc-cluster --project ci-gcloud-0546 --impersonate-service-account=kubectl Within the wrapper we should only add the flag if impersonate-sa has a valid value

terraform-google-gcloud/modules/kubectl-wrapper/scripts/kubectl_wrapper.sh

Line 53 in a3a3d16

    
           CMD="gcloud container clusters get-credentials ${CLUSTER_NAME} --project ${PROJECT_ID} --impersonate-service-account=${IMPERSONATE_SERVICE_ACCOUNT}"

  module.gke.module.gke.module.gcloud_wait_for_cluster.null_resource.module_depends_on[0]: Creating...
       module.gke.module.gke.module.gcloud_wait_for_cluster.null_resource.module_depends_on[0]: Creation complete after 0s [id=655032079276369943]
       module.gke.module.gke.module.gcloud_delete_default_kube_dns_configmap.null_resource.module_depends_on[0]: Creating...
       module.gke.module.gke.module.gcloud_delete_default_kube_dns_configmap.null_resource.module_depends_on[0]: Creation complete after 0s [id=6235501171251231086]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.module_depends_on[0]: Creating...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.module_depends_on[0]: Creation complete after 0s [id=1356930797925402823]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_destroy_command[0]: Creating...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0]: Creating...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0]: Provisioning with 'local-exec'...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_destroy_command[0]: Creation complete after 0s [id=9113003221126712514]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n../../../modules/kubectl-wrapper/scripts/kubectl_wrapper.sh gke-on-vpc-cluster us-central1 ci-gcloud-0546 false false kubectl run --generator=run-pod/v1 nginx-imperative --image=nginx\n"]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + '[' 10 -lt 5 ']'
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + CLUSTER_NAME=gke-on-vpc-cluster
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + LOCATION=us-central1
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + PROJECT_ID=ci-gcloud-0546
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + INTERNAL=false
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + USE_EXISTING_CONTEXT=false
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + IMPERSONATE_SERVICE_ACCOUNT=kubectl
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + shift 5
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + false
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + RANDOM_ID=14311_22329
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + export TMPDIR=/tmp/kubectl_wrapper_14311_22329
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + TMPDIR=/tmp/kubectl_wrapper_14311_22329
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + trap cleanup EXIT
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + mkdir /tmp/kubectl_wrapper_14311_22329
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.additional_components_destroy[0]: Creating...
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.additional_components_destroy[0]: Creation complete after 0s [id=3831978060019404227]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + export KUBECONFIG=/tmp/kubectl_wrapper_14311_22329/config
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + KUBECONFIG=/tmp/kubectl_wrapper_14311_22329/config
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): ++ wc -l
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): ++ grep -o -
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + LOCATION_TYPE=1
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + CMD='gcloud container clusters get-credentials gke-on-vpc-cluster --project ci-gcloud-0546 --impersonate-service-account=kubectl'
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + [[ 1 -eq 2 ]]
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + CMD+=' --region us-central1'
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + false
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + gcloud container clusters get-credentials gke-on-vpc-cluster --project ci-gcloud-0546 --impersonate-service-account=kubectl --region us-central1
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): WARNING: This command is using service account impersonation. All API calls will be executed as [kubectl].
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): ERROR: (gcloud.container.clusters.get-credentials) INVALID_ARGUMENT: Request contains an invalid argument.
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + cleanup
       module.gke.module.kubectl-imperative.module.gcloud_kubectl.null_resource.run_command[0] (local-exec): + rm -rf /tmp/kubectl_wrapper_14311_22329
       
       
       Error: Error running command 'PATH=/google-cloud-sdk/bin:$PATH
       ../../../modules/kubectl-wrapper/scripts/kubectl_wrapper.sh gke-on-vpc-cluster us-central1 ci-gcloud-0546 false false kubectl run --generator=run-pod/v1 nginx-imperative --image=nginx
       ': exit status 1. Output: + '[' 10 -lt 5 ']'
       + CLUSTER_NAME=gke-on-vpc-cluster
       + LOCATION=us-central1
       + PROJECT_ID=ci-gcloud-0546
       + INTERNAL=false
       + USE_EXISTING_CONTEXT=false
       + IMPERSONATE_SERVICE_ACCOUNT=kubectl
       + shift 5
       + false
       + RANDOM_ID=14311_22329
       + export TMPDIR=/tmp/kubectl_wrapper_14311_22329
       + TMPDIR=/tmp/kubectl_wrapper_14311_22329
       + trap cleanup EXIT
       + mkdir /tmp/kubectl_wrapper_14311_22329
       + export KUBECONFIG=/tmp/kubectl_wrapper_14311_22329/config
       + KUBECONFIG=/tmp/kubectl_wrapper_14311_22329/config
       ++ wc -l
       ++ grep -o -
       + LOCATION_TYPE=1
       + CMD='gcloud container clusters get-credentials gke-on-vpc-cluster --project ci-gcloud-0546 --impersonate-service-account=kubectl'
       + [[ 1 -eq 2 ]]
       + CMD+=' --region us-central1'
       + false
       + gcloud container clusters get-credentials gke-on-vpc-cluster --project ci-gcloud-0546 --impersonate-service-account=kubectl --region us-central1
       WARNING: This command is using service account impersonation. All API calls will be executed as [kubectl].
       ERROR: (gcloud.container.clusters.get-credentials) INVALID_ARGUMENT: Request contains an invalid argument.
       + cleanup
       + rm -rf /tmp/kubectl_wrapper_14311_22329
       ```

yashbhutwala · 2021-05-07T20:23:49Z

ah interesting, thanks @bharathkkb, but I'm not quite sure how I'd check for that 🤔 . It'd be easier if I made it a breaking change by changing the base_cmd, but I'm not sure if I can do it without a breaking change.

bharathkkb · 2021-05-07T20:29:31Z

Perhaps something like this

https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/734ce5d285c517a33c9f1881390a1b8f59df0dd1/modules/asm/scripts/install_asm.sh#L45

yashbhutwala · 2021-05-07T21:01:42Z

@bharathkkb looks like CI is green finally ✅ 😄! Can you re-review please?

modules/kubectl-wrapper/scripts/kubectl_wrapper.sh

bharathkkb

Thanks for the contribution @yashbhutwala

yashbhutwala · 2021-05-08T21:12:50Z

Thank you @bharathkkb for your review! Can you possibly release this in a point release so we can use it in the gke module please? 😃

yashbhutwala · 2021-05-20T15:07:22Z

@bharathkkb can you please create a release with this PR, so we can use it to resolve terraform-google-modules/terraform-google-kubernetes-engine#874?

yashbhutwala · 2021-05-20T15:39:40Z

Thank you @bharathkkb, please review terraform-google-modules/terraform-google-kubernetes-engine#903 when you get a chance.

add ability to impersonate for kubectl-wrapper module

676eefd

use make generate_docs

d31912b

bharathkkb requested changes Apr 25, 2021

View reviewed changes

modules/kubectl-wrapper/main.tf Outdated Show resolved Hide resolved

make it backwards compatible

12c820c

bharathkkb reviewed Apr 30, 2021

View reviewed changes

yashbhutwala added 3 commits April 30, 2021 11:43

fix conditional and ci

d02b918

flip the ternary

f6dd8f3

terraform fmt

a3a3d16

yashbhutwala requested a review from bharathkkb May 7, 2021 19:57

bharathkkb reviewed May 7, 2021

View reviewed changes

try with a true flag

6bb2bf3

yashbhutwala requested a review from bharathkkb May 7, 2021 20:41

yashbhutwala commented May 7, 2021

View reviewed changes

modules/kubectl-wrapper/scripts/kubectl_wrapper.sh Show resolved Hide resolved

shift additional 2 in case of service impersonation

fbfe837

bharathkkb approved these changes May 8, 2021

View reviewed changes

bharathkkb merged commit 0d4e6f3 into terraform-google-modules:master May 8, 2021

yashbhutwala mentioned this pull request May 20, 2021

Bump up kubectl-wrapper and gcloud module version to add ability to impersonate service account terraform-google-modules/terraform-google-kubernetes-engine#903

Merged

yashbhutwala deleted the 90-add-impersonation branch May 20, 2021 15:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add ability to impersonate for kubectl-wrapper module #91

add ability to impersonate for kubectl-wrapper module #91

yashbhutwala commented Apr 24, 2021

comment-bot-dev commented Apr 24, 2021 •

edited

Loading

yashbhutwala commented Apr 24, 2021 •

edited

Loading

bharathkkb left a comment

yashbhutwala commented Apr 25, 2021

bharathkkb left a comment

yashbhutwala commented Apr 30, 2021 •

edited

Loading

yashbhutwala commented May 7, 2021

bharathkkb left a comment

yashbhutwala commented May 7, 2021 •

edited

Loading

bharathkkb commented May 7, 2021

yashbhutwala commented May 7, 2021

bharathkkb left a comment

yashbhutwala commented May 8, 2021

yashbhutwala commented May 20, 2021

yashbhutwala commented May 20, 2021

add ability to impersonate for kubectl-wrapper module #91

add ability to impersonate for kubectl-wrapper module #91

Conversation

yashbhutwala commented Apr 24, 2021

comment-bot-dev commented Apr 24, 2021 • edited Loading

yashbhutwala commented Apr 24, 2021 • edited Loading

bharathkkb left a comment

Choose a reason for hiding this comment

yashbhutwala commented Apr 25, 2021

bharathkkb left a comment

Choose a reason for hiding this comment

yashbhutwala commented Apr 30, 2021 • edited Loading

yashbhutwala commented May 7, 2021

bharathkkb left a comment

Choose a reason for hiding this comment

yashbhutwala commented May 7, 2021 • edited Loading

bharathkkb commented May 7, 2021

yashbhutwala commented May 7, 2021

bharathkkb left a comment

Choose a reason for hiding this comment

yashbhutwala commented May 8, 2021

yashbhutwala commented May 20, 2021

yashbhutwala commented May 20, 2021

comment-bot-dev commented Apr 24, 2021 •

edited

Loading

yashbhutwala commented Apr 24, 2021 •

edited

Loading

yashbhutwala commented Apr 30, 2021 •

edited

Loading

yashbhutwala commented May 7, 2021 •

edited

Loading