Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Attach KMS Key in Safer IAP GKE cluster #1614

Merged
merged 12 commits into from
May 23, 2023
Merged
7 changes: 7 additions & 0 deletions examples/safer_cluster_iap_bastion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ This end to end example aims to showcase access patterns to a [Safer Cluster](..

Additionally we deploy a [tinyproxy](https://tinyproxy.github.io/) daemon which allows `kubectl` commands to be piped through the bastion host allowing ease of development from a local machine with the security of GKE Private Clusters.

GKE Autopilot clusters are deployed with Application-layer Secrets Encryption that protects your secrets in etcd with a key you manage in [Cloud KMS](https://github.com/terraform-google-modules/terraform-google-kms/blob/master/README.md).

## Setup

To deploy this example:
Expand Down Expand Up @@ -41,6 +43,8 @@ To deploy this example:
| cluster\_name | The name of the cluster | `string` | `"safer-cluster-iap-bastion"` | no |
| ip\_range\_pods\_name | The secondary ip range to use for pods | `string` | `"ip-range-pods"` | no |
| ip\_range\_services\_name | The secondary ip range to use for pods | `string` | `"ip-range-svc"` | no |
| keyring | Keyring name. | `string` | `"gke-keyring"` | no |
| keys | Key names. | `list(string)` | <pre>[<br> "gke-key"<br>]</pre> | no |
| network\_name | The name of the network being created to host the cluster in | `string` | `"safer-cluster-network"` | no |
| project\_id | The project ID to host the cluster in | `string` | n/a | yes |
| region | The region to host the cluster in | `string` | `"us-central1"` | no |
Expand All @@ -59,6 +63,9 @@ To deploy this example:
| cluster\_name | Cluster name |
| endpoint | Cluster endpoint |
| get\_credentials\_command | gcloud get-credentials command to generate kubeconfig for the private cluster |
| keyring | The name of the keyring. |
| keyring\_resource | The location of the keyring. |
| keys | Map of key name => key self link. |
| location | Cluster location (region if regional cluster, zone if zonal cluster) |
| master\_authorized\_networks\_config | Networks from which access to master is permitted |
| network\_name | The name of the VPC being created |
Expand Down
1 change: 1 addition & 0 deletions examples/safer_cluster_iap_bastion/apis.tf
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,6 @@ module "enabled_google_apis" {
"binaryauthorization.googleapis.com",
"stackdriver.googleapis.com",
"iap.googleapis.com",
"cloudkms.googleapis.com",
]
}
6 changes: 6 additions & 0 deletions examples/safer_cluster_iap_bastion/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ module "gke" {
cidr_block = "${module.bastion.ip_address}/32"
display_name = "Bastion Host"
}]
database_encryption = [
{
"key_name" : module.kms.keys[var.keys[0]],
"state" : "ENCRYPTED"
}
]
grant_registry_access = true
node_pools = [
{
Expand Down
39 changes: 39 additions & 0 deletions examples/safer_cluster_iap_bastion/kms.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

data "google_project" "project" {}

locals {
gke_sa = "serviceAccount:service-${data.google_project.project.number}@container-engine-robot.iam.gserviceaccount.com"
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can generate this using service identities from the project services module in apis.tf and use the enabled_api_identities output to retrieve the sa email.

Copy link
Contributor Author

@avinashkumar1289 avinashkumar1289 May 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @bharathkkb Thanks for the response. activate_api_identities requires the api and the role. I can pass the api but what should be the role I need to pass ? Because this service account needs the KMS encrypter/Decrypter role which is taken care at the KMS Module
Something like. this

 encrypters = [
   "serviceAccount:${module.enabled_google_apis.enabled_api_identities["container.googleapis.com"]}",
  ]
  decrypters = [
  "serviceAccount:${module.enabled_google_apis.enabled_api_identities["container.googleapis.com"]}",
  ]
  
  activate_api_identities     = [{
    api = "container.googleapis.com",
    roles = [????],
  }]

Copy link
Contributor Author

@avinashkumar1289 avinashkumar1289 May 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating the role at api_identities and removed thee role from KMS module

module "kms" {
source = "terraform-google-modules/kms/google"
version = "~> 2.2.1"
project_id = var.project_id
location = var.region
keyring = var.keyring
keys = var.keys
prevent_destroy = false
set_decrypters_for = var.keys
set_encrypters_for = var.keys
encrypters = [
local.gke_sa,
]
decrypters = [
local.gke_sa,
]
}
15 changes: 15 additions & 0 deletions examples/safer_cluster_iap_bastion/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,18 @@ output "bastion_kubectl_command" {
description = "kubectl command using the local proxy once the bastion_ssh command is running"
value = "HTTPS_PROXY=localhost:8888 kubectl get pods --all-namespaces"
}

output "keyring" {
description = "The name of the keyring."
value = module.kms.keyring
}

output "keyring_resource" {
description = "The location of the keyring."
value = module.kms.keyring_resource
}

output "keys" {
description = "Map of key name => key self link."
value = module.kms.keys
}
12 changes: 12 additions & 0 deletions examples/safer_cluster_iap_bastion/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,15 @@ variable "bastion_members" {
description = "List of users, groups, SAs who need access to the bastion host"
default = []
}

variable "keyring" {
description = "Keyring name."
type = string
default = "gke-keyring"
}

variable "keys" {
description = "Key names."
type = list(string)
default = ["gke-key"]
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are transitioning to inline these variables so you can directly provide them in kms.tf

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bharathkkb updated the comment