terraform-google-modules · milesmatthias · Oct 23, 2019 · Oct 23, 2019 · Oct 25, 2019 · Oct 25, 2019
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,99 @@
+# Contributing
+
+This document provides guidelines for contributing to the module.
+
+## Dependencies
+
+The following dependencies must be installed on the development system:
+
+- [Docker Engine][docker-engine]
+- [Google Cloud SDK][google-cloud-sdk]
+- [make]
+
+## Generating Documentation for Inputs and Outputs
+
+The Inputs and Outputs tables in the READMEs of the root module,
+submodules, and example modules are automatically generated based on
+the `variables` and `outputs` of the respective modules. These tables
+must be refreshed if the module interfaces are changed.
+
+### Execution
+
+Run `make generate_docs` to generate new Inputs and Outputs tables.
+
+## Integration Testing
+
+Integration tests are used to verify the behaviour of the root module,
+submodules, and example modules. Additions, changes, and fixes should
+be accompanied with tests.
+
+The integration tests are run using [Kitchen][kitchen],
+[Kitchen-Terraform][kitchen-terraform], and [InSpec][inspec]. These
+tools are packaged within a Docker image for convenience.
+
+The general strategy for these tests is to verify the behaviour of the
+[example modules](./examples/), thus ensuring that the root module,
+submodules, and example modules are all functionally correct.
+
+### Test Environment
+The easiest way to test the module is in an isolated test project. The setup for such a project is defined in [test/setup](./test/setup/) directory.
+
+To use this setup, you need a service account with Project Creator access on a folder. Export the Service Account credentials to your environment like so:
+
+```
+export SERVICE_ACCOUNT_JSON=$(< credentials.json)
+```
+
+You will also need to set a few environment variables:
+```
+export TF_VAR_org_id="your_org_id"
+export TF_VAR_folder_id="your_folder_id"
+export TF_VAR_billing_account="your_billing_account_id"
+```
+
+With these settings in place, you can prepare a test project using Docker:
+```
+make docker_test_prepare
+```
+
+### Noninteractive Execution
+
+Run `make docker_test_integration` to test all of the example modules
+noninteractively, using the prepared test project.
+
+### Interactive Execution
+
+1. Run `make docker_run` to start the testing Docker container in
+   interactive mode.
+
+1. Run `kitchen_do create <EXAMPLE_NAME>` to initialize the working
+   directory for an example module.
+
+1. Run `kitchen_do converge <EXAMPLE_NAME>` to apply the example module.
+
+1. Run `kitchen_do verify <EXAMPLE_NAME>` to test the example module.
+
+1. Run `kitchen_do destroy <EXAMPLE_NAME>` to destroy the example module
+   state.
+
+## Linting and Formatting
+
+Many of the files in the repository can be linted or formatted to
+maintain a standard of quality.
+
+### Execution
+
+Run `make docker_test_lint`.
+
+[docker-engine]: https://www.docker.com/products/docker-engine
+[flake8]: http://flake8.pycqa.org/en/latest/
+[gofmt]: https://golang.org/cmd/gofmt/
+[google-cloud-sdk]: https://cloud.google.com/sdk/install
+[hadolint]: https://github.com/hadolint/hadolint
+[inspec]: https://inspec.io/
+[kitchen-terraform]: https://github.com/newcontext-oss/kitchen-terraform
+[kitchen]: https://kitchen.ci/
+[make]: https://en.wikipedia.org/wiki/Make_(software)
+[shellcheck]: https://www.shellcheck.net/
+[terraform-docs]: https://github.com/segmentio/terraform-docs
+[terraform]: https://terraform.io/
diff --git a/Makefile b/Makefile
@@ -1,4 +1,4 @@
-# Copyright 2018 Google LLC
+# Copyright 2019 Google LLC
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -12,121 +12,74 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# Please note that this file was generated from [terraform-google-module-template](https://github.com/terraform-google-modules/terraform-google-module-template).
+# Please make sure to contribute relevant changes upstream!
+
 # Make will use bash instead of sh
 SHELL := /usr/bin/env bash
 
-# Docker build config variables
-CREDENTIALS_PATH 			?= /cft/workdir/credentials.json
-DOCKER_ORG 				:= gcr.io/cloud-foundation-cicd
-DOCKER_TAG_BASE_KITCHEN_TERRAFORM 	?= 2.3.0
-DOCKER_REPO_BASE_KITCHEN_TERRAFORM 	:= ${DOCKER_ORG}/cft/kitchen-terraform:${DOCKER_TAG_BASE_KITCHEN_TERRAFORM}
-
-all: check_shell check_python check_golang check_terraform test_check_headers check_headers check_trailing_whitespace generate_docs ## Run all linters and update documentation
-
-# The .PHONY directive tells make that this isn't a real target and so
-# the presence of a file named 'check_shell' won't cause this target to stop
-# working
-.PHONY: check_shell
-check_shell: ## Lint shell scripts
-	@source test/make.sh && check_shell
-
-.PHONY: check_python
-check_python: ## Lint Python source files
-	@source test/make.sh && check_python
-
-.PHONY: check_golang
-check_golang: ## Lint Go source files
-	@source test/make.sh && golang
-
-.PHONY: check_terraform
-check_terraform: ## Lint Terraform source files
-	@source test/make.sh && check_terraform
-
-.PHONY: check_shebangs
-check_shebangs: ## Check that scripts have correct shebangs
-	@source test/make.sh && check_bash
-
-.PHONY: check_trailing_whitespace
-check_trailing_whitespace:
-	@source test/make.sh && check_trailing_whitespace
-
-.PHONY: test_check_headers
-test_check_headers:
-	@echo "Testing the validity of the header check"
-	@source test/make.sh && check_headers
+DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.4.6
+DOCKER_IMAGE_DEVELOPER_TOOLS := cft/developer-tools
+REGISTRY_URL := gcr.io/cloud-foundation-cicd
 
-.PHONY: check_headers
-check_headers: ## Check that source files have appropriate boilerplate
-	@source test/make.sh && check_headers
-
-# Integration tests
-.PHONY: test_integration
-test_integration: ## Run integration tests
-	bundle install
-	bundle exec kitchen create
-	bundle exec kitchen converge
-	bundle exec kitchen converge
-	bundle exec kitchen verify
-	bundle exec kitchen destroy
-
-.PHONY: generate_docs
-generate_docs: ## Update README documentation for Terraform variables and outputs
-	@source test/make.sh && generate_docs
-
-# Run docker
+# Enter docker container for local development
 .PHONY: docker_run
-docker_run: ## Launch a shell within the Docker test environment
+docker_run:
 	docker run --rm -it \
 		-e SERVICE_ACCOUNT_JSON \
-		-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
-		-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-		-v $(CURDIR):/cft/workdir \
-		${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
 		/bin/bash
 
-.PHONY: docker_create
-docker_create: ## Run `kitchen create` within the Docker test environment
+# Execute prepare tests within the docker container
+.PHONY: docker_test_prepare
+docker_test_prepare:
 	docker run --rm -it \
 		-e SERVICE_ACCOUNT_JSON \
-		-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
-		-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-		-v $(CURDIR):/cft/workdir \
-		${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
-		/bin/bash -c "kitchen create"
-
-.PHONY: docker_converge
-docker_converge: ## Run `kitchen converge` within the Docker test environment
+		-e TF_VAR_org_id \
+		-e TF_VAR_folder_id \
+		-e TF_VAR_billing_account \
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/execute_with_credentials.sh prepare_environment
+
+# Clean up test environment within the docker container
+.PHONY: docker_test_cleanup
+docker_test_cleanup:
 	docker run --rm -it \
 		-e SERVICE_ACCOUNT_JSON \
-		-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
-		-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-		-v $(CURDIR):/cft/workdir \
-		${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
-		/bin/bash -c "kitchen converge && kitchen converge"
-
-.PHONY: docker_verify
-docker_verify: ## Run `kitchen verify` within the Docker test environment
+		-e TF_VAR_org_id \
+		-e TF_VAR_folder_id \
+		-e TF_VAR_billing_account \
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/execute_with_credentials.sh cleanup_environment
+
+# Execute integration tests within the docker container
+.PHONY: docker_test_integration
+docker_test_integration:
 	docker run --rm -it \
 		-e SERVICE_ACCOUNT_JSON \
-		-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
-		-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-		-v $(CURDIR):/cft/workdir \
-		${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
-		/bin/bash -c "kitchen verify"
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/test_integration.sh
 
-.PHONY: docker_destroy
-docker_destroy: ## Run `kitchen destroy` within the Docker test environment
+# Execute lint tests within the docker container
+.PHONY: docker_test_lint
+docker_test_lint:
 	docker run --rm -it \
-		-e SERVICE_ACCOUNT_JSON \
-		-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
-		-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
-		-v $(CURDIR):/cft/workdir \
-		${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
-		/bin/bash -c "kitchen destroy"
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/test_lint.sh
 
-.PHONY: test_integration_docker
-test_integration_docker: docker_create docker_converge docker_verify docker_destroy ## Run a full integration test cycle
-	@echo "Running test-kitchen tests in docker"
+# Generate documentation
+.PHONY: docker_generate_docs
+docker_generate_docs:
+	docker run --rm -it \
+		-v "$(CURDIR)":/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate_docs'
 
-help: ## Prints help for targets with comments
-	@grep -E '^[a-zA-Z._-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
+# Alias for backwards compatibility
+.PHONY: generate_docs
+generate_docs: docker_generate_docs
diff --git a/README.md b/README.md
@@ -1,25 +1,28 @@
-# [Google Cloud Memorystore Terraform Module](https://registry.terraform.io/modules/terraform-google-modules/memorystore/google/)
+# terraform-google-memorystore
 
-A Terraform module for creating a fully functional Google Memorystore (redis) instance.
+This module was generated from [terraform-google-module-template](https://github.com/terraform-google-modules/terraform-google-module-template/), which by default generates a module that simply creates a GCS bucket. As the module develops, this README should be updated.
 
-## Compatibility
-This module is meant for use with Terraform 0.12. If you haven't [upgraded](https://www.terraform.io/upgrade-guides/0-12.html) and need a Terraform 0.11.x-compatible version of this module, the last released version intended for Terraform 0.11.x
-is [0.1.0](https://registry.terraform.io/modules/terraform-google-modules/memorystore/google/0.1.0).
+The resources/services/activations/deletions that this module will create/trigger are:
+
+- Create a GCS bucket with the provided name
 
 ## Usage
 
-Check the [examples/](./examples/) directory for more.
+Basic usage of this module is as follows:
 
 ```hcl
 module "memorystore" {
   source  = "terraform-google-modules/memorystore/google"
-  version = "1.0.0"
+  version = "~> 0.1"
 
-  name    = "my-memorystore"
-  project = "my-gcp-project"
+  project_id  = "<PROJECT ID>"
+  bucket_name = "gcs-test-bucket"
 }
 ```
 
+Functional examples are included in the
+[examples](./examples/) directory.
+
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 ## Inputs
 
@@ -50,55 +53,44 @@ module "memorystore" {
 
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 
-## File structure
-
-The project has the following folders and files:
+## Requirements
 
-- /: root folder
-- /examples: examples for using this module
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file)
-- /test: Folders with files for testing the module (see Testing section on this file)
-- /helpers: Optional helper scripts for ease of use
-- /main.tf: main file for this module, contains all the resources to create
-- /variables.tf: all the variables for the module
-- /output.tf: the outputs of the module
-- /readme.md: this file
+These sections describe requirements for using this module.
 
-## Testing
+### Software
 
-### Requirements
+The following dependencies must be available:
 
-- Terraform is [installed](#software-dependencies) on the machine where Terraform is executed.
-- An existing google cloud project
-- The `redis.googleapis.com` API should be enabled
-- A service account key
-- Docker
+- [Terraform][terraform] v0.12
+- [Terraform Provider for GCP][terraform-provider-gcp] plugin v2.0
 
-### Software Dependencies
-### Terraform
-- [Terraform](https://www.terraform.io/downloads.html) >= 0.12.0
-- [terraform-provider-google](https://github.com/terraform-providers/terraform-provider-google) >= v2.5.0
+### Service Account
 
-### Running Integration Tests
+A service account with the following roles must be used to provision
+the resources of this module:
 
-Tests are run inside of an existing google cloud project.
+- Storage Admin: `roles/storage.admin`
 
-1. Create tfvars file for test:
+The [Project Factory module][project-factory-module] and the
+[IAM module][iam-module] may be used in combination to provision a
+service account with the necessary roles applied.
 
-```sh
-cp test/fixtures/minimal/terraform.tfvars.example test/fixtures/minimal/terraform.tfvars
-```
+### APIs
 
-2. Edit new `test/fixtures/minimal/terraform.tfvars` and add project id to run test in.
+A project with the following APIs enabled must be used to host the
+resources of this module:
 
-3. Copy a service account key (`credentials.json` file) into root of this repo.
+- Google Cloud Storage JSON API: `storage-api.googleapis.com`
 
-4. Run `make test_integration_docker`.
+The [Project Factory module][project-factory-module] can be used to
+provision a project with the necessary APIs enabled.
 
-## Autogeneration of documentation from .tf files
+## Contributing
 
-Run:
+Refer to the [contribution guidelines](./CONTRIBUTING.md) for
+information on contributing to this module.
 
-```
-make generate_docs
-```
+[iam-module]: https://registry.terraform.io/modules/terraform-google-modules/iam/google
+[project-factory-module]: https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
+[terraform-provider-gcp]: https://www.terraform.io/docs/providers/google/index.html
+[terraform]: https://www.terraform.io/downloads.html