diff --git a/docs/terraform/details.md b/docs/terraform/details.md new file mode 100644 index 00000000..624b3516 --- /dev/null +++ b/docs/terraform/details.md @@ -0,0 +1,415 @@ +### Terraform Configurations from CPLN Templates + +#### Providers + +Terraform provider configurations are controlled via `required_providers.tf` and `providers.tf`: + +- **`required_providers.tf`** + +```hcl +terraform { + required_providers { + cpln = { + source = "controlplane-com/cpln" + version = "~> 1.0" + } + } +} +``` + +- **`providers.tf`** + +```hcl +provider "cpln" { + org = "org-name-example" +} +``` + +#### GVC (Global Virtual Cloud) + +CPLN template in YAML format: + +```yaml +kind: gvc +name: app-name +description: app-description +tags: + tag-name-1: "tag-value-1" + tag-name-2: "tag-value-2" +spec: + domain: "app.example.com" + env: + - name: DATABASE_URL + value: "postgres://the_user:the_password@postgres.app-name.cpln.local:5432/app-name" + - name: RAILS_ENV + value: production + - name: RAILS_SERVE_STATIC_FILES + value: "true" + staticPlacement: + locationLinks: + - "//location/aws-us-west-2" + pullSecretLinks: + - "/org/org-name/secret/some-secret" + loadBalancer: + dedicated: true + trustedProxies: 0 +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_gvc" "app-name" { + name = "app-name" + description = "app-description" + tags = { + tag_name_1 = "tag-value-1" + tag_name_2 = "tag-value-2" + } + domain = "app.example.com" + locations = ["aws-us-west-2"] + pull_secrets = ["cpln_secret.some-secret.name"] + env = { + DATABASE_URL = "postgres://the_user:the_password@postgres.app-name.cpln.local:5432/app-name" + RAILS_ENV = "production" + RAILS_SERVE_STATIC_FILES = "true" + } + load_balancer { + dedicated = true + trusted_proxies = 0 + } +} +``` + +#### Identity + +CPLN template in YAML format: + +```yaml +kind: identity +name: postgres-poc-identity +description: postgres-poc-identity +tags: + tag-name-1: "tag-value-1" + tag-name-2: "tag-value-2" +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_identity" "postgres-poc-identity" { + name = "postgres-poc-identity" + description = "postgres-poc-identity" + tags = { + tag_name_1 = "tag-value-1" + tag_name_2 = "tag-value-2" + } +} +``` + +#### Secret + +CPLN template in YAML format + +**For `aws` secret:** + +```yaml +kind: secret +name: aws +description: aws +type: aws +data: + accessKey: 'AccessKeyExample' + externalId: 'ExternalIdExample' + roleArn: arn:awskey + secretKey: 'SecretKeyExample' +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_secret" "aws" { + name = "aws" + description = "aws" + aws { + secret_key = "SecretKeyExample" + access_key = "AccessKeyExample" + role_arn = "arn:awskey" + external_id = "ExternalIdExample" + } +} +``` + +**For `azure-connector` secret:** + +```yaml +kind: secret +name: azure-connector +description: azure_connector +tags: + tag1: tag-val +type: azure-connector +data: + code: 'CodeExample' + url: https://example.com +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_secret" "azure-connector" { + name = "azure-connector" + description = "azure_connector" + tags = { + tag1 = "tag-val" + } + azure_connector { + url = "https://example.com" + code = "CodeExample" + } +} +``` + +**For `azure-sdk-secret` secret:** + +```yaml +kind: secret +name: azure-sdk-secret +description: azure-sdk-secret +type: azure-sdk +data: >- + {"subscriptionId":"subscriptionId","tenantId":"tenantId","clientId":"clientId","clientSecret":"CONFIDENTIAL"} +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_secret" "azure-sdk-secret" { + name = "azure-sdk-secret" + description = "azure-sdk-secret" + azure_sdk = "{"subscriptionId":"subscriptionId","tenantId":"tenantId","clientId":"clientID","clientSecret":"CONFIDENTIAL"}" +} +``` + +**For `dictionary` secret:** + +```yaml +kind: secret +name: dictionary +description: dictionary +tags: {} +type: dictionary +data: + example: 'value' +``` + +Will transform to Terraform config: + +```hcl +resource "cpln_secret" "dictionary" { + name = "dictionary" + description = "dictionary" + tags = { + } + dictionary = { + example = "value" + } +} +``` + +Supported all types of the secrets which can be configured in Control Plane. + +#### Policy + +CPLN template in YAML format: + +```yaml +kind: policy +name: policy-name +description: policy description +tags: + tag1: tag1_value + tag2: tag2_value +target: all +targetKind: secret +targetLinks: +- "//secret/postgres-poc-credentials" +- "//secret/postgres-poc-entrypoint-script" +bindings: + - permissions: + - reveal + - view + - use + principalLinks: + - "//gvc/{{APP_NAME}}/identity/postgres-poc-identity" + - permissions: + - view + principalLinks: + - user/fake-user@fake-email.com +``` + +Will be transformed to Terraform config: + +```hcl +resource "cpln_policy" "policy-name" { + name = "policy-name" + description = "policy description" + tags = { + tag1 = "tag1_value" + tag2 = "tag2_value" + } + target_kind = "secret" + gvc = cpln_gvc.app-name.name + target = "all" + target_links = ["postgres-poc-credentials", "postgres-poc-entrypoint-script"] + binding { + permissions = ["reveal", "view", "use"] + principal_links = ["gvc/app-name/identity/postgres-poc-identity"] + } + binding { + permissions = ["view"] + principal_links = ["user/fake-user@fake-email.com"] + } +} +``` + +#### Volumeset + +CPLN template in YAML format: + +```yaml +kind: volumeset +name: postgres-poc-vs +description: postgres-poc-vs +spec: + autoscaling: + maxCapacity: 1000 + minFreePercentage: 1 + scalingFactor: 1.1 + fileSystemType: ext4 + initialCapacity: 10 + performanceClass: general-purpose-ssd + snapshots: + createFinalSnapshot: true + retentionDuration: 7d +``` + +Will be transformed to Terraform config: + +```hcl +resource "cpln_volume_set" "postgres-poc-vs" { + gvc = cpln_gvc.app-name.name + name = "postgres-poc-vs" + description = "postgres-poc-vs" + initial_capacity = 10 + performance_class = "general-purpose-ssd" + file_system_type = "ext4" + snapshots { + create_final_snapshot = true + retention_duration = "7d" + } + autoscaling { + max_capacity = 1000 + min_free_percentage = 1 + scaling_factor = 1.1 + } +} +``` + +#### Workload + +CPLN template in YAML format: + +```yaml +kind: workload +name: rails +spec: + type: standard + containers: + - name: rails + cpu: 300m + env: + - name: LOG_LEVEL + value: debug + inheritEnv: true + image: {{APP_IMAGE_LINK}} + memory: 512Mi + ports: + - number: 3000 + protocol: http + defaultOptions: + autoscaling: + maxScale: 1 + capacityAI: false + firewallConfig: + external: + inboundAllowCIDR: + - 0.0.0.0/0 + outboundAllowCIDR: + - 0.0.0.0/0 +``` + +Will be transformed to Terraform configs: + +- **`rails.tf`** + +```hcl +module "rails" { + source = "../workload" + type = "standard" + name = "rails" + gvc = cpln_gvc.my-app-production.name + containers = { + rails: { + image: "/org/shakacode-demo/image/my-app-production:rails", + cpu: "300m", + memory: "512Mi", + inherit_env: true, + envs: local.rails_envs, + ports: [ + { + number: 3000, + protocol: "http" + } + ] + } + } + options = { + autoscaling: { + max_scale: 1 + } + capacity_ai: false + } + firewall_spec = { + external: { + inbound_allow_cidr: [ + "0.0.0.0/0" + ], + outbound_allow_cidr: [ + "0.0.0.0/0" + ] + } + } +} +``` + +Notice the `source: ../workload` line - there is a common `workload` module which is used for generating Terraform configs from workload templates: +``` +workload/ +├── main.tf -- Configurable workload resource in HCL +├── required_providers.tf -- Required providers for Terraform in HCL +├── variables.tf -- Variables used to configure workload resource above +``` + +- **`rails_envs.tf`** + +```hcl +locals { + rails_envs = { + LOG_LEVEL = "debug" + } +} +``` + +### References + +- [Control Plane Terraform Provider](https://registry.terraform.io/providers/controlplane-com/cpln/latest/docs) diff --git a/docs/terraform/example/.controlplane/controlplane.yml b/docs/terraform/example/.controlplane/controlplane.yml new file mode 100644 index 00000000..2f257e3f --- /dev/null +++ b/docs/terraform/example/.controlplane/controlplane.yml @@ -0,0 +1,29 @@ +allow_org_override_by_env: true +allow_app_override_by_env: true + +aliases: + common: &common + cpln_org: my-org-staging + default_location: aws-us-east-2 + setup_app_templates: + - app + - postgres + - rails + one_off_workload: rails + app_workloads: + - rails + additional_workloads: + - postgres +apps: + rails-app-staging: + <<: *common + hooks: + post_creation: bundle exec rake db:prepare + pre_deletion: bundle exec rake db:drop + + rails-app-production: + <<: *common + allow_org_override_by_env: false + allow_app_override_by_env: false + cpln_org: my-org-production + upstream: rails-app-staging diff --git a/docs/terraform/example/.controlplane/templates/app.yml b/docs/terraform/example/.controlplane/templates/app.yml new file mode 100644 index 00000000..132e94a6 --- /dev/null +++ b/docs/terraform/example/.controlplane/templates/app.yml @@ -0,0 +1,38 @@ +kind: gvc +name: {{APP_NAME}} +description: Global Virtual Cloud for Rails Application +spec: + env: + - name: DATABASE_URL + value: "postgres://user:password@postgres.{{APP_NAME}}.cpln.local:5432/{{APP_NAME}}" + - name: RAILS_ENV + value: production + - name: RAILS_SERVE_STATIC_FILES + value: "true" + staticPlacement: + locationLinks: + - {{APP_LOCATION_LINK}} + pullSecretLinks: + - "/org/org-name/secret/rails-app-secret" + loadBalancer: + dedicated: true + trustedProxies: 0 + +--- + +kind: identity +name: rails-app-identity +description: Identity for Rails Application +tags: + environment: production + +--- + +kind: secret +name: rails-app-secret +description: Secret for Rails Application +type: aws +data: + accessKey: 'AccessKeyExample' + secretKey: 'SecretKeyExample' + region: 'us-west-2' diff --git a/docs/terraform/example/.controlplane/templates/postgres.yml b/docs/terraform/example/.controlplane/templates/postgres.yml new file mode 100644 index 00000000..69371175 --- /dev/null +++ b/docs/terraform/example/.controlplane/templates/postgres.yml @@ -0,0 +1,30 @@ +kind: workload +name: postgres +spec: + type: standard + containers: + - name: postgres + cpu: 500m + env: + - name: POSTGRES_USER + value: "user" + - name: POSTGRES_PASSWORD + value: "password" + - name: POSTGRES_DB + value: "rails_app" + inheritEnv: true + image: "postgres:latest" + memory: 1Gi + ports: + - number: 5432 + protocol: tcp + defaultOptions: + autoscaling: + maxScale: 1 + capacityAI: false + firewallConfig: + external: + inboundAllowCIDR: + - 0.0.0.0/0 + outboundAllowCIDR: + - 0.0.0.0/0 diff --git a/docs/terraform/example/.controlplane/templates/rails.yml b/docs/terraform/example/.controlplane/templates/rails.yml new file mode 100644 index 00000000..83b56acf --- /dev/null +++ b/docs/terraform/example/.controlplane/templates/rails.yml @@ -0,0 +1,26 @@ +kind: workload +name: rails +spec: + type: standard + containers: + - name: rails + cpu: 300m + env: + - name: LOG_LEVEL + value: debug + inheritEnv: true + image: {{APP_IMAGE_LINK}} + memory: 512Mi + ports: + - number: 3000 + protocol: http + defaultOptions: + autoscaling: + maxScale: 1 + capacityAI: false + firewallConfig: + external: + inboundAllowCIDR: + - 0.0.0.0/0 + outboundAllowCIDR: + - 0.0.0.0/0 diff --git a/docs/terraform/overview.md b/docs/terraform/overview.md new file mode 100644 index 00000000..02878d94 --- /dev/null +++ b/docs/terraform/overview.md @@ -0,0 +1,105 @@ +# Terraform + +## Overview + +The Terraform feature in this project allows you to manage your Control Plane (CPLN) configurations using Terraform by: +1. Generating Terraform configuration files from existing CPLN YAML configuration files +2. Easily importing existing infrastructure into Terraform management + +You can continue working with CPLN configuration files in YAML format and start using Terraform at any time. + +## Benefits of Using Terraform Over YAML Configs + +1. **State Management**: Terraform maintains a state file that tracks the current state of your infrastructure, making it easier to manage changes and updates. +2. **Dependency Management**: Terraform automatically handles dependencies between resources, ensuring that they are created or destroyed in the correct order. +3. **Multi-Cloud Support**: With Terraform, you can manage resources across multiple cloud providers seamlessly, allowing for a more flexible architecture. +4. **Plan and Apply**: Terraform provides a clear plan of what changes will be made before applying them, reducing the risk of unintended modifications. + +## Usage + +Let's take a look at how to deploy a [simple Rails application](example/.controlplane/controlplane.yml) on CPLN using Terraform: + +``` +.controlplane/ +├── templates/ +│ ├── app.yml -- GVC config +│ ├── postgres.yml -- Workload config for PostgreSQL +│ └── rails.yml -- Workload config for Rails +└── controlplane.yml -- Configs for overall application +``` + +### Generating Terraform configurations + +To generate Terraform configurations, run the following command from the project root: + +```sh +cpflow terraform generate +``` + +Invoking this command will generate a new `terraform` folder with subfolders containing Terraform configurations for each application described in `controlplane.yml`: + +``` +terraform/ +├── rails-app-production/ -- Terraform configurations for production environment +│ ├── gvc.tf -- GVC config in HCL +│ ├── identities.tf -- Identities config in HCL +│ ├── postgres.tf -- Postgres workload config in HCL +│ ├── postgres_envs.tf -- ENV variables for Postgres workload in HCL +│ ├── providers.tf -- Providers config in HCL +│ ├── rails.tf -- Rails workload config in HCL +│ ├── rails_envs.tf -- ENV variables for Rails workload in HCL +│ ├── required_providers.tf -- Required providers config in HCL +│ └── secrets.tf -- Secrets config in HCL +├── rails-app-staging/ -- Terraform configurations for staging environment +│ ├── gvc.tf -- GVC config in HCL +│ ├── identities.tf -- Identities config in HCL +│ ├── postgres.tf -- Postgres workload config in HCL +│ ├── postgres_envs.tf -- ENV variables for Postgres workload in HCL +│ ├── providers.tf -- Providers config in HCL +│ ├── rails.tf -- Rails workload config in HCL +│ ├── rails_envs.tf -- ENV variables for Rails workload in HCL +│ ├── required_providers.tf -- Required providers config in HCL +│ └── secrets.tf -- Secrets config in HCL +├── workload/ -- Terraform configurations for workload module +│ ├── main.tf -- Main config for workload resource in HCL +│ ├── required_providers.tf -- Required providers for Terraform in HCL +│ └── variables.tf -- Variables used to create config for workload resource in HCL +``` + +### Importing existing infrastructure + +Now we need to import existing infrastructure into Terraform management because some resources can already exist on CPLN and Terraform needs to know about this: + +```sh +cpflow terraform import +``` + +This command will initialize Terraform and import resources defined in your `controlplane.yml` and `templates` folder into the Terraform state for each application. + +Please note that during the import process, you may encounter errors indicating that non-existing resources are being imported. This is expected behavior and can be safely ignored. + +### Application deployment using Terraform + +Preparations are complete, and now we can use Terraform commands directly to deploy our application. + +1. **Navigate to the Application Folder**: + ```sh + cd terraform/rails-app-staging + ``` + +2. **Plan the Deployment**: + ```sh + terraform plan + ``` + +3. **Apply the Configuration**: + ```sh + terraform apply + ``` + +You can visit [Details](details.md) to learn more about how CPLN templates in YAML format are transformed to Terraform configurations. + +## References + +- [Terraform Provider Plugin](https://shakadocs.controlplane.com/terraform/installation#terraform-provider-plugin) +- [Terraform - Control Plane Examples](https://github.com/controlplane-com/examples/tree/main/terraform)