Skip to content
This repository was archived by the owner on Jul 5, 2023. It is now read-only.

add documentation around archiving configuration versions #2157

Merged
merged 13 commits into from
Mar 12, 2022
7 changes: 7 additions & 0 deletions content/cloud-docs/api-docs/changelog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ page_id: api-changelog

Keep track of changes to the API for Terraform Cloud and Terraform Enterprise.

### 2022-03-11

* Introduced [Archiving Configuration Versions](/cloud-docs/workspaces/configurations#archiving-configuration-versions).
* Updated [Configuration Versions](/cloud-docs/api-docs/configuration-versions#attributes) to add new `fetching` and `archived` states.
* Updated [Runs](/cloud-docs/api-docs/runs#attributes) to add new `fetching` state.
* Added the [Archive a Configuration Version](/cloud-docs/api-docs/configuration-versions#archive-a-configuration-version) endpoint.

### 2021-12-09

* Added `variables` field for POST /runs and the run resource, enabling run-specific variable values.
Expand Down
60 changes: 50 additions & 10 deletions content/cloud-docs/api-docs/configuration-versions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,22 @@ You need read runs permission to list and view configuration versions for a work

[permissions-citation]: #intentionally-unused---keep-for-maintainers

## Attributes

### Configuration Version States

The configuration version state is found in `data.attributes.status`, and you can reference the following list of possible states.

A configuration version created through the API or CLI can only be used in runs if it is in an `uploaded` state. A configuration version created through a linked VCS repository may also be used in runs if it is in an `archived` state.

| State | Description |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pending` | The initial status of a configuration version after it has been created. Pending configuration versions cannot be used to create new runs. |
| `fetching` | For configuration versions created from a commit to a connected VCS repository, Terraform Cloud is currently fetching the associated files from VCS. |
| `uploaded` | The configuration version is fully processed and can be used in runs. |
| `archived` | All immediate runs are complete and Terraform Cloud has discarded the files associated with this configuration version. If the configuration version was created through a connected VCS repository, it can still be used in new runs. In those cases, Terraform Cloud will re-fetch the files from VCS. |
| `errored` | Terraform Cloud could not process this configuration version, and it cannot be used to create new runs. You can try again by pushing a new commit to your linked VCS repository or creating a new configuration version with the API or CLI. |

## List Configuration Versions

`GET /workspaces/:workspace_id/configuration-versions`
Expand Down Expand Up @@ -316,16 +332,6 @@ curl \

Once a configuration version is created, use the `upload-url` attribute to [upload configuration files](#upload-configuration-files) associated with that version. The `upload-url` attribute is only provided in the response when creating configuration versions.

### Configuration Version Status

A configuration version will be in the `pending` status when initially created. It will remain pending until configuration files are supplied via [upload](#upload-configuration-files), and while they are processed.

If upload and processing succeed, the configuration version status will then be `uploaded`. An `uploaded` configuration version is ready for use.

If upload and processing fail, the status will be `errored`, indicating that something went wrong.

Runs cannot be created using `pending` or `errored` configuration versions.

## Upload Configuration Files

-> **Note**: If `auto-queue-runs` was either not provided or set to `true` during creation of the configuration version, a run using this configuration version will be automatically queued on the workspace. If `auto-queue-runs` was set to `false` explicitly, then it is necessary to [create a run on the workspace](/cloud-docs/api-docs/run#create-a-run) manually after the configuration version is uploaded.
Expand All @@ -346,6 +352,40 @@ curl \
https://archivist.terraform.io/v1/object/4c44d964-eba7-4dd5-ad29-1ece7b99e8da
```

## Archive a Configuration Version

`POST /configuration-versions/:configuration_version_id/actions/archive`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this something we want to add a version note on for when this is actually available for enterprise side folks? e.g., like is done here for example

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I added it in the changelog but totally forgot to add it here 😆


| Parameter | Description |
| --------------------------- | ---------------------------------------------- |
| `configuration_version_id` | The ID of the configuration version to archive. |

This endpoint notifies Terraform Cloud to discard the uploaded `.tar.gz` file associated with this configuration version. This endpoint can only archive configuration versions that were created with the API or CLI, are in an `uploaded` [state](#configuration-version-states), and have no runs in progress. Otherwise, calling this endpoint will result in an error.

Terraform Cloud automatically archives configuration versions created through VCS when associated runs are complete and then re-fetches the files for subsequent runs.

-> **Note:** This endpoint is available in Terraform Enterprise v202203-1 or later.

| Status | Response | Reason(s) |
| ------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| [202][] | none | Successfully initiated the archive process. |
| [409][] | [JSON API error object][] | Configuration version was in a non-archivable state or the configuration version was created with VCS and cannot be archived through the API. |
| [404][] | [JSON API error object][] | Configuration version was not found or user not authorized. |

### Request Body

This POST endpoint does not take a request body.

### Sample Request

```shell
curl \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/vnd.api+json" \
--request POST \
https://app.terraform.io/api/v2/configuration-versions/cv-ntv3HbhJqvFzamy7/actions/archive
```

## Available Related Resources

The GET endpoints above can optionally return related resources, if requested with [the `include` query parameter](/cloud-docs/api-docs/#inclusion-of-related-resources). The following resource types are available:
Expand Down
1 change: 1 addition & 0 deletions content/cloud-docs/api-docs/run.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ The run state is found in `data.attributes.status`, and you can reference the fo
| State | Description |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pending` | The initial status of a run once it has been created. |
| `fetching` | The run is waiting for Terraform Cloud to fetch the configuration from VCS. |
| `plan_queued` | Once a workspace has the availability to start a new run, the next run will transition to `plan_queued`. This status indicates that the it should start as soon as the backend services that run terraform have available capacity. In Terraform Cloud, you should seldom see this status, as our aim is to always have capacity. However, in Terraform Enterprise this status will be more common due to the self-hosted nature. |
| `planning` | The planning phase of a run is in progress. |
| `planned` | The planning phase of a run has completed. |
Expand Down
26 changes: 15 additions & 11 deletions content/cloud-docs/run/states.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,26 +19,30 @@ _States in this stage:_

_Leaving this stage:_

- Proceeds automatically to the plan stage (**Planning** state) when it becomes the first run in the queue.
- Can skip to completion (**Discarded** state) if a user discards it before it starts.
- If the user discards the run before it starts, the run does not continue (**Discarded** state).
- If the run is first in the queue, it proceeds automatically to the plan stage (**Planning** state).

## 2. The Plan Stage

A run goes through different steps during the plan stage depending on whether or not Terraform Cloud needs to fetch the configuration from VCS. Terraform Cloud automatically archives configuration versions created through VCS when all runs are complete and then re-fetches the files for subsequent runs.

_States in this stage:_

- **Fetching:** If Terraform Cloud has not yet fetched the configuration from VCS, the run will go into this state until the configuration is available.
- **Planning:** Terraform Cloud is currently running `terraform plan`.
- **Needs Confirmation:** `terraform plan` has finished. Runs sometimes pause in this state, depending on the workspace and organization settings.

_Leaving this stage:_

- If the `terraform plan` command failed, the run skips to completion (**Plan Errored** state).
- If a user canceled the plan by pressing the "Cancel Run" button, the run skips to completion (**Canceled** state).
- If the plan succeeded and neither cost estimation nor Sentinel policy checks will be done, the run skips to completion (**Planned** state).
- If Terraform Cloud encounters an error when fetching the configuration from VCS, the run does not continue (**Plan Errored** state).
- If the `terraform plan` command failed, the run does not continue (**Plan Errored** state).
- If a user canceled the plan by pressing the "Cancel Run" button, the run does not continue (**Canceled** state).
- If the plan succeeded with no changes and neither cost estimation nor Sentinel policy checks will be done, Terraform Cloud considers the run complete (**Planned and Finished** state).
- If the plan succeeded and requires changes:
- If cost estimation is enabled, the run proceeds automatically to the cost estimation stage.
- If cost estimation is disabled and [Sentinel policies][] are enabled, the run proceeds automatically to the policy check stage.
- If there are no Sentinel policies and the plan can be auto-applied, the run proceeds automatically to the apply stage. Plans can be auto-applied if the auto-apply setting is enabled on the workspace and the plan was queued by a new VCS commit or by a user with permission to apply runs. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions))
- If there are no Sentinel policies and the plan can't be auto-applied, the run pauses in the **Needs Confirmation** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) The run proceeds to the apply stage if they approve the apply, or skips to completion (**Discarded** state) if they reject the apply.
- If there are no Sentinel policies and Terraform Cloud cannot auto-apply the plan, the run pauses in the **Needs Confirmation** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) If an authorized user approves the apply, the run proceeds to the apply stage. If an authorized user rejects the apply, the run does not continue (**Discarded** state).

[permissions-citation]: #intentionally-unused---keep-for-maintainers

Expand All @@ -54,7 +58,7 @@ _States in this stage:_
_Leaving this stage:_

- If cost estimation succeeded or errors, the run moves to the next stage.
- If there are no policy checks or applies, the run skips to completion (**Planned (Finished)** state).
- If there are no policy checks or applies, the run does not continue (**Planned and Finished** state).

## 4. The Policy Check Stage

Expand All @@ -72,13 +76,13 @@ _States in this stage:_

_Leaving this stage:_

- If any hard-mandatory policies failed, the run skips to completion (**Plan Errored** state).
- If any hard-mandatory policies failed, the run does not continue (**Plan Errored** state).
- If any soft-mandatory policies failed, the run pauses in the **Policy Override** state.
- If a user with permission to manage policy overrides, overrides the failed policy, the run proceeds to the **Policy Checked** state. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions))
- If a user with permission to apply runs discards the run, the run skips to completion (**Discarded** state). ([More about permissions.](/cloud-docs/users-teams-organizations/permissions))
- If a user with permission to apply runs discards the run, the run does not continue (**Discarded** state). ([More about permissions.](/cloud-docs/users-teams-organizations/permissions))
- If the run reaches the **Policy Checked** state (no mandatory policies failed, or soft-mandatory policies were overridden):
- If the plan can be auto-applied, the run proceeds automatically to the apply stage. Plans can be auto-applied if the auto-apply setting is enabled on the workspace and the plan was queued by a new VCS commit or by a user with permission to apply runs. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions))
- If the plan can't be auto-applied, the run pauses in the **Policy Checked** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) The run proceeds to the apply stage if they approve the apply, or skips to completion (**Discarded** state) if they reject the apply.
- If the plan can't be auto-applied, the run pauses in the **Policy Checked** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) The run proceeds to the apply stage if they approve the apply, or does not continue (**Discarded** state) if they reject the apply.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- If the plan can't be auto-applied, the run pauses in the **Policy Checked** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) The run proceeds to the apply stage if they approve the apply, or does not continue (**Discarded** state) if they reject the apply.
- If Terraform cannot auto-apply the plan, the run pauses in the **Policy Checked** state until a user with permission to apply runs takes action. ([More about permissions.](/cloud-docs/users-teams-organizations/permissions)) If an authorized user approves the apply, the run proceeds to the apply stage. If an authorized user rejects the apply, the run does not continue (**Discarded** state).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh - the second half of this is a repeat of a bullet above - do we want to remove one of them so we're not repeating ourselves?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, am I looking at the right thing? The only direct repetition I see is the "More about permissions" link 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I made that up or got confused lol - If you read the rest of the section and it's not repeated, ignore this!


[permissions-citation]: #intentionally-unused---keep-for-maintainers

Expand All @@ -103,7 +107,7 @@ A run is considered completed if it finishes applying, if any part of the run fa
_States in this stage:_

- **Applied:** The run was successfully applied.
- **Planned:** `terraform plan`'s output already matches the current infrastructure state, so `terraform apply` doesn't need to do anything.
- **Planned and Finished:** `terraform plan`'s output already matches the current infrastructure state, so `terraform apply` doesn't need to do anything.
- **Apply Errored:** The `terraform apply` command failed, possibly due to a missing or misconfigured provider or an illegal operation on a provider.
- **Plan Errored:** The `terraform plan` command failed (usually requiring fixes to variables or code), or a hard-mandatory Sentinel policy failed. The run cannot be applied.
- **Discarded:** A user chose not to continue this run.
Expand Down
10 changes: 10 additions & 0 deletions content/cloud-docs/workspaces/configurations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,13 @@ There are also a variety of ways to handle multiple environments. The most commo
- Different environments use completely separate configurations, and shared behaviors are handled with shared Terraform modules. To protect production environments, verify new module versions in staging before updating the version used in production.

Terraform Cloud works well with all of these approaches. If you used long-lived branches, be sure to specify which branch to use in each workspace's VCS connection settings.

## Archiving Configuration Versions

Once all runs using a particular configuration version are complete, Terraform Cloud no longer needs the associated `.tar.gz` file and may discard it to save storage space. This process is handled differently depending on how the configuration version was created.

- **Created with a connected VCS repository.** Terraform Cloud will automatically archive VCS configuration versions once all runs are complete and re-fetch the configuration files from VCS as needed for new runs.

- **Created with direct uploads via the API or CLI.** Terraform Cloud does not archive CLI and API configuration versions automatically, because it cannot re-fetch the files for new runs. However, you can use the [Archive a Configuration Version](/cloud-docs/api-docs/configuration-versions#archive-a-configuration-version) endpoint to archive them manually.

-> **Note:** This functionality is available in Terraform Enterprise v202203-1 or later. In earlier versions, Terraform Enterprise keeps the `.tar.gz` files associated with configuration versions until the associated workspace is deleted.