cloud: when saving state, utilize new 'pending' state version #33336
Conversation
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
afafd00 to
58c1cb3
Compare
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
58c1cb3 to
ff9179a
Compare
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
ff9179a to
cd018f8
Compare
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
cd018f8 to
b3890b1
Compare
|
Gonna smoke test this and spy on atlas/archivist logs |
There was a problem hiding this comment.
OK, this checks out! I ran it while spying on the atlas and archivist logs, and the sequence seems right:
- atlas: GET current-state-version: 404
- atlas: POST state-versions 201
- archivist: create-object (VERSION)
- archivist: starting upload request
- archivist: PUT callback
- atlas: callback received
- atlas: setting state version as current
- atlas: stateparsing controller: parsed successfully
etc. etc.
Code also seems straightforward and good.
I'm assuming you'll want to delay this until the go-tfe version is shipped, but I'll pre-approve anyway since it's working fine; ping me for an auto- ✅ if you fixup the go.mod later.
internal/cloud/state.go
Outdated
| _, err := s.tfeClient.StateVersions.Upload(ctx, s.workspace.ID, options) | ||
| if errors.Is(err, tfe.ErrStateVersionUploadNotSupported) { | ||
| // Create the new state with content included in the request (Terraform Enterprise v202306-1 and below) | ||
| log.Println("[WARN] Trying compatibility state upload") |
There was a problem hiding this comment.
🤔 Is WARN justified for this, given that we expect compatibility mode to usually be fine? Or would INFO suffice?
There was a problem hiding this comment.
INFO or even TRACE would probably suffice.
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
b3890b1 to
7544cc3
Compare
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
7544cc3 to
00b0d78
Compare
Create a pending state version followed by a separate state upload When this version of the endpoint fails (It is not yet generally available, or when using with Terraform Enterprise) Fall back to the original call with state content included in the request. This strategy will reduce the amount of save failures due to network latency and gateway timeouts.
brandonc
force-pushed
the
TF-7056-uploading-state-directly-to-hosted-state-upload-url-when-available
branch
from
00b0d78 to
19b17ad
Compare
… PR #33336 Updating go-tfe made the mocks no longer compile, due to the new Upload method on state versions.
|
(Smoke tested both backends with the platform feature enabled and disabled using go-tfe 1.28) |
|
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
brandonc
added
the
1.5-backport
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
When using a cloud block and saving state, terraform cloud will no longer close the connection and fail to save with a gateway timeout response due to exceedingly large state or slow network operations. This utilizes a new platform feature to create a pending state version followed by uploading state content directly to the object store which has no gateway timeout. If the new feature is unavailable (during pre-release, or when using Terraform Enterprise) the legacy fallback is used.
Target Release
1.5.1
Important!⚠️
Depends on the release of hashicorp/go-tfe#717
Draft CHANGELOG entry
ENHANCEMENTS
When using a cloud block and saving state, terraform cloud will no longer close the connection and fail to save with a gateway timeout response due to exceedingly large or slow network operations.