Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use precomputed terraform plan for bundle deploy #1640

Merged
merged 89 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
4bedb54
rename Location -> Locations. Also add bundle.config.GetLocations fun…
shreyas-goenka Jul 18, 2024
7ab131f
validate.Location() -> validate.Locations()
shreyas-goenka Jul 18, 2024
39b3eea
fix TestSyncIncludeExcludeNoMatchesTest
shreyas-goenka Jul 18, 2024
14a200a
patch to transform location -> locations
shreyas-goenka Jul 18, 2024
b380344
add support for printing multiple locations in validate errors and wa…
shreyas-goenka Jul 18, 2024
50b8fa5
newlines in python_diagnostics_test.go
shreyas-goenka Jul 18, 2024
4d1e2a0
newlines in python_mutator_test.go
shreyas-goenka Jul 18, 2024
3b8fc61
clean up
shreyas-goenka Jul 18, 2024
83558a6
only select single location for undefined job_cluster_key
shreyas-goenka Jul 18, 2024
d9b8902
single location for validate_sync_patterns.go
shreyas-goenka Jul 18, 2024
5b94edd
add cyan color to location
shreyas-goenka Jul 19, 2024
52c2d2e
Fix test in python_diagnostics_test.go
shreyas-goenka Jul 19, 2024
6ed74d0
fix comment for Diagnostic.Locations
shreyas-goenka Jul 19, 2024
1202bcc
include multiple locations for unknown field warning
shreyas-goenka Jul 19, 2024
16072f6
fmt
shreyas-goenka Jul 19, 2024
6c24df8
Error on duplicate resource keys after YAML files have been loaded
shreyas-goenka Jul 19, 2024
0d76a42
Associate multiple paths with a diagnostic
shreyas-goenka Jul 22, 2024
14cd225
add tests and modify rendering
shreyas-goenka Jul 22, 2024
a550760
fix unit tests
shreyas-goenka Jul 23, 2024
21781fe
wip
shreyas-goenka Jul 22, 2024
b78754d
Merge remote-tracking branch 'origin' into error-when-dup-resource
shreyas-goenka Jul 23, 2024
a1e504f
Merge branch 'multi-path-diagnostics' into error-when-dup-resource
shreyas-goenka Jul 23, 2024
a602858
Merge remote-tracking branch 'origin' into diag-multiple-locations
shreyas-goenka Jul 23, 2024
83183fe
merge
shreyas-goenka Jul 23, 2024
a489095
first version
shreyas-goenka Jul 23, 2024
841deea
Merge remote-tracking branch 'origin' into multi-path-diagnostics
shreyas-goenka Jul 23, 2024
a1f1f5c
inline code to render paths in template
shreyas-goenka Jul 23, 2024
bb32270
Merge remote-tracking branch 'origin' into error-when-dup-resource
shreyas-goenka Jul 23, 2024
4fc6ce6
merge
shreyas-goenka Jul 24, 2024
24338e2
fmt
shreyas-goenka Jul 24, 2024
be5361c
enforce sorted mapping walks by default, and get the tests ready
shreyas-goenka Jul 24, 2024
ef2b64d
add comment for the validation mutator
shreyas-goenka Jul 24, 2024
76ed78d
coverage for the multiple resources case
shreyas-goenka Jul 24, 2024
b9aade4
Merge remote-tracking branch 'origin' into multi-path-diagnostics
shreyas-goenka Jul 24, 2024
4f0aaf0
fmt
shreyas-goenka Jul 25, 2024
04cb868
merge new paths
shreyas-goenka Jul 25, 2024
556a3d2
Merge remote-tracking branch 'origin' into error-when-dup-resource
shreyas-goenka Jul 25, 2024
b4c9379
cleanup
shreyas-goenka Jul 25, 2024
ee68e78
-
shreyas-goenka Jul 25, 2024
c2908ab
Merge remote-tracking branch 'origin' into error-when-dup-resource
shreyas-goenka Jul 25, 2024
0c2238b
nit
shreyas-goenka Jul 25, 2024
04b270b
remove unnecessary unit tests
shreyas-goenka Jul 26, 2024
447764e
Merge remote-tracking branch 'origin' into error-when-dup-resource
shreyas-goenka Jul 26, 2024
29cad8c
fix basic test
shreyas-goenka Jul 26, 2024
f1ffa59
some cleanup
shreyas-goenka Jul 26, 2024
8298f7d
fix windows tests
shreyas-goenka Jul 26, 2024
5e687c2
getting started scaffold
shreyas-goenka May 2, 2024
1b5c6a7
added more conversion
shreyas-goenka May 2, 2024
abd8924
add conversion for schemas
shreyas-goenka May 3, 2024
c61a969
added integration test
shreyas-goenka May 3, 2024
5f8d6cd
lint
shreyas-goenka May 6, 2024
782f903
-
shreyas-goenka May 6, 2024
86d5a33
fix failing test
shreyas-goenka May 6, 2024
3b9cb98
remove todo
shreyas-goenka May 6, 2024
cbd018c
cleanup comments
shreyas-goenka May 6, 2024
c8d6ff9
fix schema
shreyas-goenka May 7, 2024
4cf193b
cleanup todo
shreyas-goenka May 7, 2024
7faca08
-
shreyas-goenka May 7, 2024
8735a8a
add paths to schema
shreyas-goenka Jul 23, 2024
0c02ed1
Return early in bundle destroy if no deployment exists
shreyas-goenka Jul 8, 2024
6060989
-
shreyas-goenka Jul 8, 2024
0b26cbc
add test for the mutator
shreyas-goenka Jul 8, 2024
f57ec1e
fix seq
shreyas-goenka Jul 9, 2024
6da6e0c
some cleanup
shreyas-goenka Jul 9, 2024
028d9c4
add tests for the break signal
shreyas-goenka Jul 9, 2024
25d8a0b
more coverage for assert root path
shreyas-goenka Jul 9, 2024
76eea20
use if mutator instead
shreyas-goenka Jul 9, 2024
1f872d9
move if to bundle package
shreyas-goenka Jul 9, 2024
3e0b0d0
Move to a single prompt during `bundle destroy`
shreyas-goenka Jul 9, 2024
bc064f3
remove unnecessary flag
shreyas-goenka Jul 9, 2024
f62b875
wip adding support for prompting in UC schema creation
shreyas-goenka Jul 23, 2024
694a84a
fix build and dmt
shreyas-goenka Jul 26, 2024
3127843
cleanup todos
shreyas-goenka Jul 26, 2024
6c83903
remove paths from schema resources
shreyas-goenka Jul 26, 2024
5d213dd
merge
shreyas-goenka Jul 29, 2024
eeafcb4
cleanup
shreyas-goenka Jul 29, 2024
52889fb
modify schema tag comment
shreyas-goenka Jul 29, 2024
a2b9a77
prompting done (initial)
shreyas-goenka Jul 29, 2024
fc031e7
fix lock releases and setting force_destroy
shreyas-goenka Jul 29, 2024
5575ca8
fix TestConvertSchema
shreyas-goenka Jul 30, 2024
aa5f4fa
cleanup
shreyas-goenka Jul 30, 2024
9752431
support for auto-approve
shreyas-goenka Jul 30, 2024
caf318d
add test for non-tty error
shreyas-goenka Jul 30, 2024
7969a77
undo sorted pairs change
shreyas-goenka Jul 30, 2024
89f3e30
address comments
shreyas-goenka Jul 31, 2024
e3826e8
rename approvalForDeploy -> approvalForUcSchemaDelete
shreyas-goenka Jul 31, 2024
03d5f1d
Use precomputed terraform plan for `bundle deploy`
shreyas-goenka Jul 31, 2024
50c640f
merge
shreyas-goenka Jul 31, 2024
223d2e8
handle tf state being absent
shreyas-goenka Jul 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions bundle/deploy/terraform/apply.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"context"

"github.com/databricks/cli/bundle"
"github.com/databricks/cli/libs/cmdio"
"github.com/databricks/cli/libs/diag"
"github.com/databricks/cli/libs/log"
"github.com/hashicorp/terraform-exec/tfexec"
Expand All @@ -17,28 +16,32 @@ func (w *apply) Name() string {
}

func (w *apply) Apply(ctx context.Context, b *bundle.Bundle) diag.Diagnostics {
// return early if plan is empty
if b.Plan.IsEmpty {
log.Debugf(ctx, "No changes in plan. Skipping terraform apply.")
return nil
}

tf := b.Terraform
if tf == nil {
return diag.Errorf("terraform not initialized")
}

cmdio.LogString(ctx, "Deploying resources...")

err := tf.Init(ctx, tfexec.Upgrade(true))
if err != nil {
return diag.Errorf("terraform init: %v", err)
if b.Plan.Path == "" {
return diag.Errorf("no plan found")
}

err = tf.Apply(ctx)
// Apply terraform according to the computed plan
err := tf.Apply(ctx, tfexec.DirOrPlan(b.Plan.Path))
if err != nil {
return diag.Errorf("terraform apply: %v", err)
}

log.Infof(ctx, "Resource deployment completed")
log.Infof(ctx, "terraform apply completed")
return nil
}

// Apply returns a [bundle.Mutator] that runs the equivalent of `terraform apply`
// Apply returns a [bundle.Mutator] that runs the equivalent of `terraform apply ./plan`
// from the bundle's ephemeral working directory for Terraform.
func Apply() bundle.Mutator {
return &apply{}
Expand Down
46 changes: 0 additions & 46 deletions bundle/deploy/terraform/destroy.go

This file was deleted.

8 changes: 8 additions & 0 deletions bundle/deploy/terraform/state_push.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package terraform

import (
"context"
"errors"
"io/fs"
"os"
"path/filepath"

Expand Down Expand Up @@ -34,6 +36,12 @@ func (l *statePush) Apply(ctx context.Context, b *bundle.Bundle) diag.Diagnostic

// Expect the state file to live under dir.
local, err := os.Open(filepath.Join(dir, TerraformStateFileName))
if errors.Is(err, fs.ErrNotExist) {
// The state file can be absent if terraform apply is skipped because
// there are no changes to apply in the plan.
log.Debugf(ctx, "Local terraform state file does not exist.")
return nil
}
if err != nil {
return diag.FromErr(err)
}
Expand Down
5 changes: 4 additions & 1 deletion bundle/phases/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,10 @@ func Deploy() bundle.Mutator {
// Core mutators that CRUD resources and modify deployment state. These
// mutators need informed consent if they are potentially destructive.
deployCore := bundle.Defer(
terraform.Apply(),
bundle.Seq(
bundle.LogString("Deploying resources..."),
terraform.Apply(),
),
bundle.Seq(
terraform.StatePush(),
terraform.Load(),
Expand Down
2 changes: 1 addition & 1 deletion bundle/phases/destroy.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ func approvalForDestroy(ctx context.Context, b *bundle.Bundle) (bool, error) {
func Destroy() bundle.Mutator {
// Core destructive mutators for destroy. These require informed user consent.
destroyCore := bundle.Seq(
terraform.Destroy(),
terraform.Apply(),
terraform.StatePush(),
files.Delete(),
bundle.LogString("Destroy complete!"),
Expand Down
Loading