Unable to create environments after recreating an app

[lukeledet]

I first created an app with this command: copilot app init myapp

Then I added 2 environments:

copilot env init --name staging --app myapp --profile default --default-config
copilot env init --name prod --app myapp --profile default --default-config --prod

Then I created 2 services:

copilot init -n intake-form -t "Load Balanced Web Service" -d Dockerfile
copilot init -n admin -t "Load Balanced Web Service" -d admin/Dockerfile

Then I setup Route53 and ACM manually to point a domain at the load balancer. I was able to access my intake-form service without an issue but after getting some help on gitter I realized I should have used the --domain flag when creating the app. Since everything was brand new I ran copilot app delete and wiped it all out. Then I recreated the app: copilot app init myapp --domain mydomain.com

This worked without a problem. But now when I try to add an environment to the app using the same commands above, I get this error:

✘ Failed to accept changes for the staging environment.

✘ check if changeset is empty: create change set copilot-21e5d8ef-daf6-4fd6-9dae-a2d3e3243ae5 for stack myapp-staging: ValidationError: Template format error: Unresolved resource dependencies [PublicLoadBalancer, DefaultHTTPTargetGroup] in the Resources block of the template
	status code: 400, request id: b3d7b6e5-3e22-43f9-8850-7dafeeab9f7b: describe change set copilot-21e5d8ef-daf6-4fd6-9dae-a2d3e3243ae5 for stack myapp-staging: ValidationError: Stack [myapp-staging] does not exist
	status code: 400, request id: 5e7eff9d-2fbe-437a-b959-b2bbe14bb896

I have tried with staging, prod and test in case it could only create environments with new names for some reason but nothing is working.

Is there something that does not get deleted in the account that I can manually delete to be able to use copilot again?

[efekarakus]

Heya @lukeledet !

Apologies 🙇‍♂️, we introduced a bug in v0.6.0 #1662 that causes this error and merged the fix recently which should go out in the next release: #1663

In the mean time to unblock yourself, you can downgrade to v0.5.0 if that doesn't work let us know and we can push the fixed version out:

curl -Lo /usr/local/bin/copilot https://github.com/aws/copilot-cli/releases/download/v0.5.0/copilot-darwin-v0.5.0 && chmod +x /usr/local/bin/copilot && copilot --help

[lukeledet]

Thank you @efekarakus I was able to downgrade to 0.5.0 but I still can't create a load balancer. I got the following error:

$ copilot env init --name staging --app ldh --profile ldh --default-config
✘ Failed to create the infrastructure for the staging environment.
- Virtual private cloud on 2 availability zones to hold your services and jobs     [Complete]
- Virtual private cloud on 2 availability zones to hold your services and jobs     [Complete]
  - Internet gateway to connect the network to the internet                        [Complete]
  - Public subnets for internet facing services and jobs                           [Complete]
  - Private subnets for services and jobs that can't be reached from the internet  [Complete]
  - Routing tables for services and jobs to talk with each other                   [Complete]
- ECS Cluster to hold your services and jobs                                       [Complete]
- Application load balancer to distribute traffic                                  [Failed]
  Resource creation cancelled
✘ wait until stack ldh-staging create is complete: ResourceNotReady: failed waiting for successful resource state

Running it again gave me this error:

✘ Failed to accept changes for the staging environment.

✘ check if changeset is empty: create change set copilot-81d5fa15-a370-4f71-a6eb-a43066d1d850 for stack ldh-staging: ValidationError: Stack [ldh-staging] already exists and cannot be created again with the changeSet [copilot-81d5fa15-a370-4f71-a6eb-a43066d1d850].
	status code: 400, request id: 5a1fc73b-6c80-4476-8b9e-969fb7ca95bb: describe change set copilot-81d5fa15-a370-4f71-a6eb-a43066d1d850 for stack ldh-staging: ChangeSetNotFound: ChangeSet [copilot-81d5fa15-a370-4f71-a6eb-a43066d1d850] does not exist
	status code: 404, request id: 6b891afa-79c3-4056-b366-b98d35e0dbbd

It seems like everything is rolling back correctly on these failures but subsequent runs give similar errors. I don't know if this is related or some other issue though

[efekarakus]

Ah interesting 🤔 once it rollbacks, is it possible to manually delete the environment's cloudformation stack and then retry the operation again?

We're in the process of doing a release so hopefully the fix for this issue should be out soon!

[lukeledet]

Yes, I could delete the stack and run it again. Same problem but I noticed that when it rolls back it doesn't delete the myapp-staging-CFNExecutionRole and myapp-staging-EnvManagerRole roles. After I remove those manually it works again!

[efekarakus]

@lukeledet Awesome! also we just released v0.7.0 that contains the fix for this issue: https://github.com/aws/copilot-cli/releases/tag/v0.7.0

It'd be super helpful for us if you don't mind giving it a try and see if the problem goes away :)

[lukeledet]

I had to manually delete the old myapp-infrastructure-roles stack but after that I was able to recreate everything and the process went very smoothly. Thanks for your help Efe!