Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Karpenter] Make sure to preserve the SA during chart deletion #1012

Merged
merged 1 commit into from
Jun 20, 2024

Conversation

muckelba
Copy link
Contributor

Problem

Karpenter fails to shut down its nodes properly during cluster deletion due to a permission error caused by the premature deletion of the service account:

panic: failed to setup manager: failed to determine if *v1.Lease is namespaced: failed to get restmapping: failed to get server groups: the server has asked for the client to provide credentials

goroutine 1 [running]:
github.com/samber/lo.must({0x26921c0, 0xc0009765a0}, {0xc000b17a18, 0x1, 0x1})
	github.com/samber/[email protected]/errors.go:51 +0x1bb
github.com/samber/lo.Must[...](...)
	github.com/samber/[email protected]/errors.go:65
sigs.k8s.io/karpenter/pkg/operator.NewOperator()
	sigs.k8s.io/[email protected]/pkg/operator/operator.go:183 +0x1418
main.main()
	github.com/aws/karpenter-provider-aws/cmd/controller/main.go:33 +0x25

Solution:

Updated the dependency statement to ensure the correct deletion order:

  • Helm chart
  • Service account
  • Namespace

This prevents permission errors by ensuring the service account is deleted after Karpenter components.

Testing

I wasn't able to properly test this because everytime i npm linked my local version of this repo, i got this error:

Error: There is already a Construct with name 'HasEcrPublic' in Cluster [redacted]

The stacktrace points to the first new eks.KubernetesManifest() call in my code. Is using npm link the recommended method to test local changes?

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for your contribution!

@shapirov103 shapirov103 merged commit 3ea63c8 into aws-quickstart:main Jun 20, 2024
1 check passed
@muckelba muckelba deleted the patch-1 branch July 2, 2024 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants