Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

charts/karmada: automatically clean up the static-resource Job after it completes #5442

Merged
merged 1 commit into from
Aug 29, 2024
Merged

charts/karmada: automatically clean up the static-resource Job after it completes #5442

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions charts/karmada/templates/karmada-static-resource-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ metadata:
spec:
parallelism: 1
completions: 1
{{- if semverCompare ">=1.23.0-0" .Capabilities.KubeVersion.GitVersion }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the .Capabilities.KubeVersion.GitVersion come from?

Is this version always comparable? I mean I can't assume all Kubernetes version format is x.x.y-0xxx.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the .Capabilities.KubeVersion.GitVersion come from?

It's helm's built-in objects.

Is this version always comparable? I mean I can't assume all Kubernetes version format is x.x.y-0xxx.

Yes, don't worry about the Kubernetes version format, it is semantic versioning. So the -0 suffix here is used to find pre-release versions, e.g., 1.24-beta.1 will still be considered greater than 1.23, see https://masterminds.github.io/sprig/semver.html for more information.

ttlSecondsAfterFinished: {{ .Values.staticResourceJob.ttlSecondsAfterFinished }}
{{- end }}
template:
metadata:
name: {{ $name }}
Expand Down
6 changes: 4 additions & 2 deletions charts/karmada/templates/post-install-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- if semverCompare "<1.23.0-0" .Capabilities.KubeVersion.GitVersion }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any particular reason why this was added here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

❯ kubectl version
Credentials expired, trying refresh token .. success
Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.6

This Job is no longer rendering for me

Copy link
Member Author

@iawia002 iawia002 Aug 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this Job for K8s version higher than 1.23, the static-resource Job will be cleaned automatically by ttlSecondsAfterFinished. Please read the description of this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Job is no longer rendering for me

Do you mean it can't be installed on your side?

Copy link
Contributor

@a7i a7i Aug 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is no longer rendered (via helm template).

The explanation makes sense. I didn't realize post-install was split into static-resources and post-install, essentially making the latter useless if using ttlSecondsAfterFinished.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything we need to do regarding this?

By the way, I pined you on the slack @a7i about the chart ownership thing :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing ❤️

{{- $name := include "karmada.name" . -}}
{{- $namespace := include "karmada.namespace" . -}}
{{- if eq .Values.installMode "host" }}
Expand Down Expand Up @@ -51,8 +52,8 @@ spec:
bash <<'EOF'
set -ex

# The `post-install hook job` may be applied before all karmada components are ready that rely on `static-resource job` and `hook-job secret`.
# So, we have to postpone the deletion of the `static-resource job` and `hook-job secret` until all karmada components are up and running.
# The `post-install hook job` may be applied before all karmada components are ready that rely on `static-resource job`.
# So, we have to postpone the deletion of the `static-resource job` until all karmada components are up and running.
while [[ $(kubectl get pods -n {{ $namespace }} --field-selector=status.phase!=Running,status.phase!=Succeeded -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' | grep -v static-resource | wc -l) > 0 ]];
do
echo "waiting for all pods of karmada control plane ready..."; sleep 1;
Expand All @@ -61,3 +62,4 @@ spec:
kubectl delete job {{ $name }}-static-resource -n {{ $namespace }}
EOF
{{- end }}
{{- end }}
3 changes: 3 additions & 0 deletions charts/karmada/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ preInstallJob:
staticResourceJob:
tolerations: []
nodeSelector: {}
## Set a TTL for the static-resource Job, the Job will be automatically cleaned up after this time.
## This only works on Kubernetes version 1.23 or higher.
ttlSecondsAfterFinished: 10
Comment on lines +105 to +107
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the impact if we don't clean the job?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The finished Pod holds extra resources, e.g., IP. Since this Job is useless, we need to release those resources.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the Pod IP, right? Yeah, I agree that we should try to clean up temporary resources after the installation.
I'm asking because I want to evaluate the benefit we can get from this. Thanks for the clarification.


## post-install job config
postInstallJob:
Expand Down