feat: PreferNoSchedule taint on candidates for disruption

[sadath-12]

Fixes #624 Part 2 (Candidate (PreferNoSchedule Taint) )

Description

This pr will add preferNoSchedule taints to nodes that are selected as candidates for disruption and any fails on that operation will remove this taint .

Also this pr is a follow up of pr #626

How was this change tested?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[njtran]

There's some design here that is in-flux before we can continue with this work. Generally, I'd like to see the design tackle some of the high level concerns that come with this, and some considerations on how this may evolve in the future.

This is a pretty important design space concerned with schedulability and the wide range of use-cases that come with this. Some thoughts on what I'd love to see explored in the design:

1. What use-cases are we trying to solve here?
2. How does a PreferNoSchedule taint affect scheduling in most Karpenter clusters?
3. When is the right time to add this Taint during disruption?

I can imagine most clusters are tightly packed due to the way the Consolidation algorithm converges to an optimal state. In that case, I don't think that a PreferNoSchedule taint would affect and solve all use-cases. As a follow-up to this taint addition, we might need to explore allowing the Effect here be configurable. It would be nice if the design mentioned this as well.

Can you add your design doc here in the PR so that we can iterate through the design? Since there's a bit of work needed on the maintainer side, I can imagine that there will be a lot of collaboration here to reach the desired state of the design doc, where we should eventually review this design at the working group.

[sadath-12]

sure @njtran , ill try if I can get things done within this pr itself or if necessary will close this and raise new pr

[sadath-12]

Linking the design doc here #649

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-contributor-experience at kubernetes/community.

/check-cla
/easycla

[unfor19]

@sadath-12 @njtran

Is any help needed here to merge this? This feature is awesome

[jmdeal]

Since the design has been closed, closing out the corresponding implementation PR. Work is still being done to address this issue, I'm hoping to have a design out in the next week or two.

/close

[k8s-ci-robot]

@jmdeal: Closed this PR.

In response to this:

Since the design has been closed, closing out the corresponding implementation PR. Work is still being done to address this issue, I'm hoping to have a design out in the next week or two.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.