Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm/cilium: bump tested versions #365

Merged
merged 4 commits into from
Aug 31, 2022
Merged

Conversation

tormath1
Copy link
Contributor

@tormath1 tormath1 commented Aug 30, 2022

  • kubeadm/cilium: bump CLI and tested Cilium version

    Starting from Cilium 1.12, {live,ready}ness probes are on :9879 while
    it was on :9876 on older versions. (See: cilium/cilium@22cd47e)

    CLI made this change on versions greater or equal to 1.10.12 (See: https://github.com/cilium/cilium-cli/pull/869/files) - it
    results with a port mismatch 9879/9876 if we test the version 1.11.5.

    Basically Cilium is running fine but its status is not ready/healthy
    from a Kubernetes PoV.

  • kubeadm/cilium: patch Cilium daemon set

    This is required even with Permissive mode. Can be dropped once spc_t
    is supported on Flatcar.

  • kubeadm/cilium: disable SELinux for Cilium

    Once the daemon set is started with unconfined_t it does not work with
    enforced SELinux because it hits a denial from transitioning to kernel_t to
    unconfined_t (and this normal because currently everything runs with
    kernel_t including container runtimes)

    Can be dropped once it works fine with spc_t label with the SELinux
    upgrade.


Tested multiple times on Openstack, the CI: http://jenkins.infra.kinvolk.io:8080/job/os/job/kola/job/packet/309/console and with kubernetes-1.25

Starting from Cilium 1.12, {live,ready}ness probes are on :9879 while
it was on :9876 on older versions. (See: cilium/cilium@22cd47e)

CLI made this change on versions greater or equal to 1.10.12 (See: https://github.com/cilium/cilium-cli/pull/869/files) - it
results with a port mismatch 9879/9876 if we test the version 1.11.5.

Basically Cilium is running fine but its status is not ready/healthy
from a Kubernetes PoV.

Signed-off-by: Mathieu Tortuyaux <[email protected]>
@tormath1 tormath1 self-assigned this Aug 30, 2022
@tormath1 tormath1 marked this pull request as ready for review August 30, 2022 15:48
@tormath1 tormath1 requested a review from a team August 30, 2022 15:48
@tormath1 tormath1 mentioned this pull request Aug 30, 2022
1 task
Copy link
Member

@krnowak krnowak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nitpick, otherwise looks good. Also, I think it needs an entry in the changelog.

@@ -54,8 +54,12 @@ var (
_ = c.MustSSH(controller, "/opt/bin/cilium uninstall")
version := params["CiliumVersion"].(string)
cidr := params["PodSubnet"].(string)
cmd := fmt.Sprintf("/opt/bin/cilium install --config enable-endpoint-routes=true --config cluster-pool-ipv4-cidr=%s --version=%s --encryption=ipsec --wait --wait-duration 1m", cidr, version)
_ = c.MustSSH(controller, cmd)
cmd := fmt.Sprintf("/opt/bin/cilium install --config enable-endpoint-routes=true --config cluster-pool-ipv4-cidr=%s --version=%s --encryption=ipsec --wait-duration=1s --rollback=false", cidr, version)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifying wait duration is most likely pointless if we are not waiting:

Suggested change
cmd := fmt.Sprintf("/opt/bin/cilium install --config enable-endpoint-routes=true --config cluster-pool-ipv4-cidr=%s --version=%s --encryption=ipsec --wait-duration=1s --rollback=false", cidr, version)
cmd := fmt.Sprintf("/opt/bin/cilium install --config enable-endpoint-routes=true --config cluster-pool-ipv4-cidr=%s --version=%s --encryption=ipsec --rollback=false", cidr, version)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually waits (--wait defaults to true) - but it made me think that we can --wait=false --restart-unmanaged-pods=false to not wait at all, this is enough for this test case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. If that works then feel free to merge the PR.

This is required even with Permissive mode. Can be dropped once `spc_t`
is supported on Flatcar.

Signed-off-by: Mathieu Tortuyaux <[email protected]>
Once the daemon set is started with `unconfined_t` it does not work with
enforced SELinux because it hits a denial from transitioning to `kernel_t` to
`unconfined_t` (and this normal because currently everything runs with
`kernel_t` including container runtimes)

Can be dropped once it works fine with `spc_t` label with the SELinux
upgrade.

Signed-off-by: Mathieu Tortuyaux <[email protected]>
Signed-off-by: Mathieu Tortuyaux <[email protected]>
@tormath1 tormath1 merged commit be7a6a4 into flatcar-master Aug 31, 2022
@tormath1 tormath1 deleted the tormath1/kubeadm branch August 31, 2022 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants