Set K8s distribution with Helm to simplify choosing container runtime socket

[kate-goldenring]

What this PR does / why we need it:
Now that we have removed the need to specify a crictl path #418, all that needs to be set when installing Akri is the container runtime socket, which is set with --set agent.host.containerRuntimeSock="/some/sock.sock. This varies by distribution. To simplify installation and Kubernetes cluster setup, this proposes providing a kubernetesDistro value. Simply set kubernetesDistro=$DISTRO, where $DISTRO is either k3s, microk8s, or k8s. The template will always preference agent.host.containerRuntimeSocket if set. Chart throws the following error if no distribution nor socket are set:
Please set container runtime socket by either selecting the appropriate K8s distro kubernetesDistro=<k8s|k3s|microk8s>or settingagent.host.containerRuntimeSocket=/container/runtime.sock. See https://docs.akri.sh/user-guide/cluster-setup for more information."

Alternatives:
If adding kuberntesDistro is more confusing than beneficial, just retain the agent.host.containerRuntimeSock value and set it per usual in the docs, just renaming to be associated with container runtime. These docs will need to be modified to describe setting a distro during installation anyways.

Docs to modify: PR project-akri/akri-docs#19

If applicable:

• this PR has an associated PR with documentation in akri-docs
• added code adheres to standard Rust formatting (cargo fmt)
• version has been updated appropriately (./version.sh)
• all commits pass the DCO bot check by being signed off -- see the failing DCO check for instructions on how to retroactively sign commits

[bfjelds]

i think i'd lean towards using kubernetesDistro==k8s | kubernetesDistro==microk8s | kubernetesDistro==k3s ... rather than kubernetesDistro.microk8s==true, etc.

seems like maybe a cleaner way to make sure one and only one distro is ever specified.

[kate-goldenring]

I was worried about spelling but this makes sense. Changed it all!

[romoh]

Should we default to some value? like if not set then k8s is picked.

[kate-goldenring]

Should we default to some value? like if not set then k8s is picked.

My thought is no. It seems better to error and let them decide their distro. Otherwise slot reconciliation may not work. i know this is different from what we were doing but it would reduce the number of issues folks have with slot reconciliation errors.

[kate-goldenring]

Associated documentation: project-akri/akri-docs#19

[jiria]

Please update the PR message with the updated design.

[jiria]

What if folks use containerd instead of dockershim?

[kate-goldenring]

then they can set agent.host.containerRuntimeSocket to whatever they want

[kate-goldenring]

The change in documentation should clarify that project-akri/akri-docs#19

[jiria]

Would it make sense to then use k8s-dockershim to reduce confusion? Otherwise folks with containerd might be surprised why it is not working. Just FYI, AKS has transitioned from dockershim to containerd, the future lies with containerd, not dockershim.

[kate-goldenring]

in that case i think it would be better to do the alternative i describe in the PR description. Namely, we would provide no kubernetesDistro=$DISTRO_NAME and instead recommend how to set agent.host.containerRuntimeSocket based on the distro. I think that would be better than providing multiple options per distro (k8s-containerd, k8s-dockershim, etc). @bfjelds and @romoh on thoughts?

[kate-goldenring]

@jiria @romoh @bfjelds bump here. Would be nice to get the documentation in one this once we decide on what to do here

[bfjelds]

i would say have distro=k8s|k3s|etc ... then use whatever the distro default is (dockershim or containerd) ... and document that anything that doesn't follow the defaults can be supported iwth containerRuntimeSocket. though i also have no problem with k8s-dockershim|k8s-containerd|etc.

[kate-goldenring]

I am going with brian's last comment and sticking with what we have here. If folks complain about wanting our k8s default changed, we can change it. And I'll make sure the documentation points out how to set it to containerd.

[jiria]

Sounds good to me.

[kate-goldenring]

@romoh @bfjelds instead of erroring out, I set k8s' docker as the default. I could not throw a warning with helm, but i commented in the chart manifest that the default is being used: 161cdfd

[jiria]

Sounds good to me.