Fix the test custom-domain flakyness

[juanluisvaladas]

Description

The test custom-domain has become flaky. I'm not quite sure when did this happen, but it was recently. We can see in many executions such as:

https://github.com/k0sproject/k0s/actions/runs/3742332086/jobs/6359020341
https://github.com/k0sproject/k0s/actions/runs/3740780796/jobs/6349616080
https://github.com/k0sproject/k0s/actions/runs/3731323166/attempts/2

We can solve this simply by waiting for konnectivity to be ready before actually starting the test.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update

How Has This Been Tested?

• Manual test
• Auto test added

Checklist:

• My code follows the style guidelines of this project
• My commit messages are signed-off
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have checked my code and corrected any misspellings

[mikhail-sakhnov]

By the way, there is a way to check if agent is ready (like actually ready), but it is a little bit awkard. The konnectivity-server (the process on the controller node) has /ready endpoint which gives a list of registred agents, IIRC.

[juanluisvaladas]

By the way, there is a way to check if agent is ready (like actually ready), but it is a little bit awkard. The konnectivity-server (the process on the controller node) has /ready endpoint which gives a list of registred agents, IIRC.

I don't think we need to think about this that much because in this test it's guaranteed to be only one controller and therefore if we get pod logs, Konnectivity is working for the only controller.