Conformance results for v1.20/digitalocean

[adamwg]

vendor: DigitalOcean
name: DigitalOcean Kubernetes
version: v1.20.2-do.0
website_url: https://www.digitalocean.com/products/kubernetes/
documentation_url: https://www.digitalocean.com/docs/kubernetes/
product_logo_url: https://assets.digitalocean.com/logos/DO_Logo_vertical_blue.svg
type: hosted
description: 'DigitalOcean Kubernetes is designed for developers who want a simple and cost effective way to deploy container workloads to a managed Kubernetes service.'

Signed-off-by: Adam Wolfe Gordon [email protected]

Pre-submission checklist:

Please check each of these after submitting your pull request:

• If this is a new entry, have you submitted a signed participation form?
• Did you include the product/project logo in SVG, EPS or AI format?
• Does your logo clearly state the name of the product/project and follow the other logo guidelines?
• If your product/project is open source, did you include the repo_url?
• Did you copy and paste the installation and configuration instructions into the README.md file in addition to linking to them?

[cncf-ci]

This conformance request is not yet verifiable. Please ensure that PR Title refernces the Kubernetes Release and that the supplied logs refer to the specified Release

[cncf-ci]

Found v1.20 in logs

[cncf-ci]

Automatically verified as having all required tests present and passed

[cncf-ci]

Automatically verified as having all required tests present and passed

[cncf-ci]

This conformance request failed to include all of the required tests for v1.20

[cncf-ci]

The first test found to be mssing was [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [LinuxOnly] [Conformance]

[adamwg]

Not sure what to make of the bot output here... did the list of required tests change between yesterday and today?

[riaankleinhans]

@adamwg I looked at the failure message and replicated it in another repo.

The error is valid and there is a miss-match between the conformance.yaml file in the K/K Master branch and the content of your junit file. This was caused by PR #98299 changing the conformance.yaml file.

Your data contain the test [sig-scheduling] SchedulerPredicates [Serial] validates that there is no conflict between pods with same hostPort but different hostIP and protocol [LinuxOnly] [Conformance] but that was replaced by [sig-network] HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol [LinuxOnly] [Conformance] in the PR mentioned.
Unfortunately, this will happen if the conformance.yaml get updated between the pulling of the data and the submission of the results.
You can resolve this by pulling the latest data and re-run your tests.

Sorry, for the confusing error message. The message has a logic error, reporting the test that fell out and not the missing new test. (We are working on that)

[adamwg]

@Riaankl Thanks for digging! I'm confused, though - what does the master branch of k/k have to do with this? Since we're certifying conformance for 1.20 here, shouldn't the 1.20 version of conformance.yaml be used? It hasn't been updated since November.

[brtkwr]

Same issue here

[riaankleinhans]

@adamwg It seems to be a bug.
We are trying to sort it out as soon as possible.

[adamwg]

@Riaankl Thanks for the update! Please let me know if there's any action needed from my side.

[riaankleinhans]

@adamwg the blocking PR has merged and the bug should be fixed now.
Would you please update your k/k repo and submit a new PR for conformance?

[adamwg]

@Riaankl I can re-run and submit a new PR. To be clear, though - I'm using sonobuoy per the instructions in this repo, not running tests directly from k/k. Does sonobuoy need an update to pull in the right tests?

[riaankleinhans]

Yes, you will have to update sonobuoy to retrieve the newly merge metadata updates and then run it with that.
That should solve it.

[riaankleinhans]

@adamwg I believe we figured out the issue causing the failure of PR checks.
on 16 Jan, after the release for 1.20 was cut, two commits changed the 1.20 conformance.yaml file.
This cause a mismatch between what you get out of Sonobuoy and that the bot check against the v1.20 .yaml file.
A PR was created to revert these commits, and that should stop the pain we are experiencing.
We will discuss the topic at the next conformance meeting to see how the historical conformance.yaml file should be protected against unwanted change.
Once the PR merge I believe the bot will check are label your PR correcly.

[cncf-ci]

Found v1.20 in logs

[cncf-ci]

Automatically verified as having all required tests present and passed

[cncf-ci]

Automatically verified as having all required tests present and passed

[taylorwaggoner]

You are now Certified Kubernetes