Flaky e2e-test for checking Ext4Error

[pwschuurman]

The following e2e test was causing flakes when running presubmit on #561

NPD should update problem_counter{reason:Ext4Error} and problem_gauge{type:ReadonlyFilesystem}

NPD exports a counter based on the number of records encountered that match a pattern. There is some non-determinism by the system on how many times the Ext4Error message gets written to system logs. This causes the test case to fail currently, as it expects the counter to be between 1 and 2 (inclusive). In my testing I have seen counter values of 3 and 4. I think a reasonable fix is just to check the lower bound of 1, and not worry about the upper bound, since there is some variability here.

[1] • [SLOW TEST:58.595 seconds]
[1] NPD should export Prometheus metrics.
[1] /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:36
[1]   On a clean node
[1]   /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:69
[1]     NPD should export cpu/disk/host/memory metric
[1]     /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:71
[1] ------------------------------
[1] NPD should export Prometheus metrics. When OOM kills and docker hung happen
[1]   NPD should update problem_counter and problem_gauge
[1]   /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:156
[2]
[2] • Failure [64.868 seconds]
[2] NPD should export Prometheus metrics.
[2] /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:36
[2]   When ext4 filesystem error happens
[2]   /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:121
[2]     NPD should update problem_counter{reason:Ext4Error} and problem_gauge{type:ReadonlyFilesystem} [It]
[2]     /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:130
[2]
[2]     Got value for metric problem_counter with label map[reason:Ext4Error]: 3, expect at most 2.
[2]     Expected
[2]         <float64>: 3
[2]     to be <=
[2]         <float64>: 2
[2]
[2]     /usr/local/google/home/psch/go/src/k8s.io/node-problem-detector/test/e2e/metriconly/metrics_test.go:212

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[pwschuurman]

Fixed in #570