-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
timer: TestTicker fail #46474
Comments
Is this a reproducible failure or a one off? how are you limiting CPU resources? and how loaded is the system (hardware) overall? |
From https://bugs.gentoo.org/792963 the beginning of the failure report is
This certainly suggests that there is something wrong but it's hard to know what. I don't know what tinderbox does. The environment variables you mention should not make any difference to this test. This test can fail in this way if the machine is very busy, or if it is run in an environment in which the |
Each image is given a cgroup v1 quota of usually 2.1 cpu if run with "make -j2" (https://github.com/toralf/tinderbox/blob/master/bin/bwrap.sh#L30) I can see that go spawns always much more threads as if could be used all 32 CPUs. The system is under load, yes - it is a build bot. I will repeat the tests. |
Is there some way that the Go runtime could know that it only has a limited number of CPUs available? Currently on Linux systems the Go runtime determines the number of CPUs available by using the |
Change https://golang.org/cl/339892 mentions this issue: |
TestTicker is sensitive to overloaded or slow systems, where a 20ms ticker running for 10 ticks has a total run time out of the range [110ms, 290ms]. To counter this flakiness, it tries five times to get a successful result. This is insufficient--an overloaded test machine can introduce more than 100ms of delay across the test. Reduce the five attempts to two, but use a 1s ticker for 8 ticks in the second attempt. Updates #46474. Updates #35692. Change-Id: Ibd5187b00ccceeb981b652f2af9a1c3766357b78 Reviewed-on: https://go-review.googlesource.com/c/go/+/339892 Trust: Damien Neil <[email protected]> Run-TryBot: Damien Neil <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>
Fixed. |
TestTicker is sensitive to overloaded or slow systems, where a 20ms ticker running for 10 ticks has a total run time out of the range [110ms, 290ms]. To counter this flakiness, it tries five times to get a successful result. This is insufficient--an overloaded test machine can introduce more than 100ms of delay across the test. Reduce the five attempts to two, but use a 1s ticker for 8 ticks in the second attempt. Updates golang#46474. Updates golang#35692. Change-Id: Ibd5187b00ccceeb981b652f2af9a1c3766357b78 Reviewed-on: https://go-review.googlesource.com/c/go/+/339892 Trust: Damien Neil <[email protected]> Run-TryBot: Damien Neil <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>
TestTicker is sensitive to overloaded or slow systems, where a 20ms ticker running for 10 ticks has a total run time out of the range [110ms, 290ms]. To counter this flakiness, it tries five times to get a successful result. This is insufficient--an overloaded test machine can introduce more than 100ms of delay across the test. Reduce the five attempts to two, but use a 1s ticker for 8 ticks in the second attempt. Updates golang#46474. Updates golang#35692. Change-Id: Ibd5187b00ccceeb981b652f2af9a1c3766357b78 Reviewed-on: https://go-review.googlesource.com/c/go/+/339892 Trust: Damien Neil <[email protected]> Run-TryBot: Damien Neil <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>
TestTicker is sensitive to overloaded or slow systems, where a 20ms ticker running for 10 ticks has a total run time out of the range [110ms, 290ms]. To counter this flakiness, it tries five times to get a successful result. This is insufficient--an overloaded test machine can introduce more than 100ms of delay across the test. Reduce the five attempts to two, but use a 1s ticker for 8 ticks in the second attempt. Updates golang#46474. Updates golang#35692. Change-Id: Ibd5187b00ccceeb981b652f2af9a1c3766357b78 Reviewed-on: https://go-review.googlesource.com/c/go/+/339892 Trust: Damien Neil <[email protected]> Run-TryBot: Damien Neil <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>
I do maintain and run a build bot for Gentoo packages. Inside one of the running images (bubblewrap'ed images) the mentioned test case fails. The approproiate Gentoo dev [2] is arguing that the environment of the tinderbox harms the test case scenario.
The build bot is a recent zen3 with enough RAM. For a good overall throuput however I do chain each image via cgroups to 2 CPUs. Furthermore I do set [3]
with jobs=2 or jobs=3 usually.
Does this by any chance harms the test case ?
[1] https://github.com/toralf/tinderbox
[2] https://bugs.gentoo.org/792963#c7
[3] https://github.com/toralf/tinderbox/blob/master/bin/setup_img.sh#L393
The text was updated successfully, but these errors were encountered: