Flapping backend tests on GitHub Actions #380
Comments
|
Updated the ticket with new observations. |
|
Updated with new observations, stats so far:
I'll try to fix at least the worst offenders. |
|
I haven't fixed everything, but I improved the tests code reliability a little and I hope next time we'll see a false positive, it would be easier to investigate. Resolving this issue. |
|
Below are problems that are actual and for sure not fixed at the time of writing. Seems to be reproducible only in GitHub Actions, working theory is that the reason for it is having single virtual CPU and thus goroutines which are executed in very strange sequential order instead of being run in parallel.
|
|
I think we can close this one. Tests seem to be stable now |
|
Unfortunately, as you can see from the dates in the last table, we still have false positives. Before my changes, there were "real" false positives and now we see only ones caused by GitHub limited parallelization: I believe our tests fail (almost) every time when they hit some particular kind of limited conditions on GitHub instance. If we don't want to invest time into tilting at windmills of GitHub, feel free to close this one. It's interesting to investigate that but I didn't figured out how to fix it so far. |
paskal
changed the title
Make comment consistent with code Decrease timeout param to prevent unwanted 200 response
|
Streams which is a large contributor to the false positives might be dropped in #826. |
|
Resolving this one as we haven't seen these problems recently as frequent as in the past. There are still few false positives but they are rare. |
It flaps (see #380), and with this change it will have more time to get the expected output in flaky GitHub Actions environment.
It flaps (see #380), and with this change, it will have more time to get the expected output in the flaky GitHub Actions environment.
It flaps (see #380), and with this change, it will have more time to get the expected output in the flaky GitHub Actions environment.
It flaps (see #380), and with this change, it will have more time to get the expected output in the flaky GitHub Actions environment.
It flaps (see #380), and with this change, it will have more time to get the expected output in the flaky GitHub Actions environment.
|
assert.Eventually will save us, #1122 is an example of its use. |
It flaps (see #380), and with this change, it will have more time to get the expected output in the flaky GitHub Actions environment.
I think that multiple tests might be flapping, examples:
Test_MainTest_MainTest_MainTest_MainTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestFsStore_CleanupTestMigrator_ExportTestMigrator_ExportTestMigrator_ExportTestMigrator_ExportTestMigrator_ExportTestMigrator_ExportTestMigrator_ImportWaitExpiredTestMigrator_ImportWaitExpiredTestMigrator_ImportWaitExpiredTestMigrator_ImportWaitExpiredTestRest_RunStaticSSLModeTestRest_RunStaticSSLModeTestRest_InfoStreamCancelTestRest_InfoStreamCancelTestRest_InfoStreamCancelTestRest_InfoStreamCancelTestRest_InfoStreamCancelTestRest_InfoStreamSinceTestRest_CreateWithPicturesTestRest_CreateWithPicturesTestServerAppTestServerAppTestServerAppTestServerApp_AnonModeTestService_UserRepliesTestMemData_FlagListBlockedTestBolt_FlagListBlockedTestRPC_admEnabledHndlTestRPC_admEventHndlTestRPC_deleteHndlTestRPC_listFlagsHndlTestRPC_closeHndlFrom what I see it seems like these fails are unrelated to code changes.
The text was updated successfully, but these errors were encountered: