Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UtilitiesTest JUnit can fail from time to time #1047

Open
johnbrvc opened this issue Feb 15, 2025 · 0 comments
Open

UtilitiesTest JUnit can fail from time to time #1047

johnbrvc opened this issue Feb 15, 2025 · 0 comments
Labels
bug Something isn't working Developer Issue An issue which affects PC2 Developers but doesn't directly affect users. low priority An issue that can be deferred

Comments

@johnbrvc
Copy link
Collaborator

Describe the issue:
A git CI build failed on UtilitiesTest.testgetDateTime().

To Reproduce:
Extremely difficult to reproduce (see below).

Expected behavior:
The UtilitiesTest.testgetDateTime() should always succeed.

Actual behavior:
The UtilitiesTest.testgetDateTime() can fail since the time can change between the initial Utilities.getDateTime() and the new Date() on the next line in UtilitiesTest.java. Basically, the test is obtaining a formatted time string (Utilities.DATE_TIME_FORMAT_STRING = "yyyyddMMhhmmss.SSS") from two different sources and comparing the non-millisecond part (first 15 characters), It is still possible (but extremely unlikely) that the first time string was obtained at, say, 20251502141310.999 and the next one was at 20251502141311.000. Notice that the first 15 characters, 20251502141310. and 20251502141311. will compare differently causing the JUnit test to fail, and consequently, the CI Build process to fail.

Environment:
All, but the Git CI in this case.

Log Info:

    [junit] Testsuite: edu.csus.ecs.pc2.core.UtilitiesTest
    [junit] Tests run: 29, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.347 sec
    [junit] 
    [junit] ------------- Standard Output ---------------
    [junit] Load from file /__w/pc2v9/pc2v9/samps/contests/mini/config/groups.tsv
    [junit] Load from file /__w/pc2v9/pc2v9/samps/contests/mini/config/teams.tsv
    [junit] Added 151 accounts and 12 groups.
    [junit] Load from file /__w/pc2v9/pc2v9/samps/contests/mini/config/groups.tsv
    [junit] Load from file /__w/pc2v9/pc2v9/samps/contests/mini/config/teams.tsv
    [junit] Added 151 accounts and 12 groups.
    [junit] ------------- ---------------- ---------------
    [junit] Testcase: testgetDateTime(edu.csus.ecs.pc2.core.UtilitiesTest):	FAILED
    [junit] First part of string should match 
    [junit] junit.framework.AssertionFailedError: First part of string should match 
    [junit] 	at edu.csus.ecs.pc2.core.UtilitiesTest.testgetDateTime(UtilitiesTest.java:119)
    [junit] 
    [junit] 
    [junit] Test edu.csus.ecs.pc2.core.UtilitiesTest FAILED

Screenshots:

Additional context:
Apparently the author thought about the fact that comparing the milliseconds could fail (a comment in the code: // may not match actual milliseconds.) However, no consideration was given to the fact that seconds may roll over as well (and consequently minutes, hours, etc.)

I'm not exactly sure of the best way to fix this, other than just removing the test, or, perhaps if it fails, sleep 10ms and try again?

@johnbrvc johnbrvc added bug Something isn't working Developer Issue An issue which affects PC2 Developers but doesn't directly affect users. low priority An issue that can be deferred labels Feb 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Developer Issue An issue which affects PC2 Developers but doesn't directly affect users. low priority An issue that can be deferred
Projects
None yet
Development

No branches or pull requests

1 participant