Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event Feed contains improper/invalid time values in "judgement" events #712

Open
clevengr opened this issue Mar 4, 2023 · 0 comments
Open
Assignees
Milestone

Comments

@clevengr
Copy link
Contributor

clevengr commented Mar 4, 2023

Describe the issue:

In the PacNWSpring23 contest, the following judgement event was found in the Event Feed:

{"type":"judgements", "id":"pc2-499", "op":"update", "data": {"id":"Run--3013976802286185460","submission_id":"37","start_time":"2023-02-25T19:12:14.190Z","start_contest_time":"00:12:14.232","judgement_type_id":"WA","end_time":"2023-02-25T19:12:00Z","end_contest_time":"00:12:00.000"}}

Note that in the above judgement event the (absolute) start_time is 2023-02-25T19:12:14.190Z and the (contest-relative) start_contest_time is 00:12:14.232, while the corresponding end_time and end_contest_time values are 2023-02-25T19:12:00Z and 00:12:00.000 respectively. In other words, this is saying that the judgement finished 14 seconds before it started. Clearly, that's wrong.

To Reproduce:

  • Start a contest
  • Start an Event Feed client
  • Submit a run
  • Wait a short time, then judge the run
  • Examine the judgement event in the Event Feed

Expected behavior:
The times in the judgement event should match the times required by the CLICS specification (see below).

Actual behavior:
Judgement end times come before judgement start times.

Environment:
Detected under Linux (Ubuntu 22.04), but likely applies to all environments.

Additional context:

According to the 2022-07 CLICS specification, the fields of a judgements event should be:

Name Type Description
id ID Identifier of the judgement.
submission_id ID Identifier of the submission judged.
judgement_type_id ID ? The verdict of this judgement. Required iff judgement has completed.
score number Score for this judgement. Required iff contest:scoreboard_type is score.
start_time TIME Absolute time when judgement started.
start_contest_time RELTIME Contest relative time when judgement started.
end_time TIME ? Absolute time when judgement completed. Required iff judgement_type_id is present.
end_contest_time RELTIME ? Contest relative time when judgement completed. Required iff judgement_type_id is present.
max_run_time number ? Maximum run time in seconds for any test case. Should be an integer multiple of 0.001.

The underlying issue appears to be that PC2 is not currently tracking "start judging" and "end judging" time. Further, there are two places in the code where "JSON Judgement" objects are being created: methods JSONTool.convertJudgementToJSON(Run submission) and JudgementJSON.createJSON(IInternalContest contest, Run run). It's not clear why there are two different methods handling what appears to be the same functionality. Note also that there is a comment in JSONTool.convertJudgementToJSON(Run submission) saying

        // SOMEDAY this is suppose to be when the judge retrieves it, not the submission time.
        element.put("start_time", Utilities.getIso8601formatterWithMS().format(submission.getCreateDate()));
        element.put("start_contest_time", ContestTime.formatTimeMS(submission.getElapsedMS()));

It seems likely that this the source of the bad times in the event feed...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants