Specific E2E tests failing due to data mismatches

[GoelBiju]

Description:

This is a full issue to the comment @MRichards99 made on PR #756 which had failing end-to-end tests for DataView and has been discussed with @sam-glendenning. The current issue is that table/isis/facilityCycles.spec.ts and table/isis/investigations.spec.ts fail due to data mismatches between what the CI receives in the table and what preprod contains (or when testing locally using the preprod).

The first issue is that there are timezone differences between the data which the e2e tests use on the CI and the preprod data which is on deployed development version and used when running e2e tests locally. The differences are due to a +01:00 on the test data (as shown below).

CI screenshot:

Preprod screenshot:

Another example of this is in table/isis/investigations.spec.ts on the beforeEach for the test "should be able to view details". Here the test data for CI shows the size as 10.2 GB whilst the preprod test data is 10.27 GB (shown in the screenshots below).

CI screenshot:

Preprod screenshot:

After further testing, we have also found data mismatches in the Studies table. Studies on preprod have different start dates than they do locally, causing some tests to fail that check for items first in the list. See tests for 'STUDY 4' in this describe.

Acceptance criteria:

• End to end tests for the Facility Cycles table for ISIS passes with the correct data,
• End to end tests for the Investigations table for ISIS passes with the correct data,
• There is no difference in the data between what the CI uses and the preprod.

[GoelBiju]

This commit matches the data for the affected end-to-end tests but is not a fix for the issue so the tests have been skipped by @sam-glendenning.

[sam-glendenning]

Found some further problems that indicate issues with the DataGateway API date and time code. Goel and I have found a test in studies.spec.ts that has mismatching data between the CI and preprod in one particular study that it is testing for.

Regarding this test, Study 4 is the expected data but Study 325 sometimes appears instead. If you go on preprod, you'll see that Study 325 has a start date of 2081 but the CI has Study 325 with a start date of 2000, ahead of Study 4.

What's even more confusing is when Goel and I tried to locally test this using preprod e2e settings, the data we received also has a start date of 2000 on Study 325.

[sam-glendenning]

More e2e tests have been found to fail due to data mismatches. See #758 (comment)

[MRichards99]

Found some further problems that indicate issues with the DataGateway API date and time code. Goel and I have found a test in studies.spec.ts that has mismatching data between the CI and preprod in one particular study that it is testing for.

Regarding this test, Study 4 is the expected data but Study 325 sometimes appears instead. If you go on preprod, you'll see that Study 325 has a start date of 2081 but the CI has Study 325 with a start date of 2000, ahead of Study 4.

What's even more confusing is when Goel and I tried to locally test this using preprod e2e settings, the data we received also has a start date of 2000 on Study 325.

I believe this is caused by preprod not having quite the newest version of the frontend. On master (my VSCode window), investigation start date is being displayed (as per ISIS recommended changes) but preprod is displaying the study start date (hail the lack of configured language file!)

Once preprod contains the ISIS recommended changes, I think this issue will be fixed, though I think the expected study name will need to be changed to STUDY 325.

[GoelBiju]

Found some further problems that indicate issues with the DataGateway API date and time code. Goel and I have found a test in studies.spec.ts that has mismatching data between the CI and preprod in one particular study that it is testing for.
Regarding this test, Study 4 is the expected data but Study 325 sometimes appears instead. If you go on preprod, you'll see that Study 325 has a start date of 2081 but the CI has Study 325 with a start date of 2000, ahead of Study 4.
What's even more confusing is when Goel and I tried to locally test this using preprod e2e settings, the data we received also has a start date of 2000 on Study 325.

I believe this is caused by preprod not having quite the newest version of the frontend. On master (my VSCode window), investigation start date is being displayed (as per ISIS recommended changes) but preprod is displaying the study start date (hail the lack of configured language file!)

Once preprod contains the ISIS recommended changes, I think this issue will be fixed, though I think the expected study name will need to be changed to STUDY 325.

That makes sense now @MRichards99, once the preprod has been updated with the latest changes, we can check the tests again.

[sam-glendenning]

Nice work Matt, I would never have noticed that!

[MRichards99]

The differences between the file sizes (on the more information panel) was due to preprod's TopCAT caching. Louise cleared the cache and file sizes are now the same between preprod and my local instance. Like the study tests, the sizes might have to be changed (I think they're expecting the old values) but that issue is now resolved.

Just the timezone issue to investigate, one for tomorrow me thinks.

[GoelBiju]

The differences between the file sizes (on the more information panel) was due to preprod's TopCAT caching. Louise cleared the cache and file sizes are now the same between preprod and my local instance. Like the study tests, the sizes might have to be changed (I think they're expecting the old values) but that issue is now resolved.

Just the timezone issue to investigate, one for tomorrow me thinks.

That's awesome @MRichards99! 😃

[sam-glendenning]

Ah, interesting. Sounds like another thing we'll have to remember to do if we ever regenerate the data on preprod machines - clear the cache.

[MRichards99]

The inconsistent timezone issue was occurring because the preprod machine was set to BST. Louise set the JVM timezone to UTC (we didn't want to set the system timezone to UTC in case RIG's automation changed it back) and this has fixed it. I've spent the afternoon trying to force the generator script to add UTC to all the dates it generates, but it doesn't seem to work. I'm going to add a note on the API's documentation about this but it seems ensuring we use UTC is the best/easiest workaround.

I'll unskip and update the expected data on the tests (they just need to be changed back) and then make a PR for it at some point.

[sam-glendenning]

Great work, Matt, I'll keep an eye on the process. We can say the issue will be closed when that future PR is merged.

[MRichards99]

I've made the changes for this, but it can't be reviewed until preprod has been redeployed with #756 on. @louise-davies could you let me know when this is done please? No rush from me on this, just so I know when to change the PR to 'ready for review' :)

[louise-davies]

@MRichards99 master has now been updated as of earlier today