Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Implement end-to-end testing with Playwright #420

Merged
merged 45 commits into from
Oct 10, 2024
Merged
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
1b6aea7
test: Implement end-to-end testing with Playwright
gwhitney Aug 28, 2024
1249a46
fix: Repair gallery/click on featured item test
gwhitney Aug 28, 2024
fd32bf9
fix: Repair gallery/saving a specimen test
gwhitney Aug 28, 2024
3b81741
fix: Repair scope tests.
gwhitney Aug 28, 2024
44869dc
test: Run end-to-end tests prior to any commit
gwhitney Aug 28, 2024
b98114a
test: Add image tests to e2e
gwhitney Aug 29, 2024
11a1def
doc: describe running and creating code tests
gwhitney Aug 30, 2024
c988513
test: Test that the caching mechanism won't double-calculate
gwhitney Aug 30, 2024
de5f16c
fix: Prevent ModFill from hanging on extremely large input
gwhitney Aug 30, 2024
3ca1474
doc: Update PR checklist.
gwhitney Aug 30, 2024
d8ccce2
maintenance: Remove and disallow trailing whitespace in code
gwhitney Aug 30, 2024
5cccf8c
maintenance: Make TypeScript target ES2022 in all cases.
gwhitney Aug 30, 2024
571390a
maintenance: Run typecheck in CI
gwhitney Aug 30, 2024
3d1d6fa
maintenance: Run GitHub CI on pushes to main as well
gwhitney Aug 30, 2024
044a054
test: Add OEIS Sequence-Visualizer test grid
gwhitney Sep 5, 2024
f1f7d30
refactor: better typing for param descriptions
gwhitney Sep 5, 2024
bb45388
feat: Add ExtendedBigint ParamType
gwhitney Sep 6, 2024
95d8f57
refactor: Remove generic parameters from top levels of Paramable hier…
gwhitney Sep 6, 2024
d811eba
feat: Uniform sequence bounds controls
gwhitney Sep 9, 2024
33386c0
refactor: Apply OEIS modulus on the fly, so cache can point to commun…
gwhitney Sep 10, 2024
63ddc6c
test: First working e2e tests with text and transversal
gwhitney Sep 17, 2024
bca3bf4
test: Add skipped tests reflecting known shortcomings
gwhitney Sep 18, 2024
22d77fb
fix: Generate docker image even with no existing test results
gwhitney Sep 18, 2024
b5cfc4a
chore: Attempt to run e2e tests in CI workflow
gwhitney Sep 18, 2024
8f3f58e
test: No WebGL tests in CI :-(
gwhitney Sep 18, 2024
c02cfd8
fix: format of Playwright reporter parameter
gwhitney Sep 18, 2024
4a212e3
chore: Try to grab GitHub CI actuals to make separate CI test file
gwhitney Sep 18, 2024
5214d32
fix: oops test must succeed to generate artifact
gwhitney Sep 18, 2024
acb49bb
chore: Another try to grab ci snapshots
gwhitney Sep 18, 2024
68d9294
test: Add in the extracted CI snapshots
gwhitney Sep 18, 2024
ff375c7
doc: Note that PR reviewers must run e2e tests themselves, too.
gwhitney Sep 18, 2024
345a72b
chore: Stop grabbing snapshots when we don't need them
gwhitney Sep 18, 2024
599662b
fix: Changes as per review comments
gwhitney Oct 1, 2024
7374837
doc: more info on Docker (and fix remaining typo)
gwhitney Oct 1, 2024
be2a7d5
test: Fuzz the pixel comparison in Firefox WebGL tests
gwhitney Oct 3, 2024
7567594
test: Really fuzz the pixel comparison in Firefox WebGL tests
gwhitney Oct 4, 2024
14b24f6
test: Check if docker tests passed before updating result directory
gwhitney Oct 4, 2024
31bbe08
doc: Improvements per code review
gwhitney Oct 8, 2024
94669e5
doc: show the expected output of successful end-to-end test
gwhitney Oct 8, 2024
f72c78c
chore: remove stray comment that no longer applies
gwhitney Oct 8, 2024
fbdaedc
fix: useful default camera controls for WebGL: left drag pans, wheel …
gwhitney Oct 9, 2024
feb8084
fix: correct Husky action inclusion and test it
gwhitney Oct 9, 2024
1fb03d2
fix: don't alter the URL just loaded, and reset frame limit on changes
gwhitney Oct 9, 2024
4c78deb
fix: adjust dragging, detected mouse position, and text size for zoom…
gwhitney Oct 10, 2024
4322aa7
fix: Keep the 'too many bins' message in a fixed absolute canvas posi…
gwhitney Oct 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc: show the expected output of successful end-to-end test
  • Loading branch information
gwhitney committed Oct 8, 2024
commit 94669e50c7071963ca1937a99156d9277b8eaafe
23 changes: 17 additions & 6 deletions doc/code-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ tests very convenient. To run all unit (vitest) tests, execute
`npm run test:unit`. To run all end-to-end (Playwright) tests, use
`npm run test:e2e`.

A couple of additional points about the end-to-end tests:
Here is some more detailed information about running the end-to-end tests:

- They result in accessing a considerable amount of (sequence) data over the
internet. Therefore, if you run them when your internet connection is poor
Expand All @@ -24,11 +24,22 @@ A couple of additional points about the end-to-end tests:
immediately re-execute `npm run test:e2e`, it will simply tell you that
the tests are "up to date."

- By default, they mostly run in a [Docker](https://www.docker.com/)
container, which we have found maximizes the reproducibility of the tests.
So as explained in some more detail in the [onboarding](./onboarding.md)
page, you will need to install Docker on your system and make sure the
Docker daemon is running in order to run the tests.
- When all of the end-to-end tests have passed, the last three lines of the
output should typically look as follows: (There are two copies of the
"status": "passed" line because there are two batches of tests.)

```
"status": "passed",
"status": "passed",
echo 'All tests passed' > e2e/certificate
```

- By default, the end-to-end tests mostly run in a
[Docker](https://www.docker.com/) container, which we have found maximizes
the reproducibility of the tests. So as explained in some more detail in
the [onboarding](./onboarding.md) page, you will need to install Docker on
your system and make sure the Docker daemon is running in order to run the
tests.

You can also run them directly in your ambient system with
`npm run build && npm run test:e2e:cmd` or just `npm run test:e2e:ui` for
Expand Down
Loading