Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Implement end-to-end testing with Playwright #420

Merged
merged 45 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
1b6aea7
test: Implement end-to-end testing with Playwright
gwhitney Aug 28, 2024
1249a46
fix: Repair gallery/click on featured item test
gwhitney Aug 28, 2024
fd32bf9
fix: Repair gallery/saving a specimen test
gwhitney Aug 28, 2024
3b81741
fix: Repair scope tests.
gwhitney Aug 28, 2024
44869dc
test: Run end-to-end tests prior to any commit
gwhitney Aug 28, 2024
b98114a
test: Add image tests to e2e
gwhitney Aug 29, 2024
11a1def
doc: describe running and creating code tests
gwhitney Aug 30, 2024
c988513
test: Test that the caching mechanism won't double-calculate
gwhitney Aug 30, 2024
de5f16c
fix: Prevent ModFill from hanging on extremely large input
gwhitney Aug 30, 2024
3ca1474
doc: Update PR checklist.
gwhitney Aug 30, 2024
d8ccce2
maintenance: Remove and disallow trailing whitespace in code
gwhitney Aug 30, 2024
5cccf8c
maintenance: Make TypeScript target ES2022 in all cases.
gwhitney Aug 30, 2024
571390a
maintenance: Run typecheck in CI
gwhitney Aug 30, 2024
3d1d6fa
maintenance: Run GitHub CI on pushes to main as well
gwhitney Aug 30, 2024
044a054
test: Add OEIS Sequence-Visualizer test grid
gwhitney Sep 5, 2024
f1f7d30
refactor: better typing for param descriptions
gwhitney Sep 5, 2024
bb45388
feat: Add ExtendedBigint ParamType
gwhitney Sep 6, 2024
95d8f57
refactor: Remove generic parameters from top levels of Paramable hier…
gwhitney Sep 6, 2024
d811eba
feat: Uniform sequence bounds controls
gwhitney Sep 9, 2024
33386c0
refactor: Apply OEIS modulus on the fly, so cache can point to commun…
gwhitney Sep 10, 2024
63ddc6c
test: First working e2e tests with text and transversal
gwhitney Sep 17, 2024
bca3bf4
test: Add skipped tests reflecting known shortcomings
gwhitney Sep 18, 2024
22d77fb
fix: Generate docker image even with no existing test results
gwhitney Sep 18, 2024
b5cfc4a
chore: Attempt to run e2e tests in CI workflow
gwhitney Sep 18, 2024
8f3f58e
test: No WebGL tests in CI :-(
gwhitney Sep 18, 2024
c02cfd8
fix: format of Playwright reporter parameter
gwhitney Sep 18, 2024
4a212e3
chore: Try to grab GitHub CI actuals to make separate CI test file
gwhitney Sep 18, 2024
5214d32
fix: oops test must succeed to generate artifact
gwhitney Sep 18, 2024
acb49bb
chore: Another try to grab ci snapshots
gwhitney Sep 18, 2024
68d9294
test: Add in the extracted CI snapshots
gwhitney Sep 18, 2024
ff375c7
doc: Note that PR reviewers must run e2e tests themselves, too.
gwhitney Sep 18, 2024
345a72b
chore: Stop grabbing snapshots when we don't need them
gwhitney Sep 18, 2024
599662b
fix: Changes as per review comments
gwhitney Oct 1, 2024
7374837
doc: more info on Docker (and fix remaining typo)
gwhitney Oct 1, 2024
be2a7d5
test: Fuzz the pixel comparison in Firefox WebGL tests
gwhitney Oct 3, 2024
7567594
test: Really fuzz the pixel comparison in Firefox WebGL tests
gwhitney Oct 4, 2024
14b24f6
test: Check if docker tests passed before updating result directory
gwhitney Oct 4, 2024
31bbe08
doc: Improvements per code review
gwhitney Oct 8, 2024
94669e5
doc: show the expected output of successful end-to-end test
gwhitney Oct 8, 2024
f72c78c
chore: remove stray comment that no longer applies
gwhitney Oct 8, 2024
fbdaedc
fix: useful default camera controls for WebGL: left drag pans, wheel …
gwhitney Oct 9, 2024
feb8084
fix: correct Husky action inclusion and test it
gwhitney Oct 9, 2024
1fb03d2
fix: don't alter the URL just loaded, and reset frame limit on changes
gwhitney Oct 9, 2024
4c78deb
fix: adjust dragging, detected mouse position, and text size for zoom…
gwhitney Oct 10, 2024
4322aa7
fix: Keep the 'too many bins' message in a fixed absolute canvas posi…
gwhitney Oct 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: Uniform sequence bounds controls
  All params for controlling which terms of a sequence will be used in
  the visualization are removed from individual visualizers. Instead, there
  are uniform params in the Sequence classes themselves.

  Resolves #411.

  In this implementation, several other changes are necessary and/or
  expedient to make. In particular, the type of sequence indices is changed
  to bigint and bounds to ExtendedBigint (the union of bigint and ±Infinity),
  in preparation for addressing #455. Additional changes include:

  * Removal of SequenceDefault class, as everything now derives from Cached.
  * More care in caching of OEIS sequences, to provide some help with #459.
  * Individual-parameter validation functions now take a validation status
    to update based on the finding. This refactor avoids a frequent need to
    merge status objects.
  * Individual-parameter validation functions are called in a context
    of `this` set to the Paramable object, so that other data from the
    paramable can be accessed.
  * New `math` functions for dealing with ExtendedBigints.
  * NumberGlyph visualizer now tries to display as many terms as are
    available and will fit on the screen, except in case there are infinitely
    many terms available, in which case it still defaults to 64 terms. This
    change is needed so that the general length parameter for Sequences would
    affect what NumberGlyph displayed.
  * Puts reactivation of "known" OEIS sequences into a function
    rather than at top level of sequences.ts so that the timing of
    when it occurs can be controlled.
  * Renames property `_size` of P5Visualizer to `size` as multiple derived
    classes seem to need this value (i.e., not just used internally in the base
    class).
  * Makes sure the p5 loop is restarted if need be every time `.show()` is
    called on a visualizer.
  * Temporarily disables A000521 transversal tests until we can
    get incremental OEIS loading to work.
  * Adds a test for starting a visualization deep into a sequence.
  • Loading branch information
gwhitney committed Sep 9, 2024
commit d811eba81addb372de8b7242da6b5cc5946bd519
2 changes: 1 addition & 1 deletion e2e/featured.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test.describe('Featured gallery images', () => {
await page.goto(testURL)
await expect(
page.locator('#specimen-bar-desktop').getByText('play_arrow')
).toHaveId('pause-button', {timeout: 15000})
).toHaveId('pause-button', {timeout: 20000})
expect(
await page.locator('#canvas-container').screenshot()
).toMatchSnapshot(`${short}.png`)
Expand Down
Binary file modified e2e/featured.spec.ts-snapshots/ChaosGame-chromium-linux.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified e2e/featured.spec.ts-snapshots/ChaosGame-firefox-linux.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
44 changes: 29 additions & 15 deletions e2e/idiot.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,34 @@ import {specimenQuery} from '../src/shared/browserCaching'
// These are tests in the spirit of
// https://github.com/numberscope/frontscope/issues/113, i.e.
// "idiot-proofing" Numberscope

// Each item is the argument list to specimenQuery
const testlist = {
'Way too big a number': [
'A',
'ModFill',
'Formula',
'modDimension=1111111111111111111111111111111111111111111',
'formula=12',
],
'Start deep in a sequence': [
'B',
'FactorFence',
'Formula',
'',
'first=1020000',
],
}
test.describe('Stress-test Numberscope usage', () => {
test('Way too big a number', async ({page}) => {
const query = specimenQuery(
'A',
'ModFill',
'Formula',
'modDimension=1111111111111111111111111111111111111111111',
'formula=12'
)
const testURL = '/?frames=5&' + query
await page.goto(testURL)
await expect(
page.locator('#specimen-bar-desktop').getByText('play_arrow')
).toHaveId('pause-button', {timeout: 15000})
await expect(page).toHaveScreenshot() // capture error alert, if one
})
for (const k in testlist) {
test(k, async ({page}) => {
const query = specimenQuery(...testlist[k])
const testURL = '/?frames=5&' + query
await page.goto(testURL)
await expect(
page.locator('#specimen-bar-desktop').getByText('play_arrow')
).toHaveId('pause-button', {timeout: 15000})
await expect(page).toHaveScreenshot() // capture error alert, if one
})
}
})
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion e2e/scope.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ test.describe('Scope', () => {
test('refreshing the specimen', async ({page}) => {
const oldCanvas = await page.locator('#canvas-container canvas')

oldCanvas.evaluate(canvas => {
await oldCanvas.evaluate(canvas => {
canvas.classList.add('old-canvas')
})
await page.locator('#specimen-bar-desktop #refresh-button').click()
Expand Down
4 changes: 2 additions & 2 deletions e2e/transversal.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ const challenges = [
'A114592', // Only -1,0,1, for 359 terms, then some bigger values
'A007235', // short sequence but grows fast
'A001220', // very few terms
'A000521', // short sequence with -1 offset, grows fast
// 'A000521', // long seq with -1 offset, grows fast; temp too much data
'A228060', // offset -85, many small terms
'A086677', // offset 2
'A241298', // very large positive offset
Expand Down Expand Up @@ -75,7 +75,7 @@ test.describe('Visualizer-sequence challenges', () => {
page
.locator('#specimen-bar-desktop')
.getByText('play_arrow')
).toHaveId('pause-button', {timeout: 15000})
).toHaveId('pause-button', {timeout: 20000})
expect(
await page.locator('#canvas-container').screenshot()
).toMatchSnapshot(`${viz + seq}.png`)
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions src/App.vue
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@

<script setup lang="ts">
import {RouterView} from 'vue-router'

import {reactivateOEIS} from '@/sequences/sequences'

reactivateOEIS()
</script>

<!-- Global styles. This style tag is explicitly unscoped. -->
Expand Down
Loading
Loading