Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Restore Grid visualizer to operation #522

Draft
wants to merge 43 commits into
base: ui2
Choose a base branch
from

Conversation

gwhitney
Copy link
Collaborator

By submitting this PR, I am indicating to the Numberscope maintainers that I have read and understood the contributing guidelines and that this PR follows those guidelines to the best of my knowledge. I have also read the pull request checklist and followed the instructions therein.


This PR is a work in progress, aimed at restoring the Grid visualizer. Currently it compiles and runs without crash, but still does not operate.

mbavec and others added 30 commits June 2, 2024 15:01
Adds a route that shows a panel of thumbnails of multiple different specimens. This can be used, for example, for a curated list of interesting visualizers, or for an individual user's own visualizers.
---------
Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: mbavec <[email protected]>
* Default implementation of aspect ratio in P5 visualizer template
* Responsive parameters displayed together with visualization
* New graphic design
* New typing system for parameters
* Drag and drop with tiling and windows
* Fixed name and description properties
* Setup specimen structure
* Moved the drag and drop into scope
(and multiple other small changes/fixes, too numerous to enumerate here)
---------
Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: ZiggyBeijer <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
Cause the URL of the Specimen Editor to encode the specimen name, sequence, and visualizer and all of their
parameters. Conversely, when visiting a URL with these properties encoded, decode them and re-create the
corresponding Speciment in the Editor.
---------

Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: ZiggyBeijer <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
* refactor: Add a new unused browserCaching system
Originally authored by @kaldis-berzins et. al.
* feat: resizing and URL integration

  Changes to tabs
    * Added buttons that let you control the tabs without dragging
    * Support minimizing of tabs.
    * Added a highlight to the tab that is currently selected.
    * Initial mobile compatibility in the style section of Tab.vue

  Resizing
    * Added a reset function to a specimen. This is a hard reset that is used
      when the specimen is resized and it doesn't override resized function.
    * Added a resized function to both specimens and visualizers. It is not
      required for visualizers as they can just not do anything and the
      specimen will hard reset them.

  Scope
    * Adds rudimentary mobile support.

Originally authored by @kaldis-berzins et al.

* fix: Dock back to where you were and non-overlapping initial tab positions

* doc: fix description of docking button

* refactor: Use consistent tab minimized height (and revert icon name change)
Navbar
  * Removes the navbar from App.vue. The navbar now has to be added
    to each view separately because it may be different for differrent views.
  * Makes the navbar taller and adds a slot to accommodate the specimen bar.

Specimen bar
  * Adds the specimen bar with replay, pause/resume, share, and save buttons.

Visualizers
  * Adds a continue function to be executed by the resume button.
* feat: Gallery integration

  Thumbnails
    * Adds thumbnails to gallery cards. They are essentially just specimens
      contained entirely within a little 200x200 container.
    * Thumbnails own their specimen, so the cards must retrieve the name
      of the specimen on their own.
        - Adds methods to Specimen.ts to enable this:
          getNameFromURL and getSequenceNameFromURL.

  Gallery
    * Has two sections: Saved specimens and Featured specimens.
    * Implements saving specimens and loading them from the gallery.
    * Featured specimens can be added to the list by editing
      shared/defineFeatured.ts. This file will automatically turn into cards
      on the gallery page.

  Scope
    * Current specimen is now saved in browser as well

* refactor: Merge SpecimenCard and FeaturedCard components

* style: Use 'pointer' cursor on specimen cards

* fix: Click anywhere on specimen card to load (and add featureds)

* fix: Numberscope logo goes to current specimen (and add featured)

* fix: Reduce leakage of visualizers and limit draw time in Gallery

* fix: Chaos allows no last element; P5Visualizer leave cursor be

* fix: Avoid filling in tentative values for optional parameters

* fix: Allow FactorHistogram to redraw on parameter changes

* fix: Display the fully-qualified sequence name in previews

* test: add missing browserCaching tests

* refactor: Improve Paramable API

  Prior to this commit, it was necessary in several places to manually
  validate, set isValid to true, and assignParameters. Now, whenever
  validate() is called, the isValid property is automatically updated, and
  if it happens that isValid is true, then assignParameters is
  automatically invoked.

  In addition, adds a validateIndividual() method to ParamableInterface
  that is needed by the ParamEditor. Including this method in Paramable
  itself eliminates previous code duplication between Paramable and
  ParamEditor.

* refactor: remove redundancy in defineFeatured

* fix: Specimen delete button in gallery should not open specimen
…umberscope#377)

Note that there remains an issue of the drop zone remaining visible when a Gallery specimen is loaded, but this will be addressed in the next PR.
* feat: Sequence and Visualizer switchers, first pass at cleaned

* fix: Always show the current background and stroke colors in Turtle

* fix: Don't refresh empty default parameters

* ui: move sequence search bar to right of Sequence Switcher popup title

* feat: Switchers now show thumbnails of the modified visualizers

  Implementing this feature involves abstracting the sections of the
  Gallery into a new SpecimensGallery component, and then using that
  component in the SwitcherModal. Note that in order to accommodate the
  desired captions of the SwitcherModal, the CardSpecimen interface
  used to specify the contents of a SpecimensGallery has to be generalized
  to allow arbitrary subtitles.

* fix: Equalize space between sections of Gallery

* feat: Resize Switcher modal popups to fit specimen cards cleanly

* feat: Placeholders for optional parameters

* fix: Make placeholders lighter

* ui: Pause visualizer while Switcher dialogues are open

* ui: Position Switcher popup centered over visualizer canvas

* fix: Restore Delft-submitted click behavior for closing Switcher

* ui: Invoke Switchers via current seq/vis looking and acting like entry fields

* fix: MageExchangeA icon should use current CSS color
* feat: human-readable URLs based on query strings

  To implement the planned URL search bar, it will become necessary
  to generate specimen encodings from sequence parameters without
  actually constructing the Sequence objects that have those parameters.
  That was not possible with the opaque base64 encoding.

  Hence, this PR switches to a human-readable encoding based on URL query
  strings as processed by the JavaScript standard URLSearchParams. It provides
  some functions to construct such encodings directly (rather than only
  mediated by existing Sequence and Visualizer objects). This PRI includes
  a backwards compatibility facility so that saved base64 encodings can still
  be interpreted. It also suppresses the Vue error trapping feature when
  in Workbench mode, which was and should continue to be very helpful when
  debugging.

* chore: remove unused hasStringFields function

* chore: update featured specimens to new URL scheme

* fix: parse query from path properly

* fix: Fewer url encodings -> more readable URLs

* fix: Suppress leading # in color query params

* feat: Lists separated by whitespace by default
* fix: All minor items from Delft final_fixes

* fix: Re-enable canned OEIS seqs with proper async computation

* doc: improve description of Sequence fill method

* chore: remove stray bold attribute; reword chaos comment

* fix: Right-docked tabs remain correctly positioned on zoom
…changes to Turtle and p5Template (numberscope#399)

In view of recent changes to the p5 visualizer class (parameter data structure and validation, handling aspect ratios and presketch function), this PR does the following:

    * updates the documentation making-a-visualizer.md to reflect these changes
    * updates the P5Template visualizer in the workbench to agree with said documentation and reflect these changes
    * cleans up the Turtle visualizer: make use of parameter-specific validation, reorder/prioritize the parameters according to importance, add a picture to documentation, change default values for aesthetics.

---------

Co-authored-by: Glen Whitney <[email protected]>
* refactor: move the OEIS search bar into its own component.

* doc: Basic Modfill -> two OEIS-based Specimens in Gallery

* feat: Dummy search results that pop up on input

* feat: Call a dummy search cmd on input

* feat: list of links and labels for search results

* feat: Clicking on a result adds it

* fix: When you click on link in results don't also add

* feat: Actually search the OEIS.

* feat: Cache search results and reopen search on click.

* ui: Layout and caption changes per design discussions

* fix: Never cache OEIS sequences with infinite indices
   This PR attempts to resolve numberscope#407. I added a half-dozen OEIS
   sequences as rapidly as I could and didn't get any errors.
   So hopefully this has squashed the problem, but it's hard to
   be certain.

* fix: Don't update URL until after parameter values have settled
   In order to be sure that parameter values have their "final" values
   in the case of an update (e.g., by calling the Paramable validate()
   method), it is necessary for a cascade of functions to be made async.
   This commit makes those changes and then awaits the validate() in the
   user interface, so that the URL won't be updated until all of the
   consequences of validate() have occurred (which may, for example,
   involve changing the parameter values, perhaps even the one that
   originally had its tentative value modified, necessitating the validate()
   call).

   Resolves numberscope#403.

* ui: show finger cursor over search results

* make oeis.org link open new tab

* info icon for info in search results

* ui: click off list of OEIS search results closes it

* fix: Restore sequences adding when you click on them in result list.

* fix: Various fixes to OEIS search bar/results per meeting

  - Return to the OEIS ID being a link but add the Wikipedia
    "external link" icon
  - Highlight the description of each sequence as you mouse over it,
    to provide clue you can click on it (to add as a sequence). The
    color used is a lightened version of the "primary" Numberscope color.
  - Display a message when there are no sequences to show as a result
    of a search. Note this can occur because there are _too many_ matches,
    or _no_ matches, and there is no way to tell which is the case from
    the OEIS api response, unfortunately. The message reflects this.
  - Fixed bug that results were being cached under the wrong search term
    if the searcher typed more while the search was executing.
  - Make sure that enter/clicking on magnifier always fires search

* fix: Don't make switcher cards for dummy message IDs

* fix: Really make sure magnifier puts/keeps search results up; reword message

* fix: catch errors in OEIS searching and report a 'soft' failure

---------

Co-authored-by: Kate Stange <[email protected]>
Adds a visualizer that displays the logs of sequence terms as a bar chart, with each bar a stack of sub-bars representing the prime factors of the term, with multiplicity. Mouseover and click can be used to highlight occurrences of a given prime.

Along the way, adds several enhancements to the visualizer infrastructure, including clarifying the circumstances under which presketch and setup are called; restricting some mouse and key events from reaching the visualizer to avoid interference with other widgets; allowing warnings as well as errors on parameter values; wait until initial sequence cache fill before validating visualizers; release focus from ui text entry fields when you hit enter;

Provides  docs for FactorFence, and updates general visualization authoring docs.

Adds a nice FactorFence example to the gallery

---------

Co-authored-by: Glen Whitney <[email protected]>
* test: Implement end-to-end testing with Playwright

  This is an in-progress cleaned version of the final PR (numberscope#361) of
  the Delft student user interface project, which it supersedes.

  Comments from the original PR that remain relevant:
  * The end-to-end tests run using Firefox and Chromium.
  * New tests are in the e2e folder.
  * The tests often depend on specific classes and IDs, so they may
    need to be updated upon changes to Numberscope.
  * The tests can be executed as the following npm script:
    `npm run test:e2e`
  * An interactive testing UI and debugger can be executed as the
    following npm script: `npm run test:e2e:ui`

  Caveats concerning trying this cleaned PR and its status:
  * Make certain to run `npm install` after pulling this PR.
  * Many of the tests do not yet pass, perhaps because of the "specific
    classes and IDs" point mentioned above and the fact that ui2 has
    diverged significantly from the Delft PR series.
  * Tests are not yet run automatically prior to commit.
  * I do not think there are any image tests yet, we need to try to add
    them.
  * Tests are not yet performed in the continuous integration checks
    to be run on GitHub; they should be.

* fix: Repair gallery/click on featured item test

* fix: Repair gallery/saving a specimen test

* fix: Repair scope tests.

* test: Run end-to-end tests prior to any commit

* test: Add image tests to e2e

  The strong desire for image tests led to a cascade of changes in this
  commit, mostly driven by the need to have reproducible images:

  - Removes all use of `sketch.noLoop()` and `sketch.loop()` in favor
    of the previously existing `stop()` and `continue()` visualizer methods,
    to allow:
  - Adds a `frames=NNN` query parameter to URLs to set the maximum number
    of frames a visualization may draw
  - Switches from the "static" instance of mathjs to a "dynamic" one, to
    allow its random number configuration to be controlled. In conjunction
    with this, moves all math functions into a single math module, as
    extensions of mathjs.
  - Removes all use of `Math.random()` in favor of the mathjs random
    generator
  - Adds a `randomSeed=AAAA` query parameter to URLs to make the mathjs
    random generator reproducible.
  - Documents all of the above changes.

* doc: describe running and creating code tests

  Also adds new tests for `src/shared/defineFeatured.ts` and corrects
  the documentation extraction facility for the package manager
  scripts.

  Resolves numberscope#25.
  Resolves numberscope#73.
  Resolves numberscope#246.

* test: Test that the caching mechanism won't double-calculate

   In other words, it should never call calculate twice for the
   same index. This is tested by 10K random accesses to indices
   less than 1M, followed by accessing the first 10K entries,
   followed by accessing the last 10K entries. Hopefully that
   should suffice.

   Resolves numberscope#54.

* fix: Prevent ModFill from hanging on extremely large input

  This is an initial pass at addressing numberscope#113.
  Note, however, that ModFill is not reporting to the person doing
  visualization that it is running with different parameter values
  than shown. So that still must be done, but for that part we will need
  a resolution to numberscope#112, which will be a sufficiently involve change that
  we should leave it to a spearate PR from this numberscope#420.

* doc: Update PR checklist.

  Resolves numberscope#174.

* maintenance: Remove and disallow trailing whitespace in code

  Resolves numberscope#219.

* maintenance: Make TypeScript target ES2022 in all cases.

  Resolves numberscope#226.

* maintenance: Run typecheck in CI

  Resolves numberscope#292.

* maintenance: Run GitHub CI on pushes to main as well

  Resolves numberscope#217

* test: Add OEIS Sequence-Visualizer test grid

  Use a list of "stressful sequences" to perform at least two image tests on
  each visualizer. As this change uncovered several existing errors, there
  are numerous other changes in this commit to return to a state of all tests
  passing. Here is an enumeration of other changes, in no particular order:

  * Change husky pre-commit actions so that the end-to-end tests are not
    run if there is an existing successful test since any files in the project
    last changed.
  * On most image tests, snapshot only the visualizer canvas.
  * Make 'axios' a runtime dependency, rather than just development, as it
    is used when obtaining OEIS sequence values in the running frontend.
  * So many other changes are made to Histogram.ts that this PR also adds
    hatching to show the elements with unknown factorization. Adds the
    "p5.brush" package to draw the hatching. Since this necessitates WebGL,
    adds a new 'P5GLVisualizer' base class for visualizers that wish to use
    a WebGL canvas.
  * Updates several dependencies to latest to make sure their out-of-dateness
    was not contributing to test issues.
  * Switches to the (ES) "module" import system from "commonjs" to ensure
    that was not contributing to test issues.

Resolves numberscope#294.

  * Specifies font locations in a way that vite understands how to
    relocate in both the 'dev' and 'build' versions of the app.
  * Properly marks both factor cache and value cache as empty at the
    initialization of an OEIS sequence.
  * Allows ±Infinity as valid values of INTEGER param fields for convenience.
  * Allows specimenQuery to take the output of parseSpecimenQuery to
    recreate the same specimen query as was parsed, for the sake of testing.
  * Turns off the browser default context menu on visualizers, which wasn't
    particularly useful and was interfering with the UI for some visualizers.
  * Fixes typos in Differences visualizer (this.first -> sequence.first)
  * Makes FactorHistogram visualizer a p5 WebGL-based visualizer, and adds
    hatching to the "0 factors" bar to indicate the terms with unknown
    factorizations.
  * Uses the WebGL default controls to implement panning, zooming, and (rather
    uselessly but somewhat spectacularly) rotating the plot in three
    dimensions for the FactorHistogram visualizer.
  * On FactorHistogram's first pass over the sequence data, collects the
    counts for each possible number of factors, as opposed to the number
    of factors for each entry, to avoid a possibly disastrously long loop
    on the second pass that accumulates bin counts. This also streamlines
    the computation of the largest number of factors in the data.
  * Factors out repeated code in FactorHistogram, for labeling bars and
    displaying text.
  * Fits the hover box in FactorHistogram to the text to be displayed.
  * FactorHistogram displays a temporary message if factoring is taking a
    long time.
  * Moves the bar labels of FactorHistogram just under their respective bars,
    to make sure they are visible.
  * Selects y-axis ticks at round numbers for FactorHistogram, and moves the
    tick labels closer to their ticks.
  * Prevents FactorHistogram from looping except when there is mouse activity.
  * Prevents ModFill from freezing up if too large a "mod dimension" is
    chosen. Note TODO: display a warning when a smaller mod dimension than
    requested is actually used.
  * Prevents Vue's reactivity system from attempting to modify the behavior
    of p5 sketches, using the Vue `markRaw` method. This change prevents
    some instances of infinite loops caused by cascading change notifications.
  * Tightens up the typing of P5Visualizer so that it is possible to derive
    another visualizer base class P5GLVisualizer from it. Also splits up the
    inhabit method so that P5GLVisualizer can modify it as needed.
  * Allow negative start indices for Show Factors visualizer.
  * Show at most 100 terms in Show Factors (no room on screen for more than
    that).
  * Updates TypeScript target versions of JavaScript

Resolves numberscope#226.

* refactor: better typing for param descriptions

* feat: Add ExtendedBigint ParamType

* refactor: Remove generic parameters from top levels of Paramable hierarchy

* feat: Uniform sequence bounds controls
  All params for controlling which terms of a sequence will be used in
  the visualization are removed from individual visualizers. Instead, there
  are uniform params in the Sequence classes themselves.

  Resolves numberscope#411.

  In this implementation, several other changes are necessary and/or
  expedient to make. In particular, the type of sequence indices is changed
  to bigint and bounds to ExtendedBigint (the union of bigint and ±Infinity),
  in preparation for addressing numberscope#455. Additional changes include:

  * Removal of SequenceDefault class, as everything now derives from Cached.
  * More care in caching of OEIS sequences, to provide some help with numberscope#459.
  * Individual-parameter validation functions now take a validation status
    to update based on the finding. This refactor avoids a frequent need to
    merge status objects.
  * Individual-parameter validation functions are called in a context
    of `this` set to the Paramable object, so that other data from the
    paramable can be accessed.
  * New `math` functions for dealing with ExtendedBigints.
  * NumberGlyph visualizer now tries to display as many terms as are
    available and will fit on the screen, except in case there are infinitely
    many terms available, in which case it still defaults to 64 terms. This
    change is needed so that the general length parameter for Sequences would
    affect what NumberGlyph displayed.
  * Puts reactivation of "known" OEIS sequences into a function
    rather than at top level of sequences.ts so that the timing of
    when it occurs can be controlled.
  * Renames property `_size` of P5Visualizer to `size` as multiple derived
    classes seem to need this value (i.e., not just used internally in the base
    class).
  * Makes sure the p5 loop is restarted if need be every time `.show()` is
    called on a visualizer.
  * Temporarily disables A000521 transversal tests until we can
    get incremental OEIS loading to work.
  * Adds a test for starting a visualization deep into a sequence.

* refactor: Apply OEIS modulus on the fly, so cache can point to communal one

* test: First working e2e tests with text and transversal

  Unfortunately, due to the intricacies of end-to-end testing with image
  comparison, this is a very large commit. It introduces running tests
  in a Docker container for the sake of reproducibility. (So note that
  henceforth you must have docker installed and running on your machine to
  perform testing, and hence to make a commit.)

  However, it turns out not all tests can be run in a Docker container --
  Firefox is not able to supply a WebGL context inside of Docker. Hence,
  some tests still need to be run directly on the host machine.

  To deal with all this, I felt it necessary to introduce the "make" tool.
  In particular, creating the necessary Docker image is slow, and I didn't
  want to have to repeat that except when truly necessary.

  That, in turn, means there are now many more configuration files, as
  well as auxiliary files created by the Makefile. The directory tree was
  becoming hopelessly cluttered. To keep things straight in my head, I felt
  it was really necessary to reorganize the directory structure. The biggest
  change is to put as many of the configuration files as possible in a new
  etc/ directory (the old-school unix name for where to put such things).
  I managed to get almost everything put in there. The one major holdout is
  the tsconfig files; I just couldn't find a way to get TypeScript to run
  without its config files at the top level of the project. Slightly
  annoying, but then, there are so many more annoying things about TypeScript.

  Anyhow, accomplishing the configuration file rearrangement resulted in
  updating eslint. Unfortunately, it went through a _major_ rewrite from
  version 8 to 9, and prettier-eslint has not been updated to work with
  the new version. I tried to work on that update myself, but prettier-eslint
  is incredibly intricate in the way that it handles configurations because
  it is trying to provide a great deal of flexibility and power, and it is
  trying to infer prettier configuration from eslint configuration (and maybe
  even vice-versa as well, I'm not sure). Since we were only using a tiny
  fraction of that power, it turned out to be significantly easier to just
  replace prettier-eslint with a custom script tools/prettiest.js that runs
  first prettier, then eslint. It also meant we could get rid of the
  lint-staged tool, so there is that. (While I was at it, I ran the "depcheck"
  tool and got rid of some other unneeded cruft that had accumulated.)

  But now it all works, and as a side benefit since just about every npm
  script now runs through make, you should never need to worry about
  running install before dev or build before preview, etc. Make keeps track
  of what depends on what and whether/how to redo the prerequisites before
  taking the requested action. Even better, if you just ran successful
  end-to-end tests, which take a while now, then `git commit` will not have
  to re-run them.

  Those are the major points. In addition, there are a number of minor changes
  resulting from errors uncovered in getting all the tests to work, etc:

  - Reordering of element attributes and other reformatting in vue components,
    because with the lint updates and reconfiguration, it now seems
    significantly stricter about vue templates.
  - First pass at an updated User Guide for ui2; it definitely needs more work.
  - Some other more minor documentation updates.
  - Updates tools/editor/autoformat.el so that emacs can use the new
    "prettiest" tool for formatting (even though I know I am likely the only
    one in the world who will ever use that).

* test: Add skipped tests reflecting known shortcomings

  As per Numberscope discussion today, this is the last missing element
  of the end-to-end testing PR. With these skipped tests, we are flagging
  that there are known concerns we want to resolve at least by beta, if not
  alpha. Also fixes a small gap in the docs for running from source that
  Aaron noticed in the meeting.

  With this, the PR will be marked ready for final review.

* fix: Generate docker image even with no existing test results

* chore: Attempt to run e2e tests in CI workflow

* test: No WebGL tests in CI :-(

* fix: format of Playwright reporter parameter

* chore: Try to grab GitHub CI actuals to make separate CI test file

* fix: oops test must succeed to generate artifact

* chore: Another try to grab ci snapshots

* test: Add in the extracted CI snapshots

* doc: Note that PR reviewers must run e2e tests themselves, too.

* chore: Stop grabbing snapshots when we don't need them

* fix: Changes as per review comments

* doc: more info on Docker (and fix remaining typo)

* test: Fuzz the pixel comparison in Firefox WebGL tests

* test: Really fuzz the pixel comparison in Firefox WebGL tests

* test: Check if docker tests passed before updating result directory

* doc: Improvements per code review

* doc: show the expected output of successful end-to-end test

* chore: remove stray comment that no longer applies

* fix: useful default camera controls for WebGL: left drag pans, wheel zooms

* fix: correct Husky action inclusion and test it

* fix: don't alter the URL just loaded, and reset frame limit on changes

* fix: adjust dragging, detected mouse position, and text size for zoom/pan

* fix: Keep the 'too many bins' message in a fixed absolute canvas position
…scope#463)

* doc: Harmonize guidance on git, cloning, and other workflows.

Resolves numberscope#221.

THere is still a signifcant amount of repetition/overlap among
CONTRIBUTING.md, and in the doc/ directory, onboarding.md,
running-from-source.md, and working-with-git-and-github.md. I wasn't
sure how we might compartmentalize these, because they are at
different levels of detail and for different audiences.

Since they are all now consistent with each other, I am not too fussed
by the repetitiveness. Maybe it's OK to give potential contributors multiple
places to find the information they need. But if the reviewer has suggestions
about how to reduce the redundancy, I am also open to making more changes.

* doc: Harmonize gitting-it-right with all of the other git/workflow docs

* doc: Changes pursuant to PR review
This fixes many issues with the Turtle Visualizer and implements several important new features. Here is a (probably not exhaustive) list:

* Adds "Fold rates" and "Stretch rates" parameters to allow animating the turtle instructions
* Allows all of the turtle instruction parameters to either be a single value applying to all elements of the domain or a separate value for each domain element. (Actually, it just repeats the last instruction element as many times as needed, which is occasionally useful in other ways as well.)
* Skips sequence entries not in the domain, rather than just stopping, but also checks that it is not skipping too large a percentage of entries
* Allows the drawing speed to be adjusted
* Allows the view to be panned, zoomed, and rolled
* Provides a few new (beautiful!) featured sequences
* Avoids restarting the path drawing on a resize
* Improves error checking of parameters
* Updates documentation
* Streamlines the parameter assignment scheme to minimize number of reinitializations at page load; this change improves the reliability of end-to-end tests

Resolves numberscope#76.
Resolves numberscope#223.
Resolves numberscope#412.
Resolves numberscope#453.
---------
Co-authored-by: Kate Stange <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
…ramable (numberscope#473)

* refactor: make overall status and status of each parameter part of Paramable

* doc: Describe updating CI snapshots, part 1

* doc: Describe updating CI snapshots, part 2

* fix: ModFill needs to recheck max modulus whenever modDimension changes

* fix: When size changes, presketch must be re-run

* fix: Changes as requested in review
  - Further detail on updating CI snapshots in docs
  - Catch empty formulas at validation time, not when they throw error later
  - Make descriptions and per-value validation messages each consistent:
    the former are full sentences with periods, the latter are phrases
    without periods
  - Add a FORMULA parameter type and use it for both places where formulas
    are used as parameters
  - Fix extraction of free variables in formulas
  - Rename the "modulo" parameter of ShiftCompare to "modulus"
  - Change the "log(EXPR^x)" in the default glyph growth formula in
    NumberGlyph visualizer to "log(EXPR)*x" to avoid crazy big exponentiations.
    Note this changes the visual results because the exponentiation was
    overflowing into floats large enough that the loss of accuracy made the
    following modulo operation a constant.

* chore: update GitHub CI snapshots for NumberGlyph

* doc: Fix typo and add message conventions
* fix: Make sure browser 'back' button actually updates specimen

  Prior to this change, the browser 'back' button would always update the
  URL, but the visualization would typically not change if you were going
  from one scope URL (`/?name=Blah&...`) to another.

  This PR remedies the situation by watching for route changes in the
  Scope view, and reloading the new visualizer and sequence **into** the
  specimen already being viewed. Implementing this tactic requires the
  ability to completely change the sequence and visualizer of a specimen,
  which previously did not exist.

  The change also enables delaying specimen initialization from Vue setup()
  time to just before the component is mounted. As a result, the Vue setup
  is not asynchronous, so the experimental Vue <Suspense> tag is no longer
  needed.

  Adds a test that the back button actually took effect, to the existing
  test that changes a parameter. (The extended test definitely failed in
  ui2 prior to this PR.)

  Uses the router to redirect from an empty URL to the last-viewed specimen,
  rather than the Scope component; this change avoids reloading that specimen
  when the URL is updated to reflect what specimen is being viewed.

  Finally, fixes a small previously-existing bug in which the Differences
  visualizer could never display more terms than were available in its last
  parameter settings, even if you changed the parameters to make more
  terms available.

  Resolves numberscope#414.

  Given the large number of internal changes to Specimen.ts, please
  in review do play with a veriety of visualizations to make sure all seems
  to be working well.

* fix: Make sure Thumbnails clean up their visualizers
* doc: Harmonize docsite style with Numberscope

* refactor: call checkParameters() when sequence changes

   Since checkParameters() is allowed to query the sequence, for
   consistency it must be called again when the sequence changes.

   Also updates the documentation on the "Building a Visualizer" page,
   adding an extensive lifecycle diagram that hopefully will clarify
   what happens when as a visualizer operates.

   Resolves numberscope#487.

* doc: Harmonize link styles with frontscope

* doc: Misc improvements
  * Admonition to be sure to clean up state in setup()
  * More thorough vetting of new versions in production in server admin page
  * Typos

  Resolves numberscope#319.

* doc: Update visualizer "Behind the scenes" documentation and move into source
   This commit updates the documentation of the VisualizerInterface,
   ParamableInterface, and Paramable base class, and relocates this
   documentation alongside the respective source code for ease of future
   maintenance. It also adds documentation for ParamType and ParamInterface.
   Resolves
   numberscope#425 (comment)

* doc: Split making-a-visualizer documentation as recommended

* doc: Convert diagrams to d2 and add more detail

* test: install d2 in GitHub workflows

* doc: Document INVALID_COLOR

* doc: User-guide documentation for the Sequence classes.
  These pages will be needed for numberscope#321.

* doc: fix unvisited link color in left sidebar

* doc: typo fixes etc per review

* doc: top-level page organization

* ui: Logo -> About, visualizer/sequence/documentation -> Help

* chore: update GitHub CI snapshots for new NavBar

* doc: Adjustments to NavBar per conversation

* doc: Hone Help menu further, crop sequence page images

* doc: Further revisions of lifecycle diagram as requested

* ui: Further refinements to controls, links, and popups as discussed

* doc: two tiny fixes per review

* doc: improve contributors table in about.md

* doc: add missing parens in lifecycle legend
* ui: Fix493: replace magnifying glass with help icon

* ui: Entire OEIS search result adds sequence; only link icon to OEIS

* ui: Scroll to and highlight new sequence cards, then let them fade

* feat: Add external links to OEIS for SpecimenCard and ParamEditor
numberscope#498)

* fix[Formula]: reject unknown function at parse to avoid throwing error

* feat: Convert formula calc errors to param tab warnings

* fix: Formula shouldn't throw when result is not a number.

* fix: Formula warning on conversion problem should show type.

* fix: Make sure warnings are shown every time Formula changes

* fix: Use the warnings themselves to know how many there are
* fix: Make sure warnings are shown every time Formula changes

* fix: Drag into sidebar always docks into an empty zone

  Prior to this PR, dragging a tab on top of a docked one would leave
  it floating over the docked tab. Moreover, if the empty zone in the
  sidebar was less than half the size of the tab being dragged, it would
  be impossible to dock the dragged tab at all.

  Now any drag into a sidebar with at least one empty zone results in
  a docking to the targeted zone if it is empty, or the first empty zone
  if not.

  Resolves numberscope#431.

  Note that this PR does not at the moment implement any resizing.
  There is resizing described in issue numberscope#431. However, now that this is
  working smoothly, it is making perfect visual sense to me that the
  dragged tab fits itself into the remaining empty space, leaving the
  already-docked tab alone, and in fact, this behavior seems to me
  preferable to what's described in numberscope#431. But if there remains a clear
  desire for the resizing as described in the issue, that can be
  implemented. Just let me know.

* chore: revert change to Formula.ts left over from Git manipulations

* fix: tab drags and docking/undocking preserve size whenever possible

   Also fixes bug that sometimes prevented docking in an apparently empty
   spot (the `empty` class was not being properly maintained).
@gwhitney
Copy link
Collaborator Author

OK, Grid now deals with aspect ratio better (I just insisted on square when using the Spiral path, rather than trying to implement rectangular spirals). More to come, happy to get guidance on properties whenever.

  Adds a new parameter type, ParamType.ACOLOR, for a color with alpha
  (opacity), selected with a third-party color picker. Sets property
  colors to use this new type, to provide an alternate way of visualizing
  overlapping colors. Ensures that properties are drawn overlaid in the
  order specified.

  Also allows the trace of the turtle in the Turtle visualizer to specify
  its opacity. Updates the featured Turtle specimens accordingly.
@gwhitney
Copy link
Collaborator Author

OK, now we have translucent property colors. I also made the Turtle path allow opacity setting, since we now have an rgba color ParamType (ParamType.ACOLOR). I made the "Beatty DNA" trace a bit transparent; it seemed to add some richness. I left all of the other feature specimens with opaque traces, although I think "Wait For It" also looks good at very low opacity, like hex 20. You may want to try it, and feel free to change the featured specimen if you like.

@katestange
Copy link
Member

katestange commented Dec 21, 2024

I think "Wait For It" also looks good at very low opacity, like hex 20. You may want to try it, a

Oh yes it looks very interesting with hex 20! Please go ahead and change that if you are back in the featured specimens.

Screenshot from 2024-12-21 16-52-14

@katestange
Copy link
Member

Any thoughts on which way to go with this bit of the UI design? There may of course be other good options besides any of (A)-(D) above.

The advantage of D in utility seems significant to me; it allows you to understand the presets. Maybe we could try that, and see if it causes too much confusion? All of them seem to have their points of confusion.

@gwhitney
Copy link
Collaborator Author

OK, I have done (D), and changed the Wait For It stroke. Let me know what you think. Next up on my end is to allow arbitrary formulas for properties, reducing the number of ad-hoc named properties on the list, while allowing many more possible properties.

@katestange
Copy link
Member

It looks good! I like the solution (D). It's nice because it lets you find out the settings for any preset. I wonder if we should even show the custom options all the time and simply have the preset populate them? Then when you change one of the options, you automatically move into "custom" mode? Pros: even easier and more intuitive to edit a preset. Cons: is it cluttered?

@gwhitney
Copy link
Collaborator Author

wonder if we should even show the custom options all the time and simply have the preset populate them? Then when you change one of the options, you automatically move into "custom" mode? Pros: even easier and more intuitive to edit a preset. Cons: is it cluttered?

I am open to it, but yes I am worried it will be overwhelming at first. We can defer to the next time we are all meeting together to decide maybe, or if you want me to check in a commit that shows the properties all the time, let me know.

@gwhitney
Copy link
Collaborator Author

Just a note so I don't forget: Grid in ui2 has a mathematics error in that it claims every number with an even number of 3 mod 4 prime factors to odd powers is a sum of two squares (e.g. 77), when in fact only those numbers with zero 3 mod 4 prime factors to odd powers is a sum of two squares. This needs to be fixed in this PR.

However, the math needs of the Grid overhaul to allow arbitrary formulas as properties are sufficiently strong that my plan is actually to put this on hold and do a PR that updates to mathjs 14, now with 100% more bigint support, and then rebase this once that PR gets through.

@katestange
Copy link
Member

katestange commented Dec 28, 2024

Just a note so I don't forget: Grid in ui2 has a mathematics error in that it claims every number with an even number of 3 mod 4 prime factors to odd powers is a sum of two squares (e.g. 77), when in fact only those numbers with zero 3 mod 4 prime factors to odd powers is a sum of two squares. This needs to be fixed in this PR.

Actually the correct criterion is the following: every prime divisor that is 3 mod 4 must appear to an even power. Hence 7^2 is a sum of squares. EDIT: Oh I misread what you wrote! We agree.

@katestange
Copy link
Member

However, the math needs of the Grid overhaul to allow arbitrary formulas as properties are sufficiently strong that my plan is actually to put this on hold and do a PR that updates to mathjs 14, now with 100% more bigint support, and then rebase this once that PR gets through.

Exciting!!

@gwhitney
Copy link
Collaborator Author

However, the math needs of the Grid overhaul to allow arbitrary formulas as properties are sufficiently strong that my plan is actually to put this on hold and do a PR that updates to mathjs 14, now with 100% more bigint support, and then rebase this once that PR gets through.

Exciting!!

@katestange @Vectornaut New Year's puzzle: I am fixing bigint bugs in mathjs, and one of them is that it will not choose a random bigint uniformly from an interval. If the interval is small, this choice is easy to make (at least to the quality of mathjs's existing pseudorandom number generator). However, if the the interval is very large, certainly once it's over 2^53 in length, this seems tricky to do in guaranteed finite time. My plan is to choose the smallest N so that $2^N$ is larger than (or equal to) the length of the interval, and then do N coin flips, and if I get unlucky and land outside the interval, redo the process until I land in the desired interval. But this method, of course, although it almost surely terminates, does not surely terminate. Any thoughts on a method that surely terminates? My concern is that if the interval length is a very large prime, there is no way to accurately split the choice into subchoices, since we only have 53 bits of uniform distribution in the base PRNG we are using.

I think this is mainly of academic interest, since I think in practice the "repeat N coin flips til you get lucky" method will work just fine. But I was curious if I was missing something, or if this was a known problem maybe with a known solution -- a quick web search didn't turn up anything obvious.

@Vectornaut
Copy link
Collaborator

@katestange

Any thoughts on a method that surely terminates?

Every method I've thought of so far would sacrifice a little uniformity, so you end up with a tradeoff between uniformity, speed, and memory use. You're already limited by the amount of randomness available to the base pseudo-random number generator, so maybe you could estimate how much uniformity you're losing from that; you could use the estimate to help decide how much additional loss of uniformity is acceptable.

@Vectornaut
Copy link
Collaborator

Vectornaut commented Dec 31, 2024

@katestange Here's another thought about deciding how much loss of uniformity is acceptable. The randomInt function must already be implemented for the JavaScript Number type. I'm guessing that this is the implementation (although I don't know enough TypeScript to understand how calling randomInt would lead to _randomInt being called). The uniformity of that implementation is limited by the precision of the Number type. You could choose a measure of uniformity, estimate its value for randomInt over Number, and then try to make sure that randomInt over BigInt is at least that uniform.

@katestange
Copy link
Member

My plan is to choose the smallest N so that 2 N is larger than (or equal to) the length of the interval, and then do N coin flips, and if I get unlucky and land outside the interval, redo the process until I land in the desired interval.

This is called rejection sampling and it is what is typically used. For example, in numpy, they use something called Lemire's method, which is a fancy version of rejection sampling. The underlying pseudo-random number generator is typically a bit-stream generator (one random bit per call).

@katestange
Copy link
Member

Here's another thought about deciding how much loss of uniformity is acceptable.

Thanks for the link. I feel fairly strongly that we should aim for perfect uniformity. Some of the visualizers (like Chaos) are very sensitive to the statistical properties of the PRNG. I don't see a problem with Glen's rejection sampling method, except that it could be modified a bit to avoid it taking an annoyingly long time in corner cases (e.g. random between (1,2^n+1). I'll try to find some helpful advice on that that isn't as complicated at Lemire's.

@katestange
Copy link
Member

katestange commented Dec 31, 2024

Ok I learned a bit more. I think plain rejection sampling as Glen described is fine, since the desired window is always at least 1/2 the multiple-of-2 window, so you have at least a 50% chance of win (edit: ignore, hang on, this isn't what you said). Lemire's is written up very nicely here. It is just a more efficient way of doing rejection sampling in the situation where your underlying PRNG is giving you randomness in fixed bitlengths, like bytes, and you might want to divide. So I don't think it's important for us; if that matters, it should be implemented in javascript's random integer in range functionality already.

@katestange
Copy link
Member

katestange commented Dec 31, 2024

@gwhitney I want to clarify what you said. Here's what I was imagining you said: say you want random in range [0,k]. Then take the smallest n so that 2^n > k and get random from [0,2^n). Reject if outside [0,k]. Is that what you meant? Then I think it's ok. You average one rejection since k is at least half of 2^n. But I don't see the relevance of the primeness of k in this proposal. (Note: the point is that getting random in [0,2^n) is just a bunch of calls to random in [0,2^s) concatenated, where s is small enough to invoke the javascript PRNG.)

@gwhitney
Copy link
Collaborator Author

@gwhitney I want to clarify what you said. Here's what I was imagining you said: say you want random in range [0,k]. Then take the smallest n so that 2^n > k and get random from [0,2^n). Reject if outside [0,k]. Is that what you meant? Then I think it's ok. You average one rejection since k is at least half of 2^n. But I don't see the relevance of the primeness of k in this proposal. (Note: the point is that getting random in [0,2^n) is just a bunch of calls to random in [0,2^s) concatenated, where s is small enough to invoke the javascript PRNG.)

Yes that's my proposal and I agree primeness is not relevant. It seems relevant to thr existence of a certain-to- terminate algorithm; it's hard to see how to break a choice among p things into two choices in a mathematically exact way that doesn't need just as much precision in each choice.

@katestange
Copy link
Member

Yes that's my proposal and I agree primeness is not relevant. It seems relevant to thr existence of a certain-to- terminate algorithm; it's hard to see how to break a choice among p things into two choices in a mathematically exact way that doesn't need just as much precision in each choice.

Got it. I say go with your original rejection sampling proposal.

@gwhitney
Copy link
Collaborator Author

gwhitney commented Jan 3, 2025

Got it. I say go with your original rejection sampling proposal.

Great: josdejong/mathjs#3345 submitted. This PR is basically on hold until Jos acts on that PR, so if anyone wants to submit a PR in the meantime, it would be fine.

@katestange
Copy link
Member

This PR is basically on hold until Jos acts on that PR, so if anyone wants to submit a PR in the meantime, it would be fine.

Is it a good moment for the self-similarity PR? It's not done but it's close.

@gwhitney
Copy link
Collaborator Author

gwhitney commented Jan 3, 2025

Is it a good moment for the self-similarity PR? It's not done but it's close.

Yes, perfectly fine. I would say this won't get back into gear until the second half of January, most likely.

@gwhitney
Copy link
Collaborator Author

NOTE this PR is out of sync since the update to main to correct a contributor name, and will likely need to be closed and resubmitted. But since we are likely to rebuild Grid as Spiral based on a generalized FormulaGrid visualizer anyway, that shouldn't really be a burden. Leaving this open as a placeholder until the pathway is worked out.

@gwhitney
Copy link
Collaborator Author

Note further that its first three commits had to be grabbed into the color math PR in order to get the new Turtle parameter layout to work, so those changes do not need to be made in any future reconstruction of this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants