Code

Automation MVP#879

Code

Merged

stalgiag merged 168 commits intomainw3c/aria-at-app:mainfrom automation-mvp-mergedw3c/aria-at-app:automation-mvp-mergedCopy head branch name to clipboardJan 22, 2024

+6,259-855

Commits

Commits on Aug 21, 2023

• Fix bug that would allow testPlanVersion to be updated to RECOMMENDED before the associated reports were all marked as final (but the tests were 100% done in the Test Queue)

  howard-e

  committed
  d2599d5

  Copy full SHA for d2599d5

Commits on Aug 24, 2023

• Refine raise an issue behavior (#753)

  Show description for cc1ec2d
  alflennik

  authored
  cc1ec2d

  Copy full SHA for cc1ec2d

• Fix for test results not being automatically saved when navigating through Test Run

  howard-e

  committed
  e31ecab

  Copy full SHA for e31ecab

Commits on Aug 28, 2023

• CollectionPass model added to schema, functioning getById resolver

  stalgiag

  committed
  8c862e0

  Copy full SHA for 8c862e0

• Working add mutation for CollectionJob

  stalgiag

  committed
  bc04463

  Copy full SHA for bc04463

Commits on Aug 29, 2023

• Create, find, findorcreate, findorupdateorcreate for CollectionJob

  stalgiag

  committed
  986b6bb

  Copy full SHA for 986b6bb

• Delete for CollectionJob, Add tests for schema

  stalgiag

  committed
  c4c09e1

  Copy full SHA for c4c09e1

• Routes for Automation Scheduler interaction with unit tests

  stalgiag

  committed
  bc6044c

  Copy full SHA for bc6044c

• Unit tests for updating CollectionJob

  stalgiag

  committed
  7df16d8

  Copy full SHA for 7df16d8

• Verify DB updates in Automation Scheduler unit tests, rollback DB changes after tests

  stalgiag

  committed
  1194741

  Copy full SHA for 1194741

Commits on Aug 30, 2023

• Add tests for CollectionJob spec

  stalgiag

  committed
  20a70f5

  Copy full SHA for 20a70f5

• Unit tests for CollectionJobService

  stalgiag

  committed
  6382c8c

  Copy full SHA for 6382c8c

Commits on Aug 31, 2023

• Add testPlanRun to CollectionJob schema

  stalgiag

  committed
  1886516

  Copy full SHA for 1886516

• Remove unused dependency, automation-scheduler.test

  stalgiag

  committed
  7d45396

  Copy full SHA for 7d45396

• Link CollectionJob to TestPlanRun for specific TestPlanReport

  stalgiag

  committed
  bbe3a9c

  Copy full SHA for bbe3a9c

• Populate test database with CollectionJob, Nullable relation on TestPlanRun delete

  stalgiag

  committed
  4d4b6e0

  Copy full SHA for 4d4b6e0

Commits on Sep 1, 2023

• Create services to replace resolver dependency for Test and TestResult retrieval

  stalgiag

  committed
  17cf931

  Copy full SHA for 17cf931

• Fix unit tests

  stalgiag

  committed
  549c780

  Copy full SHA for 549c780

Commits on Sep 5, 2023

• Singleton for AtLoader

  stalgiag

  committed
  5ee2515

  Copy full SHA for 5ee2515

• Singleton for BrowserLoader

  stalgiag

  committed
  9e58099

  Copy full SHA for 9e58099

• Decouple populateData and GraphQL context

  stalgiag

  committed
  5366239

  Copy full SHA for 5366239

• Service for getFinalizedTestResults

  stalgiag

  committed
  0676423

  Copy full SHA for 0676423

• findOrCreateTestResult Service

  stalgiag

  committed
  60b5c2c

  Copy full SHA for 60b5c2c

• Move auth checks into resolvers for submitTestResult and saveTestResult, avoid passing context for persistConflictCount, more work on AutomationController

  stalgiag

  committed
  be7f990

  Copy full SHA for be7f990

Commits on Sep 6, 2023

• Create test result from automated result submitted to route '/result' and test

  stalgiag

  committed
  3238796

  Copy full SHA for 3238796

• Cleanup of AutomationController

  stalgiag

  committed
  51094e1

  Copy full SHA for 51094e1

• Remove TestPlanRun when deleting CollectionJob

  stalgiag

  committed
  e981903

  Copy full SHA for e981903

Commits on Sep 7, 2023

• Remove all remaining instances of AtLoader or BrowserLoader use from GraphQLContext

  stalgiag

  committed
  be3f6df

  Copy full SHA for be3f6df

• Remove unused import

  stalgiag

  committed
  a492f79

  Copy full SHA for a492f79

Commits on Sep 8, 2023

• Spike on validation of Automated TestResult against historical Test Results

  stalgiag

  committed
  7e15cf8

  Copy full SHA for 7e15cf8

Commits on Sep 11, 2023

• Refactor to isolate state in automation scheduler tests

  stalgiag

  committed
  c5fb371

  Copy full SHA for c5fb371

• CollectionJob model testing, test association

  stalgiag

  committed
  f9ffbfb

  Copy full SHA for f9ffbfb

Commits on Sep 12, 2023

• Speed improvement on automation scheduler tests, Remove unneeded CollectionJob seeder

  stalgiag

  committed
  4bfff28

  Copy full SHA for 4bfff28

• Remove unused imports

  stalgiag

  committed
  1cf2c01

  Copy full SHA for 1cf2c01

• Cleanup AutomationController and CollectionJobService

  stalgiag

  committed
  230fc7d

  Copy full SHA for 230fc7d

• Construct Test data request body for Automation Scheduler, tests

  stalgiag

  committed
  b47c039

  Copy full SHA for b47c039

• Remove hard-coded values in automation scheduler test

  stalgiag

  committed
  e5a65b6

  Copy full SHA for e5a65b6

• Remove unused imports automation scheduler

  stalgiag

  committed
  2224915

  Copy full SHA for 2224915

• Switch tested report row for service tests to prevent intermittent deadlock issues with concurrent tests

  stalgiag

  committed
  77ca024

  Copy full SHA for 77ca024

• Secure CollectionJob resolvers

  stalgiag

  committed
  0271c28

  Copy full SHA for 0271c28

Commits on Sep 19, 2023

• Error handling middleware and more error checking for AutomationController

  stalgiag

  committed
  d6cc0fb

  Copy full SHA for d6cc0fb

• Replace Failed CollectionJobStatus with Error, use enum for statuses

  stalgiag

  committed
  d9fe999

  Copy full SHA for d9fe999

• Schedule new collection job only available through authenticated gql mutation

  stalgiag

  committed
  815aa37

  Copy full SHA for 815aa37

• Restart collection job only available through authenticated gql mutation

  stalgiag

  committed
  8a5dc17

  Copy full SHA for 8a5dc17

• Cancel collection job only available through authenticated gql mutation

  stalgiag

  committed
  8236d45

  Copy full SHA for 8236d45

• Skip schema test for cancelCollectionJob

  stalgiag

  committed
  d2a0086

  Copy full SHA for d2a0086

• Delete collection job only available through authenticated gql mutation

  stalgiag

  committed
  8b073bc

  Copy full SHA for 8b073bc

Commits on Sep 20, 2023

• Update server/graphql-schema.js

  Show description for 35feab6
  

  stalgiag

  and

  jugglinmike

  authored
  35feab6

  Copy full SHA for 35feab6

Commits on Sep 21, 2023

• Literal strings for job status in tests

  stalgiag

  committed
  10457bb

  Copy full SHA for 10457bb

• router.use handleError in automation routes

  stalgiag

  committed
  c6221a7

  Copy full SHA for c6221a7

Commits on Sep 28, 2023

• Merge branch 'main' into job-scheduler

  Show description for e8aac97
  howard-e

  committed
  e8aac97

  Copy full SHA for e8aac97

• Cleanup after merge

  howard-e

  committed
  5ae528d

  Copy full SHA for 5ae528d

Commits on Oct 3, 2023

• Merge pull request #772 from w3c/job-scheduler

  Show description for 552cf46
  stalgiag

  authored
  552cf46

  Copy full SHA for 552cf46

Commits on Oct 5, 2023

• Multiple action buttons supported and test, BasicModal

  stalgiag

  committed
  f1553b1

  Copy full SHA for f1553b1

• Schedule collection job through Test Plan Report Status Dialog

  stalgiag

  committed
  8c8e1b3

  Copy full SHA for 8c8e1b3

• Completion status for automated runs

  stalgiag

  committed
  a3189de

  Copy full SHA for a3189de

• Allow mock automation scheduler to post results

  stalgiag

  committed
  f7265c5

  Copy full SHA for f7265c5

• Real time test completion re-renders

  stalgiag

  committed
  743a37d

  Copy full SHA for 743a37d

• Finish bot run button skeleton

  stalgiag

  committed
  81399a2

  Copy full SHA for 81399a2

• Support reassigning of bot jobs, add related UI

  stalgiag

  committed
  f4b8d38

  Copy full SHA for f4b8d38

• Column in TestPlanRun for indicating that a job was initially automated

  stalgiag

  committed
  c576b35

  Copy full SHA for c576b35

• Allow assertion writing by reassigned human tester, correctly calculate possible assertions

  stalgiag

  committed
  41f1f7f

  Copy full SHA for 41f1f7f

• Render tests evaluated for reassigned bot tests

  stalgiag

  committed
  7de48fd

  Copy full SHA for 7de48fd

• Support marking collection job as finished on server

  stalgiag

  committed
  4251872

  Copy full SHA for 4251872

• Real-time status updates for automated tests

  stalgiag

  committed
  42782aa

  Copy full SHA for 42782aa

• Info boxes when reviewing bot run

  stalgiag

  committed
  b84954b

  Copy full SHA for b84954b

• Create new collection job directly from assigning bot tester

  stalgiag

  committed
  be258f2

  Copy full SHA for be258f2

• Delete test plan run from Mark Finished automation dialog

  stalgiag

  committed
  a57fce2

  Copy full SHA for a57fce2

• Test run navigation bot-specific status icons

  stalgiag

  committed
  6b8e95b

  Copy full SHA for 6b8e95b

• Rename bot user

  stalgiag

  committed
  a232d49

  Copy full SHA for a232d49

• Create FinishBotRunDialog.test.jsx

  stalgiag

  committed
  c3f5e0f

  Copy full SHA for c3f5e0f

• Test custom components with actions in BasicModal

  stalgiag

  committed
  a2b755d

  Copy full SHA for a2b755d

• Testing for AssignTesterDropdown

  stalgiag

  committed
  3a789c8

  Copy full SHA for 3a789c8

• Fix merge issues, handle alert for AssignTesterDropdown, move relevant styles to dedicated file

  stalgiag

  committed
  466c149

  Copy full SHA for 466c149

• Remove errant useRef

  stalgiag

  committed
  f741c5c

  Copy full SHA for f741c5c

• max-height instead of height for assign-menu

  stalgiag

  committed
  1983382

  Copy full SHA for 1983382

• useOnHide and Allow multiple modals to be open at a time.

  stalgiag

  committed
  d3b9c6e

  Copy full SHA for d3b9c6e

• Add tests for BotRunTestStatusList, ensure correct display

  stalgiag

  committed
  6b6c3ce

  Copy full SHA for 6b6c3ce

Commits on Oct 6, 2023

• Allow testing of historical results flow

  stalgiag

  committed
  ba9374d

  Copy full SHA for ba9374d

Commits on Oct 12, 2023

• Silence console errors related to initiatedByAutomation in client test mocks

  stalgiag

  committed
  5cc15a4

  Copy full SHA for 5cc15a4

• Ensure correct at id and browser id is used for getting historical results

  stalgiag

  committed
  13f8e96

  Copy full SHA for 13f8e96

Commits on Oct 13, 2023

• Rename 'FinishBotRun' dialog to 'ManageBotRun', add missing actions

  stalgiag

  committed
  f68b422

  Copy full SHA for f68b422

Commits on Oct 16, 2023

• Fix failing test with incorrect import, ManageBotRunDialog

  stalgiag

  committed
  e31e440

  Copy full SHA for e31e440

Commits on Oct 18, 2023

• Don't delete test results but test plan run

  stalgiag

  committed
  33f6f2a

  Copy full SHA for 33f6f2a

• Simplify and correct historical test validation

  stalgiag

  committed
  32069f0

  Copy full SHA for 32069f0

• Remove unused imports

  stalgiag

  committed
  80382b6

  Copy full SHA for 80382b6

Commits on Oct 19, 2023

• Mark as finished button behavior, allow submission of existing results without providing complete input

  stalgiag

  committed
  17735f0

  Copy full SHA for 17735f0

• Fix bug in AddTestToQueueWithConfirmation when no title present, remove jest debug

  stalgiag

  committed
  ae78118

  Copy full SHA for ae78118

• Centralize ApolloClient

  stalgiag

  committed
  53d18b1

  Copy full SHA for 53d18b1

• Setup for ManageTestBotRunDialog actions

  stalgiag

  committed
  cdbd241

  Copy full SHA for cdbd241

• Retry cancelled jobs

  stalgiag

  committed
  95fd929

  Copy full SHA for 95fd929

• Remove deprecated collection job cancellation

  stalgiag

  committed
  cffcade

  Copy full SHA for cffcade

Commits on Oct 26, 2023

• Force IPv4 for internal HTTP requests

  gnarf

  committed
  019e305

  Copy full SHA for 019e305

• fixup! Force IPv4 for internal HTTP requests

  gnarf

  committed
  549783a

  Copy full SHA for 549783a

• Merge pull request #827 from gnarf/force-ipv4

  Show description for 0f3e6a7
  stalgiag

  authored
  0f3e6a7

  Copy full SHA for 0f3e6a7

• Add updated copy in body of add test to queue modal

  stalgiag

  committed
  c5349ff

  Copy full SHA for c5349ff

Commits on Nov 1, 2023

• Fix sequencing of collectionjobs for manual id creation

  stalgiag

  committed
  7bbd9bc

  Copy full SHA for 7bbd9bc

• Unassign yourself on historical bot run does not reassign to bot

  stalgiag

  committed
  2237649

  Copy full SHA for 2237649

• Only show manage bot run button when run is not finished

  stalgiag

  committed
  e73cea2

  Copy full SHA for e73cea2

• Switch to using testCsvRow for updating results

  stalgiag

  committed
  a65ee1b

  Copy full SHA for a65ee1b

• Safely handle viewing from logged out state

  stalgiag

  committed
  8bacc68

  Copy full SHA for 8bacc68

Commits on Nov 2, 2023

• External logs url for CollectionJob

  stalgiag

  committed
  50110d0

  Copy full SHA for 50110d0

• Update AutomationController to support creating missing At or Browser Versions

  stalgiag

  committed
  8b82992

  Copy full SHA for 8b82992

• Cleanup

  stalgiag

  committed
  46b33a7

  Copy full SHA for 46b33a7

• Don't allow assigning bot runs to tester with run in same test plan report

  stalgiag

  committed
  e7b422c

  Copy full SHA for e7b422c

Commits on Nov 9, 2023

• Defer update resolution after the timeout fires in closeWithUpdate, AddTestToQueueWithConfirmation

  stalgiag

  committed
  9640882

  Copy full SHA for 9640882

• Pluralization for BotRunTestStatusUnorderedList

  stalgiag

  committed
  a55bf3f

  Copy full SHA for a55bf3f

• Plurals for 0, update unit test, rename filteredTestPlanRuns to botTestPlanRuns in BotRunTestStatusList

  stalgiag

  committed
  a4c8c62

  Copy full SHA for a4c8c62

• Remove unnecessary checks, BotRunTestStatusList

  stalgiag

  committed
  f2344c1

  Copy full SHA for f2344c1

• Correct useTriggerLoad use

  stalgiag

  committed
  143638f

  Copy full SHA for 143638f

• Better distinguish click assertions in BasicModal unit test

  stalgiag

  committed
  da6364d

  Copy full SHA for da6364d

• Prevent failing unit tests in dev

  stalgiag

  committed
  3054696

  Copy full SHA for 3054696

• getBotUserFromAtBrowser -> getBotUsernameFromAtBrowser

  stalgiag

  committed
  9e1f11c

  Copy full SHA for 9e1f11c

• Type fix in ManageBotRRunDialog

  Show description for 8b75bf1
  

  stalgiag

  and

  jugglinmike

  authored
  8b75bf1

  Copy full SHA for 8b75bf1

• Missing conflicts length conditional for Mark as Final button in TestQueueRow

  stalgiag

  committed
  d691397

  Copy full SHA for d691397

Commits on Nov 15, 2023

• Fix issue with TestNavigator recognizing test skeleton as complete

  stalgiag

  committed
  27eeb25

  Copy full SHA for 27eeb25

• Handle initial render null browserVersion graphql issue

  stalgiag

  committed
  029d592

  Copy full SHA for 029d592

• readOnly for TestRun textarea when reviewing bot run

  stalgiag

  committed
  f4fcc09

  Copy full SHA for f4fcc09

• Use test plan report runnable tests instead of test plan version tests for matching results to ids

  stalgiag

  committed
  1233bb1

  Copy full SHA for 1233bb1

• Remove unused imports from previous commit

  stalgiag

  committed
  a1a2cf4

  Copy full SHA for a1a2cf4

• Only show bots on assign dropdown for bot-supporting test plan reports

  stalgiag

  committed
  6ef71ba

  Copy full SHA for 6ef71ba

• Update client/hooks/useTestPlanRunValidatedAssertionCounts.js

  Show description for b143a34
  

  stalgiag

  and

  jugglinmike

  authored
  b143a34

  Copy full SHA for b143a34

• Simplify testsResultsLength calculation in useTestPlanRunValidatedAssertionCounts.js

  Show description for 407efd2
  

  stalgiag

  and

  jugglinmike

  authored
  407efd2

  Copy full SHA for 407efd2

• Remove unused useMemo dependencies, line length edit

  stalgiag

  committed
  13c600c

  Copy full SHA for 13c600c

• Default value for testResults in PreviouslyAutomatedTestCompletionStatus

  stalgiag

  committed
  9d582c5

  Copy full SHA for 9d582c5

• Remove deprecated props for BasicModal

  stalgiag

  committed
  f3bd906

  Copy full SHA for f3bd906

• Default parameter for new Date() in AtService

  stalgiag

  committed
  62ad736

  Copy full SHA for 62ad736

• Typo in BotRunTestStatusList, Additional testing for AssignTesterDropdown

  stalgiag

  committed
  74f0bcd

  Copy full SHA for 74f0bcd

• Fix rename in export, retryCancelledCollections

  stalgiag

  committed
  403d47c

  Copy full SHA for 403d47c

Commits on Nov 16, 2023

• Merge pull request #802 from w3c/automation-ui

  Show description for f590547
  stalgiag

  authored
  f590547

  Copy full SHA for f590547

• Merge branch 'main' into automation-mvp

  stalgiag

  committed
  798c43c

  Copy full SHA for 798c43c

• Fix issue with merge conflict resolution

  stalgiag

  committed
  0dccbf8

  Copy full SHA for 0dccbf8

• Additional fix to readOnly text area after merge conflict resolution

  stalgiag

  committed
  262a7eb

  Copy full SHA for 262a7eb

Commits on Dec 5, 2023

• Do not render zeros in place of omitted content (#853)

  Show description for 07a9269
  jugglinmike

  authored
  07a9269

  Copy full SHA for 07a9269

• Correct heuristic for identifying "finished" runs (#854)

  Show description for 0589afd
  jugglinmike

  authored
  0589afd

  Copy full SHA for 0589afd

Commits on Dec 6, 2023

• Store "No output was detected." for empty string output during automated tests (#857)

  Show description for a6b2934
  stalgiag

  authored
  a6b2934

  Copy full SHA for a6b2934

• Add cutoff date as temporary fix to handle deviation of automation-mvp from main (#858)

  stalgiag

  authored
  8756e62

  Copy full SHA for 8756e62

• Correct test identification heuristic

  Show description for 6269a96
  jugglinmike

  committed
  6269a96

  Copy full SHA for 6269a96

Commits on Dec 7, 2023

• Show tests as queued even before the job starts running, BotRunTestStatusList (#860)

  stalgiag

  authored
  42517dc

  Copy full SHA for 42517dc

• Make automation result posting unit tests and simulator use rowNumber instead of index (#862)

  Show description for d39284f
  stalgiag

  authored
  d39284f

  Copy full SHA for d39284f

• Start Automation Collector via github action (#831)

  Show description for 5e4f64e
  

  3 peopleauthored
  5e4f64e

  Copy full SHA for 5e4f64e

• Correct erroneous path in deployment script

  jugglinmike

  committed
  667934e

  Copy full SHA for 667934e

• Define automation-related variables in ENV files

  jugglinmike

  committed
  a0563b0

  Copy full SHA for a0563b0

• single source of truth for env var (#864)

  gnarf

  authored
  eb9f3b0

  Copy full SHA for eb9f3b0

• Nullable atVersion

  stalgiag

  committed
  2e62661

  Copy full SHA for 2e62661

Commits on Dec 13, 2023

• Merge branch 'main' into automation-mvp-rebase-attempt

  stalgiag

  committed
  8ff6e6b

  Copy full SHA for 8ff6e6b

Commits on Dec 14, 2023

• Fixes after merge

  stalgiag

  committed
  d63609d

  Copy full SHA for d63609d

• Remove aria-at git commit cut off date

  stalgiag

  committed
  9c99b4f

  Copy full SHA for 9c99b4f

• Merge branch 'main' into automation-mvp-merged

  stalgiag

  committed
  15c594a

  Copy full SHA for 15c594a

• Clean up leftover import

  stalgiag

  committed
  fbf8eef

  Copy full SHA for fbf8eef

Commits on Dec 18, 2023

• Correct bug in dropdown rendering (#869)

  Show description for c2a573d
  

  jugglinmike

  authored and

  stalgiag

  committed
  c2a573d

  Copy full SHA for c2a573d

• self-tester right of passage moment (#865)

  

  gnarf

  authored and

  stalgiag

  committed
  a25180e

  Copy full SHA for a25180e

Commits on Dec 19, 2023

• Handle v2 pass/fail for "tests evaluated" and "assertionsAssigned" view

  stalgiag

  committed
  bd1d8e3

  Copy full SHA for bd1d8e3

Commits on Dec 20, 2023

• Rename retryCancelledCollections to retryCanceledCollections

  stalgiag

  committed
  97ba68f

  Copy full SHA for 97ba68f

• Shorten comment

  stalgiag

  committed
  c042a20

  Copy full SHA for c042a20

• Require scenarioResults in schema

  stalgiag

  committed
  52fc072

  Copy full SHA for 52fc072

• Remove unexpected behaviors default

  stalgiag

  committed
  37c54ae

  Copy full SHA for 37c54ae

• Fix tests for null unexpectedBehavior

  stalgiag

  committed
  61e616a

  Copy full SHA for 61e616a

Commits on Dec 21, 2023

• Prevent multiple bot runs in the same test plan report (#866)

  Show description for 7e78b1b
  stalgiag

  authored
  7e78b1b

  Copy full SHA for 7e78b1b

• Update deploy readme for Workflow Automation

  stalgiag

  committed
  1986774

  Copy full SHA for 1986774

• Raise height of text area slightly to make possible scroll more obvious

  stalgiag

  committed
  ad17228

  Copy full SHA for ad17228

• JSDoc for TestResultReadService

  stalgiag

  committed
  a62c135

  Copy full SHA for a62c135

Commits on Jan 10, 2024

• Move new get tests code from resolver to service

  stalgiag

  committed
  de0148d

  Copy full SHA for de0148d

• Add NVDA bot user using a seeder (#881)

  Show description for 6689223
  

  stalgiag

  and

  jugglinmike

  authored
  6689223

  Copy full SHA for 6689223

• Support automation reporting test results by presentationNumber for v2 tests (#886)

  Show description for b48275f
  

  stalgiag

  and

  jugglinmike

  authored
  b48275f

  Copy full SHA for b48275f

• Require intentional cache clearing for AtLoader/BrowserLoader

  stalgiag

  committed
  f53015a

  Copy full SHA for f53015a

• Specify AT on test result retrieval

  stalgiag

  committed
  f1f9a2c

  Copy full SHA for f1f9a2c

• Only check at on v2 tests

  stalgiag

  committed
  6759024

  Copy full SHA for 6759024