self-tester right of passage moment #865
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stalgiag
approved these changes
Dec 13, 2023
stalgiag
pushed a commit
that referenced
this pull request
Dec 18, 2023
stalgiag
added a commit
that referenced
this pull request
Jan 22, 2024
* Fix bug that would allow testPlanVersion to be updated to RECOMMENDED before the associated reports were all marked as final (but the tests were 100% done in the Test Queue) * Refine raise an issue behavior (#753) * Refine raise an issue behavior * Address feedback * Fixed squished dot icon * Address last feedback * Hide closed issues on datamgmt page * Fix for test results not being automatically saved when navigating through Test Run * CollectionPass model added to schema, functioning getById resolver * Working add mutation for CollectionJob * Create, find, findorcreate, findorupdateorcreate for CollectionJob * Delete for CollectionJob, Add tests for schema * Routes for Automation Scheduler interaction with unit tests * Unit tests for updating CollectionJob * Verify DB updates in Automation Scheduler unit tests, rollback DB changes after tests * Add tests for CollectionJob spec * Unit tests for CollectionJobService * Add testPlanRun to CollectionJob schema * Remove unused dependency, automation-scheduler.test * Link CollectionJob to TestPlanRun for specific TestPlanReport * Populate test database with CollectionJob, Nullable relation on TestPlanRun delete * Create services to replace resolver dependency for Test and TestResult retrieval * Fix unit tests * Singleton for AtLoader * Singleton for BrowserLoader * Decouple populateData and GraphQL context * Service for getFinalizedTestResults * findOrCreateTestResult Service * Move auth checks into resolvers for submitTestResult and saveTestResult, avoid passing context for persistConflictCount, more work on AutomationController * Create test result from automated result submitted to route '/result' and test * Cleanup of AutomationController * Remove TestPlanRun when deleting CollectionJob * Remove all remaining instances of AtLoader or BrowserLoader use from GraphQLContext * Remove unused import * Spike on validation of Automated TestResult against historical Test Results * Refactor to isolate state in automation scheduler tests * CollectionJob model testing, test association * Speed improvement on automation scheduler tests, Remove unneeded CollectionJob seeder * Remove unused imports * Cleanup AutomationController and CollectionJobService * Construct Test data request body for Automation Scheduler, tests * Remove hard-coded values in automation scheduler test * Remove unused imports automation scheduler * Switch tested report row for service tests to prevent intermittent deadlock issues with concurrent tests * Secure CollectionJob resolvers * Error handling middleware and more error checking for AutomationController * Replace Failed CollectionJobStatus with Error, use enum for statuses * Schedule new collection job only available through authenticated gql mutation * Restart collection job only available through authenticated gql mutation * Cancel collection job only available through authenticated gql mutation * Skip schema test for cancelCollectionJob * Delete collection job only available through authenticated gql mutation * Update server/graphql-schema.js Co-authored-by: jugglinmike <[email protected]> * Literal strings for job status in tests * router.use handleError in automation routes * Cleanup after merge * Multiple action buttons supported and test, BasicModal * Schedule collection job through Test Plan Report Status Dialog * Completion status for automated runs * Allow mock automation scheduler to post results * Real time test completion re-renders * Finish bot run button skeleton * Support reassigning of bot jobs, add related UI * Column in TestPlanRun for indicating that a job was initially automated * Allow assertion writing by reassigned human tester, correctly calculate possible assertions * Render tests evaluated for reassigned bot tests * Support marking collection job as finished on server * Real-time status updates for automated tests * Info boxes when reviewing bot run * Create new collection job directly from assigning bot tester * Delete test plan run from Mark Finished automation dialog * Test run navigation bot-specific status icons * Rename bot user * Create FinishBotRunDialog.test.jsx * Test custom components with actions in BasicModal * Testing for AssignTesterDropdown * Fix merge issues, handle alert for AssignTesterDropdown, move relevant styles to dedicated file * Remove errant useRef * max-height instead of height for assign-menu * useOnHide and Allow multiple modals to be open at a time. * Add tests for BotRunTestStatusList, ensure correct display * Allow testing of historical results flow * Silence console errors related to initiatedByAutomation in client test mocks * Ensure correct at id and browser id is used for getting historical results * Rename 'FinishBotRun' dialog to 'ManageBotRun', add missing actions * Fix failing test with incorrect import, ManageBotRunDialog * Don't delete test results but test plan run * Simplify and correct historical test validation * Remove unused imports * Mark as finished button behavior, allow submission of existing results without providing complete input * Fix bug in AddTestToQueueWithConfirmation when no title present, remove jest debug * Centralize ApolloClient * Setup for ManageTestBotRunDialog actions * Retry cancelled jobs * Remove deprecated collection job cancellation * Force IPv4 for internal HTTP requests * fixup! Force IPv4 for internal HTTP requests * Add updated copy in body of add test to queue modal * Fix sequencing of collectionjobs for manual id creation * Unassign yourself on historical bot run does not reassign to bot * Only show manage bot run button when run is not finished * Switch to using testCsvRow for updating results * Safely handle viewing from logged out state * External logs url for CollectionJob * Update AutomationController to support creating missing At or Browser Versions * Cleanup * Don't allow assigning bot runs to tester with run in same test plan report * Defer update resolution after the timeout fires in closeWithUpdate, AddTestToQueueWithConfirmation * Pluralization for BotRunTestStatusUnorderedList * Plurals for 0, update unit test, rename filteredTestPlanRuns to botTestPlanRuns in BotRunTestStatusList * Remove unnecessary checks, BotRunTestStatusList * Correct useTriggerLoad use * Better distinguish click assertions in BasicModal unit test * Prevent failing unit tests in dev * getBotUserFromAtBrowser -> getBotUsernameFromAtBrowser * Type fix in ManageBotRRunDialog Co-authored-by: jugglinmike <[email protected]> * Missing conflicts length conditional for Mark as Final button in TestQueueRow * Fix issue with TestNavigator recognizing test skeleton as complete * Handle initial render null browserVersion graphql issue * readOnly for TestRun textarea when reviewing bot run * Use test plan report runnable tests instead of test plan version tests for matching results to ids * Remove unused imports from previous commit * Only show bots on assign dropdown for bot-supporting test plan reports * Update client/hooks/useTestPlanRunValidatedAssertionCounts.js Co-authored-by: jugglinmike <[email protected]> * Simplify testsResultsLength calculation in useTestPlanRunValidatedAssertionCounts.js Co-authored-by: jugglinmike <[email protected]> * Remove unused useMemo dependencies, line length edit * Default value for testResults in PreviouslyAutomatedTestCompletionStatus * Remove deprecated props for BasicModal * Default parameter for new Date() in AtService * Typo in BotRunTestStatusList, Additional testing for AssignTesterDropdown * Fix rename in export, retryCancelledCollections * Fix issue with merge conflict resolution * Additional fix to readOnly text area after merge conflict resolution * Do not render zeros in place of omitted content (#853) By relying on the "truthiness" of Number values to short-circuit boolean expressions, some rendering logic produced the text "0" instead of omitting any markup at all. Replace the sub-expressions whose value could be coerced to booleans with sub-expressions which directly evaluate to booleans (and which do not produce markup). * Correct heuristic for identifying "finished" runs (#854) * Correct heuristic for identifying "finished" runs Prior to this commit, the user interface reported that Test Plan Runs were "finished" when they included zero test results. This state, though difficult to observe in development environments, will be apparent whenever there is a perceivable delay while scheduling collection jobs (including when collection jobs are never scheduled due to some unexpected error). Test Admins were unable to manage Test Plan Runs from this state. Update the heuristic to identify Test Plan runs in this state as "unfinished." * fixup! Correct heuristic for identifying "finished" runs * Store "No output was detected." for empty string output during automated tests (#857) * When an empty string is recorded as an output by a bot, store as No Output string * Handle null and white space cases * Use constant for no output string * Annotate the mirrored constants and update the name of the client constant * Add cutoff date as temporary fix to handle deviation of automation-mvp from main (#858) * Correct test identification heuristic Co-author: Stalgia Grigg <[email protected]> * Show tests as queued even before the job starts running, BotRunTestStatusList (#860) * Make automation result posting unit tests and simulator use rowNumber instead of index (#862) * fixup! make automation result posting unit tests use rowNumber instead of index * Update mock automation scheduler to send rowNumber instead of index * Start Automation Collector via github action (#831) * Squash of first draft WIP for automation collector via github * Insert JWT-signing key * move github workflow service, load key on app startup * move files around, lint cleanup * trying to cleanup switchpoint/implement retry * review fixes * more conversion to trigger workflows * more review comment cleanup * removing nonce * remove TODO that wont ever be TODONE * re-add return job that was forgotten * moving mock server endpoints around * pull switchpoint into triggerWorkflow * adding testIds param to triggerWorkflow * await updateCollectionJob and handle null in restartCollectionJob * add missing await * convert fetch to axios --------- Co-authored-by: Mike Pennisi <[email protected]> Co-authored-by: Stalgia Grigg <[email protected]> * Correct erroneous path in deployment script * Define automation-related variables in ENV files * single source of truth for env var (#864) * Nullable atVersion * Fixes after merge * Remove aria-at git commit cut off date * Clean up leftover import * Correct bug in dropdown rendering (#869) Limit the insertion of horizontal padding to the specific case which requires it. * self-tester right of passage moment (#865) * Handle v2 pass/fail for "tests evaluated" and "assertionsAssigned" view * Rename retryCancelledCollections to retryCanceledCollections * Shorten comment * Require scenarioResults in schema * Remove unexpected behaviors default * Fix tests for null unexpectedBehavior * Prevent multiple bot runs in the same test plan report (#866) * Prevent multiple bot runs in AddTestToQueueWithConfirmation * Render checkmark for assigned bot * Update AddTestToQueueWithConfirmation test * More readable conditional sequence in AssignTeserDropdown * useMemo for alreadyHasBotInTestPlanReport in AddTestToQueueWithConfirmation * Update deploy readme for Workflow Automation * Raise height of text area slightly to make possible scroll more obvious * JSDoc for TestResultReadService * Move new get tests code from resolver to service * Add NVDA bot user using a seeder (#881) * Add NVDA Bot user using seeder * Populate NVDA Bot user from single source, remove code for creating user in service * Use centralized username for bot user Co-authored-by: jugglinmike <[email protected]> --------- Co-authored-by: jugglinmike <[email protected]> * Support automation reporting test results by presentationNumber for v2 tests (#886) * Support rowNumber identification using presentationNumber for v2 tests * Update error message --------- Co-authored-by: Mike Pennisi <[email protected]> * Require intentional cache clearing for AtLoader/BrowserLoader * Specify AT on test result retrieval * Only check at on v2 tests --------- Co-authored-by: Howard Edwards <[email protected]> Co-authored-by: Alexander Flenniken <[email protected]> Co-authored-by: jugglinmike <[email protected]> Co-authored-by: Mx. Corey Frang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.