Automation MVP

[stalgiag]

This PR is a combination of the Automation MVP feature currently in sandbox and the most recent changes on main. I've tested locally and could not find any issues. That said, there are substantials changes with this feature and there has been significant active development on main since this feature began to be developed separately. I would be grateful for any additional manual testing.

There are have been several steps of code review on the new feature code but more is welcome in instances where this would be helpful and not an unnecessary burden on the reviewer. Especially with regards to how merge conflicts were handled with main.

To manually test:

1. go to "Test Queue"
2. Click "Add a Test Plan to the Test Queue" and add an NVDA/Chrome test plan
3. Click "Add and run with bot"
4. Experiment with managing the bot, adding it through other means, reviewing its output, etc

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1206040249295671

[stalgiag]

I am super impressed by the approach and subsequent implementation of the automation bot.

In my review of the code, I could not find anything glaring to comment on here. However, when testing functionality I did find some things to bring to your attention:

The first thing is that the bot did not complete it's testing on 3 out of the 5 test plans for which it is able to run tests. It completed all tests for the Toggle Button V22.05.18 test plan and the Checkbox Example (Mixed-State) V23.08.23 test plan. But it did not compete the tests for Select Only Combobox Example V22.03.17: Alert Example V23.12.06: Modal Dialog Example V22.05.26:

The UI listing with the colored bullets disappears if I click the "Manage NVDA Bot" button then press "Stop Running". I think that that UI should remain for bot run tests.

That's it! Exceptional work!

Thanks for checking it out @Paul-Clue and thanks for the screenshots!

I am able to run all of those tests without issue in both local and prod environments. Is it possible that something could've happened to your app server while those tests were running? For example, a save while a simulated job is running could wipe out the timeouts running the simulated workflow. If you are pretty sure that your server isn't restarting and nothing is unexpected in your system, can you let me know if there is any info in any of the consoles (terminal running the app server and browser.console)?

[Paul-Clue]

Hey, @stalgiag, I rebuilt the database and pulled all the new changes. The bot is working fine now (completing all the tests for the tests plans that are run) and much faster too!

[alflennik]

Shall we mark this ready for review @stalgiag? Seems to be pretty close.