Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(PPDSC-2313): script to run applitools locally #321

Merged
merged 5 commits into from
Aug 2, 2022

Conversation

mstuartf
Copy link
Contributor

@mstuartf mstuartf commented Aug 1, 2022

PPDSC-2313

What

  1. Background - why this is needed
  • Lack of information in the repo on how to run Applitools tests locally
  • Pipeline debugging is slow and expensive
  1. What did you do
  • Shell script to set env variables (inc asking for api key)
  • Add scripts to package json to run comp and doc tests
  • Considered dockerising the script but felt like overkill
  1. What does the reviewers should expect
  • To be able to run applitools tests locally using a single command

The script will guide the user to run fewer tests by using the include config option, then ask for an api key:
Screenshot 2022-08-01 at 16 29 35

Then when the suite has finished the results and a link to the batch in the Applitools UI are shown:
Screenshot 2022-08-01 at 16 33 12

I have done:

  • Written unit tests against changes
  • Written functional tests against the component and/or NewsKit site
  • Updated relevant documentation

I have tested manually:

  • The feature's functionality is working as expected on Chrome, Firefox, Safari and Edge
  • The screen reader reads and flows through the elements as expected.
  • There are no new errors in the browser console coming from this PR.
  • When visual test is not added, it renders correctly on different browsers and mobile viewports (Safari, Firefox, small mobile viewport, tablet)
  • The Playground feature is working as expected

Before:

After:

Who should review this PR:

How to test:

- Shell script to set env variables (inc asking for api key)
- Add scripts to package json to run comp and doc tests
@mstuartf mstuartf added the WIP Work in progress label Aug 1, 2022
@github-actions github-actions bot added the feature This change contains a new feature label Aug 1, 2022
@mstuartf mstuartf added ready for review Please assist in getting this reviewed and removed WIP Work in progress labels Aug 1, 2022
@Vanals
Copy link
Contributor

Vanals commented Aug 2, 2022

How this script will be fitting the development flow? In which scenario can be run or should be run?
As you mentioned, I guess, we can run it when we already know it is gonna fail and we don't want to wait the whole pipeline to run? Ta!

@mstuartf
Copy link
Contributor Author

mstuartf commented Aug 2, 2022

How this script will be fitting the development flow? In which scenario can be run or should be run? As you mentioned, I guess, we can run it when we already know it is gonna fail and we don't want to wait the whole pipeline to run? Ta!

Yep. The idea is that if you know you will have to resolve some tests because you have made visual changes, you don't have to run the whole pipeline (and all >1,600 tests) before being able to resolve them. You can run them locally, resolve any changes, then when your pipeline runs they should all pass.

Alternatively, if you want to make sure that you have not made any visual changes that are going to cause the pipeline to fail, this script is a quick way to check.

The general idea is that this will make resolving and debugging Applitools quicker for devs and will reduce our usage.

@mstuartf mstuartf merged commit 1493a2b into main Aug 2, 2022
@mstuartf mstuartf deleted the feat/PPDSC-2313-run-applitools-locally branch August 2, 2022 13:07
Xin00163 pushed a commit that referenced this pull request Oct 17, 2022
* feat(PPDSC-2313): script to run applitools locally

- Shell script to set env variables (inc asking for api key)
- Add scripts to package json to run comp and doc tests

* feat(PPDSC-2313): update readme and encourage fewer tests

- Add local applitools scripts to scripts.md
- Guide user to only run a subset of tests

* feat(PPDSC-2313): warn user about unclosed batches

* feat(PPDSC-2313): fix break error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature This change contains a new feature ready for review Please assist in getting this reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants