Axe API integration with Jest and Puppeteer

docs/contributors/testing-overview.md

@@ -19,7 +19,7 @@ When writing tests consider the following:
 
 ## JavaScript Testing
 
-Tests for JavaScript use [Jest](http://facebook.github.io/jest/) as the test runner and its API for [globals](https://facebook.github.io/jest/docs/en/api.html) (`describe`, `test`, `beforeEach` and so on) [assertions](http://facebook.github.io/jest/docs/en/expect.html), [mocks](http://facebook.github.io/jest/docs/en/mock-functions.html), [spies](http://facebook.github.io/jest/docs/en/jest-object.html#jestspyonobject-methodname) and [mock functions](https://facebook.github.io/jest/docs/en/mock-function-api.html). If needed, you can also use [Enzyme](https://github.com/airbnb/enzyme) for React component testing.
+Tests for JavaScript use [Jest](https://jestjs.io/) as the test runner and its API for [globals](https://jestjs.io/docs/en/api.html) (`describe`, `test`, `beforeEach` and so on) [assertions](https://jestjs.io/docs/en/expect.html), [mocks](https://jestjs.io/docs/en/mock-functions.html), [spies](https://jestjs.io/docs/en/jest-object.html#jestspyonobject-methodname) and [mock functions](https://jestjs.io/docs/en/mock-function-api.html). If needed, you can also use [Enzyme](https://github.com/airbnb/enzyme) for React component testing.
 
 Assuming you've followed the [instructions](https://github.com/WordPress/gutenberg/blob/master/CONTRIBUTING.md) to install Node and project dependencies, tests can be run from the command-line with NPM:
 
@@ -88,9 +88,9 @@ describe( 'CheckboxWithLabel', () => {
 
 ### Setup and Teardown methods
 
-The Jest API includes some nifty [setup and teardown methods](https://facebook.github.io/jest/docs/en/setup-teardown.html) that allow you to perform tasks *before* and *after* each or all of your tests, or tests within a specific `describe` block.
+The Jest API includes some nifty [setup and teardown methods](https://jestjs.io/docs/en/setup-teardown.html) that allow you to perform tasks *before* and *after* each or all of your tests, or tests within a specific `describe` block.
 
-These methods can handle asynchronous code to allow setup that you normally cannot do inline. As with [individual test cases](https://facebook.github.io/jest/docs/en/asynchronous.html#promises), you can return a Promise and Jest will wait for it to resolve:
+These methods can handle asynchronous code to allow setup that you normally cannot do inline. As with [individual test cases](https://jestjs.io/docs/en/asynchronous.html#promises), you can return a Promise and Jest will wait for it to resolve:
 
 ```javascript
 // one-time setup for *all* tests
@@ -188,7 +188,7 @@ describe( 'The bilbo module', () => {
 
 ### Testing globals
 
-We can use [Jest spies](http://facebook.github.io/jest/docs/en/jest-object.html#jestspyonobject-methodname) to test code that calls global methods.
+We can use [Jest spies](https://jestjs.io/docs/en/jest-object.html#jestspyonobject-methodname) to test code that calls global methods.
 
 ```javascript
 import { myModuleFunctionThatOpensANewWindow } from '../my-module';
@@ -374,5 +374,5 @@ Code style in PHP is enforced using [PHP_CodeSniffer](https://github.com/squizla
 
 To run unit tests only, without the linter, use `npm run test-unit-php` instead.
 
-[snapshot testing]: https://facebook.github.io/jest/docs/en/snapshot-testing.html
-[update snapshots]: https://facebook.github.io/jest/docs/en/snapshot-testing.html#updating-snapshots
+[snapshot testing]: https://jestjs.io/docs/en/snapshot-testing.html
+[update snapshots]: https://jestjs.io/docs/en/snapshot-testing.html#updating-snapshots

docs/manifest.json

@@ -677,6 +677,12 @@
 		"markdown_source": "https://raw.githubusercontent.com/WordPress/gutenberg/master/packages/jest-preset-default/README.md",
 		"parent": "packages"
 	},
+	{
+		"title": "@wordpress/jest-puppeteer-axe",
+		"slug": "packages-jest-puppeteer-axe",
+		"markdown_source": "https://raw.githubusercontent.com/WordPress/gutenberg/master/packages/jest-puppeteer-axe/README.md",
+		"parent": "packages"
+	},
 	{
 		"title": "@wordpress/keycodes",
 		"slug": "packages-keycodes",

package.json

@@ -65,6 +65,7 @@
 		"@wordpress/eslint-plugin": "file:packages/eslint-plugin",
 		"@wordpress/jest-console": "file:packages/jest-console",
 		"@wordpress/jest-preset-default": "file:packages/jest-preset-default",
+		"@wordpress/jest-puppeteer-axe": "file:packages/jest-puppeteer-axe",
 		"@wordpress/library-export-default-webpack-plugin": "file:packages/library-export-default-webpack-plugin",
 		"@wordpress/npm-package-json-lint-config": "file:packages/npm-package-json-lint-config",
 		"@wordpress/postcss-themes": "file:packages/postcss-themes",

packages/e2e-test-utils/package.json

@@ -30,8 +30,8 @@
 		"node-fetch": "^1.7.3"
 	},
 	"peerDependencies": {
-		"jest": ">=23.0.0",
-		"puppeteer": ">=1.6.0"
+		"jest": ">=23",
+		"puppeteer": ">=1.6"
 	},
 	"publishConfig": {
 		"access": "public"

packages/e2e-tests/package.json

@@ -25,8 +25,8 @@
 		"lodash": "^4.17.10"
 	},
 	"peerDependencies": {
-		"jest": ">=23.0.0",
-		"puppeteer": ">=1.6.0"
+		"jest": ">=23",
+		"puppeteer": ">=1.6"
 	},
 	"publishConfig": {
 		"access": "public"

packages/jest-console/README.md

@@ -1,6 +1,6 @@
 # Jest Console
 
-Custom [Jest](http://facebook.github.io/jest/) matchers for the [Console](https://developer.mozilla.org/en-US/docs/Web/API/Console)
+Custom [Jest](https://jestjs.io/) matchers for the [Console](https://developer.mozilla.org/en-US/docs/Web/API/Console)
 object to test JavaScript code in WordPress.
 
 This package converts `console.error`, `console.info`, `console.log` and `console.warn` functions into mocks and tracks their calls.

packages/jest-preset-default/README.md

@@ -1,6 +1,6 @@
 # Jest Preset Default
 
-Default [Jest](https://facebook.github.io/jest/) preset for WordPress development.
+Default [Jest](https://jestjs.io/) preset for WordPress development.
 
 ## Installation
 
@@ -29,7 +29,7 @@ npm install @wordpress/jest-preset-default --save-dev
 * `setupFiles` - runs code before each test which sets up global variables required in the testing environment.
 * `setupTestFrameworkScriptFile` - runs code which adds improved support for `Console` object and `React` components to the testing framework before each test.
 * `testMatch`- includes `/test/` subfolder in the glob patterns Jest uses to detect test files. It detects only test files containing `.js` extension.
-* `timers` - use of [fake timers](https://facebook.github.io/jest/docs/en/timer-mocks.html) for functions such as `setTimeout` is enabled.
+* `timers` - use of [fake timers](https://jestjs.io/docs/en/timer-mocks.html) for functions such as `setTimeout` is enabled.
 * `transform` - adds support for [PEG.js]( https://github.com/pegjs/pegjs#javascript-api) transformed necessary for WordPress blocks. It also keeps the default [babel-jest](https://github.com/facebook/jest/tree/master/packages/babel-jest) transformer.
 * `verbose` - each individual test won't be reported during the run.
 

packages/jest-puppeteer-axe/.npmrc

@@ -0,0 +1 @@
+package-lock=false

packages/jest-puppeteer-axe/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 1.0.0 (Unreleased)
+
+- Initial release.

packages/jest-puppeteer-axe/README.md

@@ -0,0 +1,61 @@
+# Jest Puppeteer Axe
+
+[Axe](https://www.deque.com/axe/) (the Accessibility Engine) API integration with [Jest](https://jestjs.io/) and [Puppeteer](https://pptr.dev/).
+
+Defines Jest async matcher to check whether a given Puppeteer's page instance passes [Axe](https://www.deque.com/axe/) accessibility tests.
+
+## Installation
+
+Install the module
+
+```bash
+npm install @wordpress/jest-puppeteer-axe --save-dev
+```
+
+### Setup
+
+The simplest setup is to use Jest's `setupTestFrameworkScriptFile` config option:
+
+```js
+"jest": {
+  "setupTestFrameworkScriptFile": "./node_modules/@wordpress/jest-puppeteer-axe/build/index.js"
+},
+```
+
+If your project already has a script file which sets up the test framework, you will need the following import statement:
+
+```js
+import '@wordpress/jest-puppeteer-axe';
+```
+
+## Usage
+
+In your Jest test suite add the following code to the test's body:
+
+```js
+test( 'checks the test page with Axe', async () => {
+	// First, run some code which loads the content of the page.
+	loadTestPage();
+
+	await expect( page ).toPassAxeTests();
+} );
+```
+
+It is also possible to pass optional Axe API options to perform customized check:
+- `include` - CSS selector to to add the list of elements to include in analysis.
+- `exclude` - CSS selector to to add the list of elements to exclude from analysis.
+
+```js
+test( 'checks the test component with Axe excluding some button', async () => {
+
+	// First, run some code which loads the content of the page.
+	loadPageWithTestComponent();
+
+	await expect( page ).toPassAxeTests( {
+		include: '.test-component',
+		exclude: '.some-button',
+	} );
+} );
+```
+
+<br/><br/><p align="center"><img src="https://s.w.org/style/images/codeispoetry.png?1" alt="Code is Poetry." /></p>

packages/jest-puppeteer-axe/package.json

@@ -0,0 +1,38 @@
+{
+	"name": "@wordpress/jest-puppeteer-axe",
+	"version": "1.0.0-alpha.0",
+	"description": "Axe API integration with Jest and Puppeteer.",
+	"author": "The WordPress Contributors",
+	"license": "GPL-2.0-or-later",
+	"keywords": [
+		"wordpress",
+		"jest",
+		"puppeteer",
+		"axe",
+		"accessibility"
+	],
+	"homepage": "https://github.com/WordPress/gutenberg/tree/master/packages/jest-puppeteer-axe/README.md",
+	"repository": {
+		"type": "git",
+		"url": "https://github.com/WordPress/gutenberg.git"
+	},
+	"bugs": {
+		"url": "https://github.com/WordPress/gutenberg/issues"
+	},
+	"files": [
+		"build",
+		"build-module"
+	],
+	"main": "build/index.js",
+	"module": "build-module/index.js",
+	"dependencies": {
+		"axe-puppeteer": "^0.1.0"
+	},
+	"peerDependencies": {
+		"jest": ">=23",
+		"puppeteer": ">=1.6"
+	},
+	"publishConfig": {
+		"access": "public"
+	}
+}

packages/jest-puppeteer-axe/src/index.js

@@ -0,0 +1,103 @@
+/**
+ * External dependencies
+ */
+import AxePuppeteer from 'axe-puppeteer';
+
+/**
+ * Formats the list of violations object returned by Axe analysis.
+ *
+ * @param {Object} violations The object with the errors found by Axe.
+ *
+ * @return {string} The user friendly message to display when the matcher fails.
+ */
+function formatViolations( violations ) {
+	return violations.map( ( { help, id, nodes } ) => {
+		let output = `Rule: ${ id } (${ help })\n` +
+			'Affected Nodes:\n';
+
+		nodes.forEach( ( node ) => {
+			if ( node.any.length ) {
+				output += `  ${ node.target }\n`;
+				output += '    Fix ANY of the following:\n';
+				node.any.forEach( ( item ) => {
+					output += `    - ${ item.message }\n`;
+				} );
+			}
+
+			if ( node.all.length ) {
+				output += `  ${ node.target }\n`;
+				output += '    Fix ALL of the following:\n';
+				node.all.forEach( ( item ) => {
+					output += `      - ${ item.message }.\n`;
+				} );
+			}
+
+			if ( node.none.length ) {
+				output += `  ${ node.target }\n`;
+				output += '    Fix ALL of the following:\n';
+				node.none.forEach( ( item ) => {
+					output += `      - ${ item.message }.\n`;
+				} );
+			}
+		} );
+		return output;
+	} ).join( '\n' );
+}
+
+/**
+ * Defines async matcher to check whether a given Puppeteer's page instance passes Axe accessibility tests.
+ *
+ * @see https://www.deque.com/axe/
+ * It is possible to pass optional Axe API options to perform customized check.
+ *
+ * @see https://github.com/dequelabs/axe-puppeteer
+ *
+ * @param {Page}    page             Puppeteer's page instance.
+ * @param {?Object} params           Optional Axe API options.
+ * @param {?string} params.include   CSS selector to add to the list of elements
+ *                                   to include in analysis.
+ * @param {?string} params.exclude   CSS selector to add to the list of elements
+ *                                   to exclude from analysis.
+ *
+ * @return {Object} A matcher object with two keys `pass` and `message`.
+ */
+async function toPassAxeTests( page, { include, exclude } = {} ) {
+	const axe = new AxePuppeteer( page );
+
+	if ( include ) {
+		axe.include( include );
+	}
+
+	if ( exclude ) {
+		axe.exclude( exclude );
+	}
+
+	const { violations } = await axe.analyze();
+
+	const pass = violations.length === 0;
+	const message = pass ?
+		() => {
+			return this.utils.matcherHint( '.not.toPassAxeTests' ) +
+				'\n\n' +
+				'Expected page to contain accessibility check violations.\n' +
+				'No violations found.';
+		} :
+		() => {
+			return this.utils.matcherHint( '.toPassAxeTests' ) +
+				'\n\n' +
+				'Expected page to pass Axe accessibility tests.\n' +
+				'Violations found:\n' +
+				this.utils.RECEIVED_COLOR(
+					formatViolations( violations )
+				);
+		};
+
+	return {
+		message,
+		pass,
+	};
+}
+
+expect.extend( {
+	toPassAxeTests,
+} );

packages/npm-package-json-lint-config/package.json

@@ -19,7 +19,7 @@
 	},
 	"main": "index.js",
 	"peerDependencies": {
-		"npm-package-json-lint": ">= 3.3.1"
+		"npm-package-json-lint": ">=3.3.1"
 	},
 	"publishConfig": {
 		"access": "public"

packages/scripts/README.md

@@ -127,7 +127,7 @@ This is how you execute the script with presented setup:
 
 ### `test-e2e`
 
-Launches the End-To-End (E2E) test runner. It uses [Jest](https://facebook.github.io/jest/) behind the scenes and you are able to utilize all of its [CLI options](https://facebook.github.io/jest/docs/en/cli.html). You can also run `./node_modules/.bin/wp-scripts test:e2e --help` or `npm run test:e2e:help` (as presented below) to view all of the available options.
+Launches the End-To-End (E2E) test runner. It uses [Jest](https://jestjs.io/) behind the scenes and you are able to utilize all of its [CLI options](https://jestjs.io/docs/en/cli.html). You can also run `./node_modules/.bin/wp-scripts test:e2e --help` or `npm run test:e2e:help` (as presented below) to view all of the available options.
 
 Writing tests can be done using Puppeteer API:
 
@@ -157,7 +157,7 @@ We enforce that all tests run serially in the current process using [--runInBand
 
 _Alias_: `test-unit-jest` 
 
-Launches the unit test runner. It uses [Jest](https://facebook.github.io/jest/) behind the scenes and you are able to utilize all of its [CLI options](https://facebook.github.io/jest/docs/en/cli.html). You can also run `./node_modules/.bin/wp-scripts test-unit-js --help` or `npm run test:unit:help` (as presented below) to view all of the available options. By default, it uses the set of recommended options defined in [@wordpress/jest-preset-default](https://www.npmjs.com/package/@wordpress/jest-preset-default) npm package. You can override them with your own options as described in [Jest documentation](https://jestjs.io/docs/en/configuration).
+Launches the unit test runner. It uses [Jest](https://jestjs.io/) behind the scenes and you are able to utilize all of its [CLI options](https://jestjs.io/docs/en/cli.html). You can also run `./node_modules/.bin/wp-scripts test-unit-js --help` or `npm run test:unit:help` (as presented below) to view all of the available options. By default, it uses the set of recommended options defined in [@wordpress/jest-preset-default](https://www.npmjs.com/package/@wordpress/jest-preset-default) npm package. You can override them with your own options as described in [Jest documentation](https://jestjs.io/docs/en/configuration).
 
 _Example:_