Skip to content

Commit

Permalink
cherry-pick(#34537): feat: per-assertion snapshot path template in co…
Browse files Browse the repository at this point in the history
…nfig (#34551)
  • Loading branch information
dgozman authored Jan 30, 2025
1 parent 67313fa commit fbad9f7
Show file tree
Hide file tree
Showing 13 changed files with 239 additions and 110 deletions.
22 changes: 4 additions & 18 deletions docs/src/api/class-locatorassertions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2263,35 +2263,21 @@ assertThat(page.locator("body")).matchesAriaSnapshot("""

Asserts that the target element matches the given [accessibility snapshot](../aria-snapshots.md).

Snapshot is stored in a separate `.yml` file in a location configured by `expect.toMatchAriaSnapshot.pathTemplate` and/or `snapshotPathTemplate` properties in the configuration file.

**Usage**

```js
await expect(page.locator('body')).toMatchAriaSnapshot();
await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'snapshot' });
```

```python async
await expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
```

```python sync
expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
```

```csharp
await Expect(page.Locator("body")).ToMatchAriaSnapshotAsync(new { Path = "/path/to/snapshot.yml" });
```

```java
assertThat(page.locator("body")).matchesAriaSnapshot(new LocatorAssertions.MatchesAriaSnapshotOptions().setPath("/path/to/snapshot.yml"));
await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'body.yml' });
```

### option: LocatorAssertions.toMatchAriaSnapshot#2.name
* since: v1.50
* langs: js
- `name` <[string]>

Name of the snapshot to store in the snapshot (screenshot) folder corresponding to this test.
Name of the snapshot to store in the snapshot folder corresponding to this test.
Generates sequential names if not specified.

### option: LocatorAssertions.toMatchAriaSnapshot#2.timeout = %%-js-assertions-timeout-%%
Expand Down
28 changes: 21 additions & 7 deletions docs/src/api/params.md
Original file line number Diff line number Diff line change
Expand Up @@ -1758,7 +1758,9 @@ await Expect(Page.GetByTitle("Issues count")).toHaveText("25 issues");
- `type` ?<[string]>
* langs: js

This option configures a template controlling location of snapshots generated by [`method: PageAssertions.toHaveScreenshot#1`] and [`method: SnapshotAssertions.toMatchSnapshot#1`].
This option configures a template controlling location of snapshots generated by [`method: PageAssertions.toHaveScreenshot#1`], [`method: LocatorAssertions.toMatchAriaSnapshot#2`] and [`method: SnapshotAssertions.toMatchSnapshot#1`].

You can configure templates for each assertion separately in [`property: TestConfig.expect`].

**Usage**

Expand All @@ -1767,7 +1769,19 @@ import { defineConfig } from '@playwright/test';

export default defineConfig({
testDir: './tests',

// Single template for all assertions
snapshotPathTemplate: '{testDir}/__screenshots__/{testFilePath}/{arg}{ext}',

// Assertion-specific templates
expect: {
toHaveScreenshot: {
pathTemplate: '{testDir}/__screenshots__{/projectName}/{testFilePath}/{arg}{ext}',
},
toMatchAriaSnapshot: {
pathTemplate: '{testDir}/__snapshots__/{testFilePath}/{arg}{ext}',
},
},
});
```

Expand Down Expand Up @@ -1798,22 +1812,22 @@ test.describe('suite', () => {

The list of supported tokens:

* `{arg}` - Relative snapshot path **without extension**. These come from the arguments passed to the `toHaveScreenshot()` and `toMatchSnapshot()` calls; if called without arguments, this will be an auto-generated snapshot name.
* `{arg}` - Relative snapshot path **without extension**. This comes from the arguments passed to `toHaveScreenshot()`, `toMatchAriaSnapshot()` or `toMatchSnapshot()`; if called without arguments, this will be an auto-generated snapshot name.
* Value: `foo/bar/baz`
* `{ext}` - snapshot extension (with dots)
* `{ext}` - Snapshot extension (with the leading dot).
* Value: `.png`
* `{platform}` - The value of `process.platform`.
* `{projectName}` - Project's file-system-sanitized name, if any.
* Value: `''` (empty string).
* `{snapshotDir}` - Project's [`property: TestConfig.snapshotDir`].
* `{snapshotDir}` - Project's [`property: TestProject.snapshotDir`].
* Value: `/home/playwright/tests` (since `snapshotDir` is not provided in config, it defaults to `testDir`)
* `{testDir}` - Project's [`property: TestConfig.testDir`].
* Value: `/home/playwright/tests` (absolute path is since `testDir` is resolved relative to directory with config)
* `{testDir}` - Project's [`property: TestProject.testDir`].
* Value: `/home/playwright/tests` (absolute path since `testDir` is resolved relative to directory with config)
* `{testFileDir}` - Directories in relative path from `testDir` to **test file**.
* Value: `page`
* `{testFileName}` - Test file name with extension.
* Value: `page-click.spec.ts`
* `{testFilePath}` - Relative path from `testDir` to **test file**
* `{testFilePath}` - Relative path from `testDir` to **test file**.
* Value: `page/page-click.spec.ts`
* `{testName}` - File-system-sanitized test title, including parent describes but excluding file name.
* Value: `suite-test-should-work`
Expand Down
3 changes: 2 additions & 1 deletion docs/src/release-notes-js.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import LiteYouTube from '@site/src/components/LiteYouTube';
```

* New method [`method: Test.step.skip`] to disable execution of a test step.

```js
test('some test', async ({ page }) => {
await test.step('before running step', async () => {
Expand Down Expand Up @@ -49,6 +49,7 @@ import LiteYouTube from '@site/src/components/LiteYouTube';

* Option [`property: TestConfig.webServer`] added a `gracefulShutdown` field for specifying a process kill signal other than the default `SIGKILL`.
* Exposed [`property: TestStep.attachments`] from the reporter API to allow retrieval of all attachments created by that step.
* New option `pathTemplate` for `toHaveScreenshot` and `toMatchAriaSnapshot` assertions in the [`property: TestConfig.expect`] configuration.

### UI updates

Expand Down
3 changes: 3 additions & 0 deletions docs/src/test-api/class-testconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ export default defineConfig({
- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.
- `stylePath` ?<[string]|[Array]<[string]>> See [`option: Page.screenshot.style`] in [`method: Page.screenshot`].
- `threshold` ?<[float]> An acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
- `pathTemplate` ?<[string]> A template controlling location of the screenshots. See [`property: TestConfig.snapshotPathTemplate`] for details.
- `toMatchAriaSnapshot` ?<[Object]> Configuration for the [`method: LocatorAssertions.toMatchAriaSnapshot#2`] method.
- `pathTemplate` ?<[string]> A template controlling location of the aria snapshots. See [`property: TestConfig.snapshotPathTemplate`] for details.
- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.
- `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default.
- `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.
Expand Down
3 changes: 3 additions & 0 deletions docs/src/test-api/class-testproject.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,9 @@ export default defineConfig({
- `caret` ?<[ScreenshotCaret]<"hide"|"initial">> See [`option: Page.screenshot.caret`] in [`method: Page.screenshot`]. Defaults to `"hide"`.
- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.
- `stylePath` ?<[string]|[Array]<[string]>> See [`option: Page.screenshot.style`] in [`method: Page.screenshot`].
- `pathTemplate` ?<[string]> A template controlling location of the screenshots. See [`property: TestProject.snapshotPathTemplate`] for details.
- `toMatchAriaSnapshot` ?<[Object]> Configuration for the [`method: LocatorAssertions.toMatchAriaSnapshot#2`] method.
- `pathTemplate` ?<[string]> A template controlling location of the aria snapshots. See [`property: TestProject.snapshotPathTemplate`] for details.
- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.
- `threshold` ?<[float]> an acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
- `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default.
Expand Down
2 changes: 1 addition & 1 deletion docs/src/test-snapshots-js.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ The snapshot name `example-test-1-chromium-darwin.png` consists of a few parts:

- `chromium-darwin` - the browser name and the platform. Screenshots differ between browsers and platforms due to different rendering, fonts and more, so you will need different snapshots for them. If you use multiple projects in your [configuration file](./test-configuration.md), project name will be used instead of `chromium`.

The snapshot name and path can be configured with [`snapshotPathTemplate`](./api/class-testproject#test-project-snapshot-path-template) in the playwright config.
The snapshot name and path can be configured with [`property: TestConfig.snapshotPathTemplate`] in the playwright config.

## Updating screenshots

Expand Down
5 changes: 2 additions & 3 deletions packages/playwright/src/common/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ export class FullProjectInternal {
readonly fullyParallel: boolean;
readonly expect: Project['expect'];
readonly respectGitIgnore: boolean;
readonly snapshotPathTemplate: string;
readonly snapshotPathTemplate: string | undefined;
readonly ignoreSnapshots: boolean;
id = '';
deps: FullProjectInternal[] = [];
Expand All @@ -173,8 +173,7 @@ export class FullProjectInternal {
constructor(configDir: string, config: Config, fullConfig: FullConfigInternal, projectConfig: Project, configCLIOverrides: ConfigCLIOverrides, packageJsonDir: string) {
this.fullConfig = fullConfig;
const testDir = takeFirst(pathResolve(configDir, projectConfig.testDir), pathResolve(configDir, config.testDir), fullConfig.configDir);
const defaultSnapshotPathTemplate = '{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{-snapshotSuffix}{ext}';
this.snapshotPathTemplate = takeFirst(projectConfig.snapshotPathTemplate, config.snapshotPathTemplate, defaultSnapshotPathTemplate);
this.snapshotPathTemplate = takeFirst(projectConfig.snapshotPathTemplate, config.snapshotPathTemplate);

this.project = {
grep: takeFirst(projectConfig.grep, config.grep, defaultGrep),
Expand Down
6 changes: 4 additions & 2 deletions packages/playwright/src/matchers/toMatchAriaSnapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ export async function toMatchAriaSnapshot(
return { pass: !this.isNot, message: () => '', name: 'toMatchAriaSnapshot', expected: '' };

const updateSnapshots = testInfo.config.updateSnapshots;
const pathTemplate = testInfo._projectInternal.expect?.toMatchAriaSnapshot?.pathTemplate;
const defaultTemplate = '{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{ext}';

const matcherOptions = {
isNot: this.isNot,
Expand All @@ -63,15 +65,15 @@ export async function toMatchAriaSnapshot(
timeout = options.timeout ?? this.timeout;
} else {
if (expectedParam?.name) {
expectedPath = testInfo.snapshotPath(sanitizeFilePathBeforeExtension(expectedParam.name));
expectedPath = testInfo._resolveSnapshotPath(pathTemplate, defaultTemplate, [sanitizeFilePathBeforeExtension(expectedParam.name)]);
} else {
let snapshotNames = (testInfo as any)[snapshotNamesSymbol] as SnapshotNames;
if (!snapshotNames) {
snapshotNames = { anonymousSnapshotIndex: 0 };
(testInfo as any)[snapshotNamesSymbol] = snapshotNames;
}
const fullTitleWithoutSpec = [...testInfo.titlePath.slice(1), ++snapshotNames.anonymousSnapshotIndex].join(' ');
expectedPath = testInfo.snapshotPath(sanitizeForFilePath(trimLongString(fullTitleWithoutSpec)) + '.yml');
expectedPath = testInfo._resolveSnapshotPath(pathTemplate, defaultTemplate, [sanitizeForFilePath(trimLongString(fullTitleWithoutSpec)) + '.yml']);
}
expected = await fs.promises.readFile(expectedPath, 'utf8').catch(() => '');
timeout = expectedParam?.timeout ?? this.timeout;
Expand Down
3 changes: 2 additions & 1 deletion packages/playwright/src/matchers/toMatchSnapshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,8 @@ class SnapshotHelper {
outputBasePath = testInfo._getOutputPath(sanitizedName);
this.attachmentBaseName = sanitizedName;
}
this.expectedPath = testInfo.snapshotPath(...expectedPathSegments);
const defaultTemplate = '{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{-snapshotSuffix}{ext}';
this.expectedPath = testInfo._resolveSnapshotPath(configOptions.pathTemplate, defaultTemplate, expectedPathSegments);
this.legacyExpectedPath = addSuffixToFilePath(outputBasePath, '-expected');
this.previousPath = addSuffixToFilePath(outputBasePath, '-previous');
this.actualPath = addSuffixToFilePath(outputBasePath, '-actual');
Expand Down
10 changes: 8 additions & 2 deletions packages/playwright/src/worker/testInfo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -454,14 +454,15 @@ export class TestInfoImpl implements TestInfo {
return sanitizeForFilePath(trimLongString(fullTitleWithoutSpec));
}

snapshotPath(...pathSegments: string[]) {
_resolveSnapshotPath(template: string | undefined, defaultTemplate: string, pathSegments: string[]) {
const subPath = path.join(...pathSegments);
const parsedSubPath = path.parse(subPath);
const relativeTestFilePath = path.relative(this.project.testDir, this._requireFile);
const parsedRelativeTestFilePath = path.parse(relativeTestFilePath);
const projectNamePathSegment = sanitizeForFilePath(this.project.name);

const snapshotPath = (this._projectInternal.snapshotPathTemplate || '')
const actualTemplate = (template || this._projectInternal.snapshotPathTemplate || defaultTemplate);
const snapshotPath = actualTemplate
.replace(/\{(.)?testDir\}/g, '$1' + this.project.testDir)
.replace(/\{(.)?snapshotDir\}/g, '$1' + this.project.snapshotDir)
.replace(/\{(.)?snapshotSuffix\}/g, this.snapshotSuffix ? '$1' + this.snapshotSuffix : '')
Expand All @@ -477,6 +478,11 @@ export class TestInfoImpl implements TestInfo {
return path.normalize(path.resolve(this._configInternal.configDir, snapshotPath));
}

snapshotPath(...pathSegments: string[]) {
const legacyTemplate = '{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{-snapshotSuffix}{ext}';
return this._resolveSnapshotPath(undefined, legacyTemplate, pathSegments);
}

skip(...args: [arg?: any, description?: string]) {
this._modifier('skip', args);
}
Expand Down
Loading

0 comments on commit fbad9f7

Please sign in to comment.