[BUG] Incorrect display of Configuration Error message for the html reporter

[edumserrano]

System info

• Playwright Version: [v1.40.1]
• Operating System: [Windows 10]
• Browser: [All]
• Other info:

Source code

Don't think it's needed for this but I'll gladly work on providing one if you think it is.

Steps

1. Setup Playwright and add the html reporter as one of the reporters.
2. Set Playwright's outputDir to test-results
3. Set the list reporter outputFolder to test-results-html-report
4. Run Playwright tests and you will see a Configuration Error message like:

Configuration Error: HTML reporter output folder clashes with the tests output folder:

html reporter folder: C:\dev\repos\github-personal\playwright-adventures\demos\code-coverage-with-istanbul-via-webpack-babel-plugin\test-results-html-report

test results folder: C:\dev\repos\github-personal\playwright-adventures\demos\code-coverage-with-istanbul-via-webpack-babel-plugin\test-results

Expected

No Configuration Error message should be displayed.

Actual

A Configuration Error message is displayed.

Notes

• I believe that, as per the explanation that @mxschmitt gave in this comment, since the test-results-html-report directory is not a child of test-results directory then there shouldn't be any Configuration Error message.
• I believe the problem is in this if condition:

if (outputFolder.startsWith(project.outputDir) || project.outputDir.startsWith(outputFolder)) {
...
}

Any folder that startsWith the same name as the tests output directory will be flagged, even if it's not a child folder.

[matthiasg]

As predicted here #14964 (comment) I also just stumbled over this issue.

The types are also a little confusing when one did not look through the entire source code. project.outputDir is not immediately a thing under the 'projects' so seeing what is matched against takes a little longer