test_runner: support running tests in process

This commit introduces a new --experimental-test-isolation flag
that, when set to 'none', causes the test runner to execute all
tests in the same process. By default, this is the main test
runner process, but if watch mode is enabled, it spawns a separate
process that runs all of the tests.

The default value of the new flag is 'process', which uses the
existing behavior of running each test file in its own child
process.

It is worth noting that when the isolation mode is 'none', globals
and all other top level logic (such as top level before() and after()
hooks) is shared among all files.

Co-authored-by: Moshe Atlow <[email protected]>

doc/api/cli.md

@@ -1065,6 +1065,20 @@ generated as part of the test runner output. If no tests are run, a coverage
 report is not generated. See the documentation on
 [collecting code coverage from tests][] for more details.
 
+### `--experimental-test-isolation=mode`
+
+<!-- YAML
+added: REPLACEME
+-->
+
+> Stability: 1.0 - Early development
+
+Configures the type of test isolation used in the test runner. When `mode` is
+`'process'`, each test file is run in a separate child process. When `mode` is
+`'none'`, all test files run in the same process as the test runner. The default
+isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
+present.
+
 ### `--experimental-test-module-mocks`
 
 <!-- YAML
@@ -2170,7 +2184,9 @@ added:
 -->
 
 The maximum number of test files that the test runner CLI will execute
-concurrently. The default value is `os.availableParallelism() - 1`.
+concurrently. If `--experimental-test-isolation` is set to `'none'`, this flag
+is ignored and concurrency is one. Otherwise, concurrency defaults to
+`os.availableParallelism() - 1`.
 
 ### `--test-coverage-exclude`
 
@@ -2335,7 +2351,7 @@ added: v22.3.0
 
 > Stability: 1.0 - Early development
 
-Regenerates the snapshot file used by the test runner for [snapshot testing][].
+Regenerates the snapshot files used by the test runner for [snapshot testing][].
 Node.js must be started with the `--experimental-test-snapshots` flag in order
 to use this functionality.
 

doc/api/test.md

@@ -933,7 +933,7 @@ the [`--experimental-test-snapshots`][] command-line flag.
 Snapshot files are generated by starting Node.js with the
 [`--test-update-snapshots`][] command-line flag. A separate snapshot file is
 generated for each test file. By default, the snapshot file has the same name
-as `process.argv[1]` with a `.snapshot` file extension. This behavior can be
+as the test file with a `.snapshot` file extension. This behavior can be
 configured using the `snapshot.setResolveSnapshotPath()` function. Each
 snapshot assertion corresponds to an export in the snapshot file.
 
@@ -1239,6 +1239,9 @@ added:
   - v18.9.0
   - v16.19.0
 changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/53927
+    description: Added the `isolation` option.
   - version: REPLACEME
     pr-url: https://github.com/nodejs/node/pull/53866
     description: Added the `globPatterns` option.
@@ -1274,8 +1277,13 @@ changes:
   * `inspectPort` {number|Function} Sets inspector port of test child process.
     This can be a number, or a function that takes no arguments and returns a
     number. If a nullish value is provided, each process gets its own port,
-    incremented from the primary's `process.debugPort`.
-    **Default:** `undefined`.
+    incremented from the primary's `process.debugPort`. This option is ignored
+    if the `isolation` option is set to `'none'` as no child processes are
+    spawned. **Default:** `undefined`.
+  * `isolation` {string} Configures the type of test isolation. If set to
+    `'process'`, each test file is run in a separate child process. If set to
+    `'none'`, all test files run in the current process. The default isolation
+    mode is `'process'`.
   * `only`: {boolean} If truthy, the test context will only run tests that
     have the `only` option set
   * `setup` {Function} A function that accepts the `TestsStream` instance
@@ -1727,9 +1735,9 @@ added: v22.3.0
 
 * `fn` {Function} A function used to compute the location of the snapshot file.
   The function receives the path of the test file as its only argument. If the
-  `process.argv[1]` is not associated with a file (for example in the REPL),
-  the input is undefined. `fn()` must return a string specifying the location of
-  the snapshot file.
+  test is not associated with a file (for example in the REPL), the input is
+  undefined. `fn()` must return a string specifying the location of the snapshot
+  snapshot file.
 
 This function is used to customize the location of the snapshot file used for
 snapshot testing. By default, the snapshot filename is the same as the entry

doc/node.1

@@ -185,6 +185,9 @@ Enable the experimental node:sqlite module.
 .It Fl -experimental-test-coverage
 Enable code coverage in the test runner.
 .
+.It Fl -experimental-test-isolation Ns = Ns Ar mode
+Configures the type of test isolation used in the test runner.
+.
 .It Fl -experimental-test-module-mocks
 Enable module mocking in the test runner.
 .

lib/internal/main/test_runner.js

@@ -23,6 +23,7 @@ prepareMainThreadExecution(false);
 markBootstrapComplete();
 
 const {
+  isolation,
   perFileTimeout,
   runnerConcurrency,
   shard,
@@ -42,11 +43,12 @@ if (isUsingInspector()) {
 const options = {
   concurrency,
   inspectPort,
-  watch: watchMode,
+  isolation,
   setup: setupTestReporters,
-  timeout: perFileTimeout,
   shard,
   globPatterns: ArrayPrototypeSlice(process.argv, 1),
+  timeout: perFileTimeout,
+  watch: watchMode,
 };
 debug('test runner configuration:', options);
 run(options).on('test:fail', (data) => {

lib/internal/test_runner/harness.js

@@ -308,4 +308,5 @@ module.exports = {
   after: hook('after'),
   beforeEach: hook('beforeEach'),
   afterEach: hook('afterEach'),
+  startSubtestAfterBootstrap,
 };