Skip to content

Commit

Permalink
feat(betterer ✨): add reporters config option and --reporter flag (
Browse files Browse the repository at this point in the history
  • Loading branch information
phenomnomnominal authored Jun 28, 2020
1 parent e2e9e57 commit 784b2fa
Show file tree
Hide file tree
Showing 32 changed files with 495 additions and 157 deletions.
16 changes: 10 additions & 6 deletions .betterer.results
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,21 @@ exports[`no hack comments`] = {
exports[`new eslint rules`] = {
timestamp: 1593096621756,
value: `{
"packages/betterer/src/config/config.ts:4152254717": [
[34, 2, 72, "Unsafe return of an any[] typed value", "2233173152"]
"packages/betterer/src/config/config.ts:2688898251": [
[35, 2, 72, "Unsafe return of an any[] typed value", "2233173152"]
],
"packages/betterer/src/context/context.ts:1823827282": [
[29, 19, 8, "Don\'t use \`Function\` as a type. The \`Function\` type accepts any function-like value.\\nIt provides no type safety when calling the function, which can be a common source of bugs.\\nIt also accepts things like class declarations, which will throw at runtime as they will not be called with \`new\`.\\nIf you are expecting the function to accept certain arguments, you should explicitly define the function shape.", "4136871687"],
[146, 6, 9, "Unsafe assignment of an any value.", "166477157"]
],
"packages/betterer/src/errors.ts:3973991685": [
"packages/betterer/src/errors.ts:29397898": [
[7, 96, 10, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "3805115554"],
[8, 99, 11, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "2365839858"],
[9, 99, 11, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "2365839858"]
[9, 99, 11, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "2365839858"],
[11, 91, 12, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "362595323"],
[12, 70, 12, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "362595323"],
[13, 65, 8, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "92641249"],
[14, 67, 8, "Invalid type \\"string | Error | BettererError\\" of template literal expression.", "92641249"]
],
"packages/betterer/src/register.ts:845928349": [
[17, 8, 39, "Unsafe assignment of an any value.", "2552944516"],
Expand Down Expand Up @@ -67,8 +71,8 @@ exports[`new eslint rules`] = {
[96, 4, 27, "Unsafe member access .devDependencies on an any value.", "2927433783"],
[96, 55, 7, "Invalid type \\"any\\" of template literal expression.", "800248767"]
],
"packages/cli/src/watch.ts:3393502847": [
[25, 6, 15, "Promises must be handled appropriately or explicitly marked as ignored with the \`void\` operator.", "733721943"]
"packages/cli/src/watch.ts:176597381": [
[26, 6, 15, "Promises must be handled appropriately or explicitly marked as ignored with the \`void\` operator.", "733721943"]
],
"packages/extension/src/client/commands/disable.ts:1676199082": [
[11, 4, 42, "Promises must be handled appropriately or explicitly marked as ignored with the \`void\` operator.", "829060112"],
Expand Down
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,12 +183,15 @@ betterer -c ./path/to/config -r ./path/to/results -w

#### Start options

| Name | Description | Default |
| ------------------------- | ------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[.*]` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| Name | Description | Default |
| -------------------------- | --------------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD. Takes multiple values | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| `-R`, `--reporter` [value] | npm package name or file path to a Betterer reporter. Takes multiple values | Default reporter |

### Watch

Expand All @@ -200,9 +203,13 @@ betterer watch -c ./path/to/config -r ./path/to/results

#### Watch options

| Name | Description | Default |
| ------------------------- | ---------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[.*]` |
| `-i`, `--ignore` [value] | Ignore files by Glob when running in watch mode. Takes multiple values | `[]` |
| Name | Description | Default |
| -------------------------- | --------------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD. Takes multiple values | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| `-i`, `--ignore` [value] | Ignore files by Glob when running in watch mode. Takes multiple values | `[]` |
| `-R`, `--reporter` [value] | npm package name or file path to a Betterer reporter. Takes multiple values | Default reporter |
3 changes: 2 additions & 1 deletion packages/betterer/src/config/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ export function createConfig(partialConfig: BettererConfigPartial): BettererConf
ignores: toArray<string>(partialConfig.ignores || baseConfig.ignores),
cwd: partialConfig.cwd || baseConfig.cwd || process.cwd(),
silent: partialConfig.silent || baseConfig.silent || false,
update: partialConfig.update || baseConfig.update || false
update: partialConfig.update || baseConfig.update || false,
reporters: partialConfig.reporters || baseConfig.reporters || []
};
const tsconfigPath = partialConfig.tsconfigPath || baseConfig.tsconfigPath;

Expand Down
20 changes: 12 additions & 8 deletions packages/betterer/src/config/types.ts
Original file line number Diff line number Diff line change
@@ -1,25 +1,29 @@
import { BettererReporterNames } from '../reporters';

export type BettererConfigPaths = ReadonlyArray<string>;
export type BettererConfigFilters = ReadonlyArray<RegExp>;
export type BettererConfigIgnore = ReadonlyArray<string>;

export type BettererConfigPartial = Partial<{
configPaths: BettererConfigPaths | string;
resultsPath: string;
tsconfigPath: string;
cwd: string;
filters: BettererConfigFilters | ReadonlyArray<string> | string;
ignores: BettererConfigIgnore | string;
cwd: string;
update: boolean;
reporters: BettererReporterNames;
resultsPath: string;
silent: boolean;
tsconfigPath: string;
update: boolean;
}>;

export type BettererConfig = {
configPaths: BettererConfigPaths;
resultsPath: string;
tsconfigPath: string | null;
cwd: string;
filters: BettererConfigFilters;
ignores: BettererConfigIgnore;
cwd: string;
update: boolean;
reporters: BettererReporterNames;
resultsPath: string;
silent: boolean;
tsconfigPath: string | null;
update: boolean;
};
6 changes: 5 additions & 1 deletion packages/betterer/src/context/run.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import * as assert from 'assert';

import { BettererExpectedResult, NO_PREVIOUS_RESULT, deserialise } from '../results';
import { BettererExpectedResult, NO_PREVIOUS_RESULT, deserialise, diff } from '../results';
import { BettererTest } from '../test';
import { BettererFilePaths } from '../watcher';
import { BettererContext } from './context';
Expand Down Expand Up @@ -187,4 +187,8 @@ export class BettererRun {
this._hasResult = true;
this._context.runWorse(this);
}

public diff(): void {
diff(this);
}
}
5 changes: 5 additions & 0 deletions packages/betterer/src/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,8 @@ export const TEST_FUNCTION_REQUIRED = registerError(() => 'for a test to work, i
export const COULDNT_READ_CONFIG = registerError((configPath) => `could not read config from "${configPath}". 😔`);
export const COULDNT_READ_RESULTS = registerError((resultsPath) => `could not read results from "${resultsPath}". 😔`);
export const COULDNT_WRITE_RESULTS = registerError((resultsPath) => `could not write results to "${resultsPath}". 😔`);

export const COULDNT_LOAD_REPORTER = registerError((reporterName) => `could not require "${reporterName}". 😔`);
export const NO_REPORTER_LOADED = registerError((reporterName) => `"${reporterName}" didn't create a reporter. 😔`);
export const UNKNOWN_HOOK_NAME = registerError((hookName) => `"${hookName}" is not a valid reporter hook name. 😔`);
export const HOOK_NOT_A_FUNCTION = registerError((hookName) => `"${hookName}" is not a function. 😔`);
11 changes: 7 additions & 4 deletions packages/betterer/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import { logError } from '@betterer/errors';

import { BettererConfig, BettererConfigPartial, createConfig } from './config';
import { BettererContext, BettererStats, BettererRuns } from './context';
import { reporterParallel, reporterSerial } from './reporters';
import { registerExtensions } from './register';
import { loadReporters, DEFAULT_REPORTER, WATCH_REPORTER } from './reporters';
import { parallel, serial } from './runner';
import { BettererWatcher } from './watcher';
import { registerExtensions } from './register';

export * from './config/public';
export * from './context/public';
Expand All @@ -15,7 +16,8 @@ export * from './watcher/public';

export function betterer(partialConfig: BettererConfigPartial = {}): Promise<BettererStats> {
return runContext(partialConfig, async (config) => {
const context = new BettererContext(config, reporterSerial);
const reporter = loadReporters(config.reporters.length ? config.reporters : [DEFAULT_REPORTER]);
const context = new BettererContext(config, reporter);
await context.setup();
const runs = await serial(context);
const stats = await context.process(runs);
Expand All @@ -39,7 +41,8 @@ betterer.single = async function bettererSingle(

betterer.watch = function bettererWatch(partialConfig: BettererConfigPartial = {}): Promise<BettererWatcher> {
return runContext(partialConfig, async (config) => {
const context = new BettererContext(config, reporterParallel);
const reporter = loadReporters(config.reporters.length ? config.reporters : [WATCH_REPORTER]);
const context = new BettererContext(config, reporter);
const watcher = new BettererWatcher(context, async (filePaths) => {
await context.setup();
return parallel(context, filePaths);
Expand Down
3 changes: 2 additions & 1 deletion packages/betterer/src/reporters/index.ts
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
export { reporterParallel, reporterSerial } from './reporters';
export { BettererMultiReporter } from './reporter-multi';
export { loadReporters, DEFAULT_REPORTER, WATCH_REPORTER } from './loader';
export * from './types';
47 changes: 47 additions & 0 deletions packages/betterer/src/reporters/loader.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import { COULDNT_LOAD_REPORTER, NO_REPORTER_LOADED, UNKNOWN_HOOK_NAME, HOOK_NOT_A_FUNCTION } from '../errors';
import { requireUncached } from '../require';
import { BettererMultiReporter } from './reporter-multi';
import { BettererReporterNames, BettererReporter, BettererReporterModule } from './types';
import { isFunction } from '../utils';

export const DEFAULT_REPORTER = '@betterer/reporter';
export const WATCH_REPORTER = '@betterer/watch-reporter';

const HOOK_NAMES: ReadonlyArray<keyof BettererReporter> = [
'contextStart',
'contextEnd',
'contextError',
'runsStart',
'runsEnd',
'runStart',
'runEnd'
];

export function loadReporters(reporterNames: BettererReporterNames): BettererMultiReporter {
const reporters: Array<BettererReporter> = reporterNames.map((name) => {
try {
const module: BettererReporterModule = requireUncached(name);
if (!module || !module.reporter) {
throw NO_REPORTER_LOADED(name);
}
validate(module.reporter);
return module.reporter;
} catch (e) {
throw COULDNT_LOAD_REPORTER(name, e);
}
});
return new BettererMultiReporter(reporters);
}

function validate(result: unknown): asserts result is BettererReporter {
const reporter = result as BettererReporter;
Object.keys(reporter).forEach((key) => {
const hookName = key as keyof BettererReporter;
if (!HOOK_NAMES.includes(hookName)) {
throw UNKNOWN_HOOK_NAME(hookName);
}
if (!isFunction(reporter[hookName])) {
throw HOOK_NOT_A_FUNCTION(hookName);
}
});
}
31 changes: 31 additions & 0 deletions packages/betterer/src/reporters/reporter-multi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { BettererError } from '@betterer/errors';

import { BettererContext, BettererRun, BettererRuns, BettererStats } from '../context';
import { BettererFilePaths } from '../test';
import { BettererReporter } from './types';

export class BettererMultiReporter implements BettererReporter {
constructor(private _reporters: Array<BettererReporter>) {}

contextStart(context: BettererContext): void {
this._reporters.forEach((r) => r.contextStart?.(context));
}
contextEnd(context: BettererContext, stats: BettererStats): void {
this._reporters.forEach((r) => r.contextEnd?.(context, stats));
}
contextError(context: BettererContext, error: BettererError, printed: Array<string>): void {
this._reporters.forEach((r) => r.contextError?.(context, error, printed));
}
runsStart(runs: BettererRuns, files: BettererFilePaths): void {
this._reporters.forEach((r) => r.runsStart?.(runs, files));
}
runsEnd(runs: BettererRuns, files: BettererFilePaths): void {
this._reporters.forEach((r) => r.runsEnd?.(runs, files));
}
runStart(run: BettererRun): void {
this._reporters.forEach((r) => r.runStart?.(run));
}
runEnd(run: BettererRun): void {
this._reporters.forEach((r) => r.runEnd?.(run));
}
}
6 changes: 6 additions & 0 deletions packages/betterer/src/reporters/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,9 @@ export type BettererReporter = {
runStart?(run: BettererRun): void;
runEnd?(run: BettererRun): void;
};

export type BettererReporterNames = ReadonlyArray<string>;

export type BettererReporterModule = {
reporter: BettererReporter;
};
36 changes: 19 additions & 17 deletions packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,15 @@ betterer -c ./path/to/config -r ./path/to/results -w

#### Start options

| Name | Description | Default |
| -------------------------- | ------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[.*]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| Name | Description | Default |
| -------------------------- | --------------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD. Takes multiple values | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| `-R`, `--reporter` [value] | npm package name or file path to a Betterer reporter. Takes multiple values | Default reporter |

### Watch

Expand All @@ -51,12 +52,13 @@ betterer watch -c ./path/to/config -r ./path/to/results

#### Watch options

| Name | Description | Default |
| -------------------------- | ---------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[.*]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| `-i`, `--ignore` [value] | Ignore files by Glob when running in watch mode. Takes multiple values | `[]` |
| Name | Description | Default |
| -------------------------- | --------------------------------------------------------------------------- | --------------------- |
| `-c`, `--config` [value] | Path to test definition file relative to CWD. Takes multiple values | `./.betterer.ts` |
| `-r`, `--results` [value] | Path to test results file relative to CWD | `./.betterer.results` |
| `-t`, `--tsconfig` [value] | Path to TypeScript config file relative to CWD | `null` |
| `-f`, `--filter` [value] | Select tests to run by RegExp. Takes multiple values | `[]` |
| `-s`, `--silent` | Disable all logging | `false` |
| `-u`, `--update` | Force update the results file, even if things get worse | `false` |
| `-i`, `--ignore` [value] | Ignore files by Glob when running in watch mode. Takes multiple values | `[]` |
| `-R`, `--reporter` [value] | npm package name or file path to a Betterer reporter. Takes multiple values | Default reporter |
Loading

0 comments on commit 784b2fa

Please sign in to comment.