Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(migrate): regexManagers -> customManagers #24451

Merged
merged 16 commits into from
Sep 24, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
772 changes: 388 additions & 384 deletions docs/usage/configuration-options.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/usage/getting-started/use-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ This can be because:
- The package manager/file format is not supported, or
- The file format is not a standard or is proprietary

If your dependencies are not found by default, you can use our "regex" manager to set your own custom patterns to extract dependencies.
You configure the regex manager by telling it:
If your dependencies are not found by default, you can use our custom manager to set your own custom patterns to extract dependencies.
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
You configure the custom manager by telling it:
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved

- Which file pattern(s) to match
- How to find the dependency name and version from within the file
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/reading-list.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Feel free to read up on anything that looks interesting to you.
First, complete the "Beginners" and the "Intermediate" reading list.
Then read:

- Define your own regex manager with [`regexManagers`](./configuration-options.md#regexmanagers)
- Define your own custom manager with [`customManagers`](./configuration-options.md#custommanagers)
- [Shareable config presets](./config-presets.md)

## Self-hosting Renovate
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,13 @@ Renovate comes with over 50 different datasources, but the one that is important
Managers are the Renovate concept for package managers.
There isn’t an AUR or `PKGBUILD` manager, but there is a [regex manager](https://docs.renovatebot.com/modules/manager/regex/) that I can use.

I can create a `renovate.json` configuration with the following regex manager configuration:
I can create a `renovate.json` configuration with the following custom manager configuration:

```json
{
"regexManagers": [
"customManagers": [
{
"customType": "regex",
"fileMatch": ["(^|/)PKGBUILD$"],
"matchStrings": [
"pkgver=(?<currentValue>.*) # renovate: datasource=(?<datasource>.*) depName=(?<depName>.*)"
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/user-stories/swissquote.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Some features and options we enjoy:
- Each rule can be customized either globally [or specified per package](../configuration-options.md#packagerules)
- Works with your [private package registry](../getting-started/private-packages.md)
- Supports more than 70 [languages and package managers](https://docs.renovatebot.com/modules/manager/#supported-managers): Maven, Docker, npm, Docker Compose, Python
- If you are using dependencies in a custom way, [there is a special regexManager](../configuration-options.md#regexmanagers) that allows you to transform patterns into dependencies
- If you are using dependencies in a custom way, [there is a special customManager](../configuration-options.md#custommanagers) that allows you to transform patterns into dependencies
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved

There is an [on-premise option](https://www.mend.io/free-developer-tools/renovate/on-premises/), but you can also use [the Mend Renovate App](https://github.com/marketplace/renovate).
On our side, we’re not using the on-premise but rather a custom scheduler using the open source Docker image.
Expand Down
56 changes: 28 additions & 28 deletions lib/config/__snapshots__/migration.spec.ts.snap
Original file line number Diff line number Diff line change
@@ -1,5 +1,33 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`config/migration it migrates customManagers 1`] = `
{
"customManagers": [
{
"customType": "regex",
"fileMatch": [
"(^|/|\\.)Dockerfile$",
"(^|/)Dockerfile[^/]*$",
],
"matchStrings": [
"# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>[^\\s]+?)(?: lookupName=(?<packageName>[^\\s]+?))?(?: versioning=(?<versioning>[a-z-0-9]+?))?\\s(?:ENV|ARG) .+?_VERSION="?(?<currentValue>.+?)"?\\s",
],
},
{
"customType": "regex",
"fileMatch": [
"(^|/|\\.)Dockerfile$",
"(^|/)Dockerfile[^/]*$",
],
"matchStrings": [
"# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>[^\\s]+?)(?: lookupName=(?<holder>[^\\s]+?))?(?: versioning=(?<versioning>[a-z-0-9]+?))?\\s(?:ENV|ARG) .+?_VERSION="?(?<currentValue>.+?)"?\\s",
],
"packageNameTemplate": "{{{holder}}}",
},
],
}
`;

exports[`config/migration it migrates gradle-lite 1`] = `
{
"gradle": {
Expand Down Expand Up @@ -51,34 +79,6 @@ exports[`config/migration it migrates nested packageRules 1`] = `
}
`;

exports[`config/migration it migrates regexManagers 1`] = `
{
"regexManagers": [
{
"customType": "regex",
"fileMatch": [
"(^|/|\\.)Dockerfile$",
"(^|/)Dockerfile[^/]*$",
],
"matchStrings": [
"# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>[^\\s]+?)(?: lookupName=(?<packageName>[^\\s]+?))?(?: versioning=(?<versioning>[a-z-0-9]+?))?\\s(?:ENV|ARG) .+?_VERSION="?(?<currentValue>.+?)"?\\s",
],
},
{
"customType": "regex",
"fileMatch": [
"(^|/|\\.)Dockerfile$",
"(^|/)Dockerfile[^/]*$",
],
"matchStrings": [
"# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>[^\\s]+?)(?: lookupName=(?<holder>[^\\s]+?))?(?: versioning=(?<versioning>[a-z-0-9]+?))?\\s(?:ENV|ARG) .+?_VERSION="?(?<currentValue>.+?)"?\\s",
],
"packageNameTemplate": "{{{holder}}}",
},
],
}
`;

exports[`config/migration migrateConfig(config, parentConfig) does not migrate multi days 1`] = `
{
"schedule": "after 5:00pm on wednesday and thursday",
Expand Down
20 changes: 10 additions & 10 deletions lib/config/__snapshots__/validation.spec.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,15 @@ exports[`config/validation validateConfig(config) errors for unsafe fileMatches
]
`;

exports[`config/validation validateConfig(config) errors if customManager fields are missing 1`] = `
[
{
"message": "Regex Managers must contain currentValueTemplate configuration or regex group named currentValue",
"topic": "Configuration Error",
},
]
`;

exports[`config/validation validateConfig(config) errors if fileMatch has wrong parent 1`] = `
[
{
Expand Down Expand Up @@ -176,15 +185,6 @@ exports[`config/validation validateConfig(config) errors if manager objects are
]
`;

exports[`config/validation validateConfig(config) errors if regexManager fields are missing 1`] = `
[
{
"message": "Regex Managers must contain currentValueTemplate configuration or regex group named currentValue",
"topic": "Configuration Error",
},
]
`;

exports[`config/validation validateConfig(config) ignore packageRule nesting validation for presets 1`] = `[]`;

exports[`config/validation validateConfig(config) included managers of the wrong type 1`] = `
Expand Down Expand Up @@ -250,7 +250,7 @@ exports[`config/validation validateConfig(config) selectors outside packageRules
exports[`config/validation validateConfig(config) validates regEx for each fileMatch 1`] = `
[
{
"message": "Invalid regExp for regexManagers[0].fileMatch: \`***$}{]][\`",
"message": "Invalid regExp for customManagers[0].fileMatch: \`***$}{]][\`",
"topic": "Configuration Error",
},
]
Expand Down
5 changes: 3 additions & 2 deletions lib/config/migration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -643,10 +643,11 @@ describe('config/migration', () => {
expect(migratedConfig).toEqual({ extends: ['local>org/renovate-config'] });
});

it('it migrates regexManagers', () => {
it('it migrates customManagers', () => {
const config: RenovateConfig = {
regexManagers: [
customManagers: [
{
customType: 'regex',
fileMatch: ['(^|/|\\.)Dockerfile$', '(^|/)Dockerfile[^/]*$'],
matchStrings: [
'# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>[^\\s]+?)(?: lookupName=(?<lookupName>[^\\s]+?))?(?: versioning=(?<versioning>[a-z-0-9]+?))?\\s(?:ENV|ARG) .+?_VERSION="?(?<currentValue>.+?)"?\\s',
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
import { partial } from '../../../../test/util';
import type { CustomManager } from '../../../modules/manager/custom/types';
import { RegexManagersMigration } from './regex-managers-migration';
import { CustomManagersMigration } from './custom-managers-migration';

describe('config/migrations/custom/regex-managers-migration', () => {
describe('config/migrations/custom/custom-managers-migration', () => {
it('migrates', () => {
expect(RegexManagersMigration).toMigrate(
expect(CustomManagersMigration).toMigrate(
{
regexManagers: partial<CustomManager>([
customManagers: partial<CustomManager>([
{
fileMatch: ['js', '***$}{]]['],
matchStrings: ['^(?<depName>foo)(?<currentValue>bar)$'],
Expand All @@ -23,7 +23,7 @@ describe('config/migrations/custom/regex-managers-migration', () => {
]),
},
{
regexManagers: [
customManagers: [
{
customType: 'regex',
fileMatch: ['js', '***$}{]]['],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@ import is from '@sindresorhus/is';
import type { CustomManager } from '../../../modules/manager/custom/types';
import { AbstractMigration } from '../base/abstract-migration';

export class RegexManagersMigration extends AbstractMigration {
override readonly propertyName = 'regexManagers';
export class CustomManagersMigration extends AbstractMigration {
override readonly propertyName = 'customManagers';

override run(value: unknown): void {
if (is.nonEmptyArray(value)) {
const regexManagers = (value as CustomManager[]).map((mgr) => {
const customManagers = (value as CustomManager[]).map((mgr) => {
if (mgr.customType) {
return mgr;
}
return Object.assign({ customType: 'regex' }, mgr); // to make sure customType is at top, looks good when migration pr is created
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
});

this.rewrite(regexManagers);
this.rewrite(customManagers);
}
}
}
5 changes: 3 additions & 2 deletions lib/config/migrations/migrations-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { BranchNameMigration } from './custom/branch-name-migration';
import { BranchPrefixMigration } from './custom/branch-prefix-migration';
import { CompatibilityMigration } from './custom/compatibility-migration';
import { ComposerIgnorePlatformReqsMigration } from './custom/composer-ignore-platform-reqs-migration';
import { CustomManagersMigration } from './custom/custom-managers-migration';
import { DatasourceMigration } from './custom/datasource-migration';
import { DepTypesMigration } from './custom/dep-types-migration';
import { DryRunMigration } from './custom/dry-run-migration';
Expand All @@ -41,7 +42,6 @@ import { RaiseDeprecationWarningsMigration } from './custom/raise-deprecation-wa
import { RebaseConflictedPrs } from './custom/rebase-conflicted-prs-migration';
import { RebaseStalePrsMigration } from './custom/rebase-stale-prs-migration';
import { RecreateClosedMigration } from './custom/recreate-closed-migration';
import { RegexManagersMigration } from './custom/regex-managers-migration';
import { RenovateForkMigration } from './custom/renovate-fork-migration';
import { RequireConfigMigration } from './custom/require-config-migration';
import { RequiredStatusChecksMigration } from './custom/required-status-checks-migration';
Expand Down Expand Up @@ -96,6 +96,7 @@ export class MigrationsService {
['masterIssueFooter', 'dependencyDashboardFooter'],
['masterIssueTitle', 'dependencyDashboardTitle'],
['masterIssueLabels', 'dependencyDashboardLabels'],
['regexManagers', 'customManagers'],
]);

static readonly customMigrations: ReadonlyArray<MigrationConstructor> = [
Expand Down Expand Up @@ -151,7 +152,7 @@ export class MigrationsService {
RecreateClosedMigration,
StabilityDaysMigration,
FetchReleaseNotesMigration,
RegexManagersMigration,
CustomManagersMigration,
];

static run(originalConfig: RenovateConfig): RenovateConfig {
Expand Down
48 changes: 24 additions & 24 deletions lib/config/options/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2493,7 +2493,7 @@ const options: RenovateOptions[] = [
default: false,
},
{
name: 'regexManagers',
name: 'customManagers',
description: 'Custom managers using regex matching.',
type: 'array',
subType: 'object',
Expand All @@ -2505,21 +2505,21 @@ const options: RenovateOptions[] = [
{
name: 'customType',
description:
'Custom manager to use. Valid only within a `regexManagers` object.',
'Custom manager to use. Valid only within a `customManagers` object.',
type: 'string',
allowedValues: ['regex'],
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'matchStrings',
description:
'Regex capture rule to use. Valid only within a `regexManagers` object.',
'Regex capture rule to use. Valid only within a `customManagers` object.',
type: 'array',
subType: 'string',
format: 'regex',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
Expand All @@ -2529,88 +2529,88 @@ const options: RenovateOptions[] = [
type: 'string',
default: 'any',
allowedValues: ['any', 'recursive', 'combination'],
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'depNameTemplate',
description:
'Optional depName for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional depName for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'packageNameTemplate',
description:
'Optional packageName for extracted dependencies, else defaults to `depName` value. Valid only within a `regexManagers` object.',
'Optional packageName for extracted dependencies, else defaults to `depName` value. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'datasourceTemplate',
description:
'Optional datasource for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional datasource for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'depTypeTemplate',
description:
'Optional `depType` for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional `depType` for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'currentValueTemplate',
description:
'Optional `currentValue` for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional `currentValue` for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'versioningTemplate',
description:
'Optional versioning for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional versioning for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'registryUrlTemplate',
description:
'Optional registry URL for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional registry URL for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'extractVersionTemplate',
description:
'Optional `extractVersion` for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional `extractVersion` for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
{
name: 'autoReplaceStringTemplate',
description:
'Optional `extractVersion` for extracted dependencies. Valid only within a `regexManagers` object.',
'Optional `extractVersion` for extracted dependencies. Valid only within a `customManagers` object.',
type: 'string',
parent: 'regexManagers',
parent: 'customManagers',
cli: false,
env: false,
},
Expand Down
Loading