Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg] Rename port_versions to versions #15784

Merged
merged 2 commits into from
Jan 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 5 additions & 5 deletions docs/maintainers/maintainer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,16 +204,16 @@ For Example:

See our [manifest files document](manifest-files.md#port-version) for a full explanation of our conventions.

### Update the version files in `port_versions/` of any modified ports
### Update the version files in `versions/` of any modified ports

Vcpkg uses a set of metadata files to power its versioning feature.
These files are located in the following locations:
* `${VCPKG_ROOT}/port_versions/baseline.json`, (this file is common to all ports) and
* `${VCPKG_ROOT}/port_versions/${first-letter-of-portname}-/${portname}.json` (one per port).
* `${VCPKG_ROOT}/versions/baseline.json`, (this file is common to all ports) and
* `${VCPKG_ROOT}/versions/${first-letter-of-portname}-/${portname}.json` (one per port).

For example, for `zlib` the relevant files are:
* `${VCPKG_ROOT}/port_versions/baseline.json`
* `${VCPKG_ROOT}/port_versions/z-/zlib.json`
* `${VCPKG_ROOT}/versions/baseline.json`
* `${VCPKG_ROOT}/versions/z-/zlib.json`

We expect that each time you update a port, you also update its version files.

Expand Down
10 changes: 5 additions & 5 deletions docs/specifications/versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ Baselines are used to set lower boundaries on package versions. A baseline effec

But what is a baseline?

In the main registry, the baseline is a file located in `${VCPKG_ROOT}/port_versions/baseline.json`. This file contains a version declaration for each package in vcpkg. The format of this file is the following:
In the main registry, the baseline is a file located in `${VCPKG_ROOT}/versions/baseline.json`. This file contains a version declaration for each package in vcpkg. The format of this file is the following:

```json
{
Expand Down Expand Up @@ -319,9 +319,9 @@ Although the concept of package versions has always been present in vcpkg, the c

With the introduction of versioning constraints, it is now possible that a package depends on a port version that does not match the one available locally. This raises a problem as vcpkg needs to know how to acquire the port files for the requested version.

To solve this problem, a new set of metadata needs to be introduced. This specification proposes a that a new “port_versions” folder is added as part of a registry. In the main vcpkg registry, this means a new root level port_versions directory.
To solve this problem, a new set of metadata needs to be introduced. This specification proposes a that a new "versions" folder is added as part of a registry. In the main vcpkg registry, this means a new root level versions directory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

directory, not folder. (Sorry, I know you didn't edit, but still)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prolly not gonna make that change since I'm just doing the boring rename thing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only worry about it if you need to spin this again. (I view this comment as no different than you asking me to sort cmake vars in the vcpkg_minimum_required PR :) )


The port_versions directory, from here on referred as the versions database, will contain JSON files for each one of the ports available in the registry. Each file will list all the versions available for a package and contain a Git tree-ish object that vcpkg can check out to obtain that version’s portfiles.
The versions directory, from here on referred as the versions database, will contain JSON files for each one of the ports available in the registry. Each file will list all the versions available for a package and contain a Git tree-ish object that vcpkg can check out to obtain that version’s portfiles.

As part of the versioning implementation, a generator for these database files will be implemented. The generator will extract from our repository’s Git history, all the versions of each port that had been available at any moment in time and compile them into these database files.

Expand Down Expand Up @@ -349,5 +349,5 @@ Example: generated `zlib.json`
}
```

For each port, its corresponding versions file should be located in `port_versions/{first letter of port name}-/{port name}.json`. For example, zlib’s version file will be located in `port_versions/z-/zlib.json`.
Aside from port version files, the current baseline file is located in `port_versions/baseline.json`.
For each port, its corresponding versions file should be located in `versions/{first letter of port name}-/{port name}.json`. For example, zlib’s version file will be located in `versions/z-/zlib.json`.
Aside from port version files, the current baseline file is located in `versions/baseline.json`.
16 changes: 8 additions & 8 deletions scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
. "$PSScriptRoot/../end-to-end-tests-prelude.ps1"


$builtinRegistryArgs = $commonArgs + @("--x-builtin-port-versions-dir=$PSScriptRoot/../../e2e_ports/port_versions")
$builtinRegistryArgs = $commonArgs + @("--x-builtin-registry-versions-dir=$PSScriptRoot/../../e2e_ports/versions")

Run-Vcpkg install @builtinRegistryArgs 'vcpkg-internal-e2e-test-port'
Throw-IfNotFailed

# We should not look into the port_versions directory unless we have a baseline,
# We should not look into the versions directory unless we have a baseline,
# even if we pass the registries feature flag
Run-Vcpkg install @builtinRegistryArgs --feature-flags=registries 'vcpkg-internal-e2e-test-port'
Throw-IfNotFailed
Expand All @@ -28,13 +28,13 @@ Copy-Item -Recurse `
-LiteralPath "$PSScriptRoot/../../e2e_ports/vcpkg-internal-e2e-test-port" `
-Destination "$filesystemRegistry"
New-Item `
-Path "$filesystemRegistry/port_versions" `
-Path "$filesystemRegistry/versions" `
-ItemType Directory
Copy-Item `
-LiteralPath "$PSScriptRoot/../../e2e_ports/port_versions/baseline.json" `
-Destination "$filesystemRegistry/port_versions/baseline.json"
-LiteralPath "$PSScriptRoot/../../e2e_ports/versions/baseline.json" `
-Destination "$filesystemRegistry/versions/baseline.json"
New-Item `
-Path "$filesystemRegistry/port_versions/v-" `
-Path "$filesystemRegistry/versions/v-" `
-ItemType Directory

$vcpkgInternalE2eTestPortJson = @{
Expand All @@ -46,7 +46,7 @@ $vcpkgInternalE2eTestPortJson = @{
)
}
New-Item `
-Path "$filesystemRegistry/port_versions/v-/vcpkg-internal-e2e-test-port.json" `
-Path "$filesystemRegistry/versions/v-/vcpkg-internal-e2e-test-port.json" `
-ItemType File `
-Value (ConvertTo-Json -Depth 5 -InputObject $vcpkgInternalE2eTestPortJson)

Expand All @@ -68,7 +68,7 @@ try
$CurrentTest = 'git init .'
git @gitConfigOptions init .
Throw-IfFailed
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/port_versions" -Destination .
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/versions" -Destination .
Copy-Item -Recurse -LiteralPath "$PSScriptRoot/../../e2e_ports/vcpkg-internal-e2e-test-port" -Destination .

$CurrentTest = 'git add -A'
Expand Down
14 changes: 7 additions & 7 deletions scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@

# Test verify versions
mkdir $VersionFilesRoot
Copy-Item -Recurse "scripts/testing/version-files/port_versions_incomplete" $VersionFilesRoot
Copy-Item -Recurse "scripts/testing/version-files/versions_incomplete" $VersionFilesRoot
$portsRedirectArgsOK = @(
"--feature-flags=versions",
"--x-builtin-ports-root=scripts/testing/version-files/ports",
"--x-builtin-port-versions-dir=scripts/testing/version-files/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/versions"
)
$portsRedirectArgsIncomplete = @(
"--feature-flags=versions",
"--x-builtin-ports-root=scripts/testing/version-files/ports_incomplete",
"--x-builtin-port-versions-dir=$VersionFilesRoot/port_versions_incomplete"
"--x-builtin-registry-versions-dir=$VersionFilesRoot/versions_incomplete"
)
$CurrentTest = "x-verify-ci-versions (All files OK)"
Write-Host $CurrentTest
Expand Down Expand Up @@ -77,22 +77,22 @@ foreach ($opt_registries in @("",",registries"))
./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/without-default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfFailed
Require-FileNotExists $buildtreesRoot/versioning

$CurrentTest = "default baseline 2"
./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfFailed
Require-FileExists $buildtreesRoot/versioning

$CurrentTest = "using version features fails without flag"
./vcpkg $commonArgs "--feature-flags=-versions$opt_registries" install `
"--dry-run" `
"--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
"--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
"--x-builtin-registry-versions-dir=scripts/testing/version-files/default-baseline-2/versions"
Throw-IfNotFailed
}
}
2 changes: 1 addition & 1 deletion scripts/generateBaseline.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

SCRIPT_DIRECTORY = os.path.dirname(os.path.abspath(__file__))
PORTS_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../ports')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../port_versions')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../versions')


def get_version_tag(version):
Expand Down
10 changes: 5 additions & 5 deletions scripts/generatePortVersionsDb.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
MAX_PROCESSES = multiprocessing.cpu_count()
SCRIPT_DIRECTORY = os.path.dirname(os.path.abspath(__file__))
PORTS_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../ports')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../port_versions')
VERSIONS_DB_DIRECTORY = os.path.join(SCRIPT_DIRECTORY, '../versions')


def get_current_git_ref():
Expand All @@ -26,7 +26,7 @@ def get_current_git_ref():
return None


def generate_port_versions_file(port_name):
def generate_versions_file(port_name):
containing_dir = os.path.join(VERSIONS_DB_DIRECTORY, f'{port_name[0]}-')
os.makedirs(containing_dir, exist_ok=True)

Expand All @@ -43,7 +43,7 @@ def generate_port_versions_file(port_name):
output.stdout.strip(), file=sys.stderr)


def generate_port_versions_db(revision):
def generate_versions_db(revision):
start_time = time.time()

# Assume each directory in ${VCPKG_ROOT}/ports is a different port
Expand All @@ -54,7 +54,7 @@ def generate_port_versions_db(revision):
concurrency = MAX_PROCESSES / 2
print(f'Running {concurrency:.0f} parallel processes')
process_pool = multiprocessing.Pool(MAX_PROCESSES)
for i, _ in enumerate(process_pool.imap_unordered(generate_port_versions_file, port_names), 1):
for i, _ in enumerate(process_pool.imap_unordered(generate_versions_file, port_names), 1):
sys.stderr.write(
f'\rProcessed: {i}/{total_count} ({(i / total_count):.2%})')
process_pool.close()
Expand All @@ -80,7 +80,7 @@ def main():
print(f'Database files already exist for commit {revision}')
sys.exit(0)

generate_port_versions_db(revision)
generate_versions_db(revision)


if __name__ == "__main__":
Expand Down
1 change: 0 additions & 1 deletion test.cmake

This file was deleted.

4 changes: 2 additions & 2 deletions toolsrc/include/vcpkg/vcpkgcmdarguments.h
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ namespace vcpkg
std::unique_ptr<std::string> scripts_root_dir;
constexpr static StringLiteral BUILTIN_PORTS_ROOT_DIR_ARG = "x-builtin-ports-root";
std::unique_ptr<std::string> builtin_ports_root_dir;
constexpr static StringLiteral BUILTIN_PORT_VERSIONS_DIR_ARG = "x-builtin-port-versions-dir";
std::unique_ptr<std::string> builtin_port_versions_dir;
constexpr static StringLiteral BUILTIN_REGISTRY_VERSIONS_DIR_ARG = "x-builtin-registry-versions-dir";
std::unique_ptr<std::string> builtin_registry_versions_dir;

constexpr static StringLiteral DEFAULT_VISUAL_STUDIO_PATH_ENV = "VCPKG_VISUAL_STUDIO_PATH";
std::unique_ptr<std::string> default_visual_studio_path;
Expand Down
2 changes: 1 addition & 1 deletion toolsrc/include/vcpkg/vcpkgpaths.h
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ namespace vcpkg
fs::path scripts;
fs::path prefab;
fs::path builtin_ports;
fs::path builtin_port_versions;
fs::path builtin_registry_versions;

fs::path tools;
fs::path buildsystems;
Expand Down
8 changes: 4 additions & 4 deletions toolsrc/src/vcpkg-test/arguments.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ TEST_CASE ("VcpkgCmdArguments from lowercase argument sequence", "[arguments]")
"C:\\vcpkg",
"--x-scripts-root=C:\\scripts",
"--x-builtin-ports-root=C:\\ports",
"--x-builtin-port-versions-dir=C:\\port_versions",
"--x-builtin-registry-versions-dir=C:\\versions",
"--debug",
"--sendmetrics",
"--printmetrics",
Expand All @@ -28,7 +28,7 @@ TEST_CASE ("VcpkgCmdArguments from lowercase argument sequence", "[arguments]")
REQUIRE(*v.vcpkg_root_dir == "C:\\vcpkg");
REQUIRE(*v.scripts_root_dir == "C:\\scripts");
REQUIRE(*v.builtin_ports_root_dir == "C:\\ports");
REQUIRE(*v.builtin_port_versions_dir == "C:\\port_versions");
REQUIRE(*v.builtin_registry_versions_dir == "C:\\versions");
REQUIRE(v.debug);
REQUIRE(*v.debug.get());
REQUIRE(v.send_metrics);
Expand All @@ -51,7 +51,7 @@ TEST_CASE ("VcpkgCmdArguments from uppercase argument sequence", "[arguments]")
"C:\\vcpkg",
"--X-SCRIPTS-ROOT=C:\\scripts",
"--X-BUILTIN-PORTS-ROOT=C:\\ports",
"--X-BUILTIN-PORT-VERSIONS-DIR=C:\\port_versions",
"--X-BUILTIN-REGISTRY-VERSIONS-DIR=C:\\versions",
"--DEBUG",
"--SENDMETRICS",
"--PRINTMETRICS",
Expand All @@ -64,7 +64,7 @@ TEST_CASE ("VcpkgCmdArguments from uppercase argument sequence", "[arguments]")
REQUIRE(*v.vcpkg_root_dir == "C:\\vcpkg");
REQUIRE(*v.scripts_root_dir == "C:\\scripts");
REQUIRE(*v.builtin_ports_root_dir == "C:\\ports");
REQUIRE(*v.builtin_port_versions_dir == "C:\\port_versions");
REQUIRE(*v.builtin_registry_versions_dir == "C:\\versions");
REQUIRE(v.debug);
REQUIRE(*v.debug.get());
REQUIRE(v.send_metrics);
Expand Down
8 changes: 4 additions & 4 deletions toolsrc/src/vcpkg/commands.add-version.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ namespace vcpkg::Commands::AddVersion
const bool verbose = Util::Sets::contains(parsed_args.switches, OPTION_VERBOSE);

auto& fs = paths.get_filesystem();
auto baseline_path = paths.builtin_port_versions / fs::u8path("baseline.json");
auto baseline_path = paths.builtin_registry_versions / fs::u8path("baseline.json");
if (!fs.exists(VCPKG_LINE_INFO, baseline_path))
{
System::printf(
Expand Down Expand Up @@ -400,8 +400,8 @@ namespace vcpkg::Commands::AddVersion
}
const auto& git_tree = git_tree_it->second;

auto port_versions_path =
paths.builtin_port_versions / Strings::concat(port_name[0], '-') / Strings::concat(port_name, ".json");
auto port_versions_path = paths.builtin_registry_versions / fs::u8path({port_name[0], '-'}) /
fs::u8path(Strings::concat(port_name, ".json"));
update_version_db_file(
paths, port_name, schemed_version, git_tree, port_versions_path, overwrite_version, verbose, add_all);
update_baseline_version(paths, port_name, schemed_version.versiont, baseline_path, verbose);
Expand All @@ -413,4 +413,4 @@ namespace vcpkg::Commands::AddVersion
{
AddVersion::perform_and_exit(args, paths);
}
}
}
6 changes: 3 additions & 3 deletions toolsrc/src/vcpkg/commands.civerifyversions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -367,8 +367,8 @@ namespace vcpkg::Commands::CIVerifyVersions
continue;
}

auto versions_file_path =
paths.builtin_port_versions / Strings::concat(port_name[0], '-') / Strings::concat(port_name, ".json");
auto versions_file_path = paths.builtin_registry_versions / fs::u8path({port_name[0], '-'}) /
fs::u8path(Strings::concat(port_name, ".json"));
if (!fs.exists(versions_file_path))
{
System::printf(System::Color::error, "FAIL: %s\n", port_name);
Expand Down Expand Up @@ -415,4 +415,4 @@ namespace vcpkg::Commands::CIVerifyVersions
{
CIVerifyVersions::perform_and_exit(args, paths);
}
}
}
Loading