This repository has been archived by the owner on May 4, 2024. It is now read-only.
[14/x][move-package/lock] Clean-up: Avoid explicit PackageLock::unlock #790
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amnn
requested review from
gregnazario,
tnowacki,
sblackshear,
vgao1996 and
dariorussi
wrwg
reviewed
Jan 2, 2023
|
Hi @wrwg, although the wording/structure of some of the error messages is different, these changes are otherwise backwards compatible with the existing way resolution works (including package hooks). That was something we clarified when the RFC was tabled at the community sync, but I grant you that that was a while ago (mid-November last year). At the time I think you mentioned you would pass the proposal on to @gregnazario for feedback. We haven't heard back yet on that front, but I think the best way to make sure the details that are most relevant to you are looked into is for myself and Greg (or anyone from Aptos who is interested -- I have been adding @vgao1996 as well since he expressed some interest in building on top of this work to speed up fetching) to meet and go over the structure of the stack at a high level, so I can point out the PRs that could use the most scrutiny on that front, which are:
I won't land those PRs without making sure they preserve the way Aptos integrates with package resolution (which they intend to), let's discuss at the next community sync the best people to include in that discussion. |
amnn
added a commit
to amnn/move
that referenced
this pull request
Jan 9, 2023
With the introduction of external package resolvers in move-language#790, it is no longer necessary to embed calls to Movey's package resolution and authentication API into the Move CLI. Instead, Movey will maintain its own CLI that gets called during resolution, and that tool can communicate with Movey's servers and deal with Movey authentication. Test Plan: ``` $ cargo x lint && cargo xfmt && cargo xclippy --all-targets $ cargo-nextest nextest run $ cargo-nextest nextest run --features evm-backend ```
amnn
added a commit
that referenced
this pull request
Jan 10, 2023
With the introduction of external package resolvers in #790, it is no longer necessary to embed calls to Movey's package resolution and authentication API into the Move CLI. Instead, Movey will maintain its own CLI that gets called during resolution, and that tool can communicate with Movey's servers and deal with Movey authentication. Test Plan: ``` $ cargo x lint && cargo xfmt && cargo xclippy --all-targets $ cargo-nextest nextest run $ cargo-nextest nextest run --features evm-backend ```
The initial design of `DependencyGraph` only allowed for one `addr_subst` and `digest` per package in the transitive dependency graph, but in actuality there can be as many of these as there are dependency edges to that package: - Different `addr_subst` for different packages `P`, `Q` depending on `R` allows the same address in `R` to appear as different addresses in `P` and `Q`. - If `R` is a dependency of `P` and a dev-dependency of `Q`, then its source digest may differ between the two. Fixed by moving the `subst` and `digest` to be edge labels in the graph, rather than node labels. Note that this also changes the lock file format in the following ways: - A package's dependencies are now no longer just the name of the dependency, but an inline table containing the name and optionally the digest and address substitution. - The key within the `move` table that contains all the transitive dependencies is now called `package` rather than `dependency` (this was mainly driven by the naming within the schema, which required a name for the new, richer format for a package's dependencies.) Test Plan: ``` move/language/tools/move-cli$ cargo nextest move/language/tools/move-package$ cargo nextest ```
Function to return a package's immediate dependencies in the transitive dependency graph, for use by `ResolutionGraph` when resolving renamings and in-scope addresses during resolution Test Plan: New tests: ``` move/language/tools/move-package$ cargo nextest ```
Extract the logic for unifying named addresses across address assignments, substitutions and renamings from `ResolvingGraph` into its own class -- `ResolvingTable`. This is to be used when integrating `DependencyGraph` into `ResolutionGraph`. Test Plan: New unit tests: ``` move/language/tools/move-package$ cargo nextest ```
Integrate `DependencyGraph` and `ResolvingTable` into the creation of `ResolvedGraph`. This change replaces the recursive dependency traversal logic of `ResolutionGraph` that `DependencyGraph` was based on with a linear traversal of the package's transitive dependencies, extracted from an existing `DependencyGraph`. This is part of the preparation for integrating third-party package managers into the package resolution process: They can now inject further transitive dependencies into the DependencyGraph which will be fetched during resolution to create the `ResolvedGraph`. Although some error message wording changes (particularly where it previously relied on recursive calls for building up error context), the logic is otherwise unchanged. Test Plan: Existing test cases: ``` move$ cargo-nextest nextest run -E 'package(move-cli) | package(move-package)' ```
Update the README to reflect the inclusion of `DependencyGraph`, `LockFile`, and `ResolvingTable`.
…repos Another occasion where the previous implementation recursively traversed the transitive dependency graph -- the new approach discovers the graph and then iterates through its nodes to download them. The second step is currently technically unnecessary, but becomes relevant in the presence of externally resolved dependencies where third-party resolvers can introduce sub-graphs into the `DependencyGraph` without having to explore them locally. Test Plan: ``` move/language/tools/move-package$ cargo nextest ```
Adds support for dependencies that are resolved by calling out to third-party binaries. The invocation is expected to return the transitive dependency sub-graph as a lock file, which is deserialized and merged into the main graph. Externally resolved dependencies can appear in manifests but not lock files, and it is an error for a package to appear in the outputs of multiple resolvers (including the internal resolver). To differentiate this case from the previous case where the internal resolver encounters the same package via another path, packages in the DependencyGraph track their resolver via an optional field (`None` for internally resolved packages). Another consequence of this change is that the check that a dependency's name matches its package name moves from the creation of DependencyGraph to the creation of ResolvedGraph, to pick up cases where the name mismatch occurs in an externally resolved dependency. Clean-ups: - Also adds support for "progress" tests (snapshot tests for progress output). - Stricter warnings during dependency parsing, e.g. warning when a `rev` field is supplied for a local dependency and other similar cases. - Delete a stale (unused) snapshot file. Test Plan: New tests, including unit tests for the merging operation, and snapshot tests for external resolution lock file and resolved graph output: ``` move/language/tools/move-package$ cargo nextest ```
Remove the explicit unlock function and rely on the drop handler always. The only semantic difference is the order in which the locks are unlocked, but this should not be an issue (from a deadlock perspective), because the order in which they are locked is the always the same. Motivated by how easy (especially when using the `?`/try syntax) it is to accidentally skip the explicit calls to `PackageLock::unlock`. The reason why this still works is that the guards that it contains automatically unlock when they are dropped, so `PackageLock`'s drop handler should do the right thing, without need for an explicit `unlock` call. Test Plan: ``` move/language/tools/move-package$ cargo nextest ```
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Remove the explicit unlock function and rely on the drop handler always. The only semantic difference is the order in which the locks are unlocked, but this should not be an issue (from a deadlock perspective), because the order in which they are locked is the always the same.
Motivated by how easy (especially when using the
?/try syntax) it is to accidentally skip the explicit calls toPackageLock::unlock. The reason why this still works is that the guards that it contains automatically unlock when they are dropped, soPackageLock's drop handler should do the right thing, without need for an explicitunlockcall.Test Plan
Stack