Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrap: Consolidate coverage test suite steps into a single step #135097

Merged
merged 1 commit into from
Jan 7, 2025

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Jan 4, 2025

Now that I have more understanding of bootstrap steps, and a renewed distaste for unnecessary macros, I have managed to express the subtleties of the tests/coverage test suite in a single step defined in ordinary code, with no need for helper macros.

Deciding which modes to run is still a bit clunky due to limitations in existing ShouldRun/PathSet APIs, but I think it's a net improvement over having to declare several different steps to handle the suite path and aliases.

The interaction with --skip isn't as nice as I'd like, but all of the known limitations are limitations that already existed in the previous implementation.

One minor change is that by default compiletest is now invoked in coverage-run mode even when cross-compiling. However, in that situation compiletest still knows that it should skip all of the individual coverage-run tests.

r? jieyouxu (or reassign)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jan 4, 2025
@jieyouxu
Copy link
Member

jieyouxu commented Jan 4, 2025

I'm going to reroll because I'm not super sure about the PathSet matching logic. r? bootstrap

@rustbot rustbot assigned Kobzol and unassigned jieyouxu Jan 4, 2025
Copy link
Contributor

@Kobzol Kobzol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice simplification, thanks.

Something really weird is going on with the --skip logic though.
In

fn maybe_run(&self, builder: &Builder<'_>, mut pathsets: Vec<PathSet>) {
, a step is skipped only if all of its pathsets and aliases are skipped, which seems quite suspicious to me, that makes some default steps really hard to skip.

On the other hand, the logic in ensure_if_default skips the step if any of its paths is skipped, which makes much more sense to me.

At the very least, these two skip logics should probably be merged somehow.

In any case, this doesn't seem to make the status quo worse in this regard, as you stated in the PR description. I left two nits, otherwise you can r=me.

src/bootstrap/src/core/builder/tests.rs Outdated Show resolved Hide resolved
src/bootstrap/src/core/build_steps/test.rs Outdated Show resolved Hide resolved
@Zalathar Zalathar force-pushed the coverage-test-step branch from 089bd51 to 5298f85 Compare January 6, 2025 10:14
@Zalathar
Copy link
Contributor Author

Zalathar commented Jan 6, 2025

Addressed review feedback (diff).

@Kobzol
Copy link
Contributor

Kobzol commented Jan 6, 2025

Thanks! I'll approve once CI is green.

@Kobzol
Copy link
Contributor

Kobzol commented Jan 6, 2025

@bors r+

@bors
Copy link
Contributor

bors commented Jan 6, 2025

📌 Commit 5298f85 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 6, 2025
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 6, 2025
bootstrap: Consolidate coverage test suite steps into a single step

Now that I have more understanding of bootstrap steps, and a renewed distaste for unnecessary macros, I have managed to express the subtleties of the `tests/coverage` test suite in a single step defined in ordinary code, with no need for helper macros.

Deciding which modes to run is still a bit clunky due to limitations in existing ShouldRun/PathSet APIs, but I think it's a net improvement over having to declare several different steps to handle the suite path and aliases.

The interaction with `--skip` isn't as nice as I'd like, but all of the known limitations are limitations that already existed in the previous implementation.

One minor change is that by default compiletest is now invoked in `coverage-run` mode even when cross-compiling. However, in that situation compiletest still knows that it should skip all of the individual coverage-run tests.

r? jieyouxu (or reassign)
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 6, 2025
…iaskrgr

Rollup of 5 pull requests

Successful merges:

 - rust-lang#134742 (Use `PostBorrowckAnalysis` in `check_coroutine_obligations`)
 - rust-lang#134951 (Suppress host effect predicates if underlying trait doesn't hold)
 - rust-lang#135097 (bootstrap: Consolidate coverage test suite steps into a single step)
 - rust-lang#135146 (Don't enable anyhow's `backtrace` feature in opt-dist)
 - rust-lang#135157 (Move the has_errors check in rustdoc back to after TyCtxt is created)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 6, 2025
Rollup of 7 pull requests

Successful merges:

 - rust-lang#134742 (Use `PostBorrowckAnalysis` in `check_coroutine_obligations`)
 - rust-lang#134771 (Report correct `SelectionError` for `ConstArgHasType` in new solver fulfill)
 - rust-lang#134951 (Suppress host effect predicates if underlying trait doesn't hold)
 - rust-lang#135097 (bootstrap: Consolidate coverage test suite steps into a single step)
 - rust-lang#135146 (Don't enable anyhow's `backtrace` feature in opt-dist)
 - rust-lang#135153 (chore: remove redundant words in comment)
 - rust-lang#135157 (Move the has_errors check in rustdoc back to after TyCtxt is created)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 6, 2025
…iaskrgr

Rollup of 7 pull requests

Successful merges:

 - rust-lang#134742 (Use `PostBorrowckAnalysis` in `check_coroutine_obligations`)
 - rust-lang#134771 (Report correct `SelectionError` for `ConstArgHasType` in new solver fulfill)
 - rust-lang#134951 (Suppress host effect predicates if underlying trait doesn't hold)
 - rust-lang#135097 (bootstrap: Consolidate coverage test suite steps into a single step)
 - rust-lang#135146 (Don't enable anyhow's `backtrace` feature in opt-dist)
 - rust-lang#135153 (chore: remove redundant words in comment)
 - rust-lang#135157 (Move the has_errors check in rustdoc back to after TyCtxt is created)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit c31a4b9 into rust-lang:master Jan 7, 2025
6 checks passed
@rustbot rustbot added this to the 1.86.0 milestone Jan 7, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 7, 2025
Rollup merge of rust-lang#135097 - Zalathar:coverage-test-step, r=Kobzol

bootstrap: Consolidate coverage test suite steps into a single step

Now that I have more understanding of bootstrap steps, and a renewed distaste for unnecessary macros, I have managed to express the subtleties of the `tests/coverage` test suite in a single step defined in ordinary code, with no need for helper macros.

Deciding which modes to run is still a bit clunky due to limitations in existing ShouldRun/PathSet APIs, but I think it's a net improvement over having to declare several different steps to handle the suite path and aliases.

The interaction with `--skip` isn't as nice as I'd like, but all of the known limitations are limitations that already existed in the previous implementation.

One minor change is that by default compiletest is now invoked in `coverage-run` mode even when cross-compiling. However, in that situation compiletest still knows that it should skip all of the individual coverage-run tests.

r? jieyouxu (or reassign)
@Zalathar Zalathar deleted the coverage-test-step branch January 7, 2025 01:44
@Zalathar
Copy link
Contributor Author

Zalathar commented Jan 7, 2025

, a step is skipped only if all of its pathsets and aliases are skipped, which seems quite suspicious to me, that makes some default steps really hard to skip.
On the other hand, the logic in ensure_if_default skips the step if any of its paths is skipped, which makes much more sense to me.

At the very least, these two skip logics should probably be merged somehow.

Note that a single step can have multiple paths representing different tools/crates, such that the different parts of the step can be run/skipped independently.

For example, the bootstrap tools src/tools/jsondoclint and src/tools/suggest-tests are tested by the same step (test::CrateBootstrap) via different paths. But passing --skip jsondoclint shouldn't cause the tests for suggest-tests to be skipped!

@Kobzol
Copy link
Contributor

Kobzol commented Jan 7, 2025

Well, I'd probably use separate steps for these things in that case 🤷‍♂️ It would be easier to reason about if a single step did one thing and all its paths were just aliases for that one thing. But of course this wouldn't really work for granular skipping, like "skip this one UI test from the whole UI tests suite", so it's a no-go.

Still, the skip logic at those two places should probably be somehow unified, I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants