The embedded bitcode should always be prepared for LTO/ThinLTO #133250

DianQK · 2024-11-20T14:24:34Z

There are currently two methods for generating bitcode that used for LTO. One method involves using -C linker-plugin-lto to emit object files as bitcode, which is the typical setting used by cargo. The other method is through -C embed-bitcode=yes.

When using with -C embed-bitcode=yes -C lto=no, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module.

This PR is doing something similar to LLVM's buildFatLTODefaultPipeline, obtaining the bitcode for embedding after running buildThinLTOPreLinkDefaultPipeline.

r? nikic

DianQK · 2024-11-20T14:25:35Z

@bors try @rust-timer queue

[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost

bors · 2024-11-20T14:26:46Z

⌛ Trying commit e91e0c7 with merge 5f7a0d8...

bors · 2024-11-20T14:41:04Z

💔 Test failed - checks-actions

DianQK · 2024-11-21T13:01:50Z

@bors try @rust-timer queue

bors · 2024-11-21T13:03:02Z

⌛ Trying commit fff2da3 with merge 5a6a7f3...

[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost

bors · 2024-11-21T14:46:22Z

☀️ Try build successful - checks-actions
Build commit: 5a6a7f3 (5a6a7f3cfdb88d3331274cd6b5d737c993dd8a98)

rust-timer · 2024-11-21T16:41:40Z

Finished benchmarking commit (5a6a7f3): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.4%, 0.7%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.5%	[-2.4%, -0.5%]	4
All ❌✅ (primary)	0.5%	[0.4%, 0.7%]	2

Max RSS (memory usage)

Results (primary -0.2%, secondary 2.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.1%	[2.0%, 2.3%]	2
Regressions ❌ (secondary)	2.3%	[2.3%, 2.3%]	1
Improvements ✅ (primary)	-5.0%	[-5.0%, -5.0%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.2%	[-5.0%, 2.3%]	3

Cycles

Results (secondary -1.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.9%	[-1.9%, -1.9%]	1
All ❌✅ (primary)	-	-	0

Binary size

Results (primary 0.3%, secondary -2.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.3%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-2.1%, -2.1%]	1
All ❌✅ (primary)	0.3%	[0.3%, 0.3%]	4

Bootstrap: 796.195s -> 796.003s (-0.02%)
Artifact size: 335.92 MiB -> 335.87 MiB (-0.02%)

rustbot · 2024-11-24T14:06:12Z

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

bors · 2025-02-22T14:01:34Z

☔ The latest upstream changes (presumably #136428) made this pull request unmergeable. Please resolve the merge conflicts.

nikic · 2025-02-28T10:58:07Z

@bors r+

bors · 2025-02-28T10:58:10Z

📌 Commit a897cc0 has been approved by nikic

It is now in the queue for this repository.

bors · 2025-02-28T10:58:10Z

🌲 The tree is currently closed for pull requests below priority 100. This pull request will be tested once the tree is reopened.

ehuss · 2025-02-28T17:37:57Z

@bors p=4

Trying to keep some queue fairness since this PR is a bit older.

bors · 2025-03-01T04:11:11Z

⌛ Testing commit a897cc0 with merge cd3b56a...

The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`. When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module. This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`. r? nikic

rust-log-analyzer · 2025-03-01T04:57:01Z

The job dist-i686-msvc failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[RUSTC-TIMING] tracing_subscriber test:false 0.315
error: could not compile `tracing-subscriber` (lib)

Caused by:
  process didn't exit successfully: `D:\a\rust\rust\build\bootstrap\debug\rustc 'D:\a\rust\rust\build\bootstrap\debug\rustc' --crate-name tracing_subscriber --edition=2018 'C:\Users\runneradmin\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\tracing-subscriber-0.3.18\src\lib.rs' --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="ansi"' --cfg 'feature="default"' --cfg 'feature="fmt"' --cfg 'feature="nu-ansi-term"' --cfg 'feature="registry"' --cfg 'feature="sharded-slab"' --cfg 'feature="smallvec"' --cfg 'feature="std"' --cfg 'feature="thread_local"' --cfg 'feature="tracing-log"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("alloc", "ansi", "chrono", "default", "env-filter", "fmt", "json", "local-time", "matchers", "nu-ansi-term", "once_cell", "parking_lot", "regex", "registry", "serde", "serde_json", "sharded-slab", "smallvec", "std", "thread_local", "time", "tracing", "tracing-log", "tracing-serde", "valuable", "valuable-serde", "valuable_crate"))' -C metadata=63a427f8a09357b9 -C extra-filename=-d16693f2aed6feb2 --out-dir 'D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps' --target i686-pc-windows-msvc -L 'dependency=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps' -L 'dependency=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\release\deps' --extern 'nu_ansi_term=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libnu_ansi_term-3c3fad9e262d0799.rmeta' --extern 'sharded_slab=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libsharded_slab-d084c8eed8d7005f.rmeta' --extern 'smallvec=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libsmallvec-5cdee472287294fe.rmeta' --extern 'thread_local=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libthread_local-6f93ae252346acc4.rmeta' --extern 'tracing_core=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libtracing_core-1cff6475cfdfed68.rmeta' --extern 'tracing_log=D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-tools\i686-pc-windows-msvc\release\deps\libtracing_log-97d2fda80f2a2cdd.rmeta' --cap-lints allow --cfg=bootstrap --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=cfg(bootstrap)' '--check-cfg=cfg(llvm_enzyme)' '--check-cfg=cfg(rust_analyzer)' -Zmacro-backtrace -Csplit-debuginfo=packed -Ctarget-feature=+crt-static '-Clink-arg=-LIBPATH:D:\a\rust\rust\build\i686-pc-windows-msvc\llvm\lib' -Alinker-messages -Zunstable-options -Z binary-dep-depinfo` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)
[RUSTC-TIMING] anyhow test:false 1.059
[RUSTC-TIMING] syn test:false 8.935
[RUSTC-TIMING] clap_builder test:false 11.029
Build completed unsuccessfully in 0:39:06

bors · 2025-03-01T04:57:39Z

💔 Test failed - checks-actions

DianQK · 2025-03-01T05:21:57Z

@bors retry

bors · 2025-03-01T08:22:22Z

⌛ Testing commit a897cc0 with merge 0c72c0d...

bors · 2025-03-01T11:42:11Z

☀️ Test successful - checks-actions
Approved by: nikic
Pushing 0c72c0d to master...

rust-timer · 2025-03-01T12:58:08Z

Finished benchmarking commit (0c72c0d): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.7%	[-2.9%, -0.6%]	2
All ❌✅ (primary)	0.4%	[0.4%, 0.4%]	1

Max RSS (memory usage)

Results (primary 1.5%, secondary 0.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.5%	[1.5%, 1.5%]	1
Regressions ❌ (secondary)	3.3%	[3.3%, 3.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.2%	[-2.2%, -2.2%]	1
All ❌✅ (primary)	1.5%	[1.5%, 1.5%]	1

Cycles

Results (secondary -2.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.3%	[-2.3%, -2.3%]	1
All ❌✅ (primary)	-	-	0

Binary size

Results (primary 0.3%, secondary -0.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.0%, 0.7%]	66
Regressions ❌ (secondary)	0.9%	[0.0%, 5.3%]	21
Improvements ✅ (primary)	-0.2%	[-0.4%, -0.1%]	8
Improvements ✅ (secondary)	-0.5%	[-2.9%, -0.3%]	38
All ❌✅ (primary)	0.3%	[-0.4%, 0.7%]	74

Bootstrap: 772.769s -> 773.605s (0.11%)
Artifact size: 361.98 MiB -> 361.94 MiB (-0.01%)

rustbot added A-run-make Area: port run-make Makefiles to rmake.rs S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 20, 2024

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Nov 20, 2024

bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 20, 2024

Auto merge of rust-lang#133250 - DianQK:embed-bitcode-pgo, r=<try>

5f7a0d8

[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 20, 2024

DianQK force-pushed the embed-bitcode-pgo branch from e91e0c7 to 653500f Compare November 21, 2024 00:24

This comment has been minimized.

Sign in to view

DianQK force-pushed the embed-bitcode-pgo branch from 653500f to fff2da3 Compare November 21, 2024 12:48

This comment has been minimized.

Sign in to view

bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 21, 2024

Auto merge of rust-lang#133250 - DianQK:embed-bitcode-pgo, r=<try>

5a6a7f3

[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO Fixes rust-lang#115344. Fixes rust-lang#117220. r? ghost

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Nov 21, 2024

DianQK force-pushed the embed-bitcode-pgo branch from fff2da3 to 807def3 Compare November 24, 2024 14:05

rustbot assigned nikic Nov 24, 2024

DianQK marked this pull request as ready for review November 24, 2024 14:06

DianQK changed the title ~~[WIP] The embedded bitcode should always be prepared for LTO/ThinLTO~~ The embedded bitcode should always be prepared for LTO/ThinLTO Nov 24, 2024

DianQK added 5 commits February 23, 2025 21:23

The embedded bitcode should always be prepared for LTO/ThinLTO

1a99ca8

Document bitcode in OutputType

f32ca1a

Add new_regular and new_allocator to ModuleCodegen

9431427

Save pre-link bitcode to ModuleCodegen

da50297

Remove unused OutputType::ThinLinkBitcode

a897cc0

DianQK force-pushed the embed-bitcode-pgo branch from fa89142 to a897cc0 Compare February 23, 2025 13:24

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 28, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 1, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 1, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 1, 2025

bors merged commit 0c72c0d into rust-lang:master Mar 1, 2025
7 checks passed

rustbot added this to the 1.87.0 milestone Mar 1, 2025

DianQK deleted the embed-bitcode-pgo branch March 1, 2025 12:23

rustbot removed the perf-regression Performance regression. label Mar 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The embedded bitcode should always be prepared for LTO/ThinLTO #133250

The embedded bitcode should always be prepared for LTO/ThinLTO #133250

DianQK commented Nov 20, 2024 •

edited

Loading

DianQK commented Nov 20, 2024

This comment has been minimized.

bors commented Nov 20, 2024

This comment has been minimized.

This comment has been minimized.

bors commented Nov 20, 2024

This comment has been minimized.

DianQK commented Nov 21, 2024

This comment has been minimized.

bors commented Nov 21, 2024

bors commented Nov 21, 2024

This comment has been minimized.

rust-timer commented Nov 21, 2024

rustbot commented Nov 24, 2024

bors commented Feb 22, 2025

nikic commented Feb 28, 2025

bors commented Feb 28, 2025

bors commented Feb 28, 2025

ehuss commented Feb 28, 2025

bors commented Mar 1, 2025

rust-log-analyzer commented Mar 1, 2025

bors commented Mar 1, 2025

DianQK commented Mar 1, 2025

bors commented Mar 1, 2025

bors commented Mar 1, 2025

rust-timer commented Mar 1, 2025

The embedded bitcode should always be prepared for LTO/ThinLTO #133250

The embedded bitcode should always be prepared for LTO/ThinLTO #133250

Conversation

DianQK commented Nov 20, 2024 • edited Loading

DianQK commented Nov 20, 2024

This comment has been minimized.

bors commented Nov 20, 2024

This comment has been minimized.

This comment has been minimized.

bors commented Nov 20, 2024

This comment has been minimized.

DianQK commented Nov 21, 2024

This comment has been minimized.

bors commented Nov 21, 2024

bors commented Nov 21, 2024

This comment has been minimized.

rust-timer commented Nov 21, 2024

Overall result: ❌✅ regressions and improvements - please read the text below

rustbot commented Nov 24, 2024

bors commented Feb 22, 2025

nikic commented Feb 28, 2025

bors commented Feb 28, 2025

bors commented Feb 28, 2025

ehuss commented Feb 28, 2025

bors commented Mar 1, 2025

rust-log-analyzer commented Mar 1, 2025

bors commented Mar 1, 2025

DianQK commented Mar 1, 2025

bors commented Mar 1, 2025

bors commented Mar 1, 2025

rust-timer commented Mar 1, 2025

Overall result: ❌✅ regressions and improvements - no action needed

DianQK commented Nov 20, 2024 •

edited

Loading