Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for i686 msvc CI issues with latest Windows 2022 runner image 20250224.5.0+ #137733

Open
5 of 6 tasks
jieyouxu opened this issue Feb 27, 2025 · 1 comment · May be fixed by #137849
Open
5 of 6 tasks

Tracking issue for i686 msvc CI issues with latest Windows 2022 runner image 20250224.5.0+ #137733

jieyouxu opened this issue Feb 27, 2025 · 1 comment · May be fixed by #137849
Labels
A-CI Area: Our Github Actions CI C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC O-windows-msvc Toolchain: MSVC, Operating system: Windows O-x86_32 Target: x86 processors, 32 bit (like i686-*) T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Comments

@jieyouxu
Copy link
Member

jieyouxu commented Feb 27, 2025

This is a tracking issue for CI compilation errors related to missing SSE2 types encountered when trying to use latest Windows 2022 runner image 20250224.5.0+ on the i686-msvc-{1,2} and dist-i686-msvc jobs. Key difference for the image version is the Windows 11 SDK version bump from 10.0.22621.0 -> 10.0.26100.0 that modified UCRT headers (see below).

Context

Zulip thread: https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/spurious.20.28.3F.29.20i686.20msvc.20errors.

cc <= 1.2.15 emits /arch:IA32 for clang-cl on x86 arch Windows targets, which in conjunction with GitHub Runner Image versions Windows 2022 20250224.5.0+ seems to cause a bunch of failures related to compilation errors failing to using SSE2 types without enabling SSE2 via /arch:SSE2. msvc seems to silently accept this1 and happily emit SSE2 instruments but clang-cl will error.

As Fulgen pointed out (thanks!) on the zulip thread:

The UCRT headers are, which is where they differ (10.0.22621.0/ucrt/wchar.h doesn't have the vectorization, 10.0.26100.0/ucrt/wchar.h does)

Current mitigation attempts

Work needed to unblock using latest Windows 2022 runner image 20250224.5.0+ again

Warning

These steps are advisory and likely incomplete, we may find that we need to transitively bump cc versions in more deps.

Footnotes

  1. https://developercommunity.visualstudio.com/t/wcharh-from-SDK-100261000-cannot-be/10860122

@jieyouxu jieyouxu added C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Feb 27, 2025
@jieyouxu jieyouxu changed the title Tracking issue for i686 CI issues with latest Windows 2022 runner image 20250224.5.0+ Tracking issue for i686 msvc CI issues with latest Windows 2022 runner image 20250224.5.0+ Feb 27, 2025
@jieyouxu jieyouxu added O-windows-msvc Toolchain: MSVC, Operating system: Windows O-x86_32 Target: x86 processors, 32 bit (like i686-*) A-CI Area: Our Github Actions CI labels Feb 27, 2025
@Fulgen301
Copy link
Contributor

Fix cc to not emit /arch:IA32, but instead either omit it or explicitly set /arch:SSE2 for clang-cl on x86:

This has been fixed in the meantime, but to have it noted: Before this was fixed, C code with clang-cl was compiled without SSE2 support, whereas Rust code using that C code would be compiled with SSE2 enabled. The default function call ABI is likely fine, since extern "C" passes it all over the stack anyway, but I'm not an opsem person, maybe this could have broken something that nobody ever noticed.

bors added a commit to rust-lang-ci/rust that referenced this issue Feb 27, 2025
Use Windows 2019 for 32-bit MVSC CI jobs

Windows 2022 runner images have been updated today, which brought with itself an unfortunate problem with compiling C code (see rust-lang/cc-rs#1425 or rust-lang#137733 for more details).

This is a temporary hotfix to try to unblock our CI.

r? `@jieyouxu`

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 27, 2025
Use Windows 2019 for 32-bit MVSC CI jobs

Windows 2022 runner images have been updated today, which brought with itself an unfortunate problem with compiling C code (see rust-lang/cc-rs#1425 or rust-lang#137733 for more details).

This is a temporary hotfix to try to unblock our CI.

r? `@jieyouxu`

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 27, 2025
Fix 32-bit MSVC CI

By throwing more hardware at it. The large runners should still use the old image. This could buy us a couple of... hours? Days? Who knows.

See rust-lang#137733 for context.

r? `@ghost`

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
bors added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
…try>

Downgrade 32-bit Windows SDK to 10.0.22621.0 in CI

Reverts 6ea4823 and worksaround rust-lang#137733

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
@jieyouxu jieyouxu added the T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. label Feb 28, 2025
github-merge-queue bot pushed a commit to rust-lang/cargo that referenced this issue Feb 28, 2025
…15245)

### What does this PR try to resolve?

GitHub Runner Images 20250224.5.0+ ship Windows 11 SDK 10.0.26100+
compared to the previous Windows 11 SDK 10.0.22621, which bumped the
UCRT headers. The new UCRT headers use SSE2 types. However, `cc`
versions <= 1.2.15 emit `/arch:IA32` for `x86` Windows targets for
`clang-cl`, which causes compilation errors since `clang-cl` can't find
SSE2 types without `/arch:SSE2` specified (or defaulted). (Note that
MSVC at the time of writing silently accepts and emits instruments for
code using SSE2 types, as opposed to `clang-cl` hard error-ing).

`cc` 1.2.16 contains a fix for this problem,
rust-lang/cc-rs#1425, to correctly emit
`/arch:SSE2` instead of `/arch:IA32` to enable `clang-cl` to find the
SSE2 types. However, cargo's `cc` currently is still on 1.2.13.

To fix this for rust-lang/rust CI, we need to bump anything that
transitively relies on `cc` and tries to use `clang-cl` on `x86` Windows
targets to compile any C/C++ code that transitively use functions or
types that require SSE2 types, such as `<wchar.h>`.

### How should we test and review this PR?

The fix was initially intended for `rustc_{codegen_ssa,llvm}` `cc`, and
based on testing in rust-lang/rust#137724, I was
able to successfully build `rustc_{codegen_ssa,llvm}` with a forked `cc`
based on 1.2.15 which contains the fix from
rust-lang/cc-rs#1425. Note that in the same PR,
while the compiler build succeeded, the build of cargo itself failed
since it transitively used a `cc` *without* the fix to build
`curl-sys`[^dep-chain], which failed as one might expect (`curl-sys`
tries to build C code that uses `<wchar.h>` which runs into the same
problem). Hence, this PR is opened to bump cargo's `cc` to a `cc`
version containing the fix.

### Additional information

This `x86` Windows CI problem is:

- Discussed in
https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/spurious.20.28.3F.29.20i686.20msvc.20errors.
- Tracked by rust-lang/rust#137733.

#### `cc` changelog between 1.2.13 and 1.2.16

<details>
<summary>`cc` changes since 1.2.13 up to and including 1.2.16</summary>

#####
[1.2.16](rust-lang/cc-rs@cc-v1.2.15...cc-v1.2.16)
- 2025-02-28

###### Fixed

- force windows compiler to run in `out_dir` to prevent artifacts in cwd
(#1415)

###### Other

- use `/arch:SSE2` for `x86` target arch (#1425)
- Regenerate windows-sys binding
([#1422](rust-lang/cc-rs#1422))
- Regenerate target info
([#1418](rust-lang/cc-rs#1418))
- Add LIB var when compiling flag_check (#1417)
- Change flag ordering
([#1403](rust-lang/cc-rs#1403))
- Fix archiver detection for musl cross compilation
([#1404](rust-lang/cc-rs#1404))

#####
[1.2.15](rust-lang/cc-rs@cc-v1.2.14...cc-v1.2.15)
- 2025-02-21

###### Other

- Regenerate target info
([#1406](rust-lang/cc-rs#1406))
- Always read from all `CFLAGS`-style flags
([#1401](rust-lang/cc-rs#1401))
- Simplify the error output on failed `Command` invocation
([#1397](rust-lang/cc-rs#1397))

#####
[1.2.14](rust-lang/cc-rs@cc-v1.2.13...cc-v1.2.14)
- 2025-02-14

###### Other

- Regenerate target info
([#1398](rust-lang/cc-rs#1398))
- Add support for setting `-gdwarf-{version}` based on RUSTFLAGS
([#1395](rust-lang/cc-rs#1395))
- Add support for alternative network stack io-sock on QNX 7.1 aarch64
and x86_64 ([#1312](rust-lang/cc-rs#1312))

</details>

[^dep-chain]: I think the dep chain is something like git2 ->
libgit2-sys -> curl -> curl-sys?
jieyouxu added a commit to jieyouxu/rust that referenced this issue Feb 28, 2025
jieyouxu added a commit to jieyouxu/rust that referenced this issue Feb 28, 2025
…obzol

Bump `rustc_{codegen_ssa,llvm}` `cc` to 1.2.16 to fix `x86` Windows jobs on newest Windows SDK

Part of rust-lang#137733.

Bump `rustc_{codegen_ssa,llvm}` `cc` to 1.2.16 which contains rust-lang/cc-rs#1425 to help with rust-lang#137733. Previously tested in rust-lang#137724.

#### `cc` changelog between 1.2.13 and 1.2.16

<details>
<summary>`cc` changes since 1.2.13 up to and including 1.2.16</summary>

##### [1.2.16](rust-lang/cc-rs@cc-v1.2.15...cc-v1.2.16) - 2025-02-28

###### Fixed

- force windows compiler to run in `out_dir` to prevent artifacts in cwd (rust-lang#1415)

###### Other

- use `/arch:SSE2` for `x86` target arch (rust-lang#1425)
- Regenerate windows-sys binding ([rust-lang#1422](rust-lang/cc-rs#1422))
- Regenerate target info ([rust-lang#1418](rust-lang/cc-rs#1418))
- Add LIB var when compiling flag_check (rust-lang#1417)
- Change flag ordering ([rust-lang#1403](rust-lang/cc-rs#1403))
- Fix archiver detection for musl cross compilation ([rust-lang#1404](rust-lang/cc-rs#1404))

##### [1.2.15](rust-lang/cc-rs@cc-v1.2.14...cc-v1.2.15) - 2025-02-21

###### Other

- Regenerate target info ([rust-lang#1406](rust-lang/cc-rs#1406))
- Always read from all `CFLAGS`-style flags ([rust-lang#1401](rust-lang/cc-rs#1401))
- Simplify the error output on failed `Command` invocation ([rust-lang#1397](rust-lang/cc-rs#1397))

##### [1.2.14](rust-lang/cc-rs@cc-v1.2.13...cc-v1.2.14) - 2025-02-14

###### Other

- Regenerate target info ([rust-lang#1398](rust-lang/cc-rs#1398))
- Add support for setting `-gdwarf-{version}` based on RUSTFLAGS ([rust-lang#1395](rust-lang/cc-rs#1395))
- Add support for alternative network stack io-sock on QNX 7.1 aarch64 and x86_64 ([rust-lang#1312](rust-lang/cc-rs#1312))

</details>

r? `@Kobzol`
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Feb 28, 2025
Rollup merge of rust-lang#137788 - jieyouxu:bump-compiler-cc, r=lqd,Kobzol

Bump `rustc_{codegen_ssa,llvm}` `cc` to 1.2.16 to fix `x86` Windows jobs on newest Windows SDK

Part of rust-lang#137733.

Bump `rustc_{codegen_ssa,llvm}` `cc` to 1.2.16 which contains rust-lang/cc-rs#1425 to help with rust-lang#137733. Previously tested in rust-lang#137724.

#### `cc` changelog between 1.2.13 and 1.2.16

<details>
<summary>`cc` changes since 1.2.13 up to and including 1.2.16</summary>

##### [1.2.16](rust-lang/cc-rs@cc-v1.2.15...cc-v1.2.16) - 2025-02-28

###### Fixed

- force windows compiler to run in `out_dir` to prevent artifacts in cwd (rust-lang#1415)

###### Other

- use `/arch:SSE2` for `x86` target arch (rust-lang#1425)
- Regenerate windows-sys binding ([rust-lang#1422](rust-lang/cc-rs#1422))
- Regenerate target info ([rust-lang#1418](rust-lang/cc-rs#1418))
- Add LIB var when compiling flag_check (rust-lang#1417)
- Change flag ordering ([rust-lang#1403](rust-lang/cc-rs#1403))
- Fix archiver detection for musl cross compilation ([rust-lang#1404](rust-lang/cc-rs#1404))

##### [1.2.15](rust-lang/cc-rs@cc-v1.2.14...cc-v1.2.15) - 2025-02-21

###### Other

- Regenerate target info ([rust-lang#1406](rust-lang/cc-rs#1406))
- Always read from all `CFLAGS`-style flags ([rust-lang#1401](rust-lang/cc-rs#1401))
- Simplify the error output on failed `Command` invocation ([rust-lang#1397](rust-lang/cc-rs#1397))

##### [1.2.14](rust-lang/cc-rs@cc-v1.2.13...cc-v1.2.14) - 2025-02-14

###### Other

- Regenerate target info ([rust-lang#1398](rust-lang/cc-rs#1398))
- Add support for setting `-gdwarf-{version}` based on RUSTFLAGS ([rust-lang#1395](rust-lang/cc-rs#1395))
- Add support for alternative network stack io-sock on QNX 7.1 aarch64 and x86_64 ([rust-lang#1312](rust-lang/cc-rs#1312))

</details>

r? `@Kobzol`
@jieyouxu jieyouxu linked a pull request Mar 1, 2025 that will close this issue
bors added a commit to rust-lang-ci/rust that referenced this issue Mar 1, 2025
[WIP] Revert "Remove Win SDK 10.0.26100.0 from CI"

Part of rust-lang#137733.

This reverts commit 25617c7, the remove-latest-windows-sdk workaround from rust-lang#137753.

try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-i686-msvc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI Area: Our Github Actions CI C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC O-windows-msvc Toolchain: MSVC, Operating system: Windows O-x86_32 Target: x86 processors, 32 bit (like i686-*) T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants