Restore Windows CI

[tersec]

#5410

[tersec]

The core constraint here is that currently, the GitHub Action runners are 8GB, and recent commits exceed that with Nim 1.6. However, the Nim 2.0 compiler runs using the ORC memory manager, regardless of whether it generates code using refc or ORC.

With Nim 1.6 on 64-bit Linux (granted, not Windows), this is probably close to happening too:

$ ./env.sh nim --version
Nim Compiler Version 1.6.14 [Linux: arm64]
Compiled at 2023-09-10
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: 71ba2e7f3c5815d956b1ae0341b0743242b8fec6
active boot switches: -d:release
$ ./env.sh /usr/bin/time -f"%M %e" nim c -c --hints:off --warnings:off beacon_chain/nimbus_beacon_node.nim
7306780 308.05
$ ./env.sh /usr/bin/time -f"%M %e" nim c -c --hints:off --warnings:off tests/all_tests.nim
7746576 368.45

For Linux and macOS those, it's not as bad to disable GitHub Actions, because Jenkins, in theory, covers the same CI. It's nice to have both, but not critical (except that currently, unstable CI branch isn't running Jenkins at all, which stopped at some point without obvious reason and would need to be restored).

One approach would be to, at least for Windows for now, only build that in GitHub Actions using Nim 2.0 (and refc, not ORC), which should since #5142 work. This peaks at

$ ./env.sh nim --version
Nim Compiler Version 2.0.1 [Linux: arm64]
Compiled at 2023-09-10
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: 18e62ad13699cc054e087fd63e75183d3076de60
active boot switches: -d:release
$ ./env.sh /usr/bin/time -f"%M %e" nim c -c --hints:off --warnings:off beacon_chain/nimbus_beacon_node.nim && ./env.sh /usr/bin/time -f"%M %e" nim c -c --hints:off --warnings:off tests/all_tests.nim && exit
6538584 237.66
6467808 278.78

which peaks about a GB lower RSS for both the largest builds, nimbus_beacon_node and all_tests.

So one least-bad approach might be to re-enable Windows GitHub Action builds using Nim 2.0, even though it's not what ships; it at least keeps Windows nimbus-eth2 built and tested.

[tersec]

#5413

[zah]

Why are the action runners limited to 8GB of memory? There must be Windows projects out there that need more memory. Are they all using on-premise runners?

[tersec]

Why are the action runners limited to 8GB of memory? There must be Windows projects out there that need more memory. Are they all using on-premise runners?

I don't know, just empirically that this is true and has been true since #4504:

Run if [[ 'amd64' == 'amd64' ]]; then
Number of cores: 2
Physical memory: TotalPhysicalMemory  
7515721728           

Partition sizes: Name                   Size          Type                     
Disk #1, Partition #0  15030288384   Installable File System  
Disk #0, Partition #0  524288000     Installable File System  
Disk #0, Partition #1  274351521792  Installable File System  

This output is visible in the "Derive environment variables" section of any of the Windows GitHub Action CI logs.

[tersec]

Fixed by #5413 and #5419

[jangko]

probably can switch to llvm-mingw for faster runtime in Windows CI. see status-im/nimbus-eth1#1740

[tersec]

probably can switch to llvm-mingw for faster runtime in Windows CI. see status-im/nimbus-eth1#1740

Faster CI would be welcome, and if using llvm-mingw helps, that seems like a useful thing to do, but the issue here is purely due to memory usage from the nim c --compileOnly/nim c -c commands, in a single-threaded way. It's not a function in any way of the C compiler used.

This is not, generically, not a new issue; #4513 and #4531 ameliorated it 8 months ago by ensuring that the largest three build targets (all_tests, EF nimbus_beacon_node, and Gnosis nimbus_beacon_node) didn't build concurrently with any others, but that approach is now exhausted.