CSUB-1345: Migrate away from Azure in favor of Linode #5
Conversation
atodorov
force-pushed
the
ci/CSUB-1345-migrate-away-from-azure
branch
from
341e765 to
0cae581
Compare
which triggers MD039/no-space-in-links error
Running on Ubuntu 24.04 - same as gluwa/creditcoin3
the original VM was $0.56/hr in West US 3; 4 vCPU, 64 GiB RAM, Memory optimized with 512 GB disk space
atodorov
force-pushed
the
ci/CSUB-1345-migrate-away-from-azure
branch
from
0cae581 to
f45894d
Compare
|
FTR I think that the failure reported by |
atodorov
force-pushed
the
ci/CSUB-1345-migrate-away-from-azure
branch
2 times, most recently
from
f21e224 to
cbee2e5
Compare
|
FTR at https://github.com/gluwa/polkadot-sdk/actions/runs/11741349059/job/32710035675?pr=5 I see 2 unit tests failing, which however are passing over at https://github.com/gluwa/polkadot-sdk/actions/runs/11738728011/job/32701907195?pr=3. From what I can tell starting branch/commit for both PR#3 and PR#5 is the same and there are no changes related to the polkadot-sdk code base itself. IDK what contributes to the different outcomes here, nor I'm sure what do we want to do with that information. |
atodorov
requested review from
DylanVerstraete,
BradleyOlson64,
beqaabu and
rustlang-dev
The difference between the two runs is the I have been able to reproduce the same thing locally. compiling without |
what I am seeing with the move to Linode VMs is essentially this: rust-lang/cargo#9157 rust-lang/cargo#12912 B/c these new VMs have more CPU cores, 16 (new) vs 4(old), compilation is faster however this causes cargo to be overzealous and spawn too many linker processes which consume all of the available memory (on a 64 GB VM) and causes an OOM error forcing the kernel to kill the linker process and causing cargo to fail! Another alternative, which works, is using `--jobs 8`, however that is less optimal b/c it leaves unused CPU capacity and also affects the number of parallel threads when executing the test suite! WARNING: using `--release` is not an option because it breaks tests. The polkadot-sdk code uses the macro defensive! which is designed to panic when running in debug mode and multiple test scenarios rely on this behavior via #[should_panic]! WARNING: we still need the 64 GB memory!
atodorov
force-pushed
the
ci/CSUB-1345-migrate-away-from-azure
branch
from
cbee2e5 to
e335c88
Compare
atodorov
merged commit afaa29d
into
release-polkadot-v1.1.0-patch
NOTE: opening against release-polkadot-v1.1.0-patch (instead of release-polkadot-v1.1.1-patch) b/c this is what gluwa/creditcoin3 & gluwa/creditcoin3-next are compiled against).