Try to work around Debian kernel weirdness #238
Conversation
For some reason, the arm64 testbot is booting an old version of a -backports kernel, and failing to find headers because it's, well, old. So let's shove that into a separate command, and add an attempted workaround to try to install the right thing anyway, until we figure out why the AMI is doing something backwards... Signed-off-by: Rich Ercolani <[email protected]>
|
When the buildbot spins up an AMI it calls the This sure looks like a problem with the One option which would probably work is to move to Debian Bullseye now that's it's been released. There doesn't appear to be any issue with the repositories there. However, we'll may run in to other minor issues. For example, it does look like some of the names of packages we need to install have changed. Presumably the Debian |
|
That's curious. (Why does it need the newer kernel?) Sigh. I've filed a bug against Debian. In the interim, we could just fetch the old package, as I suggested, or hopping to Debian 11 would also work. |
|
My recollection is the newer kernel was primarily to get better performance in ec2. We could probably do without. Let me try and roll things forward to Debian 11. A test build went fine, the CI is idle, and we should move forward anyway. |
|
Moving forward didn't go as smoothly as I'd hoped. In the end I opted to make two CI changes to resolve these failures. I switched up back to the default kernel for aarch64, and I updated the bootstrap script so we always reboot on to the newest kernel. In practice we were already doing this for almost all of the builders anyway. 5b320ac Always reboot Linux builders on to the latest kernel |
|
Closing. Switching to the default kernel worked as expected and resolved this issue. |
Looking at this, it fails to build later because zlib1g-dev didn't get installed when the huge apt install failed because of this:
Which is curious, because the buildslave AMI itself is running 4.19.0-11-arm64, and the x86_64 testslave is getting a non-bpo kernel...
Unfortunately, I can't really see into the AMI instance that's spawned, so I can't tell what AMI it was running or why that has a BPO kernel and the x86_64 does not, just that in the ones that succeeded before it really is using those headers and not, say, the kernel version the buildbot has. Since the buildslave's AMIs are the only ones I see mentioned in the repo, I can't see where it would be getting one that is flawed like that...
So here's a workaround. Note that this should not be merged as-is - if this is going to happen, the relevant package files should be rehosted somewhere and that location pointed to instead, because otherwise the snapshot "mirror" is probably going to start banning hosts for hitting it too much.