pahole: patch to force single-threaded mode if reproducibility is desired

[delroth]

Description of changes

#223584 #227800

pahole processes DWARF -> BTF in a pseudo-random order if multi
threading is enabled (default in the Linux kernel). This can be fixed in dependent
derivation by making sure users don't ever pass "-j" or "-j N", but it
seems easier and safer to just detect whether reprodubility is desired
(by proxy: if SOURCE_DATE_EPOCH is set) and ignore the multi-threading
flag.

Tested with a zfs --check build, it passes with this patch applied!

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[delroth]

@ofborg build linuxPackages.zfs linux_latest

[martinetd]

I guess it makes sense -- have you brought it up with upstream?
I'm sure the merge process could be made reproducible if they tried.

[delroth]

I guess it makes sense -- have you brought it up with upstream? I'm sure the merge process could be made reproducible if they tried.

I think it could be but it would not be trivial from what I can tell -- the threads just grab DWARF info from the input in an uncoordinated fashion, so you'd need some major refactoring to split up the input in a consistent way, or to sort the output while merging.

I've not communicated with upstream at all at this point.

[martinetd]

I've opened acmel/dwarves#42 for now

[delroth]

Thanks a lot for the insightful review, I've updated this PR:

• Comment in the patch updated to mention it's forcing single-threaded mode, not just changing the default.
• Commit message is updated to match.
• Commit message also mentions that the issue comes from user (in this case the Linux kernel build scripts) using -j or -jN.

[martinetd]

Thanks!

Unless there's any reason to hurry let's give upstream a couple of days to reply (at least Arnaldo does this for work so we should wait for weekdays); otherwise LGTM

To whoever merges this: might want to retarget staging instead given number of packages involved

[delroth]

Unless there's any reason to hurry let's give upstream a couple of days to reply (at least Arnaldo does this for work so we should wait for weekdays); otherwise LGTM

No objection, I don't think there's any hurry. @raboof any particular timeline we should try to keep in mind re: reproducible builds and 23.05 release?

To whoever merges this: might want to retarget staging instead given number of packages involved

Done.

[raboof]

Great find! Does this slow down the kernel build or is the parallelism inconsequential here?

Unless there's any reason to hurry let's give upstream a couple of days to reply (at least Arnaldo does this for work so we should wait for weekdays); otherwise LGTM

No objection, I don't think there's any hurry. @raboof any particular timeline we should try to keep in mind re: reproducible builds and 23.05 release?

I don't think we expect to fix #188978 for 23.05, so I agree there's no hurry and we can take this at its 'natural pace' 👍

[martinetd]

Great find! Does this slow down the kernel build or is the parallelism inconsequential here?

This is more a developer comfort thing imo -- from the commit message in acmel/dwarves@f104790 it should be something like 20s -> 5s

[RaitoBezarius]

LGTM