libcdb: Load debug info and unstrip libc binary #1828
Conversation
The debug info is obtained from a debuginfod server maintained by systemtap.org. elfutils is used to unstrip the binary and merge the debug info back. Try to unstrip every libc binary downloaded by the libcdb module by default. Gallopsled#1818
|
Hmm, I'm not sure what GHA is doing to the filesystem. Sometimes it appears to be fine e.g. here. |
|
It appears to be a problem with the elfutils version installed. The version installed on focal |
| log.info_once("Using cached data from %r", cache) | ||
| return cache, True | ||
|
|
||
| def unstrip_libc(filename): |
There was a problem hiding this comment.
This needs doctests for a few examples
There was a problem hiding this comment.
"A few examples" = unstripping more than just one libc?
I've added a few more tests, is this what you had in mind?
|
Any chance for updates to address the requested changes? |
|
Since the tests seem to fail on certain buggy elfutils versions installed on the CI workers, I'm not sure how to persue with this. |
|
please try to print the full error message in case of failure, so that we can debug the buggy CI |
This will hopefully get logged during doctests.
|
So it says invalid section alignment... whatever that means.. |
|
:( should we build elfutils from source in CI to work around this? It appears to be fixed in later versions like the one in 20.10. |
I don't think we want to include functionality that requires bespoke builds / anything non-standard on 18.04 / 20.04 |
|
I think only the version packaged on 20.04 is buggy with the libc we fetch in the test. 18.04 and 20.10+ is fine. I'll try to switch to a different libc binary that works on 20.04 too for the doctests. It'd be a shame to strip this feature just because a buggy package version is being used atm. |
|
Agreed, it's a great feature to have. If it mostly-works and works in most
situations (and we can add tests) we can just deal with regressions
manually.
Pwntools used to distribute .dpkg for binutils for alternate architectures
[1] because the standard package managers didn't have e.g. mip64
cross-assembler.
[1]: https://launchpad.net/~pwntools/+archive/ubuntu/binutils
…On Tue, Jun 8, 2021 at 2:51 PM peace-maker ***@***.***> wrote:
I think only the version packaged on 20.04 is buggy with the libc we fetch
in the test. 18.04 and 20.10+ is fine. I'll try to switch to a different
libc binary that works on 20.04 too for the doctests. It'd be a shame to
strip this feature just because a buggy package version is being used atm.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1828 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APYTIFP5CK344H7OW4Z2BXDTRZYDHANCNFSM4YK3XWXA>
.
|
Use APT Pinning to select the newer elfutils version from groovy instead of the buggy 0.176 version available in focal.
|
I've fixed the CI by installing a newer elfutils package version from Ubuntu groovy explicitly now while the worker is still on focal. |
|
The CI introduced by this now breaks. I don't know why yet. |
|
Since ubuntu groovy 20.10 is EOL the repository was removed and we can't fetch the elfutils package from a newer ubuntu version anymore. The runners use 20.04 which uses an older libc than the newer ubuntu versions 21.04 and 21.10, so we can't use the same trick to just grab the newer elfutils package from those distributions. I guess we'll have to resort to building elfutils and eu-unstrip ourselves or disable those tests until the runners are updated to some newer base image. |
* libcdb: Load debug info and unstrip libc binary The debug info is obtained from a debuginfod server maintained by systemtap.org. elfutils is used to unstrip the binary and merge the debug info back. Try to unstrip every libc binary downloaded by the libcdb module by default. Gallopsled#1818 * Install elfutils on CI * Update CHANGELOG.md * Add more tests * Fix negative test * Raise an exception when eu-unstrip fails This will hopefully get logged during doctests. * Install elfutils 0.181 from groovy Use APT Pinning to select the newer elfutils version from groovy instead of the buggy 0.176 version available in focal. * Fix missing import of requests pylint? Co-authored-by: Arusekk <[email protected]>
The debug info is obtained from a debuginfod server maintained by systemtap.org. elfutils is used to unstrip the binary and merge the debug info back.
Try to unstrip every libc binary downloaded by the libcdb module by default and add a new
unstrip_libc(filename)function to handle other libc binaries.Closes #1818