Known profiling issues where unwinding fails

[IanButterworth]

Experience

Show Profile.print and most of the samples are hidden as C frames, with no depth when printed via C=true
i.e.

  127╎127   @julialib/libopenblas64_.so:?  dgemm_beta_ZEN
 5910╎5910  @julialib/libopenblas64_.so:?  dgemm_kernel_ZEN

Reason

Samples that are hidden as C frames with no depth in the Profile tree are indicative of the unwinder not succeeding to unwind back from library code through julia's code.

Potential causes

I'm just collecting comments from slack. Making dedicated issues might make sense, or just PRs to fix.

1. LLVM-libunwind doesn't have support for unwinding during prologue/epilogue

   • need to port this patch from Keno to Aarch64 deps/patches/llvm-libunwind-prologue-epilogue.patch

2. The platform for AArch64 prohibits async unwinding due to a bug in the spec

3. Apple compiles most of their libraries only with compact unwind, which by-design is not able to encode async unwinding as that makes it more compact

   • Need to get Apple to fix at their end?
   • Implement workarounds?

Quoting @vtjnash

We can make some moderately educated guesses based upon inspecting the registers and disassembling the code there, which is the usual approaches to this
E.g. if we see the instruction is a syscall, we can guess that this is libc and that we can skip this frame (just set PC=LR)
other times we can try to hope the FP points at the current frame pointer, so popping that gives the old {FP, LR}

4. dgemm kernels in openblas are missing correct unwind info. (Empty profile information on v1.11 #56327 looked like this issue but was actually a brief bug with profiling only collecting on thread 1)

Please feel free to refine this summary directly or suggest changes.

[topolarity]

@IanButterworth is (4) totally resolved at this point?

It seems like it's still pretty easy to end up with top-level dgemm_* entries if you do @profile peakflops()

[IanButterworth]

I don't think the gemm thing is fixed. I'm not aware of a PR. I think the issue got confused with a threading bug.

[mlechu]

@IanButterworth Chiming in because I was looking at this—do you know of any library with this issue other than OpenBLAS?

[gbaraldi]

System libraries like libc for example

[topolarity]

System libraries like libc for example

There's not much we can do about those right now, right? (due to (3), presumably)

It'd be especially useful to have a test case for (1), if we think we need that ported for AArch64