[loader]: Re-export symbols for C embedding, rename to libjulia-internal

[staticfloat]

We first rename libjulia to libjulia-internal, and libjulialoader
to libjulia so as to reduce confusion from the outside; libjulia is
the entrypoint for all usecases, and libjulia-internal just so happens
to be the chunk that contains all the actual code and linkage to other
libraries.

To support C linking against libjulia in a natural way, we must
re-export our symbols from libjulia-internal. In order to do this we
create function trampolines from libjulia to libjulia-internal and
we define all symbols within libjulia to be filled in by the
initialization routines of libjulia-internal.

The function and data symbols are now maintained in lists stored in
src/jl_exported_*.list. These lists are consumed by cli/Makefile
where bindings are generated for them to be re-exported. The function
trampolines are all of the form:

__attribute__((visibility("hidden"))) void * jl_foo_addr;
JL_DLLEXPORT JL_NAKED void jl_foo(void) { __asm__("jmpq *%0" : : "a"(jl_foo_addr) :); };

where the jl_foo_addr is initialized within loader_export_symbols():

jl_foo_addr = dlsym(libjulia_internal, "jl_foo");

The data symbols are merely defined as void * pointers within the
loader, then imported into libjulia-internal by the C linker. This
backwards-dependency is slightly confusing, but it allows the symbols to
live within libjulia while being initialized by libjulia-internal.
At first glance it appears to be a circular dependency, however since
libjulia loads libjulia-internal through dlopen() first, then
libjulia-internal finds libjulia already loaded, the circular
dependency is a non-issue. C code that wishes to link against
libjulia should do so directly, and the values of all symbols, while
initially being NULL, will be set to their correct values after
initialization through load_libjulia_internal() is complete.

[staticfloat]

@yuyichao I requested your review because I believe this was a central issue with regards to your objections to the libjulialoader PR. With this change we should support direct C linking/embedding again.

@JeffBezanson I re-named libjulia to libjulia-internal, and libjulialoader to libjulia. Let me know if there's something you don't like.

@vtjnash I think we're going to need to come up with function trampoline analogues that can work on any assembler. So far it looks like linux32 doesn't like jmpq. Perhaps we should use jmpl on x86?

[yuyichao]

With this change we should support direct C linking/embedding again.

No, as I said, the caller is still required to provide more information than before.
(and as an aside the tls handling is still wrong)

[staticfloat]

Looks like this is ready to merge. Keno and I confirmed that you can get unwind info from within a trampoline by segfaulting within the trampoline and seeing the correct unwind trace (despite the fact that the trampoline function itself was referred to as an unknown function).

[staticfloat]

Green flush! Let's go.

[GunnarFarneback]

The embedding tests work now! There should be no extra information needed, we now use dlpath() equivalents to get the path of libjulia.so and build our relative paths to dependencies off of that.

They work with Julia in the build tree but not with the nightly binaries. More specifically, running

cd /tmp
wget https://julialangnightlies-s3.julialang.org/bin/linux/x64/julia-latest-linux64.tar.gz
tar xvfz julia-latest-linux64.tar.gz
cd ~/julia/test/embedding/
mkdir build
JULIA=/tmp/julia-6ddc7f1ecc/bin/julia BIN=build make
./build/embedding

I get

ERROR: could not load library "/tmp/julia-6ddc7f1ecc/lib/julia/../bin/../lib/julia/sys.so"
/tmp/julia-6ddc7f1ecc/lib/julia/../bin/../lib/julia/sys.so: cannot open shared object file: No such file or directory

[nalimilan]

@staticfloat I get a similar error as @GunnarFarneback above when trying to build the Fedora RPM package for Julia 1.6.0-beta. Any idea about how we can fix out of tree builds? (More precisely, I get an error when trying to load openlibm.)

[staticfloat]

Milan, can you open a new issue with all the relevant info please? Thanks!

[nalimilan]

Sure: see #39523.