Root the function object in jlcall

[yuyichao]

It took me a while to figure out but this is what causing a segfault for the following code with threading in #14190 .

f is a global function but has a closure specialization. It's address is inlined by the caller but is then replaced by a new method, causing a segfault when calling the old function when it's trying to access the closure variables. In some sense this is a consequence of #265 since the method is otherwise guaranteed to be rooted in the method table. This might make the gc root list bigger but until #265 is fixed, this is the only fix I can come up with.

using Base.Threads

ex = quote
    for j in 1:10
        println(j)
        @threads all for i in 1:10
            global f() = i
            gc(false)
            f()
        end
    end
end

for k in 1:10000
    eval(ex)
end

[yuyichao]

Huh?...... Travis/pr lin32 has a strange ENOSPC failure.....

Exception running test file :
On worker 2:
LoadError: start_watching (FileMonitor): no space left on device (ENOSPC)
 [inlined code] from stream.jl:1024
 in start_watching at poll.jl:295
 in wait at poll.jl:355
 in test_monitor_wait at /tmp/julia/share/julia/test/file.jl:162
 [inlined code] from essentials.jl:114
 in include_string at loading.jl:355
 in include_from_node1 at ./loading.jl:395
 [inlined code] from util.jl:179
 in runtests at /tmp/julia/share/julia/test/testdefs.jl:7
 in anonymous at /tmp/julia/share/julia/test/runtests.jl:36
 in anonymous at multi.jl:1005
 in run_work_thunk at multi.jl:709
 [inlined code] from multi.jl:1005
 in anonymous at task.jl:63
while loading /tmp/julia/share/julia/test/file.jl, in expression starting on line 182
ERROR: LoadError: Some tests exited with errors.
 in anonymous at /tmp/julia/share/julia/test/runtests.jl:64
 in cd at file.jl:47
 in include at ./boot.jl:260
 in include_from_node1 at ./loading.jl:392
 in process_options at ./client.jl:277
 in _start at ./client.jl:377
while loading /tmp/julia/share/julia/test/runtests.jl, in expression starting on line 13

[tkelman]

probably hitting a disk quota due to the new caching stuff. still iterating on it a bit.

[yuyichao]

Added a non-threading repro as test...

[yuyichao]

Comment on this one?

[yuyichao]

The additional root shouldn't be too bad since both the array and the function are likely old anyway. From the sysimg size it also looks OK (sys.so increases by ~5kB).

I'll merge on Friday if no one has a better solution.

[JeffBezanson]

I do have a slightly better solution, but on jb/functions :)

[yuyichao]

I do have a slightly better solution, but on jb/functions :)

Cool. Feel free to close this if jb/functions is ready before Friday. ;-P

Actually what's the better solution? Fixing #265?

[tkelman]

This causes segfaults in jl_restore_incremental in a bunch of packages when backported to release-0.4, so removing backport pending label unless you can come up with a better idea.

[yuyichao]

@tkelman I couldn't reproduce the segfault in FixedPointNumbers but I could with the Cbc tests or using JuMP. The segfault happens on the tag of a singleton value (Base.DotMulFun() in this case) being 0x20. @vtjnash pointed out that this should be fixed recently by the commit a1e98be on jb/functions. I'm not exactly sure about the detail but the commit seems to apply on 0.4 cleanly and fixes the symptom locally.

Add back backport label. Can wait until next release. The commit linked above is probably a good candidate too.

c.c. @JeffBezanson

[JeffBezanson]

Yes, that commit should definitely be applied to master and 0.4; I tried to keep it isolated for that purpose.

[tkelman]

The FixedPointNumbers failure was a terminate timeout, that package's tests take forever if inlining is off.

[yuyichao]

I tried to keep it isolated for that purpose.

That was very helpful! Might worth adding tests if there's any simpler and deterministic repro.

The FixedPointNumbers failure was a terminate timeout.

Ahh, that makes sense then. I was also wondering why the backtrace is different.....

[coveralls]

Changes Unknown when pulling f95ffcd on yyc/gc/jlcall-root into ** on master**.