Julia 1.9: precompile prints `fatal error in type inference (type bound)` when mixing pkgimages=yes and pkgimages=no #48397

fonsp · 2023-01-23T21:59:06Z

If you precompile the same package environment twice, once with julia --pkgimages=yes, and once with julia --pkgimages=no, then calling Pkg.precompile() a third time will print lots of errors:

➜   julia19 --pkgimages=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-beta3 (2023-01-18)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.9) pkg> activate hellohello
  Activating new project at `~/temp/hellohello`

(hellohello) pkg> add HTTP
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/temp/hellohello/Project.toml`
  [cd3eb016] + HTTP v1.7.3
    Updating `~/temp/hellohello/Manifest.toml`
  [d1d4a3ce] + BitFlags v0.1.7
  [944b1d66] + CodecZlib v0.7.0
  [cd3eb016] + HTTP v1.7.3
  [83e8ac13] + IniFile v0.5.1
  [692b3bcd] + JLLWrappers v1.4.1
  [e6f89c97] + LoggingExtras v1.0.0
...
Precompiling environment...
  16 dependencies successfully precompiled in 3 seconds

(hellohello) pkg> 
➜  julia19 --pkgimages=yes
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-beta3 (2023-01-18)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.9) pkg> activate hellohello
  Activating project at `~/temp/hellohello`

(hellohello) pkg> precompile
Precompiling environment...
  16 dependencies successfully precompiled in 8 seconds

(hellohello) pkg> 
➜  julia19 --pkgimages=no 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-beta3 (2023-01-18)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.9) pkg> activate hellohello
  Activating project at `~/temp/hellohello`

(hellohello) pkg> precompile

(hellohello) pkg> ┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
┌ Error: Pkg.precompile error
│   exception =
│    fatal error in type inference (type bound)
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#212#213"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:468
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1051
│     [3] macro expansion
│       @ /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1371 [inlined]
│     [4] (::Pkg.API.var"#235#264"{Bool, Pkg.Types.Context, Vector{Task}, Pkg.API.var"#handle_interrupt#256"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#254", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, String, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API /Applications/Julia-1.9 ARM.app/Contents/Resources/julia/share/julia/stdlib/v1.9/Pkg/src/API.jl:1260
(hellohello) pkg> 

julia> 1+1
2

The text was updated successfully, but these errors were encountered:

KristofferC · 2023-01-24T16:20:05Z

Thanks! I saw this error somewhere else but didn't know how to repro it!

fredrikekre · 2023-01-24T16:36:16Z

I saw this error somewhere else

JuliaParallel/MPI.jl#707 (comment) looks to be the same.

maleadt · 2023-01-27T15:20:52Z

PkgEval is also running into this, e.g., https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2023-01/26/LLVM.against.log. I'm not sure how, as we aren't mixing precompilation images as far as I know. We do manually precompile the test environment beforehand, so maybe something goes wrong there: https://github.com/JuliaCI/PkgEval.jl/blob/1c9fa82732f50af4c62aaaf0a555cdbc476851b9/src/evaluate.jl#L381-L415

maleadt · 2023-01-27T15:48:21Z

According to PkgEval logs this started somewhere in 4e76346...1c5fa2b

Script version of reproducer:

using Pkg

julia = Base.julia_cmd().exec[1]
mktempdir() do depot
  withenv("JULIA_DEPOT_PATH" => depot) do
    run(`$julia --pkgimages=no  -e 'using Pkg; Pkg.activate("temp"; shared=true); Pkg.add("HTTP")'`)
    run(`$julia --pkgimages=yes -e 'using Pkg; Pkg.activate("temp"; shared=true); Pkg.precompile()'`)
    run(`$julia --pkgimages=no  -e 'using Pkg; Pkg.activate("temp"; shared=true); Pkg.precompile(); sleep(1)'`)
  end
end

Using that to bisect:

9de3263f8b36a43546fdb96cd873629c3b73fd7c is the first bad commit
commit 9de3263f8b36a43546fdb96cd873629c3b73fd7c
Author: DilumAluthgeBot <[email protected]>
Date:   Wed Jan 18 13:24:24 2023 -0500

    🤖 [master] Bump the Pkg stdlib from 5ae866151 to 67f39cd0a (#48335)

 deps/checksums/Pkg-5ae866151e0cfd0536d0bfbb66e7f14ea026bd31.tar.gz/md5  | 1 -
 .../Pkg-5ae866151e0cfd0536d0bfbb66e7f14ea026bd31.tar.gz/sha512          | 1 -
 deps/checksums/Pkg-67f39cd0ae2cc737d2f4a2989d8a96ec9061dbf4.tar.gz/md5  | 1 +
 .../Pkg-67f39cd0ae2cc737d2f4a2989d8a96ec9061dbf4.tar.gz/sha512          | 1 +
 stdlib/Pkg.version                                                      | 2 +-
 5 files changed, 3 insertions(+), 3 deletions(-)
 delete mode 100644 deps/checksums/Pkg-5ae866151e0cfd0536d0bfbb66e7f14ea026bd31.tar.gz/md5
 delete mode 100644 deps/checksums/Pkg-5ae866151e0cfd0536d0bfbb66e7f14ea026bd31.tar.gz/sha512
 create mode 100644 deps/checksums/Pkg-67f39cd0ae2cc737d2f4a2989d8a96ec9061dbf4.tar.gz/md5
 create mode 100644 deps/checksums/Pkg-67f39cd0ae2cc737d2f4a2989d8a96ec9061dbf4.tar.gz/sha512

KristofferC · 2023-01-27T15:56:02Z

Just to be clear, the issue was "introduced" in JuliaLang/Pkg.jl#3308 but it isn't a bug with that code per say (at least I don't think so), it seems instead that it triggers an error in inference in cases like this.

gbaraldi · 2023-01-27T16:12:02Z

Does setting a cache as stale make inference not trust it? Or is it rejected all at once?

maleadt · 2023-01-27T16:27:42Z

By looking at the IRcode I reconstructed the following MWE:

i = 3
wat() = i == 3 ? nothing : "nothing"
@noinline function default()
    if i == 1
        return true
    else
        return (Any[], wat())
    end
end

function doit(i)
    modpath = ""
    modkey = Base.PkgId("")
    modbuild_id = UInt128(1)
    cache = Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}()
    get!(cache, (Base.PkgId(""), UInt128(1), "", "")) do
        @show modpath modkey modbuild_id
        if i == 1
            return true
        else
            return (Any[], wat())
        end
    end
end

doit(2)

julia> doit(2)
modpath = ""
modkey =  [top-level]
modbuild_id = 0x00000000000000000000000000000001
ERROR: fatal error in type inference (type bound)
Stacktrace:
 [1] get!(default::var"#4#5"{Int64, UInt128, Base.PkgId, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
   @ Base ./dict.jl:469
 [2] doit(i::Int64)
   @ Main ./REPL[4]:6
 [3] top-level scope
   @ REPL[5]:1

This also fails on 1.9, but not on 1.8, so we could put this on the milestone as a regression if anybody wants.
EDIT: oh, it already is.

I'm going to reduce & bisect now.

maleadt · 2023-01-27T16:33:13Z

Reduced MWE:

i = wat() = i
cache = Dict{Tuple, Union{Bool, Tuple{Vector, String}}}()
get!(cache, ()) do
    if i == 1
        true
    else
        [], wat()
    end
end

Bisected to #44421 (cc @Keno):

96d6d866cc77572662ec5b482546981b84eb5c7c is the first bad commit
commit 96d6d866cc77572662ec5b482546981b84eb5c7c
Author: Keno Fischer <[email protected]>
Date:   Thu Mar 3 18:23:29 2022 -0500

    Make an inference hot-path slightly faster (#44421)

    This aims to improve performance of inference slightly by removing
    a dynamic dispatch from calls to `widenwrappedconditional`, which
    appears in various hot paths and showed up in profiling of inference.

    There's two changes here:

    1. Improve inlining for calls to functions of the form
    ```
    f(x::Int) = 1
    f(@nospecialize(x::Any)) = 2
    ```
    Previously, we would peel of the `x::Int` case and then
    generate a dynamic dispatch for the `x::Any` case. After
    this change, we directly emit an `:invoke` for the `x::Any`
    case (as well as enabling inlining of it in general).

    2. Refactor `widenwrappedconditional` itself to avoid a signature
    with a union in it, since ironically union splitting cannot currently
    deal with that (it can only split unions if they're manifest in the
    call arguments).

 base/compiler/ssair/inlining.jl | 73 +++++++++++++++++++++++++++++------------
 base/compiler/typelattice.jl    | 18 +++++-----
 test/compiler/inline.jl         |  9 +++++
 test/worlds.jl                  |  2 +-
 4 files changed, 72 insertions(+), 30 deletions(-)
bisect run success

#44421 changed the union-splitting to not generate a fallback dynamic dispatch call when there is any fully covered call. But It should generate it if there is any uncovered call candidate. fix #48397.

#44421 changed the union-splitting to skip generating unnecessary fallback dynamic dispatch call when there is any fully covered call. But it turned out that this is only valid when there is any fully covered call in matches for all signatures that inference split, and it is invalid if there is any union split signature against which any uncovered call is found. Consider the following example: # case 1 # def nosplit(::Any) = [...] nosplit(::Int) = [...] # call nosplit(a::Any) split1: a::Any ┬ nosplit(a::Int) └ nosplit(a::Any) # fully covers split1 # case 2 # def convert(::Type{T}, ::T) = T # call convert(::Type{Union{Bool,Tuple{Int,String}}}, a::Union{Bool,Tuple{Int,Any}}) split1: a::Bool ─ convert(::Type{Bool}, ::Bool) # fully covers split1 split2: a::Tuple{Int,Any} ─ convert(::Type{Tuple{Int,String}}, ::Tuple{Int,String}) # NOT fully covers split2 #44421 allows us to optimize the the first case, but handles the second case wrongly. This commit fixes it up while still optimizing the first case. fix #48397.

aviatesk · 2023-01-30T16:32:57Z

FWIW, the fatal error in compiler is fixed, and now it's raised as an error within Pkg.jl:

Precompiling environment...
  16 dependencies successfully precompiled in 8 seconds
  Activating project at `/var/folders/28/jpsyzp6971s99n6bd4_k6fgm0000gn/T/jl_wE65OO/environments/temp`
Precompiling environment...
  16 dependencies successfully precompiled in 18 seconds
  Activating project at `/var/folders/28/jpsyzp6971s99n6bd4_k6fgm0000gn/T/jl_wE65OO/environments/temp`
┌ Error: Pkg.precompile error
│   exception =
│    MethodError: Cannot `convert` an object of type 
│      Tuple{Vector{Any}, Nothing} to an object of type 
│      Union{Bool, Tuple{Vector{Any}, String}}
│    
│    Closest candidates are:
│      convert(::Type{T}, ::T) where T
│       @ Base Base.jl:64
│    
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#224#225"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:469
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1068
│     [3] macro expansion
│       @ [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1388 [inlined]
│     [4] (::Pkg.API.var"#244#274"{Bool, Bool, Pkg.Types.Context, Vector{Task}, Dict{Base.PkgId, String}, Pkg.API.var"#handle_interrupt#266"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#264", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1276
┌ Error: Pkg.precompile error
│   exception =
│    MethodError: Cannot `convert` an object of type 
│      Tuple{Vector{Any}, Nothing} to an object of type 
│      Union{Bool, Tuple{Vector{Any}, String}}
│    
│    Closest candidates are:
│      convert(::Type{T}, ::T) where T
│       @ Base Base.jl:64
│    
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#224#225"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:469
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1068
│     [3] macro expansion
│       @ [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1388 [inlined]
│     [4] (::Pkg.API.var"#244#274"{Bool, Bool, Pkg.Types.Context, Vector{Task}, Dict{Base.PkgId, String}, Pkg.API.var"#handle_interrupt#266"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#264", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1276
┌ Error: Pkg.precompile error
│   exception =
│    MethodError: Cannot `convert` an object of type 
│      Tuple{Vector{Any}, Nothing} to an object of type 
│      Union{Bool, Tuple{Vector{Any}, String}}
│    
│    Closest candidates are:
│      convert(::Type{T}, ::T) where T
│       @ Base Base.jl:64
│    
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#224#225"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:469
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1068
│     [3] macro expansion
│       @ [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1388 [inlined]
│     [4] (::Pkg.API.var"#244#274"{Bool, Bool, Pkg.Types.Context, Vector{Task}, Dict{Base.PkgId, String}, Pkg.API.var"#handle_interrupt#266"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#264", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1276
┌ Error: Pkg.precompile error
│   exception =
│    MethodError: Cannot `convert` an object of type 
│      Tuple{Vector{Any}, Nothing} to an object of type 
│      Union{Bool, Tuple{Vector{Any}, String}}
│    
│    Closest candidates are:
│      convert(::Type{T}, ::T) where T
│       @ Base Base.jl:64
│    
│    Stacktrace:
│     [1] get!(default::Pkg.API.var"#224#225"{UInt128, Base.PkgId, String, String}, h::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, key::Tuple{Base.PkgId, UInt128, String, String})
│       @ Base ./dict.jl:469
│     [2] _is_stale!(stale_cache::Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, paths::Vector{String}, sourcepath::String)
│       @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1068
│     [3] macro expansion
│       @ [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1388 [inlined]
│     [4] (::Pkg.API.var"#244#274"{Bool, Bool, Pkg.Types.Context, Vector{Task}, Dict{Base.PkgId, String}, Pkg.API.var"#handle_interrupt#266"{Base.Event, ReentrantLock, Base.TTY}, Pkg.API.var"#color_string#264", Base.Event, Base.Event, ReentrantLock, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, String}, Vector{Base.PkgId}, Vector{Base.PkgId}, Dict{Base.PkgId, Bool}, Dict{Base.PkgId, Base.Event}, Dict{Base.PkgId, Bool}, Vector{Pkg.Types.PackageSpec}, Dict{Base.PkgId, String}, Dict{Tuple{Base.PkgId, UInt128, String, String}, Union{Bool, Tuple{Vector{Any}, String}}}, Vector{Base.PkgId}, Bool, Base.TTY, Base.Semaphore, String, Vector{String}, Vector{Base.PkgId}, Base.PkgId})()
│       @ Pkg.API ./task.jl:514
└ @ Pkg.API [...]/julia/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1276
Process(`[...]/julia/usr/bin/julia --pkgimages=no -e 'using Pkg; Pkg.activate("temp"; shared=true); Pkg.precompile(); sleep(1)'`, ProcessExited(0))

maleadt · 2023-01-31T08:12:22Z

@KristofferC Should we track the remaining issue here (on the milestone), or on the Pkg.jl repository?

KristofferC transferred this issue from JuliaLang/Pkg.jl Jan 24, 2023

KristofferC added this to the 1.9 milestone Jan 24, 2023

KristofferC added the compiler:inference Type inference label Jan 24, 2023

KristofferC changed the title ~~Julia 1.9: precompile prints errors when mixing pkgimages=yes and pkgimages=no~~ Julia 1.9: precompile prints fatal error in type inference (type bound) when mixing pkgimages=yes and pkgimages=no Jan 24, 2023

KristofferC changed the title ~~Julia 1.9: precompile prints fatal error in type inference (type bound) when mixing pkgimages=yes and pkgimages=no~~ Julia 1.9: precompile prints fatal error in type inference (type bound) when mixing pkgimages=yes and pkgimages=no Jan 24, 2023

aviatesk mentioned this issue Jan 30, 2023

inlining: make union splitting account for uncovered call #48455

Merged

aviatesk closed this as completed in #48455 Jan 30, 2023

DilumAluthge mentioned this issue Feb 1, 2023

fix stale_cache calculation JuliaLang/Pkg.jl#3353

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Julia 1.9: precompile prints `fatal error in type inference (type bound)` when mixing pkgimages=yes and pkgimages=no #48397

Julia 1.9: precompile prints `fatal error in type inference (type bound)` when mixing pkgimages=yes and pkgimages=no #48397

fonsp commented Jan 23, 2023 •

edited by KristofferC

Loading

KristofferC commented Jan 24, 2023

fredrikekre commented Jan 24, 2023

maleadt commented Jan 27, 2023

maleadt commented Jan 27, 2023

KristofferC commented Jan 27, 2023 •

edited

Loading

gbaraldi commented Jan 27, 2023

maleadt commented Jan 27, 2023 •

edited

Loading

maleadt commented Jan 27, 2023

aviatesk commented Jan 30, 2023

maleadt commented Jan 31, 2023 •

edited

Loading

Julia 1.9: precompile prints fatal error in type inference (type bound) when mixing pkgimages=yes and pkgimages=no #48397

Julia 1.9: precompile prints fatal error in type inference (type bound) when mixing pkgimages=yes and pkgimages=no #48397

Comments

fonsp commented Jan 23, 2023 • edited by KristofferC Loading

KristofferC commented Jan 24, 2023

fredrikekre commented Jan 24, 2023

maleadt commented Jan 27, 2023

maleadt commented Jan 27, 2023

KristofferC commented Jan 27, 2023 • edited Loading

gbaraldi commented Jan 27, 2023

maleadt commented Jan 27, 2023 • edited Loading

maleadt commented Jan 27, 2023

aviatesk commented Jan 30, 2023

maleadt commented Jan 31, 2023 • edited Loading

Julia 1.9: precompile prints `fatal error in type inference (type bound)` when mixing pkgimages=yes and pkgimages=no #48397

Julia 1.9: precompile prints `fatal error in type inference (type bound)` when mixing pkgimages=yes and pkgimages=no #48397

fonsp commented Jan 23, 2023 •

edited by KristofferC

Loading

KristofferC commented Jan 27, 2023 •

edited

Loading

maleadt commented Jan 27, 2023 •

edited

Loading

maleadt commented Jan 31, 2023 •

edited

Loading