Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC/WIP] Add a secondary location for a sysimage in $HOME/.julia/$VERSION/sys.so #25324

Closed
wants to merge 1 commit into from

Conversation

vchuravy
Copy link
Member

adds a secondary location for a userdefined sysimage, making it easier for users
to bake their own sysimages and use them. Furthermore this allows the update of
stdlib packages, through building a new sysimage that contains them.

This sidesteps the issue brought up in #25248, instead of removing all stdlib packages
from the sysimage, we can now rebuild the sysimage even if the other location is in a
write protected area.

I would greatly appreciate a style review since I bungled my way through the C string handling.

Currently this PR checks for the modify time of the sysimage in the bindir or homedir and uses
the newer one, trying to ensure that during development a potential stale userimage is not loaded.

adds a secondary location for a userdefined sysimage, making it easier for users
to bake their own sysimages and use them. Furthermore this allows the update of
stdlib packages, through building a new sysimage that contains them.
@fredrikekre fredrikekre added the stdlib Julia's standard library label Dec 29, 2017
@KristofferC
Copy link
Member

KristofferC commented Feb 10, 2018

It seems that now we can load packages even if they are baked into the sysimg. For example, I have Pkg3 in my sysimg:

➜  v0.7> ~/julia/julia -q
julia> using Pkg3 # <- uses sysimg Pkg3

However, when I put a Pkg3 package in my directory:

➜  v0.7> git clone https://github.com/JuliaLang/Pkg3.jl Pkg3
➜  v0.7> ~/julia/julia -q
julia> using Pkg3 # <- usese pkgdir Pkg3
[ Info: Recompiling stale cache file /Users/kristoffer/.julia/lib/v0.7/fLKp/Pkg3.ji for module Pkg3
WARNING: Method definition isless(Base.UUID, Base.UUID) in module Types at /Users/kristoffer/julia/usr/share/julia/site/v0.7/Pkg3/src/Types.jl:27 overwritten in module Types at /Users/kristoffer/.julia/v0.7/Pkg3/src/Types.jl:27.
WARNING: Method definition in(Base.VersionNumber, Base.VersionNumber) in module Types at /Users/kristoffer/julia/usr/share/julia/site/v0.7/Pkg3/src/Types.jl:207 overwritten in module Types at /Users/kristoffer/.julia/v0.7/Pkg3/src/Types.jl:207.

Pkg3 does some type piracy now which is where the method overwrite warnings comes from. But using Pkg3 will now use the code that was in my recently cloned package.

Same with e.g. SHA:

➜  v0.7 ~/julia/julia -q
julia> import SHA

julia> @which SHA.sha256("foo")
sha256(str::AbstractString) in SHA at /Users/kristoffer/julia/usr/share/julia/site/v0.7/SHA/src/SHA.jl:58

➜  v0.7 git clone https://github.com/staticfloat/SHA.jl SHA
Cloning into 'SHA'...
remote: Counting objects: 431, done.
remote: Total 431 (delta 0), reused 0 (delta 0), pack-reused 431
Receiving objects: 100% (431/431), 90.89 KiB | 340.00 KiB/s, done.
Resolving deltas: 100% (229/229), done.
➜  v0.7 ~/julia/julia -q
julia> import SHA
[ Info: Recompiling stale cache file /Users/kristoffer/.julia/lib/v0.7/SHA.ji for module SHA

julia> @which SHA.sha256("foo")
sha256(str::AbstractString) in SHA at /Users/kristoffer/.julia/v0.7/SHA/src/SHA.jl:64

@vchuravy
Copy link
Member Author

@KristofferC That's good to know that the right package is being picked up, and removes my
concern that we can't update stdlib packages, but if you recall you were quite concerned about
the user experience and having stdlib packages in the sysimage minimises load times.

So this PR still has the benefit that we can rebuild the sysimage after updating Pkg3 and continue
to have a fast loading.
.

@vchuravy vchuravy mentioned this pull request Feb 22, 2018
@vchuravy vchuravy added the triage This should be discussed on a triage call label Feb 27, 2018
@vchuravy
Copy link
Member Author

Putting this on triage to get some eyes on this and decide if and how we want to do this.

@StefanKarpinski
Copy link
Member

It's unclear to me how this blocks 1.0...

@JeffBezanson JeffBezanson removed the triage This should be discussed on a triage call label Mar 1, 2018
@vchuravy
Copy link
Member Author

vchuravy commented Mar 1, 2018

Well this has been lacking discussion and review and I was hoping triage would spark either.

@StefanKarpinski
Copy link
Member

If you want to discuss it on a triage call perhaps it would be good to join the call? Is 2:15 on Thursdays not a possible time slot for you?

@vchuravy vchuravy closed this Mar 26, 2018
@vchuravy vchuravy deleted the vc/userimg branch August 19, 2018 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Julia's standard library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants