Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

type #pynamespace has no field mod error #505

Closed
marius311 opened this issue Jun 11, 2020 · 4 comments
Closed

type #pynamespace has no field mod error #505

marius311 opened this issue Jun 11, 2020 · 4 comments
Labels
needs reproducer Bug report can't be verified without reproducible test case

Comments

@marius311
Copy link

marius311 commented Jun 11, 2020 •
edited
Loading

I'm getting the following error on Julia 1.4.2 / Revise 2.7.2 which is 100% reproducible and doesn't involve changing any code, its just triggered when these packages are loaded. Is this error message enough for an expert to dechiper what's going on? If not let me know and I will work on coming up with a MWE.

The crux of it is that I have something like

module Ultradeep100d
@init @require Py2Call begin
   using .Py2Call
   @eval py2"..."
end
end

where Py2Call is another of my packages that provides the py2"..." string macro (which calls Python 2 using PyCall and an auxilliary Julia worker), the macro itself expanding to include another call to @eval and PyCall.@py_str.

To trigger the bug I do using Revise, Ultradeep100d, Py2Call and I get the error below. The reference to pynamespace in the actual error must be this one.

Again, let me know if a MWE is needed.

julia> using Revise, Ultradeep100d, Py2Call
Revise.LogRecord(Error, evaluation error starting at /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:335, lowered, Revise_e943ed8b, "/global/homes/m/marius/.julia/packages/Revise/tV8FE/src/lowered.jl", 106, (mod=Ultradeep100d, ex=begin
    begin
        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:333 =#
        using Py2Call
        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:335 =#
        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:335 =# @eval begin
                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:337 =#
                py2"import sptpol_software.observation.sky as sky
from sptpol_software.observation.sky import Map
"
                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:342 =#
                function readFromHDF5(filename; T = Float32, removeWeight = true, flattenPol = true)
                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:343 =#
                    py2"m = sky.Map.readFromHDF5($filename)$$((removeWeight ? \".removeWeight()\" : \"\") * (flattenPol ? \".flattenPol()\" : \"\"))
"
                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:346 =#
                    θpix = 60 * rad2deg(py2"m.pol_maps[0].getResoRad()")
                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:347 =#
                    if θpix ≈ round(Int, θpix)
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:348 =#
                        θpix = round(Int, θpix)
                    end
                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:350 =#
                    FlatIQUMap((T.(py2"m[$x].map") for x = ["T", "Q", "U"])..., θpix = θpix)
                end
                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:354 =#
                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:354 =# @doc "    load_ultradeep_data(k, i=0; kwargs...)\n    \nLoads ultradeep data, or a datasplit (like `[halfA, halfB]`). The argument\n`split` controls how (if at all) to split the data, and `diff` controls\nwhether to return each split, or to difference them. \n\nArguments: \n* `k`: one of `[\"100d\",\"500dLT\",\"500dFull\",\"500d\",\"ultradeep\"]`\n* `i`: index of the noise realization (irrelevant if `split==false` since they all coadd to the same thing)\n\nKeyword arguments: \n* `pol`: which polarization to load, one of `[:I,:P,:IP]` (default: `:IP`)\n* `split`: one of `false` (full data coadd; default), `:halves` (first/second half), or `:lr` (left/right)\n* `diff`: `true` to difference the two splits or `false` to return them both (default: `false`)\n* `θpix`: resolution to downgrade to (default: `1`)\n* `T` : element type (default: `Float32`)\n* `removeWeight/flattenPol`: see `sptpol_software.observation.Map` (default: `true/true`)\n* `Mdowngrade`: downgrade mask\n* `datadir`: directory holding data splits (filenames like `100d_halfAl_0000.h5`, etc....)\n" function load_ultradeep_data(k, i = 0; pol = :IP, split = false, diff = false, θpix = 1, T = Float32, removeWeight = true, flattenPol = true, return_weights = false, Mdowngrade = if θpix == 1
                                    1
                                else
                                    load("dat/posterior_inputs/pixel_masks.jld2", "downgrade_mask")
                                end, datadir = "dat/noisesims")
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:385 =#
                        ks = #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:385 =# @match(k, begin
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:386 =#
                                    "ultradeep" => ["100d", "500dLT", "500dFull"]
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:387 =#
                                    "500d" => ["500dLT", "500dFull"]
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:388 =#
                                    "100d" || ("500dLT" || "500dFull") => [k]
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:389 =#
                                    _ => throw(ArgumentError("invalid dataset: $(k)"))
                                end)
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:392 =#
                        splits = #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:392 =# @match(split, begin
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:393 =#
                                    false => (["Al", "Bl", "Ar", "Br"],)
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:394 =#
                                    :halves => (["Al", "Ar"], ["Bl", "Br"])
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:395 =#
                                    :lr => (["Al", "Bl"], ["Ar", "Br"])
                                    #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:396 =#
                                    _ => throw(ArgumentError("invalid split: $(split)"))
                                end)
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:399 =#
                        for (s, ss) = enumerate(splits)
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:400 =#
                            py2"m$$(s) = ($$(join([@sprintf(\"Map.readFromHDF5('%s/%s_half%s_%04i.h5')\",datadir,k,s,i) for s in ss for k in ks],\"+\")))$$((removeWeight ? \".removeWeight()\" : \"\") * (flattenPol ? \".flattenPol()\" : \"\"))
"
                        end
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:405 =#
                        function get_map(m)
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:406 =#
                            f = Mdowngrade * (FlatIQUMap((T.(py2"$$m[$x].map") for x = ["T", "Q", "U"])..., θpix = 1))[pol]
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:407 =#
                            f′ = 1.0f6 * ud_grade(f, θpix, mode = :map, anti_aliasing = true, deconv_pixwin = false)
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:408 =#
                            if return_weights
                                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:409 =#
                                weights = (sum(reshape(py2"$$m.weight", θpix, 780 ÷ θpix, θpix, 780 ÷ θpix, 3, 3); dims = (1, 3)))[1, :, 1, :, :, :]
                                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:410 =#
                                (f′, weights)
                            else
                                #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:412 =#
                                f′
                            end
                        end
                        #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:416 =#
                        if split == false
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:417 =#
                            return get_map("m1")
                        else
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:419 =#
                            (m1, m2) = (get_map("m1"), get_map("m2"))
                            #= /global/cscratch1/sd/marius/ultradeep100d/src/Ultradeep100d.jl:420 =#
                            return if diff
                                    m1 - m2
                                else
                                    [m1, m2]
                                end
                        end
                    end
            end
    end
    __pkguuid__ = "0c9542c1-17e0-4d77-9640-35145c276cf4"
end)type #pynamespace has no field mod
Stacktrace:
 [1] evaluate_call_recurse!(::Any, ::JuliaInterpreter.Frame, ::Expr; enter_generated::Bool) at /global/homes/m/marius/.julia/packages/JuliaInterpreter/ebDbv/src/interpret.jl:213)      From worker 2:	ImportError: No module named _multiarray_umath
      From worker 2:	/global/homes/m/marius/.conda/envs/sptpol_software/lib/python2.7/site-packages/pyfits/__init__.py:22: PyFITSDeprecationWarning: PyFITS is deprecated, please use astropy.io.fits
      From worker 2:	  PyFITSDeprecationWarning)  # noqa
      From worker 2:	
      From worker 2:	signal (11): Segmentation fault
      From worker 2:	in expression starting at none:0
      From worker 2:	gc_mark_loop at /global/u1/m/marius/src/julia-1.4/src/gc.c:2396
      From worker 2:	_jl_gc_collect at /global/u1/m/marius/src/julia-1.4/src/gc.c:2899
      From worker 2:	jl_gc_collect at /global/u1/m/marius/src/julia-1.4/src/gc.c:3105
      From worker 2:	maybe_collect at /global/u1/m/marius/src/julia-1.4/src/gc.c:827 [inlined]
      From worker 2:	jl_gc_pool_alloc at /global/u1/m/marius/src/julia-1.4/src/gc.c:1142
      From worker 2:	UseRef at ./compiler/ssair/ir.jl:263 [inlined]
      From worker 2:	UseRefIterator at ./compiler/ssair/ir.jl:268 [inlined]
      From worker 2:	userefs at ./compiler/ssair/ir.jl:382 [inlined]
      From worker 2:	renumber_ssa2! at ./compiler/ssair/ir.jl:810
      From worker 2:	process_node! at ./compiler/ssair/ir.jl:925
      From worker 2:	process_node! at ./compiler/ssair/ir.jl:984 [inlined]
      From worker 2:	iterate at ./compiler/ssair/ir.jl:1151
      From worker 2:	getfield_elim_pass! at ./compiler/ssair/passes.jl:713
      From worker 2:	run_passes at ./compiler/ssair/driver.jl:121
      From worker 2:	optimize at ./compiler/optimize.jl:169
      From worker 2:	typeinf at ./compiler/typeinfer.jl:33
      From worker 2:	typeinf_edge at ./compiler/typeinfer.jl:488
      From worker 2:	abstract_call_method at ./compiler/abstractinterpretation.jl:404
      From worker 2:	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:101
      From worker 2:	abstract_call_known at ./compiler/abstractinterpretation.jl:873
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:895
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:880
      From worker 2:	abstract_eval at ./compiler/abstractinterpretation.jl:974
      From worker 2:	typeinf_local at ./compiler/abstractinterpretation.jl:1227
      From worker 2:	typeinf_nocycle at ./compiler/abstractinterpretation.jl:1283
      From worker 2:	typeinf at ./compiler/typeinfer.jl:12
      From worker 2:	typeinf_edge at ./compiler/typeinfer.jl:488
      From worker 2:	abstract_call_method at ./compiler/abstractinterpretation.jl:404
      From worker 2:	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:101
      From worker 2:	abstract_call_known at ./compiler/abstractinterpretation.jl:873
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:895
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:880
      From worker 2:	abstract_eval at ./compiler/abstractinterpretation.jl:974
      From worker 2:	typeinf_local at ./compiler/abstractinterpretation.jl:1227
      From worker 2:	typeinf_nocycle at ./compiler/abstractinterpretation.jl:1283
      From worker 2:	typeinf at ./compiler/typeinfer.jl:12
      From worker 2:	typeinf_edge at ./compiler/typeinfer.jl:488
      From worker 2:	abstract_call_method at ./compiler/abstractinterpretation.jl:404
      From worker 2:	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:101
      From worker 2:	abstract_call_known at ./compiler/abstractinterpretation.jl:873
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:895
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:880
      From worker 2:	abstract_eval at ./compiler/abstractinterpretation.jl:974
      From worker 2:	typeinf_local at ./compiler/abstractinterpretation.jl:1227
      From worker 2:	typeinf_nocycle at ./compiler/abstractinterpretation.jl:1283
      From worker 2:	typeinf at ./compiler/typeinfer.jl:12
      From worker 2:	typeinf_edge at ./compiler/typeinfer.jl:488
      From worker 2:	abstract_call_method at ./compiler/abstractinterpretation.jl:404
      From worker 2:	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:101
      From worker 2:	abstract_call_known at ./compiler/abstractinterpretation.jl:873
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:895
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:880
      From worker 2:	abstract_eval at ./compiler/abstractinterpretation.jl:974
      From worker 2:	typeinf_local at ./compiler/abstractinterpretation.jl:1227
      From worker 2:	typeinf_nocycle at ./compiler/abstractinterpretation.jl:1283
      From worker 2:	typeinf at ./compiler/typeinfer.jl:12
      From worker 2:	typeinf_edge at ./compiler/typeinfer.jl:488
      From worker 2:	abstract_call_method at ./compiler/abstractinterpretation.jl:404
      From worker 2:	abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:101
      From worker 2:	abstract_call_known at ./compiler/abstractinterpretation.jl:873
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:895
      From worker 2:	abstract_call at ./compiler/abstractinterpretation.jl:880
      From worker 2:	abstract_eval at ./compiler/abstractinterpretation.jl:974
      From worker 2:	typeinf_local at ./compiler/abstractinterpretation.jl:1227
      From worker 2:	typeinf_nocycle at ./compiler/abstractinterpretation.jl:1283
      From worker 2:	typeinf at ./compiler/typeinfer.jl:12
      From worker 2:	typeinf_ext at ./compiler/typeinfer.jl:574
      From worker 2:	typeinf_ext at ./compiler/typeinfer.jl:605
      From worker 2:	jfptr_typeinf_ext_1 at /global/u1/m/marius/src/julia-1.4/usr/lib/julia/sys.so (unknown line)
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2144 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	jl_apply at /global/u1/m/marius/src/julia-1.4/src/julia.h:1700 [inlined]
      From worker 2:	jl_type_infer at /global/u1/m/marius/src/julia-1.4/src/gf.c:213
      From worker 2:	jl_compile_method_internal at /global/u1/m/marius/src/julia-1.4/src/gf.c:1887
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2153 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	serialize at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:270
      From worker 2:	serialize at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/clusterserialize.jl:191
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2158 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	serialize_any at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:642
      From worker 2:	serialize at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:621
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2144 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	serialize_msg at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/messages.jl:90
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2144 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	jl_apply at /global/u1/m/marius/src/julia-1.4/src/julia.h:1700 [inlined]
      From worker 2:	do_apply at /global/u1/m/marius/src/julia-1.4/src/builtins.c:643
      From worker 2:	jl_f__apply_latest at /global/u1/m/marius/src/julia-1.4/src/builtins.c:693
      From worker 2:	#invokelatest#1 at ./essentials.jl:712 [inlined]
      From worker 2:	invokelatest at ./essentials.jl:711 [inlined]
      From worker 2:	send_msg_ at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/messages.jl:185
      From worker 2:	send_msg_now at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/messages.jl:130 [inlined]
      From worker 2:	send_msg_now at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/messages.jl:125
      From worker 2:	deliver_result at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:111
      From worker 2:	message_handler_loop at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:212
      From worker 2:	process_tcp_streams at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:142
      From worker 2:	#97 at ./task.jl:358
      From worker 2:	_jl_invoke at /global/u1/m/marius/src/julia-1.4/src/gf.c:2144 [inlined]
      From worker 2:	jl_apply_generic at /global/u1/m/marius/src/julia-1.4/src/gf.c:2322
      From worker 2:	jl_apply at /global/u1/m/marius/src/julia-1.4/src/julia.h:1700 [inlined]
      From worker 2:	start_task at /global/u1/m/marius/src/julia-1.4/src/task.c:687
      From worker 2:	unknown function (ip: (nil))
      From worker 2:	Allocations: 8876237 (Pool: 8874580; Big: 1657); GC: 9
Worker 2 terminated.
ERROR: Distributed.ProcessExitedException(2)
Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:316
 [2] macro expansion at ./task.jl:335 [inlined]
 [3] _require_callback(::Base.PkgId) at /global/u1/m/marius/src/julia-1.4/usr/share/julia/stdlib/v1.4/Distributed/src/Distributed.jl:75
 [4] #invokelatest#1 at ./essentials.jl:712 [inlined]
 [5] invokelatest at ./essentials.jl:711 [inlined]
 [6] require(::Base.PkgId) at ./loading.jl:930
 [7] require(::Module, ::Symbol) at ./loading.jl:922
@timholy
Copy link
Owner

timholy commented Sep 9, 2020

No clue about this one. Perhaps try the revise3 branch, there are quite a few things fixed there. #497

@marius311
Copy link
Author

Thanks for the followup, I did try the other day and its still there on revise3. Let me try and get an actual MWE...

@timholy
Copy link
Owner

timholy commented Sep 9, 2020

Thanks for checking. I probably will need a MWE to comment usefully.

@timholy timholy added the needs reproducer Bug report can't be verified without reproducible test case label Sep 17, 2020
@marius311
Copy link
Author

marius311 commented Sep 23, 2020 •
edited
Loading

I got a MWE, which is not exactly the #pynamespace error but its really similar so I think might be the same. However, then I realized this as well as the original thing are fixed on 3.1.2 😄. Leaving the MWE here for future reference but closing this Issue otherwise.

First, pkg> generate test then pkg> add Requires Markdown then put this in src/test.jl:

module test

using Requires, Markdown

@init @require Requires="ae029012-a4dd-5104-9daa-d747884805df" begin

    """
    docs
    """ 
    function foo
    end
    
end

end

And finally using Revise, test triggers it (Revise 2.7.3):

Revise.LogRecord(Error, evaluation error starting at /home/marius/work/test/test/src/test.jl:7, lowered, Revise_e943ed8b, "/home/marius/.julia/packages/Revise/BqeJF/src/lowered.jl", 106, (mod=test, ex=begin
    begin
        #= /home/marius/work/test/test/src/test.jl:7 =#
        #= /home/marius/work/test/test/src/test.jl:7 =# Core.@doc "docs\n" function foo end
    end
    __pkguuid__ = "ae029012-a4dd-5104-9daa-d747884805df"
end)UndefVarError: Base_Docs_4352b6d8 not defined
Stacktrace:

@t-bltg t-bltg mentioned this issue Jun 26, 2022
Merged
@t-bltg t-bltg mentioned this issue Jul 23, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs reproducer Bug report can't be verified without reproducible test case
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@marius311 @timholy