Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Cubewcs mosaic and dask reproject #894

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
d9a1abd
enhance the cube header determinator by checking the spectral dimension
keflavich Mar 9, 2023
759918b
add option to specify header
keflavich Mar 13, 2023
b195811
try to make convolution possible _and_ use memmap'd output
keflavich Mar 13, 2023
2a9367e
need non-array truthiness
keflavich Mar 13, 2023
d534814
just raise the exception as is, don't wrap it
keflavich Mar 13, 2023
c2e5900
refactor cube mosaicing to use reproject.coadd
keflavich Mar 13, 2023
3928b8d
refactor to allow for direct FITS output
keflavich Mar 13, 2023
5e5211e
bitpix dtype
keflavich Mar 13, 2023
b05d2de
getdata doesn't support mode
keflavich Mar 13, 2023
837778c
force overwrite
keflavich Mar 13, 2023
4d86197
force overwrite
keflavich Mar 13, 2023
37c2758
mode for fits
keflavich Mar 13, 2023
0c4b93f
hdu0
keflavich Mar 13, 2023
5084171
refactor to use output_array
keflavich Mar 13, 2023
f36e007
allow setting no spectral block size
keflavich Mar 13, 2023
9b9ac0e
add flush
keflavich Mar 13, 2023
d90f55e
need to flush occasionally
keflavich Mar 13, 2023
f139574
allow channel-by-channel mosaicing instead of cube-by-cube
keflavich Mar 14, 2023
b37d628
add progressbar & frequent flush callback
keflavich Mar 14, 2023
60e8519
fix some critical typos and enable resuming from partly-written file …
keflavich Mar 14, 2023
5545fc9
restructure a bunch of things that are being done too expensively and…
keflavich Mar 14, 2023
aae16a6
more verbosity & parallelism
keflavich Mar 14, 2023
e34ad21
this is very verbose now, but still trying to make verbosity flexible
keflavich Mar 15, 2023
f3d016e
allow weights to be included
keflavich Mar 15, 2023
946b071
add explanations of some choices
keflavich Mar 20, 2023
3e2f580
when operating on lists of cubes, some may be out of range, and they …
keflavich Sep 4, 2023
c804cbb
need more checks for cubes to be in range
keflavich Sep 5, 2023
c1740a4
have to account for more corner cases for missing data
keflavich Sep 5, 2023
4adf4dd
fix more bugs with keep
keflavich Sep 6, 2023
31b3c8c
refactor neighboring channel chooser
keflavich Sep 8, 2023
7342668
WIP: try making dask do reprojection
keflavich Oct 5, 2022
faf3cb2
move reproject to top-level dask class
keflavich Sep 9, 2023
f263436
pass kwargs
keflavich Sep 10, 2023
3915fd3
comment out the wrapper
keflavich Sep 12, 2023
f76ec52
add some debug things
keflavich Nov 10, 2023
1cee139
allow a threshold to be specified for spectral agreement in cube
keflavich Dec 2, 2023
84761c3
logical bugfix
keflavich Dec 2, 2023
e7b5742
update use of reproject kwargs
keflavich Dec 3, 2023
85cd63c
fix typo in header creation
keflavich Dec 3, 2023
ed6185d
fix logical error: if the spectral axis is reversed, keeping CRPIX an…
keflavich Dec 3, 2023
59b8d7f
force dtype in memory-based merge
keflavich Dec 8, 2023
182211c
sanity checks, dtype forcing
keflavich Dec 9, 2023
b13631c
add some more checks that the output file is float
keflavich Dec 9, 2023
f15b4fa
add an explicit warning
keflavich Dec 9, 2023
d37b834
dtypes are a pain
keflavich Dec 9, 2023
6b79b6f
Add handling for VRSC -> SC with common beam; optional return footprint
e-koch Dec 29, 2023
f25ba59
Expand test_mosaic_cubes
e-koch Dec 29, 2023
f57cba5
Keep the original projection when combining headers
e-koch Dec 29, 2023
9bac12c
Ensure combined header updates units appropriately
e-koch Dec 29, 2023
4227354
The -1 for the combined spectral axis is returning the wrong shape. L…
e-koch Dec 29, 2023
9361b2f
Add test for mosaicking in spectral dimension
e-koch Dec 29, 2023
4834846
Remove CASABM after convolving to a common beam
e-koch Dec 29, 2023
207bb78
Drop the units check when one cube is dimensionless
e-koch Dec 29, 2023
a73663b
add common beam hack
keflavich Dec 20, 2023
6b73271
remove typo
keflavich Dec 20, 2023
59e2791
Add dask[distributed] to required packages
e-koch Dec 29, 2023
c1ba5c1
some robustness against beams, better error messaging
keflavich Dec 29, 2023
a8dda19
Allow passing specific dask distributed clients as a kwarg
e-koch Jan 5, 2024
e9a5078
Don't start a dask client as a kwarg
e-koch Jan 5, 2024
ba58ebf
Remove unused taper kwarg
e-koch Jan 6, 2024
e6394a9
make progressbar a little more sensible and add debugging things around
keflavich Feb 12, 2024
fea7317
add failure_returns_pointlike for beam deconvolution
keflavich Jun 15, 2024
c0e25a7
pass intermediate_memmap to reproject_and_coadd
keflavich Jun 15, 2024
2595139
add descs
keflavich Jun 16, 2024
a85cfd9
reproject_and_coadd does not accept progressbar any more
keflavich Jul 15, 2024
14fa698
argmax/argmin need to return ints [some test code was removed]
keflavich Oct 18, 2024
b242b85
workaround missing mask problem
keflavich Jan 22, 2025
c68d378
need to modify list inplace
keflavich Jan 22, 2025
14d13bf
workaround didn't work
keflavich Jan 22, 2025
01cfa5d
was supposed to return slices
keflavich Jan 22, 2025
21e564c
add some cube mosaicing tests. they fail. disaster.
keflavich Jan 24, 2025
c2ac661
tests caught a few bugs
keflavich Jan 24, 2025
a1c2253
testing work
keflavich Jan 24, 2025
59b3467
add weighted test
keflavich Jan 24, 2025
4183dc3
handle weight cube slicing - before, we were loading whole weight cub…
keflavich Jan 25, 2025
f623397
lots of debug statements and fixes. Somehow, every single case is a …
keflavich Jan 25, 2025
2f32e8e
fix weightcube cut handling
keflavich Jan 26, 2025
5e308c7
comment out debugs
keflavich Jan 26, 2025
938c016
debug statements semi-restored, and catch the case where scubes are d…
keflavich Jan 26, 2025
c2a29a6
add extrapolation tolerance parameter & some debug statements to remo…
keflavich Jan 26, 2025
9c66440
tests pass if we don't try to gather data
keflavich Jan 26, 2025
b106460
fix a dask test error (shouldn't run with dask) and turn off loud debug
keflavich Jan 26, 2025
06819cc
remove a debug statement that CI didn't like
keflavich Jan 26, 2025
ca56933
add missing fixture
keflavich Jan 26, 2025
f70e5cb
sometimes warnings are turned off - force them on
keflavich Jan 26, 2025
a82bec1
more verbose message when failing because cubes unmatched
keflavich Jan 28, 2025
8ba6b27
pass on reproject kwargs & fix logical error: sign(0)=0
keflavich Jan 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ install_requires =
numpy>=1.8.0
radio_beam>=0.3.3
dask[array]
dask[distributed]
joblib
casa-formats-io
packaging
Expand Down
Loading
Loading