Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update MultiScene to work with xarray/dask #322

Merged
merged 16 commits into from
Jun 14, 2018

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Jun 12, 2018

This PR udpates the MultiScene to work with xarray and dask. I added a few useful properties and would like to add at least one helper method to it for creating GIFs or movies using the imageio library (see #157).

If I have time I'd also like to add the functionality discussed in #310. I marked that issue as being for v0.10, but if it is straight forward I wouldn't mind it showing up in v0.9 as an experiment feature.

I need to add documentation and maybe an example notebook for various multiscene operations. I might add it to the regular sphinx docs with the title ending in "EXPERIMENTAL" until it has been further evaluated after SciPy 2018.

@djhoese djhoese added this to the v0.9 milestone Jun 12, 2018
@djhoese djhoese self-assigned this Jun 12, 2018
@djhoese djhoese requested a review from mraspaud June 12, 2018 20:45
scn[dataset_id] = blend_function(datasets)
common_datasets = self.shared_dataset_ids
for ds_id in common_datasets:
datasets = [scn[ds_id] for scn in self.scenes if ds_id in scn]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F812 list comprehension redefines 'scn' from line 72

@coveralls
Copy link

coveralls commented Jun 12, 2018

Coverage Status

Coverage increased (+0.6%) to 68.733% when pulling 68f7d67 on djhoese:bugfix-xarray-multiscene into b64eafe on pytroll:master.

@codecov
Copy link

codecov bot commented Jun 12, 2018

Codecov Report

Merging #322 into master will increase coverage by 0.55%.
The diff coverage is 89.12%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #322      +/-   ##
==========================================
+ Coverage   68.17%   68.73%   +0.55%     
==========================================
  Files         120      121       +1     
  Lines       14923    15134     +211     
==========================================
+ Hits        10174    10402     +228     
+ Misses       4749     4732      -17
Impacted Files Coverage Δ
satpy/tests/test_dataset.py 100% <100%> (ø) ⬆️
satpy/scene.py 84.25% <100%> (+0.07%) ⬆️
satpy/tests/__init__.py 92.3% <100%> (+0.3%) ⬆️
satpy/__init__.py 94.11% <100%> (+0.36%) ⬆️
satpy/dataset.py 89.21% <100%> (+0.55%) ⬆️
satpy/tests/test_scene.py 99.3% <100%> (ø) ⬆️
satpy/readers/__init__.py 95.14% <100%> (+0.18%) ⬆️
satpy/multiscene.py 81.08% <80.76%> (+81.08%) ⬆️
satpy/tests/test_multiscene.py 94.33% <94.33%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b64eafe...68f7d67. Read the comment docs.

def test_init_empty(self):
"""Test creating a multiscene with no children."""
from satpy import MultiScene
mscn = MultiScene()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'mscn' is assigned to but never used

"""Test creating a multiscene with children."""
from satpy import MultiScene
scenes = _create_test_scenes()
mscn = MultiScene(scenes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'mscn' is assigned to but never used


bigger_area = _create_test_area(shape=(20, 40))
scenes[0]['ds4'] = _create_test_dataset('ds4', shape=(20, 40),
area=bigger_area)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E127 continuation line over-indented for visual indent


area = _create_test_area()
scenes = _create_test_scenes(area=area)
ds1_id = DatasetID(name='ds1')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'ds1_id' is assigned to but never used

area = _create_test_area()
scenes = _create_test_scenes(area=area)
ds1_id = DatasetID(name='ds1')
ds2_id = DatasetID(name='ds2')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'ds2_id' is assigned to but never used

scenes = _create_test_scenes(area=area)
ds1_id = DatasetID(name='ds1')
ds2_id = DatasetID(name='ds2')
ds3_id = DatasetID(name='ds3')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'ds3_id' is assigned to but never used

ds1_id = DatasetID(name='ds1')
ds2_id = DatasetID(name='ds2')
ds3_id = DatasetID(name='ds3')
ds4_id = DatasetID(name='ds4')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F841 local variable 'ds4_id' is assigned to but never used

@mock.patch('satpy.multiscene.get_enhanced_image', _fake_get_enhanced_image)
def test_save_mp4(self):
"""Save a series of fake scenes to an mp4 video."""
from satpy import MultiScene, DatasetID
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F401 'satpy.DatasetID' imported but unused

@djhoese
Copy link
Member Author

djhoese commented Jun 14, 2018

I did not have time so I am not implementing the feature discussed in #310. I'll do that in a future PR.

@djhoese djhoese changed the title [WIP] Update MultiScene to work with xarray/dask Update MultiScene to work with xarray/dask Jun 14, 2018
@djhoese djhoese merged commit 377b5ba into pytroll:master Jun 14, 2018
@djhoese djhoese deleted the bugfix-xarray-multiscene branch June 14, 2018 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add animation example
3 participants