Work with dask arrays

[rabernat]

Thanks for providing this amazing package. It is absolutely one of the best and most useful python packages I know of!

Currently eofs only works with numpy arrays. However, its core computational algorithm, svd, is implemented in dask array. http://dask.pydata.org/en/latest/array-api.html

This means that it would theoretically be possible for eofs to leverage dask to do out of core EOFs with minimal refactoring.

Is this on your roadmap? Would be keen to help if you’re interested.

[rabernat]

Some inspiration for why this could be cool:

https://www.youtube.com/watch?v=R5CiXti_MWo

[rabernat]

Hi @ajdawson - any thoughts on this query?

[ajdawson]

This is a good suggestion @rabernat. I have no roadmap for this software, and little time to work on it. That said, I'm happy to support an effort on this front though.

I'm imagining the eofs.standard interface would be extended to support dask as well as numpy in a transparent way, that way the metadata interfaces built on top can make use of either. Is that what you had in mind in terms of basic design? I haven't really done my homework on this though!

[navidcy]

@rabernat I salute this suggestion. I read in the documentation the

The xarray interface works with DataArray objects, which are the data containers used by the xarray package.

and I wrongly assumed that this also implied support of dask. Then I spend an hour trying to make things work with xarray+dask before I noticed this issue :)

[ajdawson]

Done in #109