-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use dask instead of numpy masked arrays #58
Comments
As mentioned on slack, there has to be a decision made about this. Does pyspectral add a hard requirement on dask? Most numpy operations should work with dask, at least the arithmetic and ufunc ones. If pyspectral can be updated and assume that NaN is used as a fill value instead of depending on masks then it should be compatible with both in most cases. |
Yes, I agree. I noticed this after starting to convert the unit tests to the daskified version, and found out that there are soooo many use-cases that don't work with my dask conversion. SatPy was happy, but I need to think more if this is feasible for other (non-SatPy) use(r)s. |
Running a full disk scene of SEVIRI with SatPy to create |
OK, found the way to do it: only daskify the NIR reflectance part, and adjust the documentation so that it doesn't have any references to masked arrays. If someone has used the reflectance computations without SatPy, there's a need to adjust the calling script to a) to do |
Currently PySpectral uses masked arrays from Numpy to handle the data arrays. For better integration and SatPy support, these should be converted to dask arrays.
The text was updated successfully, but these errors were encountered: