Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix warnings in CI #188

Closed
Illviljan opened this issue Nov 5, 2022 · 6 comments · Fixed by #197
Closed

Fix warnings in CI #188

Illviljan opened this issue Nov 5, 2022 · 6 comments · Fixed by #197
Labels
help wanted Extra attention is needed

Comments

@Illviljan
Copy link
Contributor

Noticed in #151.

=============================== warnings summary ===============================
tests/test_core.py: 22 warnings
  /home/runner/work/flox/flox/flox/aggregate_flox.py:110: RuntimeWarning: divide by zero encountered in divide
    out /= nanlen(group_idx, array, size=size, axis=axis, fill_value=0)

tests/test_core.py: 207 warnings
  /home/runner/work/flox/flox/flox/aggregations.py:250: RuntimeWarning: invalid value encountered in divide
    finalize=lambda sum_, count: sum_ / count,

tests/test_core.py: 22 warnings
  /home/runner/work/flox/flox/flox/aggregate_flox.py:118: RuntimeWarning: divide by zero encountered in divide
    out /= nanlen(group_idx, array, size=size, axis=axis, fill_value=0)

tests/test_core.py: 531 warnings
tests/test_xarray.py: 88 warnings
  /home/runner/work/flox/flox/flox/aggregations.py:259: RuntimeWarning: invalid value encountered in divide
    finalize=lambda sum_, count: sum_ / count,

tests/test_core.py: 2772 warnings
  /home/runner/work/flox/flox/flox/aggregations.py:268: RuntimeWarning: invalid value encountered in divide
    result = (sumsq - (sum_**2 / count)) / (count - ddof)

tests/test_core.py: 1068 warnings
  /home/runner/work/flox/flox/flox/core.py:877: RuntimeWarning: All-NaN slice encountered
    result = getattr(np, combine)(array, axis=axis_, keepdims=True)

tests/test_core.py: 18 warnings
  /home/runner/work/flox/flox/tests/test_core.py:209: RuntimeWarning: Mean of empty slice
    expected = getattr(np, func)(array[..., ~nanmask], axis=-1, **kwargs)

tests/test_core.py: 8 warnings
tests/test_xarray.py: 12 warnings
  /home/runner/work/flox/flox/flox/aggregate_flox.py:118: RuntimeWarning: invalid value encountered in divide
    out /= nanlen(group_idx, array, size=size, axis=axis, fill_value=0)

tests/test_core.py: 18 warnings
  /home/runner/work/flox/flox/tests/test_core.py:209: RuntimeWarning: Degrees of freedom <= 0 for slice.
    expected = getattr(np, func)(array[..., ~nanmask], axis=-1, **kwargs)

tests/test_core.py: 1944 warnings
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/dask/array/numpy_compat.py:41: RuntimeWarning: invalid value encountered in divide
    x = np.divide(x1, x2, out)

tests/test_core.py: 200 warnings
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/dask/array/reductions.py:1062: RuntimeWarning: All-NaN slice encountered
    vals = func(x, axis=arg_axis, keepdims=True)

tests/test_core.py: 18 warnings
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/numpy/lib/nanfunctions.py:1878: RuntimeWarning: Degrees of freedom <= 0 for slice.
    var = nanvar(a, axis=axis, dtype=dtype, out=out, ddof=ddof,

tests/test_core.py: 342 warnings
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/dask/array/reductions.py:640: RuntimeWarning: All-NaN slice encountered
    return np.nanmax(x_chunk, axis=axis, keepdims=keepdims)

tests/test_core.py: 36 warnings
  /home/runner/work/flox/flox/tests/test_core.py:209: RuntimeWarning: All-NaN slice encountered
    expected = getattr(np, func)(array[..., ~nanmask], axis=-1, **kwargs)

tests/test_core.py: 342 warnings
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/dask/array/reductions.py:611: RuntimeWarning: All-NaN slice encountered
    return np.nanmin(x_chunk, axis=axis, keepdims=keepdims)

tests/test_xarray.py: 15 warnings
  /home/runner/work/flox/flox/flox/xarray.py:571: DeprecationWarning: flox.xarray.resample_reduce is now deprecated. Please use Xarray's resample method directly.
    warnings.warn(

tests/test_xarray.py::test_multi_index_groupby_sum[flox]
tests/test_xarray.py::test_multi_index_groupby_sum[flox]
tests/test_xarray.py::test_multi_index_groupby_sum[numpy]
tests/test_xarray.py::test_multi_index_groupby_sum[numpy]
tests/test_xarray.py::test_multi_index_groupby_sum[numba]
tests/test_xarray.py::test_multi_index_groupby_sum[numba]
  /home/runner/work/flox/flox/flox/xarray.py:252: DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'y', 'x'} to avoid an error in the future.
    ds = ds.drop_vars([var for var in maybe_drop if var in ds.variables])

tests/test_xarray.py::test_multi_index_groupby_sum[flox]
tests/test_xarray.py::test_multi_index_groupby_sum[numpy]
tests/test_xarray.py::test_multi_index_groupby_sum[numba]
  /home/runner/work/flox/flox/flox/xarray.py:252: DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'b', 'c'} to avoid an error in the future.
    ds = ds.drop_vars([var for var in maybe_drop if var in ds.variables])

tests/test_xarray.py::test_datetime_array_reduce[flox-count-False]
tests/test_xarray.py::test_datetime_array_reduce[flox-mean-False]
tests/test_xarray.py::test_datetime_array_reduce[numpy-count-False]
tests/test_xarray.py::test_datetime_array_reduce[numpy-mean-False]
tests/test_xarray.py::test_datetime_array_reduce[numba-count-False]
tests/test_xarray.py::test_datetime_array_reduce[numba-mean-False]
  /home/runner/micromamba-root/envs/flox-tests/lib/python3.8/site-packages/xarray/coding/cftime_offsets.py:1130: FutureWarning: Argument `closed` is deprecated in favor of `inclusive`.
    return pd.date_range(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

@Illviljan Illviljan changed the title Fix warning in CI Fix warnings in CI Nov 5, 2022
@dcherian dcherian added the help wanted Extra attention is needed label Nov 28, 2022
@tlogan2000
Copy link

I will comment here as I think my issue is related. Please let me know if you prefer a separate issue.

I have noticed that a recent upgrade to use flox in connection with xarray and dask has caused an inability for me to silence associated numpy warnings even via a blanket approach warnings.simplefilter("ignore").

The Xarray and dask combination have always been noisy wrt to warnings but this used to work to avoid messy jupyter notebook outputs. Since moving to flox I can't seem to achieve the same result

dcherian added a commit that referenced this issue Dec 6, 2022
dcherian added a commit that referenced this issue Dec 6, 2022
* Suppress warnings

Closes #188

* Silence a bunch of dask warnings

* Fix mypy
@dcherian
Copy link
Collaborator

dcherian commented Dec 6, 2022

@tlogan2000 I don't know how flox would affect that.

However I have now silenced every numpy warning from inside flox. So maybe that helps.

@tlogan2000
Copy link

@tlogan2000 I don't know how flox would affect that.

However I have now silenced every numpy warning from inside flox. So maybe that helps.

Excellent, thanks it definitely seems to help but I am still getting a recurring :

aggregations.py:259: RuntimeWarning: invalid value encountered in divide
  chunk=("sum", "nanlen"),

@dcherian
Copy link
Collaborator

dcherian commented Dec 6, 2022

Are you using the flox engine?

@tlogan2000
Copy link

I can't really say TBH, my experience with flox to date has been to simply install and let xarray defaults handle how and when it is employed. I am one of the xclim library https://xclim.readthedocs.io/en/stable/ developers and am basically using xarray either directly or via associated xclim calculations in my data processing. In general, performance for resampling (a core procedure for a large number of xclim climate indicator calculations) and use of dask.distributed clients has seen a large boost with flox installed ... my only difficulty has been with the warnings

@dcherian
Copy link
Collaborator

dcherian commented Dec 6, 2022

Hmm ok in that case I'll need a reproducible example to track it down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants