Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix scipy<1.15.0 pin in virtual environment configuration files once issue in SciPy is resolved #677

Closed
vschaffn opened this issue Jan 10, 2025 · 1 comment · Fixed by #678

Comments

@vschaffn
Copy link
Contributor

vschaffn commented Jan 10, 2025

Description

The latest release of SciPy (v1.15.0), published on January 3, 2025, causes errors in xdem during the execution of test_biascorr.py. Specifically, the following tests are impacted : test_biascorr__bin_and_fit_1d and test_biascorr__bin_and_fit_2d.

As a temporary fix, the scipy version has been pinned to <1.15.0 in environment.yml, dev-environment.yml and requirements.txt (#675) to prevent these errors from blocking development and CI.

To-Do

  • Monitor updates from SciPy regarding this issue.
  • Once the issue is fixed in a future SciPy release, remove the version pin and update the scipy dependency in environment.yml and dev-environment.yml to allow newer versions.
  • Re-run the tests to verify that the issue is resolved in xdem.

Note

requirements.txt will be automatically synchronized with environment.yml via a pre-commit hook.

@rhugonnet
Copy link
Member

Here are the error messages:

FAILED tests/test_coreg/test_biascorr.py::TestBiasCorr::test_biascorr__bin_and_fit_1d[nanmean-10-curve_fit-<lambda>-fit_args0] - RuntimeError: Optimal parameters not found: gtol=0.000000 is too small, func(x) is ort...
FAILED tests/test_coreg/test_biascorr.py::TestBiasCorr::test_biascorr__bin_and_fit_2d[nanmean-bin_sizes1-curve_fit-<lambda>-fit_args1] - scipy.optimize._optimize.OptimizeWarning: Covariance of the parameters could not be es...
FAILED tests/test_coreg/test_biascorr.py::TestBiasCorr::test_biascorr__bin_and_fit_2d[nanmean-bin_sizes1-curve_fit-<lambda>-fit_args2] - scipy.optimize._optimize.OptimizeWarning: Covariance of the parameters could not be es...

Two warnings and one error during scipy.optimize.curve_fit().

From past experience with another SciPy optimize error (1.12 update broke the behaviour of curve_fit with a lower bound of zero, which we had in SciKit-GStat/xDEM), they would have much more opened issues on their GitHub right now (after 2 weeks) if this was a common.
As they don't, this is a probably just a slight change in covariance estimation of the algorithm, which only affects some of our bias-correction tests that are quite sensitive as they have very low point number on purpose (to increase testing speed).

We can simply silence the warnings for the bin_and_fit_2d, as this is common.
But I'll look more into the RunTimeError...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants