Fix sparse diag by upgrading SciPy and Numpy and dropping Python 3.9

[maximelucas]

Fixes #649.

The problems seems to be due to a bug from SciPy that made setdiag() not set the whole diagonal to 0 in some cases for sparse arrays. It's been fixed by this PR since, and included in the 1.15 SciPy release.

I just upped our SciPy version in the requirements and added a test.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.45%. Comparing base (b86da86) to head (57a6a21).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #650   +/-   ##
=======================================
  Coverage   93.45%   93.45%           
=======================================
  Files          64       64           
  Lines        4993     4993           
=======================================
  Hits         4666     4666           
  Misses        327      327           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[maximelucas]

Tests for Python 3.9 fail because SciPy 1.15 requires Python 3.10-3.13 (and NumPy 1.23.5 or greater.)

Python 3.9's end of life is planned for Oct 2025.
Would dropping it already be a problem? @nwlandry

[nwlandry]

Tests for Python 3.9 fail because SciPy 1.15 requires Python 3.10-3.13 (and NumPy 1.23.5 or greater.)

Python 3.9's end of life is planned for Oct 2025. Would dropping it already be a problem? @nwlandry

It doesn't make sense to defer this essential fix to align with Python's release schedule. I am good with you removing v3.9 (just make a note in the PR title as well maybe?)

[nwlandry]

Looks great, @maximelucas! Good to squash and merge!

[maximelucas]

Agreed. I also upgraded NumPy as required by SciPy.

This was an important bug to fix, but for some reason I only ever noticed it in "email-enron", and even then, only the smaller eigenvalues were affected. So hopefully, the bug hasn't had too much of an impact.