Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resolve broken scikits.odes installation on self-hosted M-series runner #3785

Merged
merged 6 commits into from
Jan 30, 2024
Merged

Resolve broken scikits.odes installation on self-hosted M-series runner #3785

merged 6 commits into from
Jan 30, 2024

Conversation

agriyakhetarpal
Copy link
Member

Description

The M-series self-hosted runner was using a cached, un-repaired scikits.odes wheel which had been linked to the Homebrew rendition of SUNDIALS (which isn't a problem though) instead of the one we regularly use in CI. Apple has a history of sanitising runtime search paths, which caused the SUNDIALS dynamic libraries to not be found at runtime by the Python extension modules for scikits.odes.

This PR ensures that the build-time dependencies for PyBaMM are always re-compiled on each run, the SUNDIALS_INST variable is set, and that scikits.odes is built from source every time.

The M-series build passed here: https://github.com/pybamm-team/PyBaMM/actions/runs/7713296977/job/21022885593

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Key checklist:

  • No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
  • All tests pass: $ python run-tests.py --all (or $ nox -s tests)
  • The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).

Further checks:

  • Code is commented, particularly in hard-to-understand areas

@agriyakhetarpal
Try fixing M-series runner tests
6b5ffc5 
This is being done by adding SuiteSparse and SUNDIALS installations which might have been missing on the runner, which broke `scikits.odes`.
@agriyakhetarpal
Don't use Homebrew SUNDIALS, use LD_LIBRARY_PATH
6003009
@agriyakhetarpal
Don't use Homebrew to install SUNDIALS
ebdf018
@agriyakhetarpal
Force remove pip cache for scikits.odes
b8b2458
@codecov Codecov
Copy link

codecov bot commented Jan 30, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (165ec43) 99.59% compared to head (287d5c2) 99.59%.
Report is 1 commits behind head on develop.

Additional details and impacted files 
@@           Coverage Diff            @@
##           develop    #3785   +/-   ##
========================================
  Coverage    99.59%   99.59%           
========================================
  Files          257      257           
  Lines        20802    20802           
========================================
  Hits         20718    20718           
  Misses          84       84

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

agriyakhetarpal added a commit that referenced this pull request Jan 30, 2024
@agriyakhetarpal
Purge scikits.odes cached wheels when using pybamm_install_odes
165ec43 
See discussion on #3785. This command exits with a warning if a cached wheel is not present, it does not raise an error and therefore should be safe to add within a subprocess.
@agriyakhetarpal agriyakhetarpal mentioned this pull request Jan 30, 2024
Merged
6 tasks
BradyPlanden
BradyPlanden approved these changes Jan 30, 2024
View reviewed changes
Copy link
Member

@BradyPlanden BradyPlanden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, thanks for updating this @agriyakhetarpal. LGTM

@kratman
Copy link
Contributor

kratman commented Jan 30, 2024

Let's make sure to squash this so it is easy to cherry pick into releases if needed

@agriyakhetarpal agriyakhetarpal merged commit 063a383 into develop Jan 30, 2024
40 of 41 checks passed
@agriyakhetarpal agriyakhetarpal deleted the fix-m-series-scheduled-tests branch January 30, 2024 18:12
Saransh-cpp pushed a commit that referenced this pull request Jan 30, 2024
@agriyakhetarpal @kratman @Saransh-cpp
Resolve broken scikits.odes installation on self-hosted M-series ru…
e29dcc0 
…nner (#3785)

* Try fixing M-series runner tests

This is being done by adding SuiteSparse and SUNDIALS installations which might have been missing on the runner, which broke `scikits.odes`.

* Don't use Homebrew SUNDIALS, use LD_LIBRARY_PATH

* Don't use Homebrew to install SUNDIALS

* Force remove pip cache for `scikits.odes`

---------

Co-authored-by: Eric G. Kratz <[email protected]>
rtimms added a commit that referenced this pull request May 16, 2024
@valentinsulzer @brosaplanella
@Saransh-cpp @martinjrobins @agriyakhetarpal @kratman @rtimms
add temperature dependence to MSMR model (#3832)
33d9ea7 
* Bump to v23.9rc0

* Merge pull request #3412 from agriyakhetarpal/drop-i686-manylinux2014-support

Drop support for i686 manylinux

* Merge pull request #3413 from Saransh-cpp/improve-release-workflow

Improve release workflow, add a note, bump version manually

* Merge pull request #3436 from Saransh-cpp/fortnightly-wheels

Build wheels on the 1st and 15th of every month

* Merge pull request #3445 from pybamm-team/issue-3428-rename-exchange

#3428 exchange-current density error

* Merge pull request #3449 from pybamm-team/i3431-windows-wheels

Fix failing windows wheel builds

* Merge pull request #3456 from abillscmu/issue-3224-initial_soc

make initial soc work with half cell models

* Merge pull request #3467 from abillscmu/bugfix/initial_soh

* Merge pull request #3423 from jsbrittain/jax_gpu

JaxSolver fails when using GPU support with no input parameters

* Fix changelog

* Merge pull request #3475 from arjxn-py/fix-default-imports

Resolve default imports for optional dependencies

* Bump - `v23.9rc1`

* Fix date in CHANGELOG

* Bump version to v23.9

* Fix docs about Jax solver compatibility with Python versions (#3702)

* Ensure correct Python versions for Jax solver compatibility

* Simplify array of Python versions

Co-authored-by: Eric G. Kratz <[email protected]>

* Use different conjunction

Co-authored-by: Eric G. Kratz <[email protected]>

---------

Co-authored-by: Eric G. Kratz <[email protected]>

* Merge pull request #3706 from agriyakhetarpal/fix-pybamm-install-odes

Make `pybamm_install_odes` a bit more robust

* #3690 fix issue with skipped steps (#3708)

* #3690 fix issue with skipped steps

* #3690 changelog

* #3690 add test

* #3611 use actual cell volume for average total heating (#3707)

* #3611 use actual cell volume for average total heating

* #3611 changelog

* #3611 account for number of electrode pairs

* #3611 update variable names

* Improve the release workflow (#3737)

* Try fixing the release workflow

* Turn off safety

* Fix CHANGELOG

* Add OS

* Use regex for better matches

* Update instructions, add safety checks

* checkout to the version branch for the final release

* Bump to v24.1rc1

* #3630 fix interpolant shape error (#3761)

* #3630 fix interpolant shape error

* #3630 changelog

* Bump to v24.1rc2

* Bump to v24.1

* Fix doctests failures in scheduled tests (#3784)

Closes #3781

* Resolve broken `scikits.odes` installation on self-hosted M-series runner (#3785)

* Try fixing M-series runner tests

This is being done by adding SuiteSparse and SUNDIALS installations which might have been missing on the runner, which broke `scikits.odes`.

* Don't use Homebrew SUNDIALS, use LD_LIBRARY_PATH

* Don't use Homebrew to install SUNDIALS

* Force remove pip cache for `scikits.odes`

---------

Co-authored-by: Eric G. Kratz <[email protected]>

* add temperature dependence to MSMR model

* changelog

* fix tests

* fix example

* rob comments

* update notebook

---------

Co-authored-by: Ferran Brosa Planella <[email protected]>
Co-authored-by: Saransh Chopra <[email protected]>
Co-authored-by: Martin Robinson <[email protected]>
Co-authored-by: Agriya Khetarpal <[email protected]>
Co-authored-by: Eric G. Kratz <[email protected]>
Co-authored-by: Robert Timms <[email protected]>
Co-authored-by: Saransh-cpp <[email protected]>
js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024
@agriyakhetarpal @js1tr3
Purge scikits.odes cached wheels when using pybamm_install_odes
cb21a6f 
See discussion on pybamm-team#3785. This command exits with a warning if a cached wheel is not present, it does not raise an error and therefore should be safe to add within a subprocess.
js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024
@agriyakhetarpal @kratman @js1tr3
Resolve broken scikits.odes installation on self-hosted M-series ru…
f57eb1b 
…nner (pybamm-team#3785)

* Try fixing M-series runner tests

This is being done by adding SuiteSparse and SUNDIALS installations which might have been missing on the runner, which broke `scikits.odes`.

* Don't use Homebrew SUNDIALS, use LD_LIBRARY_PATH

* Don't use Homebrew to install SUNDIALS

* Force remove pip cache for `scikits.odes`

---------

Co-authored-by: Eric G. Kratz <[email protected]>
js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024
@valentinsulzer @brosaplanella
@Saransh-cpp @martinjrobins @agriyakhetarpal @kratman @rtimms @js1tr3
add temperature dependence to MSMR model (pybamm-team#3832)
4b564de 
* Bump to v23.9rc0

* Merge pull request pybamm-team#3412 from agriyakhetarpal/drop-i686-manylinux2014-support

Drop support for i686 manylinux

* Merge pull request pybamm-team#3413 from Saransh-cpp/improve-release-workflow

Improve release workflow, add a note, bump version manually

* Merge pull request pybamm-team#3436 from Saransh-cpp/fortnightly-wheels

Build wheels on the 1st and 15th of every month

* Merge pull request pybamm-team#3445 from pybamm-team/issue-3428-rename-exchange

pybamm-team#3428 exchange-current density error

* Merge pull request pybamm-team#3449 from pybamm-team/i3431-windows-wheels

Fix failing windows wheel builds

* Merge pull request pybamm-team#3456 from abillscmu/issue-3224-initial_soc

make initial soc work with half cell models

* Merge pull request pybamm-team#3467 from abillscmu/bugfix/initial_soh

* Merge pull request pybamm-team#3423 from jsbrittain/jax_gpu

JaxSolver fails when using GPU support with no input parameters

* Fix changelog

* Merge pull request pybamm-team#3475 from arjxn-py/fix-default-imports

Resolve default imports for optional dependencies

* Bump - `v23.9rc1`

* Fix date in CHANGELOG

* Bump version to v23.9

* Fix docs about Jax solver compatibility with Python versions (pybamm-team#3702)

* Ensure correct Python versions for Jax solver compatibility

* Simplify array of Python versions

Co-authored-by: Eric G. Kratz <[email protected]>

* Use different conjunction

Co-authored-by: Eric G. Kratz <[email protected]>

---------

Co-authored-by: Eric G. Kratz <[email protected]>

* Merge pull request pybamm-team#3706 from agriyakhetarpal/fix-pybamm-install-odes

Make `pybamm_install_odes` a bit more robust

* pybamm-team#3690 fix issue with skipped steps (pybamm-team#3708)

* pybamm-team#3690 fix issue with skipped steps

* pybamm-team#3690 changelog

* pybamm-team#3690 add test

* pybamm-team#3611 use actual cell volume for average total heating (pybamm-team#3707)

* pybamm-team#3611 use actual cell volume for average total heating

* pybamm-team#3611 changelog

* pybamm-team#3611 account for number of electrode pairs

* pybamm-team#3611 update variable names

* Improve the release workflow (pybamm-team#3737)

* Try fixing the release workflow

* Turn off safety

* Fix CHANGELOG

* Add OS

* Use regex for better matches

* Update instructions, add safety checks

* checkout to the version branch for the final release

* Bump to v24.1rc1

* pybamm-team#3630 fix interpolant shape error (pybamm-team#3761)

* pybamm-team#3630 fix interpolant shape error

* pybamm-team#3630 changelog

* Bump to v24.1rc2

* Bump to v24.1

* Fix doctests failures in scheduled tests (pybamm-team#3784)

Closes pybamm-team#3781

* Resolve broken `scikits.odes` installation on self-hosted M-series runner (pybamm-team#3785)

* Try fixing M-series runner tests

This is being done by adding SuiteSparse and SUNDIALS installations which might have been missing on the runner, which broke `scikits.odes`.

* Don't use Homebrew SUNDIALS, use LD_LIBRARY_PATH

* Don't use Homebrew to install SUNDIALS

* Force remove pip cache for `scikits.odes`

---------

Co-authored-by: Eric G. Kratz <[email protected]>

* add temperature dependence to MSMR model

* changelog

* fix tests

* fix example

* rob comments

* update notebook

---------

Co-authored-by: Ferran Brosa Planella <[email protected]>
Co-authored-by: Saransh Chopra <[email protected]>
Co-authored-by: Martin Robinson <[email protected]>
Co-authored-by: Agriya Khetarpal <[email protected]>
Co-authored-by: Eric G. Kratz <[email protected]>
Co-authored-by: Robert Timms <[email protected]>
Co-authored-by: Saransh-cpp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BradyPlanden BradyPlanden BradyPlanden approved these changes

@arjxn-py arjxn-py arjxn-py approved these changes

@martinjrobins martinjrobins Awaiting requested review from martinjrobins

@Saransh-cpp Saransh-cpp Awaiting requested review from Saransh-cpp

@kratman kratman Awaiting requested review from kratman

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@agriyakhetarpal @kratman @BradyPlanden @arjxn-py