Upgrade eclib to version 20241112 #38962

JohnCremona · 2024-11-13T14:48:41Z

📝 Checklist

The title is concise and informative.
[x ] The description explains in detail what this PR is about.
[ x] I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation and checked the documentation preview.

⌛ Dependencies

JohnCremona · 2024-11-15T09:21:17Z

Anyone looking at this ticket now would assume that there is something wrong with it as several automatic checks have failed. But none of the failures have (as far as I can see) anything to do with the new eclib. I myself am not going to take any further action on this unless someone tells me that I am wrong about this.

antonio-rojas · 2024-11-15T18:23:00Z

This requires patching sagelib due to API changes. See https://gitlab.archlinux.org/archlinux/packaging/packages/sagemath/-/blob/main/eclib-20241112.patch?ref_type=heads

JohnCremona · 2024-11-19T11:10:54Z

This requires patching sagelib due to API changes. See https://gitlab.archlinux.org/archlinux/packaging/packages/sagemath/-/blob/main/eclib-20241112.patch?ref_type=heads

Thanks. I will make the changes you suggest in the link and update the PR.

github-actions · 2024-11-19T14:40:57Z

Documentation preview for this PR (built with commit 893e20e; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

JohnCremona · 2024-11-19T14:59:24Z

The forced push was just after rebasing on current develop.

JohnCremona · 2024-11-19T16:53:38Z

As usual I have no idea what is making any of the failures, this time I am more confident that it is nothing to do with this PR but whi knows? I hope someone does.

tobiasdiez · 2024-11-25T04:32:36Z

Only had a quick look but at least the build failures for "Build & Test using Meson" are in the files changed in this PR. I suppose the problem is that these changes are not backwards compatible. Could you make compile sage with both the old and new version of eclib? If not, I'm afraid this PR has to wait until the new version is available on conda.

JohnCremona · 2024-11-25T09:50:07Z

Only had a quick look but at least the build failures for "Build & Test using Meson" are in the files changed in this PR. I suppose the problem is that these changes are not backwards compatible. Could you make compile sage with both the old and new version of eclib? If not, I'm afraid this PR has to wait until the new version is available on conda.

Thanks for the explanation. I had thought, apparently wrongly, that the code in spkg-configure.m4 which checks for the existence of an installed eclib version which is new enough was there precisely to deal with this situation. If you are building Sage in an environment where only a too-old eclib is available, then the build process will build its own version of eclib. No? That certainly used to be the case. If not, then I don't see how any upstream developer (e.g. me for eclib) can ever reliably upgrade their library for Sage.

tobiasdiez · 2024-11-28T09:09:45Z

If one uses conda, then all of sage-the-distro (everything under the build folder) is ignored, see https://doc.sagemath.org/html/en/installation/conda.html#using-conda-to-provide-all-dependencies-for-the-sage-library

Is it really not possible to support both the old and new version of ecm in sage?

JohnCremona · 2024-11-28T15:12:27Z

If one uses conda, then all of sage-the-distro (everything under the build folder) is ignored, see https://doc.sagemath.org/html/en/installation/conda.html#using-conda-to-provide-all-dependencies-for-the-sage-library

Is it really not possible to support both the old and new version of ecm in sage?

(eclib not ecm)

I will try -- there are very few lines which had to be different in the wrapping code.

JohnCremona · 2024-12-03T14:24:40Z

If one uses conda, then all of sage-the-distro (everything under the build folder) is ignored, see https://doc.sagemath.org/html/en/installation/conda.html#using-conda-to-provide-all-dependencies-for-the-sage-library
Is it really not possible to support both the old and new version of ecm in sage?

(eclib not ecm)

I will try -- there are very few lines which had to be different in the wrapping code.

OK, I can do away with the get_entries() methods which used to return a scalar* and now return a vector[scalar], and use the methods which deliver individual entries.

Incidentally, I would be amazed if there was anyone in the world who actually use this part of eclib in Sage: it is the conversion to a Sage matric f an eclib matrix, as returned by the Hecke operator functions on a CremonaModularSymbols space.

I will update the interface code and update this PR.

JohnCremona · 2024-12-04T08:37:50Z

OK, I rebased on current develop and made a few changes to the interface so that neither scalar* nor vector[scalar] are used. I hope this is enough.

JohnCremona · 2024-12-04T11:53:32Z

I checked the logs for failing tests and the only one is in src/sage/libs/gap/element.pyx which is not relevant to this PR, I think. In particular, it was happy with the tests in src/sage/libs/eclib.

dimpase · 2024-12-04T15:04:11Z

still needs a rebase

JohnCremona · 2024-12-04T16:48:11Z

If someone tells me what is now wrong and how to fix it, and if it is anything to do with my latest attempt to upgrade eclib (which used to be easy enough for me to do) then I will fix it.

sagemathgh-38962: Upgrade eclib to version 20241112       Fixed sagemath#38960 ### 📝 Checklist  - [x] The title is concise and informative. - [x ] The description explains in detail what this PR is about. - [ x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    URL: sagemath#38962 Reported by: John Cremona Reviewer(s): Dima Pasechnik

vbraun · 2024-12-08T00:12:50Z

Segfaults on 32-bit:

sage: E = EllipticCurve('11a') ## line 1454 ##
sage: phi = E.pollack_stevens_modular_symbol() ## line 1455 ##
------------------------------------------------------------------------
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/python3.12/site-packages/cysignals/signals.cpython-312-i386-linux-gnu.so(+0x8372)[0xf680a372]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/python3.12/site-packages/cysignals/signals.cpython-312-i386-linux-gnu.so(+0x840f)[0xf680a40f]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/python3.12/site-packages/cysignals/signals.cpython-312-i386-linux-gnu.so(+0xb354)[0xf680d354]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f5c570]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z12gauss_reducellllRlS_S_S_+0xda)[0xa4504a5a]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z10new_modratllRlS_+0x42)[0xa4504c52]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z6modratllRlS_+0x24)[0xa4504cf4]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z6modratiiRiS_+0x26)[0xa4504d26]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z4liftRK5vec_iRKiRS_+0x2c0)[0xa4525b20]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_Z4liftRK5vec_i+0x41)[0xa4581d01]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN12form_finder211make_basis2ER7ff_dataRK5vec_i+0xd1)[0xa4581ed1]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN12form_finder210make_basisER7ff_data+0x409)[0xa4582399]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN12form_finder28splitoffERKSt6vectorIlSaIlEE+0x386)[0xa4584246]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN12form_finder27recoverESt6vectorIS0_IlSaIlEESaIS2_EE+0x64)[0xa45845c4]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN8newforms16createfromcurvesEiSt6vectorI8CurveRedSaIS1_EEi+0x531)[0xa46dcb01]
/var/lib/buildbot/worker/sage_git/build/local/lib/libec.so.14(_ZN8newforms15createfromcurveEiRK8CurveRedi+0xd7)[0xa46dcf47]
/var/lib/buildbot/worker/sage_git/build/src/sage/libs/eclib/newforms.cpython-312-i386-linux-gnu.so(+0xba36)[0xa479ba36]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/libpython3.12.so.1.0(+0x12da60)[0xf7ae7a60]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/libpython3.12.so.1.0(_PyObject_MakeTpCall+0x9c)[0xf7a7ef2c]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/libpython3.12.so.1.0(PyObject_Vectorcall+0x74)[0xf7a7f264]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/libpython3.12.so.1.0(_PyEval_EvalFrameDefault+0x5025)[0xf7a29cf5]
/var/lib/buildbot/worker/sage_git/build/local/var/lib/sage/venv-python3.12.5/lib/libpython3.12.so.1.0(+0x1b7311)[0xf7b71311]

tornaria · 2024-12-24T01:22:48Z

See: JohnCremona/eclib#81 (comment)

A new version of modrat() has been introduced, and it seems to have issues with overflow. Definitely for 32 bit, and maybe only for 32 bit, but in the end I just reverted to old_modrat() with ~~https://github.com/tornaria/void-packages/blob/pari/srcpkgs/eclib/patches/fix-32bit-modrat.patch~~.

Edit: I copy here the patch, since the url above is gone.

--- a/libsrc/arith.cc
+++ b/libsrc/arith.cc
@@ -429,8 +430,8 @@ int new_modrat(long n, long m, long& a, long& b);
 
 int modrat(long n, long m, long& a, long& b)
 {
-  // return old_modrat(n, m, a, b);
-  return new_modrat(n, m, a, b);
+  return old_modrat(n, m, a, b);
+  //return new_modrat(n, m, a, b);
 }
 
 int old_modrat(long n, long m, long& a, long& b)

JohnCremona · 2025-01-13T11:11:37Z

@tornaria How about using one of the methods given in https://stackoverflow.com/questions/1505582/determining-32-vs-64-bit-in-c to choose which method to use? In fact it might be worth setting a macro in one place (such as interface.h) which can then be checked elsewhere. There's an issue about whether this should be done at compile time or run time. Since the function modrat() is used a lot it would be best to decide at compile time, and that appears to rely on having the right headers to define the correct macros.

tornaria · 2025-01-13T14:41:51Z

@tornaria How about using one of the methods given in https://stackoverflow.com/questions/1505582/determining-32-vs-64-bit-in-c to choose which method to use? In fact it might be worth setting a macro in one place (such as interface.h) which can then be checked elsewhere. There's an issue about whether this should be done at compile time or run time. Since the function modrat() is used a lot it would be best to decide at compile time, and that appears to rely on having the right headers to define the correct macros.

Are you sure this is an issue about 32-bit vs. 64-bit, or is this a bug that is just easier to trigger on 32-bit?

JohnCremona · 2025-01-13T15:10:40Z

Well, I am certain that it is possible to trigger crashes on 64-bit machines -- I do not check that whenever I multiply two long ints that the product fits into a long int!

I have a plan to replace all my integer arithmetic with FLINT integers, but that will take a long time and be very tedious. At the same time I would use FLINT for all floating point. I don't see an easy way of doing this incrementally, so I am not very enthusiastic about starting!

tornaria · 2025-01-13T17:19:37Z

Well, I am certain that it is possible to trigger crashes on 64-bit machines -- I do not check that whenever I multiply two long ints that the product fits into a long int!

I have a plan to replace all my integer arithmetic with FLINT integers, but that will take a long time and be very tedious. At the same time I would use FLINT for all floating point. I don't see an easy way of doing this incrementally, so I am not very enthusiastic about starting!

Sure, if you are going to replace arithmetic, it may not be worth fixing modrat now. Note that flint has rational reconstruction, it even special cases the 1 limb case https://github.com/flintlib/flint/blob/main/src/fmpq/reconstruct_fmpz_2.c#L258

The trick there is doing the gaussian reduction on the first row instead of the norm. This avoids the arithmetic on integers of double the size. Of course, you can't go all the way of the gcd or you would obtain a rational number of the form 1/K, but you can stop as in the half-gcd (i.e. the first time a is small enough). This is probably a win for bigints also, since it avoids doing the arithmetic with integers larger than the input size.

dimpase · 2025-01-14T00:31:26Z

Well, I am certain that it is possible to trigger crashes on 64-bit machines -- I do not check that whenever I multiply two long ints that the product fits into a long int!

I have a plan to replace all my integer arithmetic with FLINT integers, but that will take a long time and be very tedious. At the same time I would use FLINT for all floating point. I don't see an easy way of doing this incrementally, so I am not very enthusiastic about starting!

IMHO rather than moving between concrete implementations of integers, one should have templated code which allows plugging in whatever available integers (or, horror, algebraic numbers?)

JohnCremona · 2025-01-14T09:41:22Z

Well, I am certain that it is possible to trigger crashes on 64-bit machines -- I do not check that whenever I multiply two long ints that the product fits into a long int!
I have a plan to replace all my integer arithmetic with FLINT integers, but that will take a long time and be very tedious. At the same time I would use FLINT for all floating point. I don't see an easy way of doing this incrementally, so I am not very enthusiastic about starting!

IMHO rather than moving between concrete implementations of integers, one should have templated code which allows plugging in whatever available integers (or, horror, algebraic numbers?)

Good point -- in fact eclib does this for its vector, matrix and sparse matrix classes. Not using proper C++ templates though, as I wrote this before they were standard. I have no plans to implement algebraic numbers in eclib! Better would be to gradually transfer more of eclib to FLINT (e.g. all the elliptic curve classes, methods and functions.

dimpase · 2025-01-14T14:22:52Z

templates were already standard 25 years ago, e.g. CGAL was using them back then already. Another question is how hard they were to use :-)
I got an RSI debugging them...

But now C++ is basically a different language, and templates are easier to use.

JohnCremona · 2025-01-20T09:06:25Z

I am going to replace eclib version 20241112 with a new version with three extra tiny commits, and I hope that will deal with the hold-up here. (1) as @tornaria suggested JohnCremona/eclib#81, revert to the older version of modrat to avoind the 32-bit test failure; (2) as reported by @thierry-FreeBSD JohnCremona/eclib#83 (3) ding an #include to keep FLINT happy.

tornaria · 2025-01-23T14:26:13Z

I am going to replace eclib version 20241112 with a new version with three extra tiny commits, and I hope that will deal with the hold-up here. (1) as @tornaria suggested JohnCremona/eclib#81, revert to the older version of modrat to avoind the 32-bit test failure; (2) as reported by @thierry-FreeBSD JohnCremona/eclib#83 (3) ding an #include to keep FLINT happy.

I've build & tested 20250122 on i686 and it's ok now. This PR should be ok with that version. A minor detail is the upstream url has to change from https://github.com/JohnCremona/eclib/releases/download/vVERSION/eclib-VERSION.tar.bz2 to https://github.com/JohnCremona/eclib/releases/download/VERSION/eclib-VERSION.tar.bz2 (remove a "v" since the release tag in https://github.com/JohnCremona/eclib/releases has no "v" this time).

JohnCremona · 2025-01-23T14:52:49Z

I am going to replace eclib version 20241112 with a new version with three extra tiny commits, and I hope that will deal with the hold-up here. (1) as @tornaria suggested JohnCremona/eclib#81, revert to the older version of modrat to avoind the 32-bit test failure; (2) as reported by @thierry-FreeBSD JohnCremona/eclib#83 (3) ding an #include to keep FLINT happy.

I've build & tested 20250122 on i686 and it's ok now. This PR should be ok with that version. A minor detail is the upstream url has to change from https://github.com/JohnCremona/eclib/releases/download/vVERSION/eclib-VERSION.tar.bz2 to https://github.com/JohnCremona/eclib/releases/download/VERSION/eclib-VERSION.tar.bz2 (remove a "v" since the release tag in https://github.com/JohnCremona/eclib/releases has no "v" this time).

Thanks for testing, @tornaria . I myself have not yet been able to do any tests since I cannot even build Sage any more (see my post on sage-release).

Sorry about the inconsistency concerning the 'v'. I never remember whether to include it or not, and have even left myself instructions but they are inconsistent.

I think it is probably a good idea for me to just make in Issue when there is a new release of eclib but to let someone else create a suitable Sage PR. That would certainly save me many hours of wasted time.

tornaria · 2025-01-23T15:13:53Z

Thanks for testing, @tornaria . I myself have not yet been able to do any tests since I cannot even build Sage any more (see my post on sage-release).

I tested eclib as a package for void-linux. I don't run sage-the-distro and haven't for a very long time.

I don't know an easy way for you to test eclib on i686. The way it's done in void-packages is running a container with i686 void linux; all the building and testing for i686 packages is done inside this container.

Sorry about the inconsistency concerning the 'v'. I never remember whether to include it or not, and have even left myself instructions but they are inconsistent.

I would say it makes sense for tag to match the version, the "v" is not necessary for anything as far as I understand.

I think it is probably a good idea for me to just make in Issue when there is a new release of eclib but to let someone else create a suitable Sage PR. That would certainly save me many hours of wasted time.

Yes, sounds good. I would not update eclib in sage-the-distro since I don't use it. But I do test new versions of eclib with sagelib.

orlitzky · 2025-02-14T13:18:49Z

This is all done except for an update to v20250122, right?

tornaria · 2025-02-14T17:43:55Z

This is all done except for an update to v20250122, right?

Yes, and with the update the issue with 32 bit goes away. We are shipping eclib-20250122 (and 20241112 patched before that) without trouble.

For 10.6.beta5, this PR + #38749 is all I need to patch sagemath to work on current void linux. I'm also testing flintlib 3.2.0rc1 for which I need #39413 (it's ready and "easy" to review). Merging these 3 PRs would make me very happy :-)

AndreasEnge · 2025-02-14T20:47:14Z

I confirm: For the upcoming Guix package, I have also used the patches of this pull request and those of #38749, and then could use pari-2.17.1 and eclib-20250122.

JohnCremona · 2025-02-15T17:23:59Z

Thanks all -- I have reviewed #39533 so if @tornaria is happy there too, that one can have a positive review and this one can be retired.

tornaria · 2025-02-15T22:03:47Z

Thanks, closing for #39533

sagemathgh-39533: build/pkgs: update eclib to version 20250122 Update the version of eclib in sage-the-distro to 20250122 on top of sagemath#38962 **Dependencies**: * sagemath#38962 URL: sagemath#39533 Reported by: Michael Orlitzky Reviewer(s): John Cremona

github-actions bot added the s: needs review label Nov 13, 2024

antonio-rojas added s: needs work and removed s: needs review labels Nov 16, 2024

github-actions bot added s: needs review and removed s: needs work labels Nov 19, 2024

JohnCremona force-pushed the eclib-20241112 branch from 28c0400 to 6967897 Compare November 19, 2024 14:58

collares mentioned this pull request Nov 20, 2024

eclib: 20240408 -> 20250122 NixOS/nixpkgs#355572

Merged

kiwifb mentioned this pull request Nov 21, 2024

sci-mathematics/sagemath-standard-10.4-r3: build fails with sci-mathematics/eclib/eclib-20241112 cschwan/sage-on-gentoo#801

Closed

JohnCremona force-pushed the eclib-20241112 branch from 6967897 to f0dc04f Compare December 4, 2024 08:36

JohnCremona added 4 commits December 4, 2024 16:05

sagemath#38960-eclib-upgrade-20241112

6dfc26b

sagemath#38960 fix eclib interface after upgrade

d38cb48

sagemath#38960 simplify eclib interface

4f2cb3f

sagemath#38960: fix lint issue

8754c70

JohnCremona force-pushed the eclib-20241112 branch from 48bbf7e to 8754c70 Compare December 4, 2024 16:06

dimpase added the c: packages: standard label Dec 5, 2024

dimpase removed the s: needs review label Dec 5, 2024

vbraun added the s: needs work label Dec 8, 2024

Merge branch 'develop' into eclib-20241112

893e20e

github-actions bot added s: needs review and removed s: positive review s: needs work labels Jan 14, 2025

orlitzky mentioned this pull request Feb 15, 2025

build/pkgs: update eclib to version 20250122 #39533

Merged

tornaria closed this Feb 15, 2025

github-actions bot removed the s: needs review label Feb 15, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade eclib to version 20241112 #38962

Upgrade eclib to version 20241112 #38962

JohnCremona commented Nov 13, 2024

JohnCremona commented Nov 15, 2024

antonio-rojas commented Nov 15, 2024

JohnCremona commented Nov 19, 2024

github-actions bot commented Nov 19, 2024 •

edited

Loading

JohnCremona commented Nov 19, 2024

JohnCremona commented Nov 19, 2024

tobiasdiez commented Nov 25, 2024

JohnCremona commented Nov 25, 2024

tobiasdiez commented Nov 28, 2024

JohnCremona commented Nov 28, 2024

JohnCremona commented Dec 3, 2024

JohnCremona commented Dec 4, 2024

JohnCremona commented Dec 4, 2024

dimpase commented Dec 4, 2024

JohnCremona commented Dec 4, 2024

vbraun commented Dec 8, 2024

tornaria commented Dec 24, 2024 •

edited

Loading

JohnCremona commented Jan 13, 2025

tornaria commented Jan 13, 2025

JohnCremona commented Jan 13, 2025

tornaria commented Jan 13, 2025

dimpase commented Jan 14, 2025

JohnCremona commented Jan 14, 2025

dimpase commented Jan 14, 2025

JohnCremona commented Jan 20, 2025

tornaria commented Jan 23, 2025

JohnCremona commented Jan 23, 2025

tornaria commented Jan 23, 2025

orlitzky commented Feb 14, 2025

tornaria commented Feb 14, 2025

AndreasEnge commented Feb 14, 2025

JohnCremona commented Feb 15, 2025

tornaria commented Feb 15, 2025

Upgrade eclib to version 20241112 #38962

Upgrade eclib to version 20241112 #38962

Conversation

JohnCremona commented Nov 13, 2024

📝 Checklist

⌛ Dependencies

JohnCremona commented Nov 15, 2024

antonio-rojas commented Nov 15, 2024

JohnCremona commented Nov 19, 2024

github-actions bot commented Nov 19, 2024 • edited Loading

JohnCremona commented Nov 19, 2024

JohnCremona commented Nov 19, 2024

tobiasdiez commented Nov 25, 2024

JohnCremona commented Nov 25, 2024

tobiasdiez commented Nov 28, 2024

JohnCremona commented Nov 28, 2024

JohnCremona commented Dec 3, 2024

JohnCremona commented Dec 4, 2024

JohnCremona commented Dec 4, 2024

dimpase commented Dec 4, 2024

JohnCremona commented Dec 4, 2024

vbraun commented Dec 8, 2024

tornaria commented Dec 24, 2024 • edited Loading

JohnCremona commented Jan 13, 2025

tornaria commented Jan 13, 2025

JohnCremona commented Jan 13, 2025

tornaria commented Jan 13, 2025

dimpase commented Jan 14, 2025

JohnCremona commented Jan 14, 2025

dimpase commented Jan 14, 2025

JohnCremona commented Jan 20, 2025

tornaria commented Jan 23, 2025

JohnCremona commented Jan 23, 2025

tornaria commented Jan 23, 2025

orlitzky commented Feb 14, 2025

tornaria commented Feb 14, 2025

AndreasEnge commented Feb 14, 2025

JohnCremona commented Feb 15, 2025

tornaria commented Feb 15, 2025

github-actions bot commented Nov 19, 2024 •

edited

Loading

tornaria commented Dec 24, 2024 •

edited

Loading