At end of configure, suggest a list of system packages to install

[embray]

There have been a spate of reports of issues like #26996 all stemming from people trying to build Sage on Linux systems that do not already have a supported Fortran compiler installed (but easily could). Likewise, for some other packages (openblas, R, ...).

embray suggests: I would [...] just fail--with a helpful error message-at ./configure time, at least on platforms like Cygwin and most Linux distributions, to just install the system's fortran compiler, unless a flag is given to explicitly request it. [...]

In this ticket we just make it a friendly message at the end of the configure run -- not just for gfortran, but for all packages that have system packages. We will check if this already reduces the volume of messages regarding build problems on the lists.

Example:

config.status: creating directory local/lib
config.status: creating directory local/share
config.status: creating directory local/var/lib/sage/installed
configure: Hint: The following SPKGs did not find equivalent system packages:
configure:   cbc gp2c libsemigroups m4ri ninja_build pari_elldata pari_galpol pari_nftables pari_seadata perl_term_readline_gnu
checking for the package system in use... conda
configure: Hint: Installing the following system packages is recommended and may avoid building some of the above SPKGs from source:
configure:   $ conda install coincbc m4ri ninja pari-galpol pari-seadata
configure: After installation, re-run configure using:
configure:   $ ./config.status --recheck && ./config.status

Depends on #29143

CC: @jdemeyer @dimpase @vbraun @jhpalmieri @orlitzky @kiwifb

Component: packages: optional

Author: Matthias Koeppe

Branch/Commit: 35e1e7d

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/27351

[jdemeyer]

comment:1

I don't think that #26996 should be used as a reason. That's a bug in the build system, that we could just fix independently.

I'm not saying that this ticket by itself is a bad idea, just that #26996 is not a good reason for it.

[dimpase]

comment:2

It's a good idea to make it fail on !Linux/Cygwin/Homebrew/FreeBSD.

[embray]

comment:3

Replying to @jdemeyer:

I don't think that #26996 should be used as a reason. That's a bug in the build system, that we could just fix independently.

I'm not saying that this ticket by itself is a bad idea, just that #26996 is not a good reason for it.

I agree, that's not the motivation at all. My point was that it happened to expose the fact that a lot of people, more than I'd imagined was the case, were trying (and in the past, presumably, succeeding) to build gfortran when it wasn't really necessary for them to.

[embray]

comment:4

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

[mkoeppe]

comment:5

Instead of refusing to build it, I think we should in general suggest a list of system packages to install, at the end of the configure run, similar to #26964.

I have revised the ticket description accordingly.

[mkoeppe]

Branch: u/mkoeppe/at_end_of_configure__suggest_a_list_of_system_packages_to_install

[sagetrac-git]

Commit: 479c521

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

749f83e	tox.ini: Add other archs supported by docker
e941bd5	fixup for llocal
2441fe9	tox.ini: Restructure docker image:tag construction, add multiarch for Linux Docker
2f057cb	tox.ini: Add raspbian
7f122bd	build/bin/write-dockerfile.sh: src/ext has moved and is no longer needed for the configure phase
5776abf	build/bin/sage-guess-package-system: New
6807e0d	configure: Output a notice regarding recommended system packages
479c521	build/bin/sage-guess-package-system: Suppress error output

[mkoeppe]

Dependencies: #29143

[mkoeppe]

comment:8

The code on this branch is roughly what I have in mind for this functionality. Doesn't quite work yet.

[mkoeppe]

Author: Matthias Koeppe

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

cee2894

m4/sage_spkg_collect.m4: Fixup

[sagetrac-git]

Changed commit from 479c521 to cee2894

[embray]

comment:13

I'd be fine with completely reworking the package summary at the end of configure (the currently existing one that prints out every package name).

I think this could be done much more clearly, ending with a list of suggested system packages to install as you describe.

[mkoeppe]

comment:14

Replying to @embray:

I'd be fine with completely reworking the package summary at the end of configure (the currently existing one that prints out every package name).

I think this could be done much more clearly, ending with a list of suggested system packages to install as you describe.

We reworked it recently in #28788. Further ideas for improvement - please on a new ticket.
The present ticket is ready and needs review.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

9a8f693

Merge tag '9.1.beta7' into t/27351/at_end_of_configure__suggest_a_list_of_system_packages_to_install

[sagetrac-git]

Changed commit from cee2894 to 9a8f693

[mkoeppe]

comment:16

Needs review...

[jhpalmieri]

comment:17

In a brand new tarball on an Ubuntu virtual machine:

configure: Hint: The following SPKGs did not find equivalent system packages:
configure:   arb boost boost_cropped cbc eclib fplll gfan givaro glpk gp2c lcalc libatomic_ops libsemigroups nauty pari pari_elldata pari_galdata pari_galpol pari_nftables pari_seadata pari_seadata_small planarity sqlite tachyon zeromq
checking for the package system in use... debian
./configure: line 30473: build/bin/sage-get-system-packages: No such file or directory
configure: No equivalent system packages for debian are known to Sage

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

35e1e7d

Add missing file build/bin/sage-get-system-packages

[sagetrac-git]

Changed commit from 9a8f693 to 35e1e7d

[jhpalmieri]

comment:19

What about embray's suggestion to fail on some platforms if gfortran is not installed? I worry that many users will run make without first running ./configure, so the messages printed here will get lost.

I wouldn't object if Sage converted to the more standard ./configure; make procedure, but we're not there yet. On a separate ticket, we should at least encourage this workflow. (As a side note, it annoys me that make distclean also runs ./configure.)

[jhpalmieri]

comment:20

I don't see any message being printed on OS X with homebrew. Is this intentional? I guess so, since there aren't any files build/pkgs/*/distros/homebrew. Is that correct?

By the way, not related to this ticket, but is there a better place to put files like build/pkgs/arch.txt? In a directory build/pkgs/DISTROS or build/distros?

[mkoeppe]

comment:21

Replying to @jhpalmieri:

What about embray's suggestion to fail on some platforms if gfortran is not installed?

I prefer the milder solution - warning instead of error - for this ticket.

I worry that many users will run make without first running ./configure, so the messages printed here will get lost.

Good point.

I wouldn't object if Sage converted to the more standard ./configure; make procedure, but we're not there yet. On a separate ticket, we should at least encourage this workflow.

I wouldn't mind making this change. Separate ticket?

(As a side note, it annoys me that make distclean also runs ./configure.)

Right.

[mkoeppe]

comment:22

Replying to @jhpalmieri:

I don't see any message being printed on OS X with homebrew. Is this intentional? I guess so, since there aren't any files build/pkgs/*/distros/homebrew. Is that correct?

That's right, this needs to be done. That would be a follow up ticket of #29104.

By the way, not related to this ticket, but is there a better place to put files like build/pkgs/arch.txt? In a directory build/pkgs/DISTROS or build/distros?

Yes, in #29124 (Add script packages build/pkgs/_prereq, build/pkgs/_toolchain, build/pkgs/_bootstrap) I plan to move these files to a better location.

[jhpalmieri]

comment:23

Replying to @mkoeppe:

Replying to @jhpalmieri:

I wouldn't object if Sage converted to the more standard ./configure; make procedure, but we're not there yet. On a separate ticket, we should at least encourage this workflow.

I wouldn't mind making this change. Separate ticket?

Yes.

(As a side note, it annoys me that make distclean also runs ./configure.)

Right.

This is now #29310.

[jhpalmieri]

comment:24

This looks okay to me as a first step. We should do more to highlight the messages, at least by encouraging people to run ./configure and read what it says.

[jhpalmieri]

Reviewer: John Palmieri

[mkoeppe]

comment:25

Thanks!

[vbraun]

Changed branch from u/mkoeppe/at_end_of_configure__suggest_a_list_of_system_packages_to_install to 35e1e7d