build/bin/sage-spkg-info: Format dependencies file; remove redundant "dependencies" sections from SPKG.rst

[mkoeppe]

For most SPKGs, the dependencies file contains more accurate information than what can be found in the "Dependencies" section of SPKG.rst.

We simplify the format of the dependencies file by introducing new files dependencies_optional, dependencies_order_only, dependencies_check.

We extend the sage-spkg-info script so that, by simple heuristic text wrangling, the dependencies file is transformed to more human-readable form. When formatting for rst output for the "spkg" chapter of the reference manual, we format the dependencies as hyperlinks.
Example: https://d385f0db7500bdc8168e6f7f18d76741efa3742b--sagemath-tobias.netlify.app/reference/spkg/cvxopt.html#dependencies

Then we remove the redundant "dependencies" sections from the SPKG.rst files.

Depends on #33601

CC: @jhpalmieri @seblabbe @slel @dimpase @videlec @tobiasdiez

Component: build

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: c359f6b

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/31017

[mkoeppe]

Dependencies: #29655

[mkoeppe]

comment:3

Simple clean regexp processing like

for a in build/pkgs/*/dependencies; do echo $((for d in $(sed -n '2,$d;/#/d;s/[$][(][-a-z]* [^)]*[)]//g;s/[$|,()]/ /g;s,[^ /]*/[^ ]*,,g;s/findstring//g;s/OPTIONAL_INSTALLED_PACKAGES//;p' $a); do echo $d; done) | sort); done

does the job

[mkoeppe]

comment:4

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Branch: u/mkoeppe/build_bin_sage_spkg_info__format_dependencies_file__remove_redundant__dependencies__sections_from_spkg_rst

[mkoeppe]

Changed dependencies from #29655 to none

[mkoeppe]

Commit: 9761ef7

[mkoeppe]

New commits:

49ad190	m4/sage_spkg_collect.m4: Read files dependencies_{optional,order_only,check}
d016254	build/pkgs: Split dependencies_optional and dependencies_check out from dependencies
a9eed44	build/bin/sage-spkg-info: Restore message '(none known)' for packages without system package info
23e7a0b	build/bin/sage-spkg-info: Format dependencies
5f67829	build/pkgs/memory_allocator/SPKG.rst: Fix markup
e8c2830	build/pkgs/mathics/SPKG.rst: Fix title
9761ef7	build/bin/sage-spkg-info: Fix rst label

[sagetrac-git]

Changed commit from 9761ef7 to 5b2ed40

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5b2ed40

build/pkgs/[a-f]*/SPKG.rst: Remove redundant Dependencies sections; incidental cleanup

[sagetrac-git]

Changed commit from 5b2ed40 to f9e76d5

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f9e76d5

build/pkgs/*/dependencies: Remove outdated boilerplate comment

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

643fa51

src/doc/en/developer/packaging.rst: Update

[sagetrac-git]

Changed commit from f9e76d5 to 643fa51

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

36f0aff

build/pkgs/[g-z]*/SPKG.rst: Remove redundant Dependencies sections

[mkoeppe]

comment:26

Thank you!

[mkoeppe]

comment:27

Replying to @dimpase:

while we're busy with deps here, it would be good to understand why installing jupyterlab triggers re-building of sagelib (as reported recently on sage-devel)

Has this observation been reproduced? Then a ticket for it should be opened

[dimpase]

comment:28

Replying to @mkoeppe:

Replying to @dimpase:

while we're busy with deps here, it would be good to understand why installing jupyterlab triggers re-building of sagelib (as reported recently on sage-devel)

Has this observation been reproduced? Then a ticket for it should be opened

yes it has been reproduced. I'll open a ticket.

[dimpase]

comment:29

aand, while trying to reproduce the latter, I ran unto a bug:

$ make jupyterlab_widgets
...
make[1]: Entering directory '/home/scratch/scratch2/dimpase/sage/sagetrac-mirror/build/make'
make[1]: *** No rule to make target 'jupyter_packagingjupyterlab', needed by 'ipympl'.  Stop.
make[1]: Leaving directory '/home/scratch/scratch2/dimpase/sage/sagetrac-mirror/build/make'

real    0m0.018s
user    0m0.017s
sys     0m0.001s
***************************************************************
Error building Sage.

[sagetrac-git]

Changed commit from a9b3463 to 18593ec

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

18593ec

m4/sage_spkg_collect.m4: One more space

[mkoeppe]

comment:31

Thanks for catching this

[dimpase]

comment:32

ok, this works

[mkoeppe]

comment:33

Thanks!

[vbraun]

comment:34

Merge failure on top of:

3b7d542342 Trac #33916: tweak tox.ini: activate more pycodestyle checks in cython files

7c9c0f7a21 Trac #33910: installing jupyterlab causes sagelib rebuild

3af284d113 Trac #33905: pep and other details in dynamics

9a586e259b Trac #33901: sage-bootstrap-python doesn't work when pyenv shadows all usable pythons in PATH

221fee8ba0 Trac #33739: Migrate gitpod to conda

24579c942f Trac #33601: Documentation: Add 'furo' theme

490d27a212 Trac #32937: Drop Python 3.7 support in Sage 9.7

24d7d6af87 Trac #31937: Update markupsafe to 2.1.1, jinja2 to 3.1.2

4aedae2596 Trac #29941: Modularization of sagelib: Break out separate packages sagemath-environment, sagemath-repl

70ec4e6 Trac #25872: Modular decomposition bug

5fb2a6e Updated SageMath version to 9.7.beta1

merge was not clean: conflicts in build/pkgs/mathjax/SPKG.rst

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

8df9661	Upgrade mathjax package to mathjax3
43b802d	Merge branch 'spkg-mathjax3-trac25833'
89ec417	Fix MATHJAX_DIR env variable
b6c595b	Merge remote-tracking branch 'origin/public/docs/mathjax_ext' into public/doctests/furo
3bf8620	Clean furo package description
80e39a1	Merge remote-tracking branch 'origin/public/25833' into public/doctests/furo
3df515d	Small cleanup of mathjax spkg
171d2c9	Move changes to new sage_docbuild
aec8fec	Merge #33740
c359f6b	Merge #33601

[sagetrac-git]

Changed commit from 18593ec to c359f6b

[mkoeppe]

Dependencies: #33601

[vbraun]

Changed branch from u/mkoeppe/build_bin_sage_spkg_info__format_dependencies_file__remove_redundant__dependencies__sections_from_spkg_rst to c359f6b