src/doc/en/developer/doctesting.rst: Update

src/doc/en/developer/doctesting.rst

@@ -1080,7 +1080,7 @@ under the control of gdb, use the ``--gdb`` flag::
 
     [roed@localhost sage]$ ./sage -t --gdb \
                                   src/sage/schemes/elliptic_curves/constructor.py
-    exec gdb --eval-commands="run" --args /home/roed/sage/local/var/lib/sage/venv-python3.9/bin/python3 sage-runtests --serial --timeout=0 --stats_path=/home/roed/.sage/timings2.json --optional=pip,sage,sage_spkg src/sage/schemes/elliptic_curves/constructor.py
+    exec gdb --eval-commands="run" --args /home/roed/sage/local/var/lib/sage/venv-python3.9/bin/python3 sage-runtests --serial --timeout=0 --stats-path=/home/roed/.sage/timings2.json --optional=pip,sage,sage_spkg src/sage/schemes/elliptic_curves/constructor.py
     GNU gdb 6.8-debian
     Copyright (C) 2008 Free Software Foundation, Inc.
     License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
@@ -1308,6 +1308,8 @@ code loads the globals from that file into the namespace before
 running tests.  To disable this behaviour (and require imports to be
 explicitly specified), use the ``--force-lib`` option.
 
+.. _section-doctest-auxiliary-files:
+
 Auxiliary files
 ^^^^^^^^^^^^^^^
 
@@ -1338,10 +1340,11 @@ To specify a logfile (rather than use the default which is created for
         cumulative wall time: 4.3 seconds
 
 
-To give a json file storing the timings for each file, use the
-``--stats_path`` flag.  These statistics are used in sorting files so
-that slower tests are run first (and thus multiple processes are
-utilized most efficiently)::
+To give a json file storing the timings and pass/fail status for each file, use the
+``--stats-path`` flag; the default location of this file is ``~/.sage/timings2.json``.
+The doctester reads it if it exists, for the purpose of sorting the files
+so that slower tests are run first (and thus multiple processes are utilized most
+efficiently)::
 
     [roed@localhost sage]$ ./sage -tp 2 --stats-path ~/.sage/timings2.json --all
     Running doctests with ID 2012-07-07-01-28-34-2df4251d.
@@ -1350,6 +1353,22 @@ utilized most efficiently)::
     Doctesting 2067 files using 2 threads.
     ...
 
+At the end of the doctest run, Sage updates the json file if it exists or creates
+a new one.
+
+The recorded pass/fail status of the files can be used for running only those files
+that failed their most recent test by using the ``--failed`` flag (``-f`` for short).
+
+Using the option ``--baseline-stats-path known-test-failures.json``,
+it is possible to distinguish files with known doctest failures
+from new failures. The file ``known-test-failures.json`` should be
+prepared in the same format as ``timings2.json``.
+
+Source files that marked as failed there will be marked as "[failed in baseline]"
+failures in the doctest report; and if there are only baseline failures, no
+new failures, then ``sage -t`` will exit with status code 0 (success).
+
+
 .. _section-doctesting-venv:
 
 Options for testing in virtual environments
@@ -1655,3 +1674,17 @@ to the doctest.
 
 Likewise, when the doctester runs into a :class:`ModuleNotFoundError`,
 the doctest fixer will automatically add a ``# needs ...`` tag.
+
+
+Updating baseline files
+-----------------------
+
+The modularized distribution packages ``pkgs/sagemath-categories`` and
+``pkgs/sagemath-repl`` contain files ``known-test-failures*.json`` for use
+with the option ``--baseline-stats-path``, see section
+:ref:`section-doctest-auxiliary-files`.
+
+These files can be updated using the command::
+
+    [mkoeppe@localhost sage]$ ./sage --fixdoctests --no-test                        \
+                                --update-known-test-failures --distribution all