Auto-generate API index page

.github/workflows/docs_build_and_deploy.yml

@@ -30,6 +30,7 @@ jobs:
       - uses: neuroinformatics-unit/actions/build_sphinx_docs@main
         with:
           python-version: 3.11
+          use-make: true
 
   deploy_sphinx_docs:
     name: Deploy Sphinx Docs
@@ -42,3 +43,4 @@ jobs:
       - uses: neuroinformatics-unit/actions/deploy_sphinx_docs@main
         with:
           secret_input: ${{ secrets.GITHUB_TOKEN }}
+          use-make: true

.gitignore

@@ -60,6 +60,7 @@ instance/
 docs/build/
 docs/source/examples/
 docs/source/api/
+docs/source/api_index.rst
 sg_execution_times.rst
 
 # MkDocs documentation

CONTRIBUTING.md

@@ -209,32 +209,14 @@ If it is not yet defined and you have multiple external links pointing to the sa
 
 
 ### Updating the API reference
-If your PR introduces new public modules, or renames existing ones,
-make sure to add them to the `docs/source/api_index.rst` page, so that they are included in the [API reference](target-api), e.g.:
-
-```rst
-API Reference
-=============
-
-Information on specific functions, classes, and methods.
-
-.. rubric:: Modules
-
-.. autosummary::
-    :toctree: api
-    :recursive:
-    :nosignatures:
-
-    movement.move_accessor
-    movement.io.load_poses
-    movement.io.save_poses
-    movement.your_new_module
-```
-
-The API reference is auto-generated by the `sphinx-autodoc` and `sphinx-autosummary` plugins, based on docstrings.
+The API reference is auto-generated by the `docs/make_api_index.py` script, and the `sphinx-autodoc` and `sphinx-autosummary` plugins.
+The script generates the `docs/source/api_index.rst` file containing the list of modules to be included in the [API reference](target-api).
+The plugins then generate the API reference pages for each module listed in `api_index.rst`, based on the docstrings in the source code.
 So make sure that all your public functions/classes/methods have valid docstrings following the [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html) style.
 Our `pre-commit` hooks include some checks (`ruff` rules) that ensure the docstrings are formatted consistently.
 
+If your PR introduces new modules that should *not* be documented in the [API reference](target-api), or if there are changes to existing modules that necessitate their removal from the documentation, make sure to update the `exclude_modules` list within the `docs/make_api_index.py` script to reflect these exclusions.
+
 ### Updating the examples
 We use [sphinx-gallery](sphinx-gallery:)
 to create the [examples](target-examples).
@@ -252,28 +234,68 @@ examples are run. See the relevant section of the
 
 ### Building the documentation locally
 We recommend that you build and view the documentation website locally, before you push it.
-To do so, first install the requirements for building the documentation:
+To do so, first navigate to `docs/`.
+All subsequent commands should be run from within this directory.
+```sh
+cd docs
+```
+Install the requirements for building the documentation:
 ```sh
-pip install -r docs/requirements.txt
+pip install -r requirements.txt
 ```
 
-Then, from the root of the repository, run:
+Build the documentation:
+
+::::{tab-set}
+:::{tab-item} Unix platforms with `make`
+```sh
+make html
+```
+The local build can be viewed by opening `docs/build/html/index.html` in a browser.
+:::
+
+:::{tab-item} All platforms
 ```sh
-sphinx-build docs/source docs/build
+python make_api_index.py && sphinx-build source build
 ```
+The local build can be viewed by opening `docs/build/index.html` in a browser.
+:::
+::::
 
-You can view the local build by opening `docs/build/index.html` in a browser.
-To refresh the documentation, after making changes, remove the `docs/build` folder and re-run the above command:
+To refresh the documentation after making changes, remove all generated files in `docs/`,
+including the auto-generated API index `source/api_index.rst`, and those in `build/`, `source/api/`, and `source/examples/`.
+Then, re-run the above command to rebuild the documentation.
 
+::::{tab-set}
+:::{tab-item} Unix platforms with `make`
 ```sh
-rm -rf docs/build && sphinx-build docs/source docs/build
+make clean html
 ```
+:::
+
+:::{tab-item} All platforms
+```sh
+rm -f source/api_index.rst && rm -rf build && rm -rf source/api && rm -rf source/examples
+python make_api_index.py && sphinx-build source build
+```
+:::
+::::
 
 To check that external links are correctly resolved, run:
 
+::::{tab-set}
+:::{tab-item} Unix platforms with `make`
 ```sh
-sphinx-build docs/source docs/build -b linkcheck
+make linkcheck
 ```
+:::
+
+:::{tab-item} All platforms
+```sh
+sphinx-build source build -b linkcheck
+```
+:::
+::::
 
 If the linkcheck step incorrectly marks links with valid anchors as broken, you can skip checking the anchors in specific links by adding the URLs to `linkcheck_anchors_ignore_for_url` in `docs/source/conf.py`, e.g.:
 
@@ -332,7 +354,6 @@ To add a new file, you will need to:
 5. Add your new files to the `poses`, `bboxes`, `videos` and/or `frames` folders as appropriate. Follow the existing file naming conventions as closely as possible.
 6. Determine the sha256 checksum hash of each new file. You can do this in a terminal by running:
     ::::{tab-set}
-
     :::{tab-item} Ubuntu
       ```bash
       sha256sum <filename>

docs/Makefile

@@ -14,7 +14,18 @@ help:
 
 .PHONY: help Makefile
 
+# Generate the API index file
+api_index.rst:
+	python make_api_index.py
+
+# Remove all generated files
+clean:
+	rm -rf ./build
+	rm -f ./source/api_index.rst
+	rm -rf ./source/api
+	rm -rf ./source/examples
+
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
-%: Makefile
+%: Makefile api_index.rst
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -25,6 +25,9 @@ if errorlevel 9009 (
 
 if "%1" == "" goto help
 
+echo "Generating API index..."
+python make_api_index.py
+
 %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
 goto end
 

docs/make_api_index.py

@@ -0,0 +1,44 @@
+"""Generate the API index page for all ``movement`` modules."""
+
+import os
+
+# Modules to exclude from the API index
+exclude_modules = ["cli_entrypoint"]
+
+# Set the current working directory to the directory of this script
+script_dir = os.path.dirname(os.path.abspath(__file__))
+os.chdir(script_dir)
+
+
+def make_api_index():
+    """Create a doctree of all ``movement`` modules."""
+    doctree = "\n"
+
+    for root, _, files in os.walk("../movement"):
+        # Remove leading "../"
+        root = root[3:]
+        for file in sorted(files):
+            if file.endswith(".py") and not file.startswith("_"):
+                # Convert file path to module name
+                module_name = os.path.join(root, file)
+                module_name = module_name[:-3].replace(os.sep, ".")
+                # Check if the module should be excluded
+                if not any(
+                    file.startswith(exclude_module)
+                    for exclude_module in exclude_modules
+                ):
+                    doctree += f"    {module_name}\n"
+
+    # Get the header
+    with open("./source/_templates/api_index_head.rst") as f:
+        api_head = f.read()
+    # Write api_index.rst with header + doctree
+    with open("./source/api_index.rst", "w") as f:
+        f.write("..\n  This file is auto-generated.\n\n")
+        f.write(api_head)
+        f.write(doctree)
+    print(os.path.abspath("./source/api_index.rst"))
+
+
+if __name__ == "__main__":
+    make_api_index()

docs/source/_templates/api_index_head.rst

@@ -0,0 +1,13 @@
+.. _target-api:
+
+API Reference
+=============
+
+Information on specific functions, classes, and methods.
+
+.. rubric:: Modules
+
+.. autosummary::
+    :toctree: api
+    :recursive:
+    :nosignatures: