Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Refactor the build & build customization pages (Diátaxis) #10028

Merged
merged 6 commits into from
Feb 15, 2023
Merged

Docs: Refactor the build & build customization pages (Diátaxis) #10028

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1660a71
Refactor the build & build customization pages
ericholscher Feb 14, 2023
9541b93
Fix up titles
ericholscher Feb 14, 2023
bb54e79
Update docs/user/build-customization.rst
ericholscher Feb 15, 2023
7540cb4
Apply suggestions from code review
ericholscher Feb 15, 2023
4cfbfa1
Merge branch 'main' into relabel-build
ericholscher Feb 15, 2023
23da90d
Apply suggestions from code review
benjaoming Feb 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 69 additions & 62 deletions docs/user/build-customization.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
Build customization
===================
Build process customization
===========================

Read the Docs has a :doc:`well-defined build process <builds>` that works for many projects,
but we offer additional customization to support more uses of our platform.
This page explains how to extend the build process using :term:`user-defined build jobs` to execute custom commands,
and also how to override the build process completely:
Read the Docs has a :doc:`well-defined build process </builds>` that works for many projects.
We also allow customization of builds in primary ways:

`Extend the build process`_
If you are using Sphinx or Mkdocs and need to execute additional commands.
This is useful if you want to use our deafult build process,
but just run additional commands.

`Override the build process`_
If you want full control over your build. This option supports any tool that generates HTML as part of the build.

.. contents:: Table of contents
:local:
This option gives you *full control* over your build.
Read the Docs supports any tool that generates HTML.

Extend the build process
------------------------

In the normal build process,
the pre-defined jobs ``checkout``, ``system_dependencies``, ``create_environment``, ``install``, ``build`` and ``upload`` are executed.
However, Read the Docs exposes extra jobs to users so they can customize the build process by running shell commands.
These extra jobs are:
Read the Docs also exposes additional ujobs,
which allow you to customize the build process by running shell commands.

The jobs where users can customize our default build process are:

.. list-table::
:header-rows: 1
:widths: 25, 75

* - Step
- Customizable jobs
Expand All @@ -39,23 +39,22 @@ These extra jobs are:
* - Build
- ``pre_build``, ``post_build``
* - Upload
- There are no customizable jobs currently
- No customizable jobs currently

.. note::

Currently, the pre-defined jobs (``checkout``, ``system_dependencies``, etc) executed by Read the Docs cannot be overridden or skipped.

The pre-defined jobs (``checkout``, ``system_dependencies``, etc) cannot be overridden or skipped.
You can fully customize things in :ref:`build-customization:override the build process`.

These jobs can be declared by using a :doc:`/config-file/index` with the :ref:`config-file/v2:build.jobs` key on it.
Let's say the project requires commands to be executed *before* installing any dependency into the Python environment and *after* the build has finished.
In that case, a config file similar to this one can be used:
These jobs are defined using the :doc:`/config-file/v2` with the :ref:`config-file/v2:build.jobs` key.
This example configuration defines commands to be executed *before* installing and *after* the build has finished:

.. code-block:: yaml
:caption: .readthedocs.yaml

version: 2
build:
os: "ubuntu-20.04"
os: "ubuntu-22.04"
tools:
python: "3.10"
jobs:
Expand All @@ -67,7 +66,8 @@ In that case, a config file similar to this one can be used:
-F "version=${READTHEDOCS_VERSION}" https://example.com/webhooks/readthedocs/


There are some caveats to knowing when using user-defined jobs:
User-defined job limitations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The current working directory is at the root of your project's cloned repository
* Environment variables are expanded for each individual command (see :doc:`/reference/environment-variables`)
Expand All @@ -78,19 +78,19 @@ There are some caveats to knowing when using user-defined jobs:


``build.jobs`` examples
+++++++++++++++++++++++
~~~~~~~~~~~~~~~~~~~~~~~

We've included some common examples where using :ref:`config-file/v2:build.jobs` will be useful.
These examples may require some adaptation for each projects' use case,
we recommend you use them as a starting point.


Unshallow clone
~~~~~~~~~~~~~~~
Unshallow git clone
^^^^^^^^^^^^^^^^^^^

Read the Docs does not perform a full clone on ``checkout`` job to reduce network data and speed up the build process.
Because of this, extensions that depend on the full Git history will fail.
To avoid this, it's possible to unshallow the clone done by Read the Docs:
To avoid this, it's possible to unshallow the git clone:

.. code-block:: yaml
:caption: .readthedocs.yaml
Expand All @@ -106,7 +106,7 @@ To avoid this, it's possible to unshallow the clone done by Read the Docs:


Cancel build based on a condition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When a command exits with code ``183``,
Read the Docs will cancel the build immediately.
Expand Down Expand Up @@ -170,7 +170,7 @@ This other example shows how to cancel a build if the commit message contains ``


Generate documentation from annotated sources with Doxygen
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It's possible to run Doxygen as part of the build process to generate documentation from annotated sources:

Expand All @@ -190,7 +190,7 @@ It's possible to run Doxygen as part of the build process to generate documentat


Use MkDocs extensions with extra required steps
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There are some MkDocs extensions that require specific commands to be run to generate extra pages before performing the build.
For example, `pydoc-markdown <http://niklasrosenstein.github.io/pydoc-markdown/>`_
Expand All @@ -209,7 +209,7 @@ For example, `pydoc-markdown <http://niklasrosenstein.github.io/pydoc-markdown/>


Avoid having a dirty Git index
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Read the Docs needs to modify some files before performing the build to be able to integrate with some of its features.
Because of this reason, it could happen the Git index gets dirty (it will detect modified files).
Expand All @@ -231,7 +231,7 @@ In that case, the Git index can be updated to ignore the files that Read the Doc


Perform a check for broken links
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Sphinx comes with a `linkcheck <https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.linkcheck.CheckExternalLinksBuilder>`_ builder that checks for broken external links included in the project's documentation.
This helps ensure that all external links are still valid and readers aren't linked to non-existent pages.
Expand All @@ -251,7 +251,7 @@ This helps ensure that all external links are still valid and readers aren't lin


Support Git LFS (Large File Storage)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In case the repository contains large files that are tracked with Git LFS,
there are some extra steps required to be able to download their content.
Expand Down Expand Up @@ -284,7 +284,7 @@ It's possible to use ``post_checkout`` user-defined job for this.


Install Node.js dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It's possible to install Node.js together with the required dependencies by using :term:`user-defined build jobs`.
To setup it, you need to define the version of Node.js to use and install the dependencies by using ``build.jobs.post_install``:
Expand All @@ -307,7 +307,7 @@ To setup it, you need to define the version of Node.js to use and install the de


Install dependencies with Poetry
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Projects managed with `Poetry <https://python-poetry.org/>`__,
can use the ``post_create_environment`` user-defined job to use Poetry for installing Python dependencies.
Expand Down Expand Up @@ -344,30 +344,53 @@ Override the build process

.. warning::

This feature is in a *beta phase* and could suffer incompatible changes or even removed completely in the near feature.
It does not yet support some of the Read the Docs' features like the :term:`flyout menu`, and ads.
However, we do plan to support these features in the future.
Use this feature at your own risk.

This feature is in *beta* and could change without warning.
It does not yet support some of the Read the Docs' features like the :term:`flyout menu`.
We do our best to not break existing configurations,
but use this feature at your own risk.

If your project requires full control of the build process,
and :ref:`extending the build process <build-customization:extend the build process>` is not enough,
all the commands executed during builds can be overridden using the :ref:`config-file/v2:build.commands` configuration file key.
all the commands executed during builds can be overridden using the :ref:`config-file/v2:build.commands`.

As Read the Docs does not have control over the build process,
you are responsible for running all the commands required to install requirements and build your project properly.
Once the build process finishes, the contents of the ``_readthedocs/html/`` directory will be hosted.
you are responsible for running all the commands required to install requirements and build your project.

Where to put files
~~~~~~~~~~~~~~~~~~

It is your responsibility to generate HTML and other formats of your documentation using :ref:`config-file/v2:build.commands`.
The contents of the ``_readthedocs/<format>/`` directory will be hosted as part of your documentation.

Supported :ref:`formats <downloadable-documentation:accessing offline formats>` are:

* ``html``
* ``htmlzip``
* ``pdf``
* ``epub``

Search support
~~~~~~~~~~~~~~

Read the Docs will automatically index the content of all your HTML files,
respecting the :ref:`search <config-file/v2:search>` option.

You can access the search from the Read the Docs :term:`dashboard`,
or by using the :doc:`/server-side-search/api`.

.. note::

In order for Read the Docs to index your HTML files correctly,
they should follow the conventions described at :doc:`rtd-dev:search-integration`.

``build.commands`` examples
+++++++++++++++++++++++++++
~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section contains some examples that showcase what is possible with :ref:`config-file/v2:build.commands`.
This section contains examples that showcase what is possible with :ref:`config-file/v2:build.commands`.
Note that you may need to modify and adapt these examples depending on your needs.


Pelican
~~~~~~~
^^^^^^^

`Pelican <https://blog.getpelican.com/>`__ is a well-known static site generator that's commonly used for blogs and landing pages.
If you are building your project with Pelican you could use a configuration file similar to the following:
Expand All @@ -386,7 +409,7 @@ If you are building your project with Pelican you could use a configuration file


Docsify
~~~~~~~
^^^^^^^

`Docsify <https://docsify.js.org/>`__ generates documentation websites on the fly, without the need to build static HTML.
These projects can be built using a configuration file like this:
Expand All @@ -402,19 +425,3 @@ These projects can be built using a configuration file like this:
commands:
- mkdir --parents _readthedocs/html/
- cp --recursive docs/* _readthedocs/html/


Search support
++++++++++++++

Read the Docs will automatically index the content of all your HTML files,
respecting the :ref:`search <config-file/v2:search>` options from your config file.

You can access the search results from the :guilabel:`Search` tab of your project,
or by using the :doc:`/server-side-search/api`.

.. note::

In order for Read the Docs to index your HTML files correctly,
they should follow some of the conventions described
at :doc:`rtd-dev:search-integration`.
Loading