Skip to content

Commit

Permalink
Docs: Split Custom Domains as Explanation and How-to Guide (Diátaxis) (…
Browse files Browse the repository at this point in the history 
…#9676)

* Diatáxis refactor: Move Custom Domains to the How-to section

* Proposal: Split up Custom Domains into two articles, no new contents added (step 1 of 2)

* Try to make "Custom Domain" feature description more self-contained

* Update cross-references to match new location

* Update docs/user/guides/custom-domains.rst

Co-authored-by: Anthony <[email protected]>

* Update docs/user/custom-domains.rst

Co-authored-by: Anthony <[email protected]>

* Update seealso:: box with a better pattern

* Remove redundant section label

* Add custom-domains to explanation section, add a "What to consider" section

* Replace "central" with "reliable"

* Remove the example, there's already enough explanation

* Add mention of canonical domains

* Highlight **canonical** at first apperance, then emphasize its importance

* Try to simplify explanation and adds CNAME to the equation.

* Fix language

* Remove "by default"

* Apply suggestions from code review @ericholscher

Co-authored-by: Eric Holscher <[email protected]>

* Use Dashboard term

* Fix up the canonical domain explanation

* Change to use :guilabel: in explanation of where to add Custom Domains

* Try to build an SVG diagram with sphinxcontrib-mermaid and mmdc

* Start a real diagram, only build with mmdc if in a Read the Docs build

* Disable SVG generation with mermaid-js

* Swap direction of diagram, add some more icons

* Also illustrate the build process

* Sketching: Move contents of Canonical URLs to Custom Domains explanation, it reads well...

* Tweak text, sembr

* Canonical URLs again in separate article. More cross-references in Custom Domains. conf.py Comment that :doc: isn't supported yet

* Make sure to mention cache invalidation for Read the Docs for Business

* Replace "See more" with a better caption

* Refer using seealso::

* Text tweaks to Canonical URLs explanation

* Move How-to content to a separate how-to

* Updates to the original Canonical URLs how-to sections and further references

* Revert back to previous phrase, it reads better

* Update docs/user/canonical-urls.rst

Co-authored-by: Anthony <[email protected]>

* Apply suggestions from code review from @agjohnson and @benjaoming

Co-authored-by: Anthony <[email protected]>

* Update docs/user/custom-domains.rst

Co-authored-by: Anthony <[email protected]>

* Apply suggestions from @ericholscher code review

Co-authored-by: Eric Holscher <[email protected]>

* Downgrade cross-reference from a seealso to normal text re:@ericholsher

* Add a cautious message to anyone that would want to define html_baseurl.

* parenthesis clarified

* Add a shortcut convention for referencing issues

* Add example code for MkDocs which unfortunately requires a bit of elaboration

* Updates diagram and intro text to be easier to digest

* Add a bold text that a project needs to be rebuilt. Move the note about ANAME/CNAME.

* Turn support seealso into a note

* Apply suggestions from @ericholscher code review

Co-authored-by: Eric Holscher <[email protected]>

* Convert Mermaid diagram to SVG, remove sphinxcontrib-mermaid

* Add a closing backtick, remove image alignment (causes wrong vertical margins)

* Update docs/user/guides/canonical-urls.rst

* Manually create a PNG version (not as straight-forward as assumed)

Co-authored-by: Anthony <[email protected]>
Co-authored-by: Eric Holscher <[email protected]>
  • Loading branch information
@benjaoming @agjohnson @ericholscher
3 people authored Dec 9, 2022
1 parent 20a42cd commit fb53cab
Show file tree
Hide file tree
Showing 13 changed files with 312 additions and 131 deletions.
12 changes: 10 additions & 2 deletions docs/conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"multiproject",
"sphinx.ext.autosectionlabel",
"sphinx.ext.autodoc",
"sphinx.ext.extlinks",
"sphinx.ext.intersphinx",
"sphinxcontrib.httpdomain",
"sphinxcontrib.video",
Expand Down Expand Up @@ -159,8 +160,11 @@
hoverxref_domains = ["py"]
hoverxref_roles = [
"option",
"doc", # Documentation pages
"term", # Glossary terms
# Documentation pages
# Not supported yet: https://github.com/readthedocs/sphinx-hoverxref/issues/18
"doc",
# Glossary terms
"term",
]
hoverxref_role_types = {
"mod": "modal", # for Python Sphinx Domain
Expand Down Expand Up @@ -206,3 +210,7 @@
# This page is under login
r"https://readthedocs\.org/accounts/gold",
]

extlinks = {
"rtd-issue": ("https://github.com/readthedocs/readthedocs.org/issues/%s", "#"),
}
49 changes: 20 additions & 29 deletions docs/user/canonical-urls.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,34 @@ Canonical URLs

A `canonical URL`_
allows you to specify the preferred version of a web page to prevent duplicated content.
They are mainly used by search engines to link users to the correct
version and domain of your documentation.
Here are some examples of when a canonical URL is used:

If canonical URL's aren't used,
- Search engines use your canonical URL to link users to the correct version and domain of your documentation.
- Many popular chat clients and social media networks generate link previews,
using your canonical URL as the final destination.

If canonical URLs aren't used,
it's easy for outdated documentation to be the top search result for various pages in your documentation.
This is not a perfect solution for this problem,
but generally people finding outdated documentation is a big problem,
and this is one of the suggested ways to solve it from search engines.

.. _canonical URL: https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urls

.. tip::

In most cases, Read the Docs will automatically generate a canonical URL for Sphinx projects.
Most Sphinx users do not need to take further action.

.. seealso::

:doc:`/guides/canonical-urls`
More information on how to enable canonical URLs in your project.

How Read the Docs generates canonical URLs
------------------------------------------

The canonical URL takes into account:
The canonical URL takes the following into account:

* The default version of your project (usually "latest" or "stable").
* The canonical :doc:`custom domain </custom-domains>` if you have one,
Expand All @@ -40,32 +53,10 @@ thus avoiding duplicating the content.
Implementation
--------------

The canonical URL is set in HTML with a ``link`` element.
For example, this page has a canonical URL of:
A canonical URL is automatically specified in the HTML output with a ``<link>`` element.
For instance, regardless of whether you are viewing this page on ``/en/latest`` or ``/en/stable``,
the following HTML header data will be present:

.. code-block:: html

<link rel="canonical" href="https://docs.readthedocs.io/en/stable/canonical-urls.html" />

Sphinx
~~~~~~

If you are using :doc:`Sphinx </intro/getting-started-with-sphinx>`,
Read the Docs will set the value of the html_baseurl_ setting (if isn't already set) to your canonical domain.
If you already have ``html_baseurl`` set, you need to ensure that the value is correct.

.. _html_baseurl: https://www.sphinx-doc.org/page/usage/configuration.html#confval-html_baseurl

Mkdocs
~~~~~~

For :doc:`MkDocs </intro/getting-started-with-mkdocs>` this isn't done automatically,
but you can use the site_url_ setting to set a similar value.

.. _site_url: https://www.mkdocs.org/user-guide/configuration/#site_url

.. warning::

If you change your default version or canonical domain,
you'll need to re-build all your versions in order to update their
canonical URL to the new one.
161 changes: 68 additions & 93 deletions docs/user/custom-domains.rst
View file
Original file line number Diff line number Diff line change
@@ -1,126 +1,101 @@
Custom Domains
==============

Custom domains allow you to serve your documentation from your own domain.
This is great for maintaining a consistent brand for your documentation and application.
You can serve your documentation project from your own domain,
for instance ``docs.example.com``.
This is great for maintaining a consistent brand for your product and its documentation.

By default, your documentation is served from a Read the Docs :ref:`subdomain <hosting:subdomain support>` using the project's :term:`slug`:
*By default*, your documentation is served from a Read the Docs :ref:`subdomain <hosting:subdomain support>` using the project's :term:`slug`:

* ``<slug>.readthedocs.io`` for |org_brand|
* ``<slug>.readthedocs-hosted.com`` for |com_brand|.

For example if you import your project and it gets the :term:`slug` ``example-docs``, it will be served from ``https://example-docs.readthedocs.io``.
.. seealso::

.. contents:: Contents
:local:
:doc:`/guides/custom-domains`
Information on creating and managing custom domains,
and common configurations you might use to set up your domain

Adding a custom domain
----------------------
How custom domains work
-----------------------

To setup your custom domain, follow these steps:
To use a custom domain, two actions are needed from you:

#. Go the :guilabel:`Admin` tab of your project.
#. Click on :guilabel:`Domains`.
#. Enter the domain where you want to serve the documentation from (e.g. ``docs.example.com``).
#. Mark the :guilabel:`Canonical` option if you want use this domain
as your :doc:`canonical domain </canonical-urls>`.
#. Click on :guilabel:`Add`.
#. At the top of the next page you'll find the value of the DNS record that you need to point your domain to.
For |org_brand| this is ``readthedocs.io``, and for :doc:`/commercial/index`
the record is in the form of ``<hash>.domains.readthedocs.com``.
#. Enter the domain in your Read the Docs project's :guilabel:`Admin`
#. Update your DNS provider with a new DNS entry. The name and value of the DNS entry is found in Read the Docs' :guilabel:`Admin`.

.. note::
Once the new DNS record has propagated,
Read the Docs automatically issues an SSL certificate through Cloudflare and starts serving your documentation.

For a subdomain like ``docs.example.com`` add a CNAME record,
and for a root domain like ``example.com`` use an ANAME or ALIAS record.
.. image:: img/mermaid-custom-domains.png
:alt: Diagram of the process of adding a custom domain on Read the Docs

By default, we provide a validated SSL certificate for the domain,
managed by `Cloudflare <https://www.cloudflare.com/>`_.
The SSL certificate issuance should happen within a few minutes,
but might take up to one hour.
See `SSL certificate issue delays`_ for more troubleshooting options.
..
We have generated an PNG version of the following diagram using mermaid.live
Firstly, we generate an SVG, then we render it in a browser, then we take a screenshot,
then we paste it into GIMP or similar and make the background transparent.
As an example, our blog's DNS record looks like this:
If you wish to sketch diagrams locally, you can add sphinxcontrib-mermaid to
this project's extensions and keep using the below code.

.. prompt:: bash $, auto
SVG does not work because it embeds fontawesome from CDN (which is blocked by CSP)

$ dig +short CNAME blog.readthedocs.com
readthedocs.io.
PLEASE KEEP THIS SOURCE CODE UPDATED
.. mermaid::

.. warning::
graph TD
subgraph rtd [On Read the Docs]
A(fa:fa-pencil Add docs.example.com as Custom Domain)
end
subgraph dns [On your domain's DNS administration]
B(fa:fa-pencil Edit/add a DNS entry for docs.example.com<br>making it point to Read the Docs)
end

We don't support pointing subdomains or root domains to a project using A records.
DNS A records require a static IP address and our IPs may change without notice.
rtd & dns-->C(fa:fa-spinner Wait for DNS propagation.<br>Usually just a few minutes)

direction LR
subgraph automatic [fa:fa-paper-plane The rest is handled automatically]
direction TB
D(fa:fa-spinner The next time your project is built,<br>its Canonical URLs use docs.example.com)
D-->E(Visit https://docs.example.com)
E-->F(fa:fa-lock Correct SSL Certificate <br>automatically used)
F-->G(fa:fa-check Read the Docs knows<br> to serve your project <br>at docs.example.com)
end

Removing a custom domain
------------------------
C-->automatic

To remove a custom domain:

#. Go the :guilabel:`Admin` tab of your project.
#. Click on :guilabel:`Domains`.
#. Click the :guilabel:`Remove` button next to the domain.
#. Click :guilabel:`Confirm` on the confirmation page.
Your documentation can have multiple secondary domains but only one **canonical** domain name.
Additional domains or subdomains will redirect to the canonical domain.

.. warning::
To make this work, Read the Docs generates a special text that you are responsible for copy-pasting to your domain's DNS.
In most cases, the ``CNAME`` record is used.
This is all that's needed for a web browser to resolve your domain name to Read the Docs' servers and for our servers to match the right documentation project.
You can find step-by-step instructions for this in :doc:`/guides/custom-domains`.

Once a domain is removed,
your previous documentation domain is no longer served by Read the Docs,
and any request for it will return a 404 Not Found!
Read the Docs uses a :ref:`hosting:Content Delivery Network (CDN)` to host and serve your documentation pages.
This final step isn't changed by a custom domain
and therefore the response times are unaffected as the delivery of resources happens through the same CDN setup.

Strict Transport Security (HSTS) and other custom headers
---------------------------------------------------------
Considerations for custom domain usage
--------------------------------------

By default, we do not return a `Strict Transport Security header`_ (HSTS) for user custom domains.
This is a conscious decision as it can be misconfigured in a not easily reversible way.
For both |org_brand| and |com_brand|, HSTS and other custom headers can be set upon request.
Some open source projects have seen their domains expire.
Even prominent ones.
**It's important that you give the responsibility for managing your domain to someone reliable in your organization.**

We always return the HSTS header with a max-age of at least one year
for our own domains including ``*.readthedocs.io``, ``*.readthedocs-hosted.com``, ``readthedocs.org`` and ``readthedocs.com``.
The **canonical domain** feature allows you to have several domains and the canonical domain will be indexed by search engines.
The domain that you choose as your canonical domain is by far the most important one.
If you lose the canonical domain,
someone else can set up a website that search results will end up referring to.

Please contact :doc:`support` if you want to add a custom header to your domain.
.. seealso::

.. _Strict Transport Security header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security
In a URL, both the domain and the path (``https://<domain>/<path>``) are important.
In combination, they are referred to as the *canonical URL* of a resource.

Multiple documentation sites as sub-folders of a domain
-------------------------------------------------------
Most documentation projects are versioned.
Therefore, it's important to ensure that incoming links and search engine results point to the canonical URL of the resource
and not a specific version that becomes outdated.

You may host multiple documentation repositories as **sub-folders of a single domain**.
For example, ``docs.example.org/projects/repo1`` and ``docs.example.org/projects/repo2``.
This is `a way to boost the SEO of your website <https://moz.com/blog/subdomains-vs-subfolders-rel-canonical-vs-301-how-to-structure-links-optimally-for-seo-whiteboard-friday>`_.

See :doc:`subprojects` for more information.

Troubleshooting
---------------

SSL certificate issue delays
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The status of your domain validation and certificate can always be seen on the details page for your domain
under :guilabel:`Admin` > :guilabel:`Domains` > :guilabel:`YOURDOMAIN.TLD (details)`.

Domains are usually validated and a certificate issued within minutes.
However, if you setup the domain in Read the Docs without provisioning the necessary DNS changes
and then update DNS hours or days later,
this can cause a delay in validating because there is an exponential back-off in validation.

.. tip::

Loading the domain details in the Read the Docs dashboard and saving the domain again will force a revalidation.

The validation process period has ended
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

After you add a new custom domain, you have 30 days to complete the configuration.
Once that period has ended, we will stop trying to validate your domain.
If you still want to complete the configuration,
go to your domain and click on :guilabel:`Save` to restart the process.

Migrating from GitBook
~~~~~~~~~~~~~~~~~~~~~~

If your custom domain was previously used in GitBook, contact GitBook support (via live chat in their website)
to remove the domain name from their DNS Zone in order for your domain name to work with Read the Docs,
else it will always redirect to GitBook.
To learn more about canonical URLs, see: :doc:`/canonical-urls`
6 changes: 6 additions & 0 deletions docs/user/glossary.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ Glossary
and hypens. You can retreive your project or version slugs from
:doc:`our API <api/v3>`.

subproject
Project A can be configured such that when requesting a URL ``/projects/<subproject-slug>``,
the root of project B is returned.
In this case, *project B* is the subproject.
Read more in :doc:`/subprojects`.

root URL
Home URL of your documentation without the ``/<lang>`` and ``/<version>`` segments.
For projects without custom domains, the one ending in ``.readthedocs.io/``
Expand Down
2 changes: 2 additions & 0 deletions docs/user/guides/administrators.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ have a look at our :doc:`/tutorial/index`.
.. toctree::
:maxdepth: 1

Manage Custom Domains <custom-domains>
Enable Canonical URLs <canonical-urls>
technical-docs-seo-guide
manage-translations-sphinx
hiding-a-version
Expand Down
58 changes: 58 additions & 0 deletions docs/user/guides/canonical-urls.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
How to Enable Canonical URLs
============================

In this guide, we introduce relevant settings for enabling canonical URLs in popular documentation frameworks.

If you need to customize the domain from which your documentation project is served,
please refer to :doc:`/guides/custom-domains`.

Sphinx
~~~~~~

If you are using :doc:`Sphinx </intro/getting-started-with-sphinx>`,
Read the Docs will automatically add a default value of the html_baseurl_ setting matching your canonical domain.

If you are using a custom ``html_baseurl`` in your ``conf.py``,
you have to ensure that the value is correct.
This can be complex,
supporting pull request builds (which are published on a separate domain),
special branches
or if you are using :term:`subproject` s or :ref:`translations <localization:Localization of Documentation>`.
We recommend not including a ``html_baseurl`` in your ``conf.py``,
and letting Read the Docs define it.

.. _html_baseurl: https://www.sphinx-doc.org/page/usage/configuration.html#confval-html_baseurl

MkDocs
~~~~~~

For :doc:`MkDocs </intro/getting-started-with-mkdocs>` we do not define your canonical domain automatically,
but you can use the site_url_ setting to set a similar value.

In your ``mkdocs.yml``, define the following:

.. code-block:: yaml
# Canonical URL, adjust as need with respect to your slug, language,
# default branch and if you use a custom domain.
site_url: https://<slug>.readthedocs.io/en/stable/
Note that this will define the same canonical URL for all your branches and versions.
According to MkDocs, defining site_url_ will only define the canonical URL of a website and does not affect the base URL of generated links, CSS, or Javascript files.

.. note::

2 known issues are currently making it impossible to use `environment variables in MkDocs configuration`_.
Once these issues are solved, it will be easier.

- Support for ``!ENV``: :rtd-issue:`8529`
- Add environment variable for canonical URL: :rtd-issue:`9781`

.. _site_url: https://www.mkdocs.org/user-guide/configuration/#site_url
.. _environment variables in MkDocs configuration: https://www.mkdocs.org/user-guide/configuration/#environment-variables

.. warning::

If you change your default version or canonical domain,
you'll need to re-build all your versions in order to update their
canonical URL to the new one.
Loading

0 comments on commit fb53cab

Please sign in to comment.