Merging upstream version 7.4.7.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

10 files changed, 312 insertions, 231 deletions

diff --git a/doc/development/tutorials/recipe.rst b/doc/development/tutorials/adding_domain.rst
index 683cc8c..8a00211 100644
--- a/doc/development/tutorials/recipe.rst
+++ b/doc/development/tutorials/adding_domain.rst
@@ -1,5 +1,7 @@
-Developing a "recipe" extension
-===============================
+.. _tutorial-adding-domain:
+
+Adding a reference domain
+=========================
 
 The objective of this tutorial is to illustrate roles, directives and domains.
 Once complete, we will be able to use this extension to describe a recipe and
@@ -41,7 +43,9 @@ For that, we will need to add the following elements to Sphinx:
 Prerequisites
 -------------
 
-We need the same setup as in :doc:`the previous extensions <todo>`. This time,
+We need the same setup as in
+:ref:`the previous extensions <tutorial-extend-build>`.
+This time,
 we will be putting out extension in a file called :file:`recipe.py`.
 
 Here is an example of the folder structure you might obtain:
@@ -77,7 +81,8 @@ The first thing to examine is the ``RecipeDirective`` directive:
    :linenos:
    :pyobject: RecipeDirective
 
-Unlike :doc:`helloworld` and :doc:`todo`, this directive doesn't derive from
+Unlike :ref:`tutorial-extending-syntax` and :ref:`tutorial-extend-build`,
+this directive doesn't derive from
 :class:`docutils.parsers.rst.Directive` and doesn't define a ``run`` method.
 Instead, it derives from :class:`sphinx.directives.ObjectDescription` and
 defines  ``handle_signature`` and ``add_target_and_index`` methods. This is
@@ -90,9 +95,10 @@ for this node.
 
 We also see that this directive defines ``has_content``, ``required_arguments``
 and ``option_spec``. Unlike the ``TodoDirective`` directive added in the
-:doc:`previous tutorial <todo>`, this directive takes a single argument, the
-recipe name, and an option, ``contains``, in addition to the nested
-reStructuredText in the body.
+:ref:`previous tutorial <tutorial-extend-build>`,
+this directive takes a single argument,
+the recipe name, and an option, ``contains``,
+in addition to the nested reStructuredText in the body.
 
 .. rubric:: The index classes
 
@@ -167,7 +173,8 @@ indices and our cross-referencing code use this feature.
 
 .. currentmodule:: sphinx.application
 
-:doc:`As always <todo>`, the ``setup`` function is a requirement and is used to
+:ref:`As always <tutorial-extend-build>`,
+the ``setup`` function is a requirement and is used to
 hook the various parts of our extension into Sphinx. Let's look at the
 ``setup`` function for this extension.
 
@@ -224,4 +231,7 @@ Further reading
 For more information, refer to the `docutils`_ documentation and
 :doc:`/extdev/index`.
 
+If you wish to share your extension across multiple projects or with others,
+check out the :ref:`third-party-extensions` section.
+
 .. _docutils: https://docutils.sourceforge.io/docs/
diff --git a/doc/development/tutorials/autodoc_ext.rst b/doc/development/tutorials/autodoc_ext.rst
index cfd23e7..fb2a917 100644
--- a/doc/development/tutorials/autodoc_ext.rst
+++ b/doc/development/tutorials/autodoc_ext.rst
@@ -1,7 +1,7 @@
 .. _autodoc_ext_tutorial:
 
-Developing autodoc extension for IntEnum
-========================================
+Developing autodoc extensions
+=============================
 
 The objective of this tutorial is to create an extension that adds
 support for new type for autodoc. This autodoc extension will format
@@ -27,8 +27,10 @@ We want to add following to autodoc:
 Prerequisites
 -------------
 
-We need the same setup as in :doc:`the previous extensions <todo>`. This time,
-we will be putting out extension in a file called :file:`autodoc_intenum.py`.
+We need the same setup as in
+:ref:`the previous extensions <tutorial-extend-build>`.
+This time, we will be putting out extension
+in a file called :file:`autodoc_intenum.py`.
 The :file:`my_enums.py` will contain the sample enums we will document.
 
 Here is an example of the folder structure you might obtain:
@@ -139,3 +141,9 @@ This will be the documentation file with auto-documentation directive:
    :caption: index.rst
 
    .. autointenum:: my_enums.Colors
+
+Further reading
+---------------
+
+If you wish to share your extension across multiple projects or with others,
+check out the :ref:`third-party-extensions` section.
diff --git a/doc/development/tutorials/examples/autodoc_intenum.py b/doc/development/tutorials/examples/autodoc_intenum.py
index c52bb4c..7a19a23 100644
--- a/doc/development/tutorials/examples/autodoc_intenum.py
+++ b/doc/development/tutorials/examples/autodoc_intenum.py
@@ -9,6 +9,7 @@ if TYPE_CHECKING:
     from docutils.statemachine import StringList
 
     from sphinx.application import Sphinx
+    from sphinx.util.typing import ExtensionMetadata
 
 
 class IntEnumDocumenter(ClassDocumenter):
@@ -52,6 +53,10 @@ class IntEnumDocumenter(ClassDocumenter):
             self.add_line('', source_name)
 
 
-def setup(app: Sphinx) -> None:
+def setup(app: Sphinx) -> ExtensionMetadata:
     app.setup_extension('sphinx.ext.autodoc')  # Require autodoc extension
     app.add_autodocumenter(IntEnumDocumenter)
+    return {
+        'version': '1',
+        'parallel_read_safe': True,
+    }
diff --git a/doc/development/tutorials/examples/helloworld.py b/doc/development/tutorials/examples/helloworld.py
index da29562..3f7e504 100644
--- a/doc/development/tutorials/examples/helloworld.py
+++ b/doc/development/tutorials/examples/helloworld.py
@@ -1,18 +1,33 @@
+from __future__ import annotations
+
 from docutils import nodes
-from docutils.parsers.rst import Directive
 
 from sphinx.application import Sphinx
+from sphinx.util.docutils import SphinxDirective, SphinxRole
 from sphinx.util.typing import ExtensionMetadata
 
 
-class HelloWorld(Directive):
-    def run(self):
-        paragraph_node = nodes.paragraph(text='Hello World!')
+class HelloRole(SphinxRole):
+    """A role to say hello!"""
+
+    def run(self) -> tuple[list[nodes.Node], list[nodes.system_message]]:
+        node = nodes.inline(text=f'Hello {self.text}!')
+        return [node], []
+
+
+class HelloDirective(SphinxDirective):
+    """A directive to say hello!"""
+
+    required_arguments = 1
+
+    def run(self) -> list[nodes.Node]:
+        paragraph_node = nodes.paragraph(text=f'hello {self.arguments[0]}!')
         return [paragraph_node]
 
 
 def setup(app: Sphinx) -> ExtensionMetadata:
-    app.add_directive('helloworld', HelloWorld)
+    app.add_role('hello', HelloRole())
+    app.add_directive('hello', HelloDirective)
 
     return {
         'version': '0.1',
diff --git a/doc/development/tutorials/examples/recipe.py b/doc/development/tutorials/examples/recipe.py
index 28d25f2..baf85fe 100644
--- a/doc/development/tutorials/examples/recipe.py
+++ b/doc/development/tutorials/examples/recipe.py
@@ -122,6 +122,7 @@ class RecipeDomain(Domain):
         'recipes': [],  # object list
         'recipe_ingredients': {},  # name -> object
     }
+    data_version = 0
 
     def get_full_qualified_name(self, node):
         return f'recipe.{node.arguments[0]}'
diff --git a/doc/development/tutorials/examples/todo.py b/doc/development/tutorials/examples/todo.py
index 2baac5c..4e9dc66 100644
--- a/doc/development/tutorials/examples/todo.py
+++ b/doc/development/tutorials/examples/todo.py
@@ -38,7 +38,7 @@ class TodoDirective(SphinxDirective):
 
         todo_node = todo('\n'.join(self.content))
         todo_node += nodes.title(_('Todo'), _('Todo'))
-        self.state.nested_parse(self.content, self.content_offset, todo_node)
+        todo_node += self.parse_content_to_nodes()
 
         if not hasattr(self.env, 'todo_all_todos'):
             self.env.todo_all_todos = []
@@ -132,6 +132,7 @@ def setup(app: Sphinx) -> ExtensionMetadata:
 
     return {
         'version': '0.1',
+        'env_version': 1,
         'parallel_read_safe': True,
         'parallel_write_safe': True,
     }
diff --git a/doc/development/tutorials/todo.rst b/doc/development/tutorials/extending_build.rst
index f23d8ad..a81c84b 100644
--- a/doc/development/tutorials/todo.rst
+++ b/doc/development/tutorials/extending_build.rst
@@ -1,14 +1,20 @@
-Developing a "TODO" extension
-=============================
+.. _tutorial-extend-build:
 
-The objective of this tutorial is to create a more comprehensive extension than
-that created in :doc:`helloworld`. Whereas that guide just covered writing a
-custom :term:`directive`, this guide adds multiple directives, along with custom
-nodes, additional config values and custom event handlers. To this end, we will
-cover a ``todo`` extension that adds capabilities to include todo entries in the
-documentation, and to collect these in a central place. This is similar the
-``sphinxext.todo`` extension distributed with Sphinx.
+Extending the build process
+===========================
 
+The objective of this tutorial is to create a more comprehensive extension than
+that created in :ref:`tutorial-extending-syntax`.
+Whereas that guide just covered writing
+a custom :term:`role` and :term:`directive`,
+this guide covers a more complex extension to the Sphinx build process;
+adding multiple directives,
+along with custom nodes, additional config values and custom event handlers.
+
+To this end, we will cover a ``todo`` extension
+that adds capabilities to include todo entries in the documentation,
+and to collect these in a central place.
+This is similar to the :mod:`sphinx.ext.todo` extension distributed with Sphinx.
 
 Overview
 --------
@@ -47,7 +53,8 @@ For that, we will need to add the following elements to Sphinx:
 Prerequisites
 -------------
 
-As with :doc:`helloworld`, we will not be distributing this plugin via PyPI so
+As with :ref:`tutorial-extending-syntax`,
+we will not be distributing this plugin via PyPI so
 once again we need a Sphinx project to call this from. You can use an existing
 project or create a new one using :program:`sphinx-quickstart`.
 
@@ -83,7 +90,8 @@ explain in detail shortly:
    :language: python
    :linenos:
 
-This is far more extensive extension than the one detailed in :doc:`helloworld`,
+This is far more extensive extension than the one detailed in
+:ref:`tutorial-extending-syntax`,
 however, we will will look at each piece step-by-step to explain what's
 happening.
 
@@ -250,7 +258,8 @@ ID as the anchor name.
 
 .. currentmodule:: sphinx.application
 
-As noted :doc:`previously <helloworld>`, the ``setup`` function is a requirement
+As noted :ref:`previously <tutorial-extending-syntax>`,
+the ``setup`` function is a requirement
 and is used to plug directives into Sphinx. However, we also use it to hook up
 the other parts of our extension. Let's look at our ``setup`` function:
 
@@ -361,6 +370,9 @@ Further reading
 For more information, refer to the `docutils`_ documentation and
 :doc:`/extdev/index`.
 
+If you wish to share your extension across multiple projects or with others,
+check out the :ref:`third-party-extensions` section.
+
 
 .. _docutils: https://docutils.sourceforge.io/docs/
 .. _Python path: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
diff --git a/doc/development/tutorials/extending_syntax.rst b/doc/development/tutorials/extending_syntax.rst
new file mode 100644
index 0000000..bab8037
--- /dev/null
+++ b/doc/development/tutorials/extending_syntax.rst
@@ -0,0 +1,223 @@
+.. _tutorial-extending-syntax:
+
+Extending syntax with roles and directives
+==========================================
+
+Overview
+--------
+
+The syntax of both reStructuredText and MyST can be extended
+by creating new **directives** - for block-level elements -
+and **roles** - for inline elements.
+
+In this tutorial we shall extend Sphinx to add:
+
+* A ``hello`` role, that will simply output the text ``Hello {text}!``.
+* A ``hello`` directive, that will simply output the text ``Hello {text}!``,
+  as a paragraph.
+
+For this extension, you will need some basic understanding of Python,
+and we shall also introduce aspects of the docutils_ API.
+
+Setting up the project
+----------------------
+
+You can either use an existing Sphinx project
+or create a new one using :program:`sphinx-quickstart`.
+
+With this we will add the extension to the project,
+within the :file:`source` folder:
+
+#. Create an :file:`_ext` folder in :file:`source`
+#. Create a new Python file in the :file:`_ext` folder called
+   :file:`helloworld.py`
+
+Here is an example of the folder structure you might obtain:
+
+.. code-block:: text
+
+      └── source
+          ├── _ext
+          │   └── helloworld.py
+          ├── conf.py
+          ├── index.rst
+
+
+Writing the extension
+---------------------
+
+Open :file:`helloworld.py` and paste the following code in it:
+
+.. literalinclude:: examples/helloworld.py
+   :language: python
+   :linenos:
+
+Some essential things are happening in this example:
+
+The role class
+...............
+
+Our new role is declared in the ``HelloRole`` class.
+
+.. literalinclude:: examples/helloworld.py
+   :language: python
+   :linenos:
+   :pyobject: HelloRole
+
+This class extends the :class:`.SphinxRole` class.
+The class contains a ``run`` method,
+which is a requirement for every role.
+It contains the main logic of the role and it
+returns a tuple containing:
+
+- a list of inline-level docutils nodes to be processed by Sphinx.
+- an (optional) list of system message nodes
+
+The directive class
+...................
+
+Our new directive is declared in the ``HelloDirective`` class.
+
+.. literalinclude:: examples/helloworld.py
+   :language: python
+   :linenos:
+   :pyobject: HelloDirective
+
+This class extends the :class:`.SphinxDirective` class.
+The class contains a ``run`` method,
+which is a requirement for every directive.
+It contains the main logic of the directive and it
+returns a list of block-level docutils nodes to be processed by Sphinx.
+It also contains a ``required_arguments`` attribute,
+which tells Sphinx how many arguments are required for the directive.
+
+What are docutils nodes?
+........................
+
+When Sphinx parses a document,
+it creates an "Abstract Syntax Tree" (AST) of nodes
+that represent the content of the document in a structured way,
+that is generally independent of any one
+input (rST, MyST, etc) or output (HTML, LaTeX, etc) format.
+It is a tree because each node can have children nodes, and so on:
+
+.. code-block:: xml
+
+      <document>
+         <paragraph>
+            <text>
+               Hello world!
+
+The docutils_ package provides many `built-in nodes <docutils nodes_>`_,
+to represent different types of content such as
+text, paragraphs, references, tables, etc.
+
+Each node type generally only accepts a specific set of direct child nodes,
+for example the ``document`` node should only contain "block-level" nodes,
+such as ``paragraph``, ``section``, ``table``, etc,
+whilst the ``paragraph`` node should only contain "inline-level" nodes,
+such as ``text``, ``emphasis``, ``strong``, etc.
+
+.. seealso::
+
+   The docutils documentation on
+   `creating directives <docutils directives_>`_, and
+   `creating roles <docutils roles_>`_.
+
+The ``setup`` function
+......................
+
+This function is a requirement.
+We use it to plug our new directive into Sphinx.
+
+.. literalinclude:: examples/helloworld.py
+   :language: python
+   :pyobject: setup
+
+The simplest thing you can do is to call the
+:meth:`.Sphinx.add_role` and :meth:`.Sphinx.add_directive` methods,
+which is what we've done here.
+For this particular call, the first argument is the name of the role/directive itself
+as used in a reStructuredText file.
+In this case, we would use ``hello``. For example:
+
+.. code-block:: rst
+
+   Some intro text here...
+
+   .. hello:: world
+
+   Some text with a :hello:`world` role.
+
+We also return the :ref:`extension metadata <ext-metadata>` that indicates the
+version of our extension, along with the fact that it is safe to use the
+extension for both parallel reading and writing.
+
+Using the extension
+-------------------
+
+The extension has to be declared in your :file:`conf.py` file to make Sphinx
+aware of it. There are two steps necessary here:
+
+#. Add the :file:`_ext` directory to the `Python path`_ using
+   ``sys.path.append``. This should be placed at the top of the file.
+
+#. Update or create the :confval:`extensions` list and add the extension file
+   name to the list
+
+For example:
+
+.. code-block:: python
+
+   import os
+   import sys
+
+   sys.path.append(os.path.abspath("./_ext"))
+
+   extensions = ['helloworld']
+
+.. tip::
+
+   Because we haven't installed our extension as a `Python package`_, we need to
+   modify the `Python path`_ so Sphinx can find our extension. This is why we
+   need the call to ``sys.path.append``.
+
+You can now use the extension in a file. For example:
+
+.. code-block:: rst
+
+   Some intro text here...
+
+   .. hello:: world
+
+   Some text with a :hello:`world` role.
+
+The sample above would generate:
+
+.. code-block:: text
+
+   Some intro text here...
+
+   Hello world!
+
+   Some text with a hello world! role.
+
+
+Further reading
+---------------
+
+This is the very basic principle of an extension
+that creates a new role and directive.
+
+For a more advanced example, refer to :ref:`tutorial-extend-build`.
+
+If you wish to share your extension across multiple projects or with others,
+check out the :ref:`third-party-extensions` section.
+
+.. _docutils: https://docutils.sourceforge.io/
+.. _docutils roles: https://docutils.sourceforge.io/docs/howto/rst-roles.html
+.. _docutils directives: https://docutils.sourceforge.io/docs/howto/rst-directives.html
+.. _docutils nodes: https://docutils.sourceforge.io/docs/ref/doctree.html
+.. _PyPI: https://pypi.org/
+.. _Python package: https://packaging.python.org/
+.. _Python path: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
diff --git a/doc/development/tutorials/helloworld.rst b/doc/development/tutorials/helloworld.rst
deleted file mode 100644
index 8940e3d..0000000
--- a/doc/development/tutorials/helloworld.rst
+++ /dev/null
@@ -1,189 +0,0 @@
-Developing a "Hello world" extension
-====================================
-
-The objective of this tutorial is to create a very basic extension that adds a
-new directive. This directive will output a paragraph containing "hello world".
-
-Only basic information is provided in this tutorial. For more information, refer
-to the :doc:`other tutorials <index>` that go into more details.
-
-.. warning::
-
-   For this extension, you will need some basic understanding of docutils_
-   and Python.
-
-
-Overview
---------
-
-We want the extension to add the following to Sphinx:
-
-* A ``helloworld`` directive, that will simply output the text "hello world".
-
-
-Prerequisites
--------------
-
-We will not be distributing this plugin via `PyPI`_ and will instead include it
-as part of an existing project. This means you will need to use an existing
-project or create a new one using :program:`sphinx-quickstart`.
-
-We assume you are using separate source (:file:`source`) and build
-(:file:`build`) folders. Your extension file could be in any folder of your
-project. In our case, let's do the following:
-
-#. Create an :file:`_ext` folder in :file:`source`
-#. Create a new Python file in the :file:`_ext` folder called
-   :file:`helloworld.py`
-
-Here is an example of the folder structure you might obtain:
-
-.. code-block:: text
-
-      └── source
-          ├── _ext
-          │   └── helloworld.py
-          ├── _static
-          ├── conf.py
-          ├── somefolder
-          ├── index.rst
-          ├── somefile.rst
-          └── someotherfile.rst
-
-
-Writing the extension
----------------------
-
-Open :file:`helloworld.py` and paste the following code in it:
-
-.. literalinclude:: examples/helloworld.py
-   :language: python
-   :linenos:
-
-Some essential things are happening in this example, and you will see them for
-all directives.
-
-.. rubric:: The directive class
-
-Our new directive is declared in the ``HelloWorld`` class.
-
-.. literalinclude:: examples/helloworld.py
-   :language: python
-   :linenos:
-   :lines: 5-9
-
-This class extends the docutils_' ``Directive`` class. All extensions that
-create directives should extend this class.
-
-.. seealso::
-
-   `The docutils documentation on creating directives <docutils directives_>`_
-
-This class contains a ``run`` method.  This method is a requirement and it is
-part of every directive.  It contains the main logic of the directive and it
-returns a list of docutils nodes to be processed by Sphinx. These nodes are
-docutils' way of representing the content of a document. There are many types of
-nodes available: text, paragraph, reference, table, etc.
-
-.. seealso::
-
-   `The docutils documentation on nodes <docutils nodes_>`_
-
-The ``nodes.paragraph`` class creates a new paragraph node. A paragraph
-node typically contains some text that we can set during instantiation using
-the ``text`` parameter.
-
-.. rubric:: The ``setup`` function
-
-.. currentmodule:: sphinx.application
-
-This function is a requirement. We use it to plug our new directive into
-Sphinx.
-
-.. literalinclude:: examples/helloworld.py
-   :language: python
-   :linenos:
-   :lines: 12-
-
-The simplest thing you can do is to call the :meth:`~Sphinx.add_directive` method,
-which is what we've done here. For this particular call, the first argument is
-the name of the directive itself as used in a reST file. In this case, we would
-use ``helloworld``. For example:
-
-.. code-block:: rst
-
-   Some intro text here...
-
-   .. helloworld::
-
-   Some more text here...
-
-We also return the :ref:`extension metadata <ext-metadata>` that indicates the
-version of our extension, along with the fact that it is safe to use the
-extension for both parallel reading and writing.
-
-
-Using the extension
--------------------
-
-The extension has to be declared in your :file:`conf.py` file to make Sphinx
-aware of it. There are two steps necessary here:
-
-#. Add the :file:`_ext` directory to the `Python path`_ using
-   ``sys.path.append``. This should be placed at the top of the file.
-
-#. Update or create the :confval:`extensions` list and add the extension file
-   name to the list
-
-For example:
-
-.. code-block:: python
-
-   import os
-   import sys
-
-   sys.path.append(os.path.abspath("./_ext"))
-
-   extensions = ['helloworld']
-
-.. tip::
-
-   We're not distributing this extension as a `Python package`_, we need to
-   modify the `Python path`_ so Sphinx can find our extension. This is why we
-   need the call to ``sys.path.append``.
-
-You can now use the extension in a file. For example:
-
-.. code-block:: rst
-
-   Some intro text here...
-
-   .. helloworld::
-
-   Some more text here...
-
-The sample above would generate:
-
-.. code-block:: text
-
-   Some intro text here...
-
-   Hello World!
-
-   Some more text here...
-
-
-Further reading
----------------
-
-This is the very basic principle of an extension that creates a new directive.
-
-For a more advanced example, refer to :doc:`todo`.
-
-
-.. _docutils: https://docutils.sourceforge.io/
-.. _docutils directives: https://docutils.sourceforge.io/docs/howto/rst-directives.html
-.. _docutils nodes: https://docutils.sourceforge.io/docs/ref/doctree.html
-.. _PyPI: https://pypi.org/
-.. _Python package: https://packaging.python.org/
-.. _Python path: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
diff --git a/doc/development/tutorials/index.rst b/doc/development/tutorials/index.rst
index a7eee48..0c5c920 100644
--- a/doc/development/tutorials/index.rst
+++ b/doc/development/tutorials/index.rst
@@ -1,17 +1,12 @@
 .. _extension-tutorials-index:
 
-Extension tutorials
-===================
-
-Refer to the following tutorials to get started with extension development.
-
+Tutorials
+=========
 
 .. toctree::
-   :caption: Directive tutorials
-   :maxdepth: 1
+   :maxdepth: 2
 
-   helloworld
-   todo
-   recipe
+   extending_syntax
+   extending_build
+   adding_domain
    autodoc_ext
-