Adding upstream version 7.4.7.upstream/7.4.7upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

8 files changed, 593 insertions, 340 deletions

diff --git a/doc/extdev/appapi.rst b/doc/extdev/appapi.rst
index 10d030b..682db4e 100644
--- a/doc/extdev/appapi.rst
+++ b/doc/extdev/appapi.rst
@@ -1,4 +1,4 @@
-.. highlight:: rest
+.. highlight:: rst
 
 Application API
 ===============
@@ -137,294 +137,12 @@ The application object also provides runtime information as attributes.
 
    Directory for storing built document.
 
-
-.. _events:
+.. autoattribute:: Sphinx.fresh_env_used
 
 Sphinx core events
 ------------------
 
-These events are known to the core.  The arguments shown are given to the
-registered event handlers.  Use :meth:`.Sphinx.connect` in an extension's
-``setup`` function (note that ``conf.py`` can also have a ``setup`` function) to
-connect handlers to the events.  Example:
-
-.. code-block:: python
-
-   def source_read_handler(app, docname, source):
-       print('do something here...')
-
-   def setup(app):
-       app.connect('source-read', source_read_handler)
-
-
-Below is an overview of each event that happens during a build. In the list
-below, we include the event name, its callback parameters, and the input and output
-type for that event:
-
-.. code-block:: none
-
-   1. event.config-inited(app,config)
-   2. event.builder-inited(app)
-   3. event.env-get-outdated(app, env, added, changed, removed)
-   4. event.env-before-read-docs(app, env, docnames)
-
-   for docname in docnames:
-      5. event.env-purge-doc(app, env, docname)
-
-      if doc changed and not removed:
-         6. source-read(app, docname, source)
-         7. run source parsers: text -> docutils.document
-            - parsers can be added with the app.add_source_parser() API
-         8. apply transforms based on priority: docutils.document -> docutils.document
-            - event.doctree-read(app, doctree) is called in the middle of transforms,
-              transforms come before/after this event depending on their priority.
-
-   9. event.env-merge-info(app, env, docnames, other)
-      - if running in parallel mode, this event will be emitted for each process
-
-   10. event.env-updated(app, env)
-   11. event.env-get-updated(app, env)
-   12. event.env-check-consistency(app, env)
-
-   # The updated-docs list can be builder dependent, but generally includes all new/changed documents,
-   # plus any output from `env-get-updated`, and then all "parent" documents in the ToC tree
-   # For builders that output a single page, they are first joined into a single doctree before post-transforms
-   # or the doctree-resolved event is emitted
-   for docname in updated-docs:
-      13. apply post-transforms (by priority): docutils.document -> docutils.document
-      14. event.doctree-resolved(app, doctree, docname)
-          - In the event that any reference nodes fail to resolve, the following may emit:
-          - event.missing-reference(env, node, contnode)
-          - event.warn-missing-reference(domain, node)
-
-   15. Generate output files
-   16. event.build-finished(app, exception)
-
-Here is a more detailed list of these events.
-
-.. event:: builder-inited (app)
-
-   Emitted when the builder object has been created.  It is available as
-   ``app.builder``.
-
-.. event:: config-inited (app, config)
-
-   Emitted when the config object has been initialized.
-
-   .. versionadded:: 1.8
-
-.. event:: env-get-outdated (app, env, added, changed, removed)
-
-   Emitted when the environment determines which source files have changed and
-   should be re-read.  *added*, *changed* and *removed* are sets of docnames
-   that the environment has determined.  You can return a list of docnames to
-   re-read in addition to these.
-
-   .. versionadded:: 1.1
-
-.. event:: env-purge-doc (app, env, docname)
-
-   Emitted when all traces of a source file should be cleaned from the
-   environment, that is, if the source file is removed or before it is freshly
-   read.  This is for extensions that keep their own caches in attributes of the
-   environment.
-
-   For example, there is a cache of all modules on the environment.  When a
-   source file has been changed, the cache's entries for the file are cleared,
-   since the module declarations could have been removed from the file.
-
-   .. versionadded:: 0.5
-
-.. event:: env-before-read-docs (app, env, docnames)
-
-   Emitted after the environment has determined the list of all added and
-   changed files and just before it reads them.  It allows extension authors to
-   reorder the list of docnames (*inplace*) before processing, or add more
-   docnames that Sphinx did not consider changed (but never add any docnames
-   that are not in ``env.found_docs``).
-
-   You can also remove document names; do this with caution since it will make
-   Sphinx treat changed files as unchanged.
-
-   .. versionadded:: 1.3
-
-.. event:: source-read (app, docname, source)
-
-   Emitted when a source file has been read.  The *source* argument is a list
-   whose single element is the contents of the source file.  You can process the
-   contents and replace this item to implement source-level transformations.
-
-   For example, if you want to use ``$`` signs to delimit inline math, like in
-   LaTeX, you can use a regular expression to replace ``$...$`` by
-   ``:math:`...```.
-
-   .. versionadded:: 0.5
-
-.. event:: include-read (app, relative_path, parent_docname, content)
-
-   Emitted when a file has been read with the :dudir:`include` directive.
-   The *relative_path* argument is a :py:class:`~pathlib.Path` object representing
-   the relative path of the included file from the :term:`source directory`.
-   The *parent_docname* argument is the name of the document that
-   contains the :dudir:`include` directive.
-   The *source* argument is a list whose single element is
-   the contents of the included file.
-   You can process the contents and replace this item
-   to transform the included content,
-   as with the :event:`source-read` event.
-
-   .. versionadded:: 7.2.5
-
-   .. seealso:: The :dudir:`include` directive and the :event:`source-read` event.
-
-.. event:: object-description-transform (app, domain, objtype, contentnode)
-
-   Emitted when an object description directive has run.  The *domain* and
-   *objtype* arguments are strings indicating object description of the object.
-   And *contentnode* is a content for the object.  It can be modified in-place.
-
-   .. versionadded:: 2.4
-
-.. event:: doctree-read (app, doctree)
-
-   Emitted when a doctree has been parsed and read by the environment, and is
-   about to be pickled.  The *doctree* can be modified in-place.
-
-.. event:: missing-reference (app, env, node, contnode)
-
-   Emitted when a cross-reference to an object cannot be resolved.
-   If the event handler can resolve the reference, it should return a
-   new docutils node to be inserted in the document tree in place of the node
-   *node*.  Usually this node is a :class:`~nodes.reference` node containing
-   *contnode* as a child.
-   If the handler can not resolve the cross-reference,
-   it can either return ``None`` to let other handlers try,
-   or raise :class:`~sphinx.errors.NoUri` to prevent other handlers in
-   trying and suppress a warning about this cross-reference being unresolved.
-
-   :param env: The build environment (``app.builder.env``).
-   :param node: The :class:`~sphinx.addnodes.pending_xref` node to be resolved.
-      Its ``reftype``, ``reftarget``, ``modname`` and ``classname`` attributes
-      determine the type and target of the reference.
-   :param contnode: The node that carries the text and formatting inside the
-      future reference and should be a child of the returned reference node.
-
-   .. versionadded:: 0.5
-
-.. event:: warn-missing-reference (app, domain, node)
-
-   Emitted when a cross-reference to an object cannot be resolved even after
-   :event:`missing-reference`.  If the event handler can emit warnings for
-   the missing reference, it should return ``True``. The configuration variables
-   :confval:`nitpick_ignore` and :confval:`nitpick_ignore_regex` prevent the
-   event from being emitted for the corresponding nodes.
-
-   .. versionadded:: 3.4
-
-.. event:: doctree-resolved (app, doctree, docname)
-
-   Emitted when a doctree has been "resolved" by the environment, that is, all
-   references have been resolved and TOCs have been inserted.  The *doctree* can
-   be modified in place.
-
-   Here is the place to replace custom nodes that don't have visitor methods in
-   the writers, so that they don't cause errors when the writers encounter them.
-
-.. event:: env-merge-info (app, env, docnames, other)
-
-   This event is only emitted when parallel reading of documents is enabled.  It
-   is emitted once for every subprocess that has read some documents.
-
-   You must handle this event in an extension that stores data in the
-   environment in a custom location.  Otherwise the environment in the main
-   process will not be aware of the information stored in the subprocess.
-
-   *other* is the environment object from the subprocess, *env* is the
-   environment from the main process.  *docnames* is a set of document names
-   that have been read in the subprocess.
-
-   .. versionadded:: 1.3
-
-.. event:: env-updated (app, env)
-
-   Emitted after reading all documents, when the environment and all
-   doctrees are now up-to-date.
-
-   You can return an iterable of docnames from the handler.  These documents
-   will then be considered updated, and will be (re-)written during the writing
-   phase.
-
-   .. versionadded:: 0.5
-
-   .. versionchanged:: 1.3
-      The handlers' return value is now used.
-
-.. event:: env-check-consistency (app, env)
-
-   Emitted when Consistency checks phase.  You can check consistency of
-   metadata for whole of documents.
-
-   .. versionadded:: 1.6
-
-      As a **experimental** event
-
-.. event:: html-collect-pages (app)
-
-   Emitted when the HTML builder is starting to write non-document pages.  You
-   can add pages to write by returning an iterable from this event consisting of
-   ``(pagename, context, templatename)``.
-
-   .. versionadded:: 1.0
-
-.. event:: html-page-context (app, pagename, templatename, context, doctree)
-
-   Emitted when the HTML builder has created a context dictionary to render a
-   template with -- this can be used to add custom elements to the context.
-
-   The *pagename* argument is the canonical name of the page being rendered,
-   that is, without ``.html`` suffix and using slashes as path separators.  The
-   *templatename* is the name of the template to render, this will be
-   ``'page.html'`` for all pages from reST documents.
-
-   The *context* argument is a dictionary of values that are given to the
-   template engine to render the page and can be modified to include custom
-   values.  Keys must be strings.
-
-   The *doctree* argument will be a doctree when the page is created from a reST
-   documents; it will be ``None`` when the page is created from an HTML template
-   alone.
-
-   You can return a string from the handler, it will then replace
-   ``'page.html'`` as the HTML template for this page.
-
-   .. note:: You can install JS/CSS files for the specific page via
-             :meth:`Sphinx.add_js_file` and :meth:`Sphinx.add_css_file` since
-             v3.5.0.
-
-   .. versionadded:: 0.4
-
-   .. versionchanged:: 1.3
-      The return value can now specify a template name.
-
-.. event:: linkcheck-process-uri (app, uri)
-
-   Emitted when the linkcheck builder collects hyperlinks from document.  *uri*
-   is a collected URI.  The event handlers can modify the URI by returning a
-   string.
-
-   .. versionadded:: 4.1
-
-.. event:: build-finished (app, exception)
-
-   Emitted when a build has finished, before Sphinx exits, usually used for
-   cleanup.  This event is emitted even when the build process raised an
-   exception, given as the *exception* argument.  The exception is reraised in
-   the application after the event handlers have run.  If the build process
-   raised no exception, *exception* will be ``None``.  This allows to customize
-   cleanup actions depending on the exception status.
-
-   .. versionadded:: 0.5
-
+.. note:: Moved to :ref:`events`.
 
 Checking the Sphinx version
 ---------------------------
diff --git a/doc/extdev/builderapi.rst b/doc/extdev/builderapi.rst
index 5c5a525..9259aaa 100644
--- a/doc/extdev/builderapi.rst
+++ b/doc/extdev/builderapi.rst
@@ -3,15 +3,20 @@
 Builder API
 ===========
 
-.. todo:: Expand this.
-
 .. currentmodule:: sphinx.builders
 
 .. class:: Builder
 
    This is the base class for all builders.
 
-   These attributes should be set on builder classes:
+   It follows this basic workflow:
+
+   .. graphviz:: /_static/diagrams/sphinx_build_flow.dot
+      :caption: UML for the standard Sphinx build workflow
+
+   .. rubric:: Overridable Attributes
+
+   These attributes should be set on builder sub-classes:
 
    .. autoattribute:: name
    .. autoattribute:: format
@@ -22,24 +27,39 @@ Builder API
    .. autoattribute:: supported_data_uri_images
    .. autoattribute:: default_translator_class
 
-   These methods are predefined and will be called from the application:
+   .. rubric:: Core Methods
+
+   These methods are predefined and should generally not be overridden,
+   since they form the core of the build process:
 
-   .. automethod:: get_relative_uri
    .. automethod:: build_all
    .. automethod:: build_specific
    .. automethod:: build_update
    .. automethod:: build
+   .. automethod:: read
+   .. automethod:: read_doc
+   .. automethod:: write_doctree
 
-   These methods can be overridden in concrete builder classes:
+   .. rubric:: Overridable Methods
+
+   These must be implemented in builder sub-classes:
 
-   .. automethod:: init
    .. automethod:: get_outdated_docs
-   .. automethod:: get_target_uri
    .. automethod:: prepare_writing
    .. automethod:: write_doc
+   .. automethod:: get_target_uri
+
+   These methods can be overridden in builder sub-classes:
+
+   .. automethod:: init
+   .. automethod:: write
+   .. automethod:: copy_assets
+   .. automethod:: get_relative_uri
    .. automethod:: finish
 
-   **Attributes**
+   .. rubric:: Attributes
+
+   Attributes that are callable from the builder instance:
 
    .. attribute:: events
 
diff --git a/doc/extdev/deprecated.rst b/doc/extdev/deprecated.rst
index 1476ce8..7504be3 100644
--- a/doc/extdev/deprecated.rst
+++ b/doc/extdev/deprecated.rst
@@ -1769,7 +1769,7 @@ The following is a list of deprecated interfaces.
      - 3.0
      - ``warning()``
 
-   * - :confval:`source_parsers`
+   * - :confval:`!source_parsers`
      - 1.8
      - 3.0
      - :meth:`~sphinx.application.Sphinx.add_source_parser()`
diff --git a/doc/extdev/envapi.rst b/doc/extdev/envapi.rst
index d7ec239..971e506 100644
--- a/doc/extdev/envapi.rst
+++ b/doc/extdev/envapi.rst
@@ -45,6 +45,8 @@ Build environment API
 
    .. autoattribute:: docname
 
+   .. autoattribute:: parser
+
    **Utility methods**
 
    .. automethod:: doc2path
diff --git a/doc/extdev/event_callbacks.rst b/doc/extdev/event_callbacks.rst
new file mode 100644
index 0000000..1edade4
--- /dev/null
+++ b/doc/extdev/event_callbacks.rst
@@ -0,0 +1,424 @@
+.. _events:
+
+Event callbacks API
+===================
+
+Connecting callback functions to events is a simple way to extend Sphinx,
+by hooking into the build process at various points.
+
+Use :meth:`.Sphinx.connect` in an extension's ``setup`` function,
+or a ``setup`` function in your projects :file:`conf.py`,
+to connect functions to the events:
+
+.. code-block:: python
+
+   def source_read_handler(app, docname, source):
+       print('do something here...')
+
+   def setup(app):
+       app.connect('source-read', source_read_handler)
+
+.. seealso::
+
+   Extensions can add their own events by using :meth:`.Sphinx.add_event`,
+   and calling them them with
+   :meth:`.Sphinx.emit` or :meth:`.Sphinx.emit_firstresult`.
+
+Core events overview
+--------------------
+
+Below is an overview of the core event that happens during a build.
+
+.. code-block:: none
+
+   1. event.config-inited(app,config)
+   2. event.builder-inited(app)
+   3. event.env-get-outdated(app, env, added, changed, removed)
+   4. event.env-before-read-docs(app, env, docnames)
+
+   for docname in docnames:
+      5. event.env-purge-doc(app, env, docname)
+
+      if doc changed and not removed:
+         6. source-read(app, docname, source)
+         7. run source parsers: text -> docutils.document
+            - parsers can be added with the app.add_source_parser() API
+            - event.include-read(app, relative_path, parent_docname, content)
+              is called for each include directive
+         8. apply transforms based on priority: docutils.document -> docutils.document
+            - event.doctree-read(app, doctree) is called in the middle of transforms,
+              transforms come before/after this event depending on their priority.
+
+   9. event.env-merge-info(app, env, docnames, other)
+      - if running in parallel mode, this event will be emitted for each process
+
+   10. event.env-updated(app, env)
+   11. event.env-get-updated(app, env)
+
+   if environment is written to disk:
+      12. event.env-check-consistency(app, env)
+
+   13. event.write-started(app, builder)
+       - This is called after ``app.parallel_ok`` has been set,
+         which must not be altered by any event handler.
+
+   # The updated-docs list can be builder dependent, but generally includes all new/changed documents,
+   # plus any output from `env-get-updated`, and then all "parent" documents in the ToC tree
+   # For builders that output a single page, they are first joined into a single doctree before post-transforms
+   # or the doctree-resolved event is emitted
+   for docname in updated-docs:
+      14. apply post-transforms (by priority): docutils.document -> docutils.document
+      15. event.doctree-resolved(app, doctree, docname)
+          - In the event that any reference nodes fail to resolve, the following may emit:
+          - event.missing-reference(env, node, contnode)
+          - event.warn-missing-reference(domain, node)
+
+   16. Generate output files
+   17. event.build-finished(app, exception)
+
+Here is also a flow diagram of the events,
+within the context of the Sphinx build process:
+
+.. graphviz:: /_static/diagrams/sphinx_core_events_flow.dot
+   :caption: Sphinx core events flow
+
+Core event details
+------------------
+
+Here is a more detailed list of these events.
+
+.. event:: config-inited (app, config)
+
+   :param app: :class:`.Sphinx`
+   :param config: :class:`.Config`
+
+   Emitted when the config object has been initialized.
+
+   .. versionadded:: 1.8
+
+.. event:: builder-inited (app)
+
+   :param app: :class:`.Sphinx`
+
+   Emitted when the builder object has been created
+   (available as ``app.builder``).
+
+.. event:: env-get-outdated (app, env, added, changed, removed)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :param added: ``set[str]``
+   :param changed: ``set[str]``
+   :param removed: ``set[str]``
+   :returns: ``list[str]`` of additional docnames to re-read
+
+   Emitted when the environment determines which source files have changed and
+   should be re-read.
+   *added*, *changed* and *removed* are sets of docnames
+   that the environment has determined.
+   You can return a list of docnames to re-read in addition to these.
+
+   .. versionadded:: 1.1
+
+.. event:: env-purge-doc (app, env, docname)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :param docname: ``str``
+
+   Emitted when all traces of a source file should be cleaned from the
+   environment, that is, if the source file is removed or before it is freshly read.
+   This is for extensions that keep their own caches
+   in attributes of the environment.
+
+   For example, there is a cache of all modules on the environment.
+   When a source file has been changed, the cache's entries for the file are cleared,
+   since the module declarations could have been removed from the file.
+
+   .. versionadded:: 0.5
+
+.. event:: env-before-read-docs (app, env, docnames)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :param docnames: ``list[str]``
+
+   Emitted after the environment has determined the list of all added and
+   changed files and just before it reads them.
+   It allows extension authors to reorder
+   the list of docnames (*inplace*) before processing,
+   or add more docnames that Sphinx did not consider changed
+   (but never add any docnames that are not in :attr:`.found_docs`).
+
+   You can also remove document names; do this with caution since it will make
+   Sphinx treat changed files as unchanged.
+
+   .. versionadded:: 1.3
+
+.. event:: source-read (app, docname, content)
+
+   :param app: :class:`.Sphinx`
+   :param docname: ``str``
+   :param content: ``list[str]``
+      with a single element,
+      representing the content of the included file.
+
+   Emitted when a source file has been read.
+
+   You can process the ``content`` and
+   replace this item to implement source-level transformations.
+
+   For example, if you want to use ``$`` signs to delimit inline math, like in
+   LaTeX, you can use a regular expression to replace ``$...$`` by
+   ``:math:`...```.
+
+   .. versionadded:: 0.5
+
+.. event:: include-read (app, relative_path, parent_docname, content)
+
+   :param app: :class:`.Sphinx`
+   :param relative_path: :class:`~pathlib.Path`
+      representing the included file
+      relative to the :term:`source directory`.
+   :param parent_docname: ``str``
+      of the document name that
+      contains the :dudir:`include` directive.
+   :param content: ``list[str]``
+      with a single element,
+      representing the content of the included file.
+
+   Emitted when a file has been read with the :dudir:`include` directive.
+
+   You can process the ``content`` and replace this item
+   to transform the included content, as with the :event:`source-read` event.
+
+   .. versionadded:: 7.2.5
+
+   .. seealso:: The :dudir:`include` directive and the :event:`source-read` event.
+
+.. event:: object-description-transform (app, domain, objtype, contentnode)
+
+   :param app: :class:`.Sphinx`
+   :param domain: ``str``
+   :param objtype: ``str``
+   :param contentnode: :class:`.desc_content`
+
+   Emitted when an object description directive has run.  The *domain* and
+   *objtype* arguments are strings indicating object description of the object.
+   And *contentnode* is a content for the object.  It can be modified in-place.
+
+   .. versionadded:: 2.4
+
+.. event:: doctree-read (app, doctree)
+
+   :param app: :class:`.Sphinx`
+   :param doctree: :class:`docutils.nodes.document`
+
+   Emitted when a doctree has been parsed and read by the environment, and is
+   about to be pickled.
+   The ``doctree`` can be modified in-place.
+
+.. event:: missing-reference (app, env, node, contnode)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :param node: The :class:`.pending_xref` node to be resolved.
+      Its ``reftype``, ``reftarget``, ``modname`` and ``classname`` attributes
+      determine the type and target of the reference.
+   :param contnode: The node that carries the text and formatting inside the
+      future reference and should be a child of the returned reference node.
+   :returns: A new node to be inserted in the document tree in place of the node,
+      or ``None`` to let other handlers try.
+
+   Emitted when a cross-reference to an object cannot be resolved.
+   If the event handler can resolve the reference, it should return a
+   new docutils node to be inserted in the document tree in place of the node
+   *node*.  Usually this node is a :class:`~nodes.reference` node containing
+   *contnode* as a child.
+   If the handler can not resolve the cross-reference,
+   it can either return ``None`` to let other handlers try,
+   or raise :class:`~sphinx.errors.NoUri` to prevent other handlers in
+   trying and suppress a warning about this cross-reference being unresolved.
+
+   .. versionadded:: 0.5
+
+.. event:: warn-missing-reference (app, domain, node)
+
+   :param app: :class:`.Sphinx`
+   :param domain: The :class:`.Domain` of the missing reference.
+   :param node: The :class:`.pending_xref` node that could not be resolved.
+   :returns: ``True`` if a warning was emitted, else ``None``
+
+   Emitted when a cross-reference to an object cannot be resolved even after
+   :event:`missing-reference`.
+   If the event handler can emit warnings for the missing reference,
+   it should return ``True``.
+   The configuration variables
+   :confval:`nitpick_ignore` and :confval:`nitpick_ignore_regex`
+   prevent the event from being emitted for the corresponding nodes.
+
+   .. versionadded:: 3.4
+
+.. event:: doctree-resolved (app, doctree, docname)
+
+   :param app: :class:`.Sphinx`
+   :param doctree: :class:`docutils.nodes.document`
+   :param docname: ``str``
+
+   Emitted when a doctree has been "resolved" by the environment, that is, all
+   references have been resolved and TOCs have been inserted.  The *doctree* can
+   be modified in place.
+
+   Here is the place to replace custom nodes that don't have visitor methods in
+   the writers, so that they don't cause errors when the writers encounter them.
+
+.. event:: env-merge-info (app, env, docnames, other)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :param docnames: ``list[str]``
+   :param other: :class:`.BuildEnvironment`
+
+   This event is only emitted when parallel reading of documents is enabled.  It
+   is emitted once for every subprocess that has read some documents.
+
+   You must handle this event in an extension that stores data in the
+   environment in a custom location.  Otherwise the environment in the main
+   process will not be aware of the information stored in the subprocess.
+
+   *other* is the environment object from the subprocess, *env* is the
+   environment from the main process.  *docnames* is a set of document names
+   that have been read in the subprocess.
+
+   .. versionadded:: 1.3
+
+.. event:: env-updated (app, env)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :returns: iterable of ``str``
+
+   Emitted after reading all documents, when the environment and all
+   doctrees are now up-to-date.
+
+   You can return an iterable of docnames from the handler.  These documents
+   will then be considered updated, and will be (re-)written during the writing
+   phase.
+
+   .. versionadded:: 0.5
+
+   .. versionchanged:: 1.3
+      The handlers' return value is now used.
+
+.. event:: env-get-updated (app, env)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+   :returns: iterable of ``str``
+
+   Emitted when the environment determines which source files have changed and
+   should be re-read.
+   You can return an iterable of docnames to re-read.
+
+.. event:: env-check-consistency (app, env)
+
+   :param app: :class:`.Sphinx`
+   :param env: :class:`.BuildEnvironment`
+
+   Emitted when Consistency checks phase.  You can check consistency of
+   metadata for whole of documents.
+
+   .. versionadded:: 1.6
+
+.. event:: write-started (app, builder)
+
+   :param app: :class:`.Sphinx`
+   :param builder: :class:`.Builder`
+
+   Emitted before the builder starts to
+   resolve and write documents.
+
+   .. versionadded:: 7.4
+
+.. event:: build-finished (app, exception)
+
+   :param app: :class:`.Sphinx`
+   :param exception: ``Exception`` or ``None``
+
+   Emitted when a build has finished, before Sphinx exits, usually used for
+   cleanup.  This event is emitted even when the build process raised an
+   exception, given as the *exception* argument.  The exception is reraised in
+   the application after the event handlers have run.  If the build process
+   raised no exception, *exception* will be ``None``.  This allows to customize
+   cleanup actions depending on the exception status.
+
+   .. versionadded:: 0.5
+
+Builder specific events
+-----------------------
+
+These events are emitted by specific builders.
+
+.. event:: html-collect-pages (app)
+
+   :param app: :class:`.Sphinx`
+   :returns: iterable of ``(pagename, context, templatename)``
+      where *pagename* and *templatename* are strings and
+      *context* is a ``dict[str, Any]``.
+
+   Emitted when the HTML builder is starting to write non-document pages.
+
+   You can add pages to write by returning an iterable from this event.
+
+   .. versionadded:: 1.0
+
+.. event:: html-page-context (app, pagename, templatename, context, doctree)
+
+   :param app: :class:`.Sphinx`
+   :param pagename: ``str``
+   :param templatename: ``str``
+   :param context: ``dict[str, Any]``
+   :param doctree: :class:`docutils.nodes.document` or ``None``
+   :returns: ``str`` or ``None``
+
+   Emitted when the HTML builder has created a context dictionary to render a
+   template with -- this can be used to add custom elements to the context.
+
+   The *pagename* argument is the canonical name of the page being rendered,
+   that is, without ``.html`` suffix and using slashes as path separators.
+   The *templatename* is the name of the template to render, this will be
+   ``'page.html'`` for all pages from reStructuredText documents.
+
+   The *context* argument is a dictionary of values that are given to the
+   template engine to render the page and can be modified to include custom
+   values.
+
+   The *doctree* argument will be a doctree when
+   the page is created from a reStructuredText documents;
+   it will be ``None`` when the page is created from an HTML template alone.
+
+   You can return a string from the handler, it will then replace
+   ``'page.html'`` as the HTML template for this page.
+
+   .. tip::
+
+      You can install JS/CSS files for the specific page via
+      :meth:`.Sphinx.add_js_file` and :meth:`.Sphinx.add_css_file`
+      (since v3.5.0).
+
+   .. versionadded:: 0.4
+
+   .. versionchanged:: 1.3
+      The return value can now specify a template name.
+
+.. event:: linkcheck-process-uri (app, uri)
+
+   :param app: :class:`.Sphinx`
+   :param uri: ``str`` of the collected URI
+   :returns: ``str`` or ``None``
+
+   Emitted when the linkcheck builder collects hyperlinks from document.
+
+   The event handlers can modify the URI by returning a string.
+
+   .. versionadded:: 4.1
diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst
index 8332315..9617dd4 100644
--- a/doc/extdev/index.rst
+++ b/doc/extdev/index.rst
@@ -1,7 +1,7 @@
 .. _dev-extensions:
 
-Sphinx Extensions API
-=====================
+Sphinx API
+==========
 
 Since many projects will need special features in their documentation, Sphinx
 is designed to be extensible on several levels.
@@ -85,7 +85,7 @@ extension. These are:
    The config is available as ``app.config`` or ``env.config``.
 
 To see an example of use of these objects, refer to
-:doc:`../development/tutorials/index`.
+:ref:`the tutorials <extension-tutorials-index>`.
 
 .. _build-phases:
 
@@ -147,7 +147,7 @@ the individual nodes of each doctree and produces some output in the process.
    that checks external links does not need anything more than the parsed
    doctrees and therefore does not have phases 2--4.
 
-To see an example of application, refer to :doc:`../development/tutorials/todo`.
+To see an example of application, refer to :ref:`tutorial-extend-build`.
 
 .. _ext-metadata:
 
@@ -204,6 +204,7 @@ disposal when developing Sphinx extensions. Some are core to Sphinx
    :maxdepth: 2
 
    appapi
+   event_callbacks
    projectapi
    envapi
    builderapi
diff --git a/doc/extdev/markupapi.rst b/doc/extdev/markupapi.rst
index 072760c..7aa6324 100644
--- a/doc/extdev/markupapi.rst
+++ b/doc/extdev/markupapi.rst
@@ -1,12 +1,53 @@
 Docutils markup API
 ===================
 
-This section describes the API for adding ReST markup elements (roles and
-directives).
+This section describes the API for adding reStructuredText markup elements
+(roles and directives).
+
 
 Roles
 -----
 
+Roles follow the interface described below.
+They have to be registered by an extension using
+:meth:`.Sphinx.add_role` or :meth:`.Sphinx.add_role_to_domain`.
+
+
+.. code-block:: python
+
+   def role_function(
+       role_name: str, raw_source: str, text: str,
+       lineno: int, inliner: Inliner,
+       options: dict = {}, content: list = [],
+   ) -> tuple[list[Node], list[system_message]]:
+       elements = []
+       messages = []
+       return elements, messages
+
+The *options* and *content* parameters are only used for custom roles
+created via the :dudir:`role` directive.
+The return value is a tuple of two lists,
+the first containing the text nodes and elements from the role,
+and the second containing any system messages generated.
+For more information, see the `custom role overview`_ from Docutils.
+
+.. _custom role overview: https://docutils.sourceforge.io/docs/howto/rst-roles.html
+
+
+Creating custom roles
+^^^^^^^^^^^^^^^^^^^^^
+
+Sphinx provides two base classes for creating custom roles,
+:class:`~sphinx.util.docutils.SphinxRole` and :class:`~sphinx.util.docutils.ReferenceRole`.
+
+These provide a class-based interface for creating roles,
+where the main logic must be implemented in your ``run()`` method.
+The classes provide a number of useful methods and attributes,
+such as ``self.text``, ``self.config``, and ``self.env``.
+The ``ReferenceRole`` class implements Sphinx's ``title <target>`` logic,
+exposing ``self.target`` and ``self.title`` attributes.
+This is useful for creating cross-reference roles.
+
 
 Directives
 ----------
@@ -85,68 +126,106 @@ using :meth:`.Sphinx.add_directive` or :meth:`.Sphinx.add_directive_to_domain`.
       The state and state machine which controls the parsing.  Used for
       ``nested_parse``.
 
+.. seealso::
+
+   `Creating directives`_ HOWTO of the Docutils documentation
+
+   .. _Creating directives: https://docutils.sourceforge.io/docs/howto/rst-directives.html
+
+
+.. _parsing-directive-content-as-rest:
+
+Parsing directive content as reStructuredText
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-ViewLists
-^^^^^^^^^
+Many directives will contain more markup that must be parsed.
+To do this, use one of the following APIs from the :meth:`~Directive.run` method:
 
-Docutils represents document source lines in a class
-``docutils.statemachine.ViewList``.  This is a list with extended functionality
--- for one, slicing creates views of the original list, and also the list
-contains information about the source line numbers.
+* :py:meth:`.SphinxDirective.parse_content_to_nodes()`
+* :py:meth:`.SphinxDirective.parse_text_to_nodes()`
 
-The :attr:`Directive.content` attribute is a ViewList.  If you generate content
-to be parsed as ReST, you have to create a ViewList yourself.  Important for
-content generation are the following points:
+The first method parses all the directive's content as markup,
+whilst the second only parses the given *text* string.
+Both methods return the parsed Docutils nodes in a list.
 
-* The constructor takes a list of strings (lines) and a source (document) name.
+The methods are used as follows:
 
-* The ``.append()`` method takes a line and a source name as well.
+.. code-block:: python
 
+   def run(self) -> list[Node]:
+       # either
+       parsed = self.parse_content_to_nodes()
+       # or
+       parsed = self.parse_text_to_nodes('spam spam spam')
+       return parsed
 
-Parsing directive content as ReST
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+
+   The above utility methods were added in Sphinx 7.4.
+   Prior to Sphinx 7.4, the following methods should be used to parse content:
 
-Many directives will contain more markup that must be parsed.  To do this, use
-one of the following APIs from the :meth:`Directive.run` method:
+   * ``self.state.nested_parse``
+   * :func:`sphinx.util.nodes.nested_parse_with_titles` -- this allows titles in
+     the parsed content.
 
-* ``self.state.nested_parse``
-* :func:`sphinx.util.nodes.nested_parse_with_titles` -- this allows titles in
-  the parsed content.
+   .. code-block:: python
 
-Both APIs parse the content into a given node. They are used like this::
+      def run(self) -> list[Node]:
+          container = docutils.nodes.Element()
+          # either
+          nested_parse_with_titles(self.state, self.result, container)
+          # or
+          self.state.nested_parse(self.result, 0, container)
+          parsed = container.children
+          return parsed
 
-   node = docutils.nodes.paragraph()
-   # either
-   nested_parse_with_titles(self.state, self.result, node)
-   # or
-   self.state.nested_parse(self.result, 0, node)
+To parse inline markup,
+use :py:meth:`~sphinx.util.docutils.SphinxDirective.parse_inline()`.
+This must only be used for text which is a single line or paragraph,
+and does not contain any structural elements
+(headings, transitions, directives, etc).
 
 .. note::
 
-   ``sphinx.util.docutils.switch_source_input()`` allows to change a target file
-   during nested_parse.  It is useful to mixed contents.  For example, ``sphinx.
-   ext.autodoc`` uses it to parse docstrings::
+   ``sphinx.util.docutils.switch_source_input()`` allows changing
+   the source (input) file during parsing content in a directive.
+   It is useful to parse mixed content, such as in ``sphinx.ext.autodoc``,
+   where it is used to parse docstrings.
+
+   .. code-block:: python
 
-       from sphinx.util.docutils import switch_source_input
+      from sphinx.util.docutils import switch_source_input
+      from sphinx.util.parsing import nested_parse_to_nodes
 
-       # Switch source_input between parsing content.
-       # Inside this context, all parsing errors and warnings are reported as
-       # happened in new source_input (in this case, ``self.result``).
-       with switch_source_input(self.state, self.result):
-           node = docutils.nodes.paragraph()
-           self.state.nested_parse(self.result, 0, node)
+      # Switch source_input between parsing content.
+      # Inside this context, all parsing errors and warnings are reported as
+      # happened in new source_input (in this case, ``self.result``).
+      with switch_source_input(self.state, self.result):
+          parsed = nested_parse_to_nodes(self.state, self.result)
 
    .. deprecated:: 1.7
 
       Until Sphinx 1.6, ``sphinx.ext.autodoc.AutodocReporter`` was used for this
       purpose.  It is replaced by ``switch_source_input()``.
 
-If you don't need the wrapping node, you can use any concrete node type and
-return ``node.children`` from the Directive.
 
+.. _ViewLists:
 
-.. seealso::
+ViewLists and StringLists
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
-   `Creating directives`_ HOWTO of the Docutils documentation
+Docutils represents document source lines in a ``StringList`` class,
+which inherits from ``ViewList``, both in the ``docutils.statemachine`` module.
+This is a list with extended functionality,
+including that slicing creates views of the original list and
+that the list contains information about source line numbers.
+
+The :attr:`Directive.content` attribute is a ``StringList``.
+If you generate content to be parsed as reStructuredText,
+you have to create a ``StringList`` for the Docutils APIs.
+The utility functions provided by Sphinx handle this automatically.
+Important for content generation are the following points:
 
-.. _Creating directives: https://docutils.sourceforge.io/docs/howto/rst-directives.html
+* The ``ViewList`` constructor takes a list of strings (lines)
+  and a source (document) name.
+* The ``ViewList.append()`` method takes a line and a source name as well.
diff --git a/doc/extdev/utils.rst b/doc/extdev/utils.rst
index ff8dc4e..9e10a0d 100644
--- a/doc/extdev/utils.rst
+++ b/doc/extdev/utils.rst
@@ -3,6 +3,7 @@ Utilities
 
 Sphinx provides utility classes and functions to develop extensions.
 
+
 Base classes for components
 ---------------------------
 
@@ -30,12 +31,20 @@ components (e.g. :class:`.Config`, :class:`.BuildEnvironment` and so on) easily.
 .. autoclass:: sphinx.transforms.post_transforms.images.ImageConverter
    :members:
 
+
 Utility components
 ------------------
 
 .. autoclass:: sphinx.events.EventManager
    :members:
 
+
+Utility functions
+-----------------
+
+.. autofunction:: sphinx.util.parsing.nested_parse_to_nodes
+
+
 Utility types
 -------------