diff options
Diffstat (limited to 'doc/development/tutorials')
| -rw-r--r-- | doc/development/tutorials/autodoc_ext.rst | 140 | ||||
| -rw-r--r-- | doc/development/tutorials/examples/README.rst | 11 | ||||
| -rw-r--r-- | doc/development/tutorials/examples/autodoc_intenum.py | 54 | ||||
| -rw-r--r-- | doc/development/tutorials/examples/helloworld.py | 19 | ||||
| -rw-r--r-- | doc/development/tutorials/examples/recipe.py | 159 | ||||
| -rw-r--r-- | doc/development/tutorials/examples/todo.py | 135 | ||||
| -rw-r--r-- | doc/development/tutorials/helloworld.rst | 189 | ||||
| -rw-r--r-- | doc/development/tutorials/index.rst | 17 | ||||
| -rw-r--r-- | doc/development/tutorials/recipe.rst | 226 | ||||
| -rw-r--r-- | doc/development/tutorials/todo.rst | 367 |
10 files changed, 1317 insertions, 0 deletions
diff --git a/doc/development/tutorials/autodoc_ext.rst b/doc/development/tutorials/autodoc_ext.rst new file mode 100644 index 0000000..8de2e4d --- /dev/null +++ b/doc/development/tutorials/autodoc_ext.rst @@ -0,0 +1,140 @@ +.. _autodoc_ext_tutorial: + +Developing autodoc extension for IntEnum +======================================== + +The objective of this tutorial is to create an extension that adds +support for new type for autodoc. This autodoc extension will format +the ``IntEnum`` class from Python standard library. (module ``enum``) + +Overview +-------- + +We want the extension that will create auto-documentation for IntEnum. +``IntEnum`` is the integer enum class from standard library ``enum`` module. + +Currently this class has no special auto documentation behavior. + +We want to add following to autodoc: + +* A new ``autointenum`` directive that will document the ``IntEnum`` class. +* The generated documentation will have all the enum possible values + with names. +* The ``autointenum`` directive will have an option ``:hex:`` which will + cause the integers be printed in hexadecimal form. + + +Prerequisites +------------- + +We need the same setup as in :doc:`the previous extensions <todo>`. This time, +we will be putting out extension in a file called :file:`autodoc_intenum.py`. +The :file:`my_enums.py` will contain the sample enums we will document. + +Here is an example of the folder structure you might obtain: + +.. code-block:: text + + └── source + ├── _ext + │ └── autodoc_intenum.py + ├── conf.py + ├── index.rst + └── my_enums.py + + +Writing the extension +--------------------- + +Start with ``setup`` function for the extension. + +.. literalinclude:: examples/autodoc_intenum.py + :language: python + :linenos: + :pyobject: setup + + +The :meth:`~Sphinx.setup_extension` method will pull the autodoc extension +because our new extension depends on autodoc. :meth:`~Sphinx.add_autodocumenter` +is the method that registers our new auto documenter class. + +We want to import certain objects from the autodoc extension: + +.. literalinclude:: examples/autodoc_intenum.py + :language: python + :linenos: + :lines: 1-7 + + +There are several different documenter classes such as ``MethodDocumenter`` +or ``AttributeDocumenter`` available in the autodoc extension but +our new class is the subclass of ``ClassDocumenter`` which a +documenter class used by autodoc to document classes. + +This is the definition of our new the auto-documenter class: + +.. literalinclude:: examples/autodoc_intenum.py + :language: python + :linenos: + :pyobject: IntEnumDocumenter + + +Important attributes of the new class: + +**objtype** + This attribute determines the ``auto`` directive name. In + this case the auto directive will be ``autointenum``. + +**directivetype** + This attribute sets the generated directive name. In + this example the generated directive will be ``.. :py:class::``. + +**priority** + the larger the number the higher is the priority. We want our + documenter be higher priority than the parent. + +**option_spec** + option specifications. We copy the parent class options and + add a new option *hex*. + + +Overridden members: + +**can_document_member** + This member is important to override. It should + return *True* when the passed object can be documented by this class. + +**add_directive_header** + This method generates the directive header. We add + **:final:** directive option. Remember to call **super** or no directive + will be generated. + +**add_content** + This method generates the body of the class documentation. + After calling the super method we generate lines for enum description. + + +Using the extension +------------------- + +You can now use the new autodoc directive to document any ``IntEnum``. + +For example, you have the following ``IntEnum``: + +.. code-block:: python + :caption: my_enums.py + + class Colors(IntEnum): + """Colors enumerator""" + NONE = 0 + RED = 1 + GREEN = 2 + BLUE = 3 + + +This will be the documentation file with auto-documentation directive: + +.. code-block:: rst + :caption: index.rst + + .. autointenum:: my_enums.Colors diff --git a/doc/development/tutorials/examples/README.rst b/doc/development/tutorials/examples/README.rst new file mode 100644 index 0000000..2b9c01b --- /dev/null +++ b/doc/development/tutorials/examples/README.rst @@ -0,0 +1,11 @@ +:orphan: + +Tutorial examples +================= + +This directory contains a number of examples used in the tutorials. These are +intended to be increasingly complex to demonstrate the various features of +Sphinx, but should aim to be as complicated as necessary but no more. +Individual sections are referenced by line numbers, meaning if you make changes +to the source files, you should update the references in the documentation +accordingly. diff --git a/doc/development/tutorials/examples/autodoc_intenum.py b/doc/development/tutorials/examples/autodoc_intenum.py new file mode 100644 index 0000000..a23f9ce --- /dev/null +++ b/doc/development/tutorials/examples/autodoc_intenum.py @@ -0,0 +1,54 @@ +from enum import IntEnum +from typing import Any, Optional + +from docutils.statemachine import StringList + +from sphinx.application import Sphinx +from sphinx.ext.autodoc import ClassDocumenter, bool_option + + +class IntEnumDocumenter(ClassDocumenter): + objtype = 'intenum' + directivetype = ClassDocumenter.objtype + priority = 10 + ClassDocumenter.priority + option_spec = dict(ClassDocumenter.option_spec) + option_spec['hex'] = bool_option + + @classmethod + def can_document_member(cls, + member: Any, membername: str, + isattr: bool, parent: Any) -> bool: + try: + return issubclass(member, IntEnum) + except TypeError: + return False + + def add_directive_header(self, sig: str) -> None: + super().add_directive_header(sig) + self.add_line(' :final:', self.get_sourcename()) + + def add_content(self, + more_content: Optional[StringList], + no_docstring: bool = False + ) -> None: + + super().add_content(more_content, no_docstring) + + source_name = self.get_sourcename() + enum_object: IntEnum = self.object + use_hex = self.options.hex + self.add_line('', source_name) + + for the_member_name, enum_member in enum_object.__members__.items(): + the_member_value = enum_member.value + if use_hex: + the_member_value = hex(the_member_value) + + self.add_line( + f"**{the_member_name}**: {the_member_value}", source_name) + self.add_line('', source_name) + + +def setup(app: Sphinx) -> None: + app.setup_extension('sphinx.ext.autodoc') # Require autodoc extension + app.add_autodocumenter(IntEnumDocumenter) diff --git a/doc/development/tutorials/examples/helloworld.py b/doc/development/tutorials/examples/helloworld.py new file mode 100644 index 0000000..d6d81fd --- /dev/null +++ b/doc/development/tutorials/examples/helloworld.py @@ -0,0 +1,19 @@ +from docutils import nodes +from docutils.parsers.rst import Directive + + +class HelloWorld(Directive): + + def run(self): + paragraph_node = nodes.paragraph(text='Hello World!') + return [paragraph_node] + + +def setup(app): + app.add_directive("helloworld", HelloWorld) + + return { + 'version': '0.1', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/doc/development/tutorials/examples/recipe.py b/doc/development/tutorials/examples/recipe.py new file mode 100644 index 0000000..8dc53fd --- /dev/null +++ b/doc/development/tutorials/examples/recipe.py @@ -0,0 +1,159 @@ +from collections import defaultdict + +from docutils.parsers.rst import directives + +from sphinx import addnodes +from sphinx.directives import ObjectDescription +from sphinx.domains import Domain, Index +from sphinx.roles import XRefRole +from sphinx.util.nodes import make_refnode + + +class RecipeDirective(ObjectDescription): + """A custom directive that describes a recipe.""" + + has_content = True + required_arguments = 1 + option_spec = { + 'contains': directives.unchanged_required, + } + + def handle_signature(self, sig, signode): + signode += addnodes.desc_name(text=sig) + return sig + + def add_target_and_index(self, name_cls, sig, signode): + signode['ids'].append('recipe' + '-' + sig) + if 'contains' in self.options: + ingredients = [ + x.strip() for x in self.options.get('contains').split(',')] + + recipes = self.env.get_domain('recipe') + recipes.add_recipe(sig, ingredients) + + +class IngredientIndex(Index): + """A custom index that creates an ingredient matrix.""" + + name = 'ingredient' + localname = 'Ingredient Index' + shortname = 'Ingredient' + + def generate(self, docnames=None): + content = defaultdict(list) + + recipes = {name: (dispname, typ, docname, anchor) + for name, dispname, typ, docname, anchor, _ + in self.domain.get_objects()} + recipe_ingredients = self.domain.data['recipe_ingredients'] + ingredient_recipes = defaultdict(list) + + # flip from recipe_ingredients to ingredient_recipes + for recipe_name, ingredients in recipe_ingredients.items(): + for ingredient in ingredients: + ingredient_recipes[ingredient].append(recipe_name) + + # convert the mapping of ingredient to recipes to produce the expected + # output, shown below, using the ingredient name as a key to group + # + # name, subtype, docname, anchor, extra, qualifier, description + for ingredient, recipe_names in ingredient_recipes.items(): + for recipe_name in recipe_names: + dispname, typ, docname, anchor = recipes[recipe_name] + content[ingredient].append( + (dispname, 0, docname, anchor, docname, '', typ)) + + # convert the dict to the sorted list of tuples expected + content = sorted(content.items()) + + return content, True + + +class RecipeIndex(Index): + """A custom index that creates an recipe matrix.""" + + name = 'recipe' + localname = 'Recipe Index' + shortname = 'Recipe' + + def generate(self, docnames=None): + content = defaultdict(list) + + # sort the list of recipes in alphabetical order + recipes = self.domain.get_objects() + recipes = sorted(recipes, key=lambda recipe: recipe[0]) + + # generate the expected output, shown below, from the above using the + # first letter of the recipe as a key to group thing + # + # name, subtype, docname, anchor, extra, qualifier, description + for _name, dispname, typ, docname, anchor, _priority in recipes: + content[dispname[0].lower()].append( + (dispname, 0, docname, anchor, docname, '', typ)) + + # convert the dict to the sorted list of tuples expected + content = sorted(content.items()) + + return content, True + + +class RecipeDomain(Domain): + + name = 'recipe' + label = 'Recipe Sample' + roles = { + 'ref': XRefRole() + } + directives = { + 'recipe': RecipeDirective, + } + indices = { + RecipeIndex, + IngredientIndex + } + initial_data = { + 'recipes': [], # object list + 'recipe_ingredients': {}, # name -> object + } + + def get_full_qualified_name(self, node): + return '{}.{}'.format('recipe', node.arguments[0]) + + def get_objects(self): + yield from self.data['recipes'] + + def resolve_xref(self, env, fromdocname, builder, typ, target, node, + contnode): + match = [(docname, anchor) + for name, sig, typ, docname, anchor, prio + in self.get_objects() if sig == target] + + if len(match) > 0: + todocname = match[0][0] + targ = match[0][1] + + return make_refnode(builder, fromdocname, todocname, targ, + contnode, targ) + else: + print('Awww, found nothing') + return None + + def add_recipe(self, signature, ingredients): + """Add a new recipe to the domain.""" + name = '{}.{}'.format('recipe', signature) + anchor = 'recipe-{}'.format(signature) + + self.data['recipe_ingredients'][name] = ingredients + # name, dispname, type, docname, anchor, priority + self.data['recipes'].append( + (name, signature, 'Recipe', self.env.docname, anchor, 0)) + + +def setup(app): + app.add_domain(RecipeDomain) + + return { + 'version': '0.1', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/doc/development/tutorials/examples/todo.py b/doc/development/tutorials/examples/todo.py new file mode 100644 index 0000000..15368f4 --- /dev/null +++ b/doc/development/tutorials/examples/todo.py @@ -0,0 +1,135 @@ +from docutils import nodes +from docutils.parsers.rst import Directive + +from sphinx.locale import _ +from sphinx.util.docutils import SphinxDirective + + +class todo(nodes.Admonition, nodes.Element): + pass + + +class todolist(nodes.General, nodes.Element): + pass + + +def visit_todo_node(self, node): + self.visit_admonition(node) + + +def depart_todo_node(self, node): + self.depart_admonition(node) + + +class TodolistDirective(Directive): + + def run(self): + return [todolist('')] + + +class TodoDirective(SphinxDirective): + + # this enables content in the directive + has_content = True + + def run(self): + targetid = 'todo-%d' % self.env.new_serialno('todo') + targetnode = nodes.target('', '', ids=[targetid]) + + todo_node = todo('\n'.join(self.content)) + todo_node += nodes.title(_('Todo'), _('Todo')) + self.state.nested_parse(self.content, self.content_offset, todo_node) + + if not hasattr(self.env, 'todo_all_todos'): + self.env.todo_all_todos = [] + + self.env.todo_all_todos.append({ + 'docname': self.env.docname, + 'lineno': self.lineno, + 'todo': todo_node.deepcopy(), + 'target': targetnode, + }) + + return [targetnode, todo_node] + + +def purge_todos(app, env, docname): + if not hasattr(env, 'todo_all_todos'): + return + + env.todo_all_todos = [todo for todo in env.todo_all_todos + if todo['docname'] != docname] + + +def merge_todos(app, env, docnames, other): + if not hasattr(env, 'todo_all_todos'): + env.todo_all_todos = [] + if hasattr(other, 'todo_all_todos'): + env.todo_all_todos.extend(other.todo_all_todos) + + +def process_todo_nodes(app, doctree, fromdocname): + if not app.config.todo_include_todos: + for node in doctree.findall(todo): + node.parent.remove(node) + + # Replace all todolist nodes with a list of the collected todos. + # Augment each todo with a backlink to the original location. + env = app.builder.env + + if not hasattr(env, 'todo_all_todos'): + env.todo_all_todos = [] + + for node in doctree.findall(todolist): + if not app.config.todo_include_todos: + node.replace_self([]) + continue + + content = [] + + for todo_info in env.todo_all_todos: + para = nodes.paragraph() + filename = env.doc2path(todo_info['docname'], base=None) + description = ( + _('(The original entry is located in %s, line %d and can be found ') % + (filename, todo_info['lineno'])) + para += nodes.Text(description) + + # Create a reference + newnode = nodes.reference('', '') + innernode = nodes.emphasis(_('here'), _('here')) + newnode['refdocname'] = todo_info['docname'] + newnode['refuri'] = app.builder.get_relative_uri( + fromdocname, todo_info['docname']) + newnode['refuri'] += '#' + todo_info['target']['refid'] + newnode.append(innernode) + para += newnode + para += nodes.Text('.)') + + # Insert into the todolist + content.append(todo_info['todo']) + content.append(para) + + node.replace_self(content) + + +def setup(app): + app.add_config_value('todo_include_todos', False, 'html') + + app.add_node(todolist) + app.add_node(todo, + html=(visit_todo_node, depart_todo_node), + latex=(visit_todo_node, depart_todo_node), + text=(visit_todo_node, depart_todo_node)) + + app.add_directive('todo', TodoDirective) + app.add_directive('todolist', TodolistDirective) + app.connect('doctree-resolved', process_todo_nodes) + app.connect('env-purge-doc', purge_todos) + app.connect('env-merge-info', merge_todos) + + return { + 'version': '0.1', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/doc/development/tutorials/helloworld.rst b/doc/development/tutorials/helloworld.rst new file mode 100644 index 0000000..8940e3d --- /dev/null +++ b/doc/development/tutorials/helloworld.rst @@ -0,0 +1,189 @@ +Developing a "Hello world" extension +==================================== + +The objective of this tutorial is to create a very basic extension that adds a +new directive. This directive will output a paragraph containing "hello world". + +Only basic information is provided in this tutorial. For more information, refer +to the :doc:`other tutorials <index>` that go into more details. + +.. warning:: + + For this extension, you will need some basic understanding of docutils_ + and Python. + + +Overview +-------- + +We want the extension to add the following to Sphinx: + +* A ``helloworld`` directive, that will simply output the text "hello world". + + +Prerequisites +------------- + +We will not be distributing this plugin via `PyPI`_ and will instead include it +as part of an existing project. This means you will need to use an existing +project or create a new one using :program:`sphinx-quickstart`. + +We assume you are using separate source (:file:`source`) and build +(:file:`build`) folders. Your extension file could be in any folder of your +project. In our case, let's do the following: + +#. Create an :file:`_ext` folder in :file:`source` +#. Create a new Python file in the :file:`_ext` folder called + :file:`helloworld.py` + +Here is an example of the folder structure you might obtain: + +.. code-block:: text + + └── source + ├── _ext + │ └── helloworld.py + ├── _static + ├── conf.py + ├── somefolder + ├── index.rst + ├── somefile.rst + └── someotherfile.rst + + +Writing the extension +--------------------- + +Open :file:`helloworld.py` and paste the following code in it: + +.. literalinclude:: examples/helloworld.py + :language: python + :linenos: + +Some essential things are happening in this example, and you will see them for +all directives. + +.. rubric:: The directive class + +Our new directive is declared in the ``HelloWorld`` class. + +.. literalinclude:: examples/helloworld.py + :language: python + :linenos: + :lines: 5-9 + +This class extends the docutils_' ``Directive`` class. All extensions that +create directives should extend this class. + +.. seealso:: + + `The docutils documentation on creating directives <docutils directives_>`_ + +This class contains a ``run`` method. This method is a requirement and it is +part of every directive. It contains the main logic of the directive and it +returns a list of docutils nodes to be processed by Sphinx. These nodes are +docutils' way of representing the content of a document. There are many types of +nodes available: text, paragraph, reference, table, etc. + +.. seealso:: + + `The docutils documentation on nodes <docutils nodes_>`_ + +The ``nodes.paragraph`` class creates a new paragraph node. A paragraph +node typically contains some text that we can set during instantiation using +the ``text`` parameter. + +.. rubric:: The ``setup`` function + +.. currentmodule:: sphinx.application + +This function is a requirement. We use it to plug our new directive into +Sphinx. + +.. literalinclude:: examples/helloworld.py + :language: python + :linenos: + :lines: 12- + +The simplest thing you can do is to call the :meth:`~Sphinx.add_directive` method, +which is what we've done here. For this particular call, the first argument is +the name of the directive itself as used in a reST file. In this case, we would +use ``helloworld``. For example: + +.. code-block:: rst + + Some intro text here... + + .. helloworld:: + + Some more text here... + +We also return the :ref:`extension metadata <ext-metadata>` that indicates the +version of our extension, along with the fact that it is safe to use the +extension for both parallel reading and writing. + + +Using the extension +------------------- + +The extension has to be declared in your :file:`conf.py` file to make Sphinx +aware of it. There are two steps necessary here: + +#. Add the :file:`_ext` directory to the `Python path`_ using + ``sys.path.append``. This should be placed at the top of the file. + +#. Update or create the :confval:`extensions` list and add the extension file + name to the list + +For example: + +.. code-block:: python + + import os + import sys + + sys.path.append(os.path.abspath("./_ext")) + + extensions = ['helloworld'] + +.. tip:: + + We're not distributing this extension as a `Python package`_, we need to + modify the `Python path`_ so Sphinx can find our extension. This is why we + need the call to ``sys.path.append``. + +You can now use the extension in a file. For example: + +.. code-block:: rst + + Some intro text here... + + .. helloworld:: + + Some more text here... + +The sample above would generate: + +.. code-block:: text + + Some intro text here... + + Hello World! + + Some more text here... + + +Further reading +--------------- + +This is the very basic principle of an extension that creates a new directive. + +For a more advanced example, refer to :doc:`todo`. + + +.. _docutils: https://docutils.sourceforge.io/ +.. _docutils directives: https://docutils.sourceforge.io/docs/howto/rst-directives.html +.. _docutils nodes: https://docutils.sourceforge.io/docs/ref/doctree.html +.. _PyPI: https://pypi.org/ +.. _Python package: https://packaging.python.org/ +.. _Python path: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH diff --git a/doc/development/tutorials/index.rst b/doc/development/tutorials/index.rst new file mode 100644 index 0000000..a7eee48 --- /dev/null +++ b/doc/development/tutorials/index.rst @@ -0,0 +1,17 @@ +.. _extension-tutorials-index: + +Extension tutorials +=================== + +Refer to the following tutorials to get started with extension development. + + +.. toctree:: + :caption: Directive tutorials + :maxdepth: 1 + + helloworld + todo + recipe + autodoc_ext + diff --git a/doc/development/tutorials/recipe.rst b/doc/development/tutorials/recipe.rst new file mode 100644 index 0000000..0e96a45 --- /dev/null +++ b/doc/development/tutorials/recipe.rst @@ -0,0 +1,226 @@ +Developing a "recipe" extension +=============================== + +The objective of this tutorial is to illustrate roles, directives and domains. +Once complete, we will be able to use this extension to describe a recipe and +reference that recipe from elsewhere in our documentation. + +.. note:: + + This tutorial is based on a guide first published on `opensource.com`_ and + is provided here with the original author's permission. + + .. _opensource.com: https://opensource.com/article/18/11/building-custom-workflows-sphinx + + +Overview +-------- + +We want the extension to add the following to Sphinx: + +* A ``recipe`` :term:`directive`, containing some content describing the recipe + steps, along with a ``:contains:`` option highlighting the main ingredients + of the recipe. + +* A ``ref`` :term:`role`, which provides a cross-reference to the recipe + itself. + +* A ``recipe`` :term:`domain`, which allows us to tie together the above role + and domain, along with things like indices. + +For that, we will need to add the following elements to Sphinx: + +* A new directive called ``recipe`` + +* New indexes to allow us to reference ingredient and recipes + +* A new domain called ``recipe``, which will contain the ``recipe`` directive + and ``ref`` role + + +Prerequisites +------------- + +We need the same setup as in :doc:`the previous extensions <todo>`. This time, +we will be putting out extension in a file called :file:`recipe.py`. + +Here is an example of the folder structure you might obtain: + +.. code-block:: text + + └── source + ├── _ext + │ └── recipe.py + ├── conf.py + └── index.rst + + +Writing the extension +--------------------- + +Open :file:`recipe.py` and paste the following code in it, all of which we will +explain in detail shortly: + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + +Let's look at each piece of this extension step-by-step to explain what's going +on. + +.. rubric:: The directive class + +The first thing to examine is the ``RecipeDirective`` directive: + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + :pyobject: RecipeDirective + +Unlike :doc:`helloworld` and :doc:`todo`, this directive doesn't derive from +:class:`docutils.parsers.rst.Directive` and doesn't define a ``run`` method. +Instead, it derives from :class:`sphinx.directives.ObjectDescription` and +defines ``handle_signature`` and ``add_target_and_index`` methods. This is +because ``ObjectDescription`` is a special-purpose directive that's intended +for describing things like classes, functions, or, in our case, recipes. More +specifically, ``handle_signature`` implements parsing the signature of the +directive and passes on the object's name and type to its superclass, while +``add_taget_and_index`` adds a target (to link to) and an entry to the index +for this node. + +We also see that this directive defines ``has_content``, ``required_arguments`` +and ``option_spec``. Unlike the ``TodoDirective`` directive added in the +:doc:`previous tutorial <todo>`, this directive takes a single argument, the +recipe name, and an option, ``contains``, in addition to the nested +reStructuredText in the body. + +.. rubric:: The index classes + +.. currentmodule:: sphinx.domains + +.. todo:: Add brief overview of indices + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + :pyobject: IngredientIndex + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + :pyobject: RecipeIndex + +Both ``IngredientIndex`` and ``RecipeIndex`` are derived from :class:`Index`. +They implement custom logic to generate a tuple of values that define the +index. Note that ``RecipeIndex`` is a simple index that has only one entry. +Extending it to cover more object types is not yet part of the code. + +Both indices use the method :meth:`Index.generate` to do their work. This +method combines the information from our domain, sorts it, and returns it in a +list structure that will be accepted by Sphinx. This might look complicated but +all it really is is a list of tuples like ``('tomato', 'TomatoSoup', 'test', +'rec-TomatoSoup',...)``. Refer to the :doc:`domain API guide +</extdev/domainapi>` for more information on this API. + +These index pages can be referred by combination of domain name and its +``name`` using :rst:role:`ref` role. For example, ``RecipeIndex`` can be +referred by ``:ref:`recipe-recipe```. + +.. rubric:: The domain + +A Sphinx domain is a specialized container that ties together roles, +directives, and indices, among other things. Let's look at the domain we're +creating here. + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + :pyobject: RecipeDomain + +There are some interesting things to note about this ``recipe`` domain and domains +in general. Firstly, we actually register our directives, roles and indices +here, via the ``directives``, ``roles`` and ``indices`` attributes, rather than +via calls later on in ``setup``. We can also note that we aren't actually +defining a custom role and are instead reusing the +:class:`sphinx.roles.XRefRole` role and defining the +:class:`sphinx.domains.Domain.resolve_xref` method. This method takes two +arguments, ``typ`` and ``target``, which refer to the cross-reference type and +its target name. We'll use ``target`` to resolve our destination from our +domain's ``recipes`` because we currently have only one type of node. + +Moving on, we can see that we've defined ``initial_data``. The values defined in +``initial_data`` will be copied to ``env.domaindata[domain_name]`` as the +initial data of the domain, and domain instances can access it via +``self.data``. We see that we have defined two items in ``initial_data``: +``recipes`` and ``recipe2ingredient``. These contain a list of all objects +defined (i.e. all recipes) and a hash that maps a canonical ingredient name to +the list of objects. The way we name objects is common across our extension and +is defined in the ``get_full_qualified_name`` method. For each object created, +the canonical name is ``recipe.<recipename>``, where ``<recipename>`` is the +name the documentation writer gives the object (a recipe). This enables the +extension to use different object types that share the same name. Having a +canonical name and central place for our objects is a huge advantage. Both our +indices and our cross-referencing code use this feature. + +.. rubric:: The ``setup`` function + +.. currentmodule:: sphinx.application + +:doc:`As always <todo>`, the ``setup`` function is a requirement and is used to +hook the various parts of our extension into Sphinx. Let's look at the +``setup`` function for this extension. + +.. literalinclude:: examples/recipe.py + :language: python + :linenos: + :pyobject: setup + +This looks a little different to what we're used to seeing. There are no calls +to :meth:`~Sphinx.add_directive` or even :meth:`~Sphinx.add_role`. Instead, we +have a single call to :meth:`~Sphinx.add_domain` followed by some +initialization of the :ref:`standard domain <domains-std>`. This is because we +had already registered our directives, roles and indexes as part of the +directive itself. + + +Using the extension +------------------- + +You can now use the extension throughout your project. For example: + +.. code-block:: rst + :caption: index.rst + + Joe's Recipes + ============= + + Below are a collection of my favourite recipes. I highly recommend the + :recipe:ref:`TomatoSoup` recipe in particular! + + .. toctree:: + + tomato-soup + +.. code-block:: rst + :caption: tomato-soup.rst + + The recipe contains `tomato` and `cilantro`. + + .. recipe:recipe:: TomatoSoup + :contains: tomato, cilantro, salt, pepper + + This recipe is a tasty tomato soup, combine all ingredients + and cook. + +The important things to note are the use of the ``:recipe:ref:`` role to +cross-reference the recipe actually defined elsewhere (using the +``:recipe:recipe:`` directive. + + +Further reading +--------------- + +For more information, refer to the `docutils`_ documentation and +:doc:`/extdev/index`. + +.. _docutils: https://docutils.sourceforge.io/docs/ diff --git a/doc/development/tutorials/todo.rst b/doc/development/tutorials/todo.rst new file mode 100644 index 0000000..c0a5c0d --- /dev/null +++ b/doc/development/tutorials/todo.rst @@ -0,0 +1,367 @@ +Developing a "TODO" extension +============================= + +The objective of this tutorial is to create a more comprehensive extension than +that created in :doc:`helloworld`. Whereas that guide just covered writing a +custom :term:`directive`, this guide adds multiple directives, along with custom +nodes, additional config values and custom event handlers. To this end, we will +cover a ``todo`` extension that adds capabilities to include todo entries in the +documentation, and to collect these in a central place. This is similar the +``sphinxext.todo`` extension distributed with Sphinx. + + +Overview +-------- + +.. note:: + To understand the design of this extension, refer to + :ref:`important-objects` and :ref:`build-phases`. + +We want the extension to add the following to Sphinx: + +* A ``todo`` directive, containing some content that is marked with "TODO" and + only shown in the output if a new config value is set. Todo entries should not + be in the output by default. + +* A ``todolist`` directive that creates a list of all todo entries throughout + the documentation. + +For that, we will need to add the following elements to Sphinx: + +* New directives, called ``todo`` and ``todolist``. + +* New document tree nodes to represent these directives, conventionally also + called ``todo`` and ``todolist``. We wouldn't need new nodes if the new + directives only produced some content representable by existing nodes. + +* A new config value ``todo_include_todos`` (config value names should start + with the extension name, in order to stay unique) that controls whether todo + entries make it into the output. + +* New event handlers: one for the :event:`doctree-resolved` event, to + replace the todo and todolist nodes, one for :event:`env-merge-info` + to merge intermediate results from parallel builds, and one for + :event:`env-purge-doc` (the reason for that will be covered later). + + +Prerequisites +------------- + +As with :doc:`helloworld`, we will not be distributing this plugin via PyPI so +once again we need a Sphinx project to call this from. You can use an existing +project or create a new one using :program:`sphinx-quickstart`. + +We assume you are using separate source (:file:`source`) and build +(:file:`build`) folders. Your extension file could be in any folder of your +project. In our case, let's do the following: + +#. Create an :file:`_ext` folder in :file:`source` +#. Create a new Python file in the :file:`_ext` folder called :file:`todo.py` + +Here is an example of the folder structure you might obtain: + +.. code-block:: text + + └── source + ├── _ext + │ └── todo.py + ├── _static + ├── conf.py + ├── somefolder + ├── index.rst + ├── somefile.rst + └── someotherfile.rst + + +Writing the extension +--------------------- + +Open :file:`todo.py` and paste the following code in it, all of which we will +explain in detail shortly: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + +This is far more extensive extension than the one detailed in :doc:`helloworld`, +however, we will will look at each piece step-by-step to explain what's +happening. + +.. rubric:: The node classes + +Let's start with the node classes: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 8-21 + +Node classes usually don't have to do anything except inherit from the standard +docutils classes defined in :mod:`docutils.nodes`. ``todo`` inherits from +``Admonition`` because it should be handled like a note or warning, ``todolist`` +is just a "general" node. + +.. note:: + + Many extensions will not have to create their own node classes and work fine + with the nodes already provided by `docutils + <https://docutils.sourceforge.io/docs/ref/doctree.html>`__ and :ref:`Sphinx + <nodes>`. + +.. attention:: + + It is important to know that while you can extend Sphinx without + leaving your ``conf.py``, if you declare an inherited node right + there, you'll hit an unobvious :py:class:`PickleError`. So if + something goes wrong, please make sure that you put inherited nodes + into a separate Python module. + + For more details, see: + + - https://github.com/sphinx-doc/sphinx/issues/6751 + - https://github.com/sphinx-doc/sphinx/issues/1493 + - https://github.com/sphinx-doc/sphinx/issues/1424 + +.. rubric:: The directive classes + +A directive class is a class deriving usually from +:class:`docutils.parsers.rst.Directive`. The directive interface is also +covered in detail in the `docutils documentation`_; the important thing is that +the class should have attributes that configure the allowed markup, and a +``run`` method that returns a list of nodes. + +Looking first at the ``TodolistDirective`` directive: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 24-27 + +It's very simple, creating and returning an instance of our ``todolist`` node +class. The ``TodolistDirective`` directive itself has neither content nor +arguments that need to be handled. That brings us to the ``TodoDirective`` +directive: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 30-53 + +Several important things are covered here. First, as you can see, we're now +subclassing the :class:`~sphinx.util.docutils.SphinxDirective` helper class +instead of the usual :class:`~docutils.parsers.rst.Directive` class. This +gives us access to the :ref:`build environment instance <important-objects>` +using the ``self.env`` property. Without this, we'd have to use the rather +convoluted ``self.state.document.settings.env``. Then, to act as a link target +(from ``TodolistDirective``), the ``TodoDirective`` directive needs to return a +target node in addition to the ``todo`` node. The target ID (in HTML, this will +be the anchor name) is generated by using ``env.new_serialno`` which returns a +new unique integer on each call and therefore leads to unique target names. The +target node is instantiated without any text (the first two arguments). + +On creating admonition node, the content body of the directive are parsed using +``self.state.nested_parse``. The first argument gives the content body, and +the second one gives content offset. The third argument gives the parent node +of parsed result, in our case the ``todo`` node. Following this, the ``todo`` +node is added to the environment. This is needed to be able to create a list of +all todo entries throughout the documentation, in the place where the author +puts a ``todolist`` directive. For this case, the environment attribute +``todo_all_todos`` is used (again, the name should be unique, so it is prefixed +by the extension name). It does not exist when a new environment is created, so +the directive must check and create it if necessary. Various information about +the todo entry's location are stored along with a copy of the node. + +In the last line, the nodes that should be put into the doctree are returned: +the target node and the admonition node. + +The node structure that the directive returns looks like this:: + + +--------------------+ + | target node | + +--------------------+ + +--------------------+ + | todo node | + +--------------------+ + \__+--------------------+ + | admonition title | + +--------------------+ + | paragraph | + +--------------------+ + | ... | + +--------------------+ + +.. rubric:: The event handlers + +Event handlers are one of Sphinx's most powerful features, providing a way to +do hook into any part of the documentation process. There are many events +provided by Sphinx itself, as detailed in :ref:`the API guide <events>`, and +we're going to use a subset of them here. + +Let's look at the event handlers used in the above example. First, the one for +the :event:`env-purge-doc` event: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 56-61 + +Since we store information from source files in the environment, which is +persistent, it may become out of date when the source file changes. Therefore, +before each source file is read, the environment's records of it are cleared, +and the :event:`env-purge-doc` event gives extensions a chance to do the same. +Here we clear out all todos whose docname matches the given one from the +``todo_all_todos`` list. If there are todos left in the document, they will be +added again during parsing. + +The next handler, for the :event:`env-merge-info` event, is used +during parallel builds. As during parallel builds all threads have +their own ``env``, there's multiple ``todo_all_todos`` lists that need +to be merged: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 64-68 + + +The other handler belongs to the :event:`doctree-resolved` event: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 71-113 + +The :event:`doctree-resolved` event is emitted at the end of :ref:`phase 3 +(resolving) <build-phases>` and allows custom resolving to be done. The handler +we have written for this event is a bit more involved. If the +``todo_include_todos`` config value (which we'll describe shortly) is false, +all ``todo`` and ``todolist`` nodes are removed from the documents. If not, +``todo`` nodes just stay where and how they are. ``todolist`` nodes are +replaced by a list of todo entries, complete with backlinks to the location +where they come from. The list items are composed of the nodes from the +``todo`` entry and docutils nodes created on the fly: a paragraph for each +entry, containing text that gives the location, and a link (reference node +containing an italic node) with the backreference. The reference URI is built +by :meth:`sphinx.builders.Builder.get_relative_uri` which creates a suitable +URI depending on the used builder, and appending the todo node's (the target's) +ID as the anchor name. + +.. rubric:: The ``setup`` function + +.. currentmodule:: sphinx.application + +As noted :doc:`previously <helloworld>`, the ``setup`` function is a requirement +and is used to plug directives into Sphinx. However, we also use it to hook up +the other parts of our extension. Let's look at our ``setup`` function: + +.. literalinclude:: examples/todo.py + :language: python + :linenos: + :lines: 116- + +The calls in this function refer to the classes and functions we added earlier. +What the individual calls do is the following: + +* :meth:`~Sphinx.add_config_value` lets Sphinx know that it should recognize the + new *config value* ``todo_include_todos``, whose default value should be + ``False`` (this also tells Sphinx that it is a boolean value). + + If the third argument was ``'html'``, HTML documents would be full rebuild if the + config value changed its value. This is needed for config values that + influence reading (build :ref:`phase 1 (reading) <build-phases>`). + +* :meth:`~Sphinx.add_node` adds a new *node class* to the build system. It also + can specify visitor functions for each supported output format. These visitor + functions are needed when the new nodes stay until :ref:`phase 4 (writing) + <build-phases>`. Since the ``todolist`` node is always replaced in + :ref:`phase 3 (resolving) <build-phases>`, it doesn't need any. + +* :meth:`~Sphinx.add_directive` adds a new *directive*, given by name and class. + +* Finally, :meth:`~Sphinx.connect` adds an *event handler* to the event whose + name is given by the first argument. The event handler function is called + with several arguments which are documented with the event. + +With this, our extension is complete. + + +Using the extension +------------------- + +As before, we need to enable the extension by declaring it in our +:file:`conf.py` file. There are two steps necessary here: + +#. Add the :file:`_ext` directory to the `Python path`_ using + ``sys.path.append``. This should be placed at the top of the file. + +#. Update or create the :confval:`extensions` list and add the extension file + name to the list + +In addition, we may wish to set the ``todo_include_todos`` config value. As +noted above, this defaults to ``False`` but we can set it explicitly. + +For example: + +.. code-block:: python + + import os + import sys + + sys.path.append(os.path.abspath("./_ext")) + + extensions = ['todo'] + + todo_include_todos = False + +You can now use the extension throughout your project. For example: + +.. code-block:: rst + :caption: index.rst + + Hello, world + ============ + + .. toctree:: + somefile.rst + someotherfile.rst + + Hello world. Below is the list of TODOs. + + .. todolist:: + +.. code-block:: rst + :caption: somefile.rst + + foo + === + + Some intro text here... + + .. todo:: Fix this + +.. code-block:: rst + :caption: someotherfile.rst + + bar + === + + Some more text here... + + .. todo:: Fix that + +Because we have configured ``todo_include_todos`` to ``False``, we won't +actually see anything rendered for the ``todo`` and ``todolist`` directives. +However, if we toggle this to true, we will see the output described +previously. + + +Further reading +--------------- + +For more information, refer to the `docutils`_ documentation and +:doc:`/extdev/index`. + + +.. _docutils: https://docutils.sourceforge.io/docs/ +.. _Python path: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH +.. _docutils documentation: https://docutils.sourceforge.io/docs/ref/rst/directives.html |