Merging upstream version 2.1.8~rc1.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

7 files changed, 284 insertions, 47 deletions

diff --git a/doc/sphinx/Pacemaker_Development/c.rst b/doc/sphinx/Pacemaker_Development/c.rst
index b03ddae..8bc5e80 100644
--- a/doc/sphinx/Pacemaker_Development/c.rst
+++ b/doc/sphinx/Pacemaker_Development/c.rst
@@ -752,12 +752,35 @@ Function names should be unique across the entire project, to allow for
 individual tracing via ``PCMK_trace_functions``, and make it easier to search
 code and follow detail logs.
 
-A common function signature is a comparison function that returns 0 if its
-arguments are equal for sorting purposes, -1 if the first argument should sort
-first, and 1 is the second argument should sort first. Such a function should
-have ``cmp`` in its name, to parallel ``strcmp()``; ``sort`` should only be
-used in the names of functions that sort an entire list (typically using a
-``cmp`` function).
+.. _sort_func:
+
+Sorting
+^^^^^^^
+
+A function that sorts an entire list should have ``sort`` in its name. It sorts
+elements using a :ref:`comparison <compare_func>` function, which may be either
+hard-coded or passed as an argument.
+
+.. _compare_func:
+
+Comparison
+^^^^^^^^^^
+
+A comparison function for :ref:`sorting <sort_func>` should have ``cmp`` in its
+name and should *not* have ``sort`` in its name.
+
+.. _constructor_func:
+
+Constructors
+^^^^^^^^^^^^
+
+A constructor creates a new dynamically allocated object. It may perform some
+initialization procedure on the new object.
+
+* If the constructor always creates an independent object instance, its name
+  should include ``new``.
+* If the constructor may add the new object to some existing object, its name
+  should include ``create``.
 
 
 Function Definitions
@@ -832,6 +855,12 @@ messages and converting from one to another, can be found in
 Of course, functions may have return values that aren't success/failure
 indicators, such as a pointer, integer count, or bool.
 
+:ref:`Comparison <compare_func>` functions should return
+
+* a negative integer if the first argument should sort first
+* 0 if its arguments are equal for sorting purposes
+* a positive integer is the second argument should sort first
+
 
 Public API Functions
 ____________________
@@ -880,6 +909,30 @@ __________________________________
 * The convenience macros ``pcmk__plural_s()`` and ``pcmk__plural_alt()`` are
   handy when logging a word that may be singular or plural.
 
+Log Levels
+__________
+
+When to use each log level:
+
+* **critical:** fatal error (usually something that would make a daemon exit)
+* **error:** failure of something that affects the cluster (such as a resource
+  action, fencing action, etc.) or daemon operation
+* **warning:** minor, potential, or recoverable failures (such as something
+  only affecting a daemon client, or invalid configuration that can be left to
+  default)
+* **notice:** important successful events (such as a node joining or leaving,
+  resource action results, or configuration changes)
+* **info:** events that would be helpful with troubleshooting (such as status
+  section updates or elections)
+* **debug:** information that would be helpful for debugging code or complex
+  problems
+* **trace:** like debug but for very noisy or low-level stuff
+
+By default, critical through notice are logged to the system log and detail
+log, info is logged to the detail log only, and debug and trace are not logged
+(if enabled, they go to the detail log only).
+
+
 Logging
 _______
 
@@ -912,6 +965,34 @@ using libqb's "extended logging" feature:
             pcmk_rc_str(rc), rc, id);
 
 
+Assertion Logging
+_________________
+
+``CRM_ASSERT(expr)``
+  If ``expr`` is false, this will call <code>crm_err()</code> with a "Triggered
+  fatal assert" message (with details), then abort execution. This should be
+  used for logic errors that should be impossible (such as a NULL function
+  argument where not accepted) and environmental errors that can't be handled
+  gracefully (for example, memory allocation failures, though returning
+  ``ENOMEM`` is often better).
+
+``CRM_LOG_ASSERT(expr)``
+  If ``expr`` is false, this will generally log a message without aborting. If
+  the log level is below trace, it just calls ``crm_err()`` with a "Triggered
+  assert" message (with details). If the log level is trace, and the caller is
+  a daemon, then it will fork a child process in which to dump core, as well as
+  logging the message. If the log level is trace, and the caller is not a
+  daemon, then it will behave like ``CRM_ASSERT()`` (i.e. log and abort). This
+  should be used for logic or protocol errors that require no special handling.
+
+``CRM_CHECK(expr, failed_action)``
+  If ``expr`` is false, behave like ``CRM_LOG_ASSERT(expr)`` (that is, log a
+  message and dump core if requested) then perform ``failed_action`` (which
+  must not contain ``continue``, ``break``, or ``errno``). This should be used
+  for logic or protocol errors that can be handled, usually by returning an
+  error status.
+
+
 Output
 ______
 
@@ -924,12 +1005,40 @@ A custom message can be defined with a unique string identifier, plus
 implementation functions for each supported format. The caller invokes the
 message using the identifier. The user selects the output format via
 ``--output-as``, and the output code automatically calls the appropriate
-implementation function.
+implementation function. Custom messages are useful when you want to output 
+messages that are more complex than a one-line error or informational message, 
+reproducible, and automatically handled by the output formatting system. 
+Custom messages can contain other custom messages.
+
+Custom message functions are implemented as follows: Start with the macro 
+``PCMK__OUTPUT_ARGS``, whose arguments are the message name, followed by the 
+arguments to the message. Then there is the function declaration, for which the 
+arguments are the pointer to the current output object, then a variable argument 
+list.
+
+To output a custom message, you first need to create, i.e. register, the custom 
+message that you want to output. Either call ``register_message``, which 
+registers a custom message at runtime, or make use of the collection of 
+predefined custom messages in ``fmt_functions``, which is defined in 
+``lib/pacemaker/pcmk_output.c``. Once you have the message to be outputted, 
+output it by calling ``message``.
+
+Note: The ``fmt_functions`` functions accommodate all of the output formats; 
+the default implementation accommodates any format that isn't explicitly 
+accommodated. The default output provides valid output for any output format, 
+but you may still want to implement a specific output, i.e. xml, text, or html. 
+The ``message`` function automatically knows which implementation to use, 
+because the ``pcmk__output_s`` contains this information.
 
 The interface (most importantly ``pcmk__output_t``) is declared in
 ``include/crm/common/output*h``. See the API comments and existing tools for
-examples.
+examples. 
 
+Some of its important member functions are ``err``, which formats error messages 
+and ``info``, which formats informational messages. Also, ``list_item``, 
+which formats list items, ``begin_list``, which starts lists, and ``end_list``, 
+which ends lists, are important because lists can be useful, yet differently 
+handled by the different output types.
 
 .. index::
    single: Makefile.am
diff --git a/doc/sphinx/Pacemaker_Development/components.rst b/doc/sphinx/Pacemaker_Development/components.rst
index 5086fa8..ce6b36b 100644
--- a/doc/sphinx/Pacemaker_Development/components.rst
+++ b/doc/sphinx/Pacemaker_Development/components.rst
@@ -27,10 +27,10 @@ As might be expected, it has the most code of any of the daemons.
 Join sequence
 _____________
 
-Most daemons track their cluster peers using Corosync's membership and CPG
-only. The controller additionally requires peers to `join`, which ensures they
-are ready to be assigned tasks. Joining proceeds through a series of phases
-referred to as the `join sequence` or `join process`.
+Most daemons track their cluster peers using Corosync's membership and
+:term:`CPG` only. The controller additionally requires peers to `join`, which
+ensures they are ready to be assigned tasks. Joining proceeds through a series
+of phases referred to as the `join sequence` or `join process`.
 
 A node's current join phase is tracked by the ``join`` member of ``crm_node_t``
 (used in the peer cache). It is an ``enum crm_join_phase`` that (ideally)
@@ -141,7 +141,7 @@ _______________
 
 The function calls for a fencing request go something like this:
 
-The local fencer receives the client's request via an IPC or messaging
+The local fencer receives the client's request via an :term:`IPC` or messaging
 layer callback, which calls
 
 * ``stonith_command()``, which (for requests) calls
@@ -199,8 +199,8 @@ __________________
 
 Each ``STONITH_OP_FENCE`` request goes something like this:
 
-The chosen peer fencer receives the ``STONITH_OP_FENCE`` request via IPC or
-messaging layer callback, which calls:
+The chosen peer fencer receives the ``STONITH_OP_FENCE`` request via
+:term:`IPC` or messaging layer callback, which calls:
 
 * ``stonith_command()``, which (for requests) calls
 
@@ -240,7 +240,7 @@ returns, and calls
 Fencing replies
 _______________
 
-The original fencer receives the ``STONITH_OP_FENCE`` reply via IPC or
+The original fencer receives the ``STONITH_OP_FENCE`` reply via :term:`IPC` or
 messaging layer callback, which calls:
 
 * ``stonith_command()``, which (for replies) calls
@@ -295,10 +295,10 @@ The purpose of the scheduler is to take a CIB as input and generate a
 transition graph (list of actions that need to be taken) as output.
 
 The controller invokes the scheduler by contacting the scheduler daemon via
-local IPC. Tools such as ``crm_simulate``, ``crm_mon``, and ``crm_resource``
-can also invoke the scheduler, but do so by calling the library functions
-directly. This allows them to run using a ``CIB_file`` without the cluster
-needing to be active.
+local :term:`IPC`. Tools such as ``crm_simulate``, ``crm_mon``, and
+``crm_resource`` can also invoke the scheduler, but do so by calling the
+library functions directly. This allows them to run using a ``CIB_file``
+without the cluster needing to be active.
 
 The main entry point for the scheduler code is
 ``lib/pacemaker/pcmk_scheduler.c:pcmk__schedule_actions()``. It sets
@@ -315,7 +315,7 @@ defaults and calls a series of functions for the scheduling. Some key steps:
   the CIB status section. This is used to decide whether certain
   actions need to be done, such as deleting orphan resources, forcing a restart
   when a resource definition changes, etc.
-* ``assign_resources()`` assigns resources to nodes.
+* ``assign_resources()`` :term:`assigns <assign>` resources to nodes.
 * ``schedule_resource_actions()`` schedules resource-specific actions (which
   might or might not end up in the final graph).
 * ``pcmk__apply_orderings()`` processes ordering constraints in order to modify
@@ -364,7 +364,7 @@ Resources
 _________
 
 ``pcmk_resource_t`` is the data object representing cluster resources. A
-resource has a variant: primitive (a.k.a. native), group, clone, or bundle.
+resource has a variant: :term:`primitive`, group, clone, or :term:`bundle`.
 
 The resource object has members for two sets of methods,
 ``resource_object_functions_t`` from the ``libpe_status`` public API, and
@@ -374,9 +374,9 @@ The resource object has members for two sets of methods,
 The object functions have basic capabilities such as unpacking the resource
 XML, and determining the current or planned location of the resource.
 
-The assignment functions have more obscure capabilities needed for scheduling,
-such as processing location and ordering constraints. For example,
-``pcmk__create_internal_constraints()`` simply calls the
+The :term:`assignment <assign>` functions have more obscure capabilities needed
+for scheduling, such as processing location and ordering constraints. For
+example, ``pcmk__create_internal_constraints()`` simply calls the
 ``internal_constraints()`` method for each top-level resource in the cluster.
 
 .. index::
@@ -385,9 +385,10 @@ such as processing location and ordering constraints. For example,
 Nodes
 _____
 
-Assignment of resources to nodes is done by choosing the node with the highest
-score for a given resource. The scheduler does a bunch of processing to
-generate the scores, then the actual assignment is straightforward.
+:term:`Assignment <assign>` of resources to nodes is done by choosing the node
+with the highest :term:`score` for a given resource. The scheduler does a bunch
+of processing to generate the scores, then the actual assignment is
+straightforward.
 
 Node lists are frequently used. For example, ``pcmk_scheduler_t`` has a
 ``nodes`` member which is a list of all nodes in the cluster, and
@@ -435,8 +436,8 @@ ___________
 
 Colocation constraints come into play in these parts of the scheduler code:
 
-* When sorting resources for assignment, so resources with highest node score
-  are assigned first (see ``cmp_resources()``)
+* When sorting resources for :term:`assignment <assign>`, so resources with
+  highest node :term:`score` are assigned first (see ``cmp_resources()``)
 * When updating node scores for resource assigment or promotion priority
 * When assigning resources, so any resources to be colocated with can be
   assigned first, and so colocations affect where the resource is assigned
@@ -449,7 +450,8 @@ The resource assignment functions have several methods related to colocations:
   dependent's allowed node scores (if called while resources are being
   assigned) or the dependent's priority (if called while choosing promotable
   instance roles). It can behave differently depending on whether it is being
-  called as the primary's method or as the dependent's method.
+  called as the :term:`primary's <primary>` method or as the :term:`dependent's
+  <dependent>` method.
 * ``add_colocated_node_scores():`` This updates a table of nodes for a given
   colocation attribute and score. It goes through colocations involving a given
   resource, and updates the scores of the nodes in the table with the best
diff --git a/doc/sphinx/Pacemaker_Development/documentation.rst b/doc/sphinx/Pacemaker_Development/documentation.rst
new file mode 100644
index 0000000..6880bb0
--- /dev/null
+++ b/doc/sphinx/Pacemaker_Development/documentation.rst
@@ -0,0 +1,35 @@
+.. index::
+   pair: documentation; guidelines
+
+Documentation Guidelines
+------------------------
+
+See `doc/README.md
+<https://github.com/ClusterLabs/pacemaker/blob/main/doc/README.md>`_ in the
+source code repository for the kinds of documentation that Pacemaker provides.
+
+Books
+#####
+
+The ``doc/sphinx`` subdirectory has a subdirectory for each book by title. Each
+book's directory contains .rst files, which are the chapter sources in
+`reStructuredText
+<https://www.sphinx-doc.org/en/master/usage/restructuredtext/>`_ format (with
+index.rst as the starting point).
+
+Once you have edited the sources as desired, run ``make`` in the ``doc`` or
+``doc/sphinx`` directory to generate all the books locally. You can view the
+results by pointing your web browser to (replacing PATH\_TO\_CHECKOUT and
+BOOK\_TITLE appropriately):
+
+    file:///PATH_TO_CHECKOUT/doc/sphinx/BOOK_TITLE/_build/html/index.html
+
+See the comments at the top of ``doc/sphinx/Makefile.am`` for options you can
+use.
+
+Recommended practices:
+
+* Use ``list-table`` instead of ``table`` for tables
+* When documenting newly added features and syntax, add "\*(since X.Y.Z)\*"
+  with the version introducing them. These comments can be removed when rolling
+  upgrades from that version are no longer supported.
diff --git a/doc/sphinx/Pacemaker_Development/faq.rst b/doc/sphinx/Pacemaker_Development/faq.rst
index e738b7d..b1b1e5a 100644
--- a/doc/sphinx/Pacemaker_Development/faq.rst
+++ b/doc/sphinx/Pacemaker_Development/faq.rst
@@ -32,21 +32,20 @@ Frequently Asked Questions
 
 :Q: What are the different Git branches and repositories used for?
 :A: * The `main branch <https://github.com/ClusterLabs/pacemaker/tree/main>`_
-      is the primary branch used for development.
-    * The `2.1 branch <https://github.com/ClusterLabs/pacemaker/tree/2.1>`_ is
-      the current release branch. Normally, it does not receive any changes, but
-      during the release cycle for a new release, it will contain release
-      candidates. During the release cycle, certain bug fixes will go to the
-      2.1 branch first (and be pulled into main later).
+      is used for all new development.
+    * The `3.0 <https://github.com/ClusterLabs/pacemaker/tree/3.0>`_ and
+      `2.1 <https://github.com/ClusterLabs/pacemaker/tree/2.1>`_ branches are
+      for the currently supported major and minor version release series.
+      Normally, they do not receive any changes, but during the release cycle
+      for a new release, they will contain release candidates. The main branch
+      is pulled into 3.0 just before the first release candidate of a new
+      release, but otherwise, separate pull requests must be submitted to
+      backport changes from the main branch into a release branch.
     * The `2.0 branch <https://github.com/ClusterLabs/pacemaker/tree/2.0>`_,
       `1.1 branch <https://github.com/ClusterLabs/pacemaker/tree/1.1>`_,
       and separate
       `1.0 repository <https://github.com/ClusterLabs/pacemaker-1.0>`_
       are frozen snapshots of earlier release series, no longer being developed.
-    * Messages will be posted to the
-      `developers@ClusterLabs.org <https://lists.ClusterLabs.org/mailman/listinfo/developers>`_
-      mailing list during the release cycle, with instructions about which
-      branches to use when submitting requests.
 
 ----
 
@@ -163,9 +162,5 @@ Frequently Asked Questions
 
 :Q: What if I still have questions?
 :A: Ask on the
-    `developers@ClusterLabs.org <https://lists.ClusterLabs.org/mailman/listinfo/developers>`_
-    mailing list for development-related questions, or on the
-    `users@ClusterLabs.org <https://lists.ClusterLabs.org/mailman/listinfo/users>`_
-    mailing list for general questions about using Pacemaker.
-    Developers often also hang out on the
-    [ClusterLabs IRC channel](https://wiki.clusterlabs.org/wiki/ClusterLabs_IRC_channel).
+    `ClusterLabs mailing lists
+    <https://projects.clusterlabs.org/w/clusterlabs/clusterlabs_mailing_lists/>`_.
diff --git a/doc/sphinx/Pacemaker_Development/general.rst b/doc/sphinx/Pacemaker_Development/general.rst
index 9d9dcec..94015c9 100644
--- a/doc/sphinx/Pacemaker_Development/general.rst
+++ b/doc/sphinx/Pacemaker_Development/general.rst
@@ -38,3 +38,13 @@ may put more specific copyright notices in their commit messages if desired.
        `"Updating Copyright Notices"
        <https://techwhirl.com/updating-copyright-notices/>`_ for a more
        readable summary.
+
+Terminology
+###########
+
+Pacemaker is extremely complex, and it helps to use terminology consistently
+throughout documentation, symbol names and comments in code, and so forth. It
+also helps to use natural language when practical instead of technical jargon
+and acronyms.
+
+For specific recommendations, see the :ref:`glossary`.
diff --git a/doc/sphinx/Pacemaker_Development/glossary.rst b/doc/sphinx/Pacemaker_Development/glossary.rst
new file mode 100644
index 0000000..6f73e96
--- /dev/null
+++ b/doc/sphinx/Pacemaker_Development/glossary.rst
@@ -0,0 +1,84 @@
+.. index::
+   single: glossary
+
+.. _glossary:
+
+Glossary
+--------
+
+.. glossary::
+
+    assign
+      In the scheduler, this refers to associating a resource with a node. Do
+      not use *allocate* for this purpose.
+
+    bundle
+      The collective resource type associating instances of a container with
+      storage and networking. Do not use :term:`container` when referring to
+      the bundle as a whole.
+
+    cluster layer
+      The layer of the :term:`cluster stack` that provides membership and
+      messaging capabilities (such as Corosync).
+
+    cluster stack
+      The core components of a high-availability cluster: the
+      :term:`cluster layer` at the "bottom" of the stack, then Pacemaker, then
+      resource agents, and then the actual services managed by the cluster at
+      the "top" of the stack. Do not use *stack* for the cluster layer alone.
+
+    CPG
+      Corosync Process Group. This is the messaging layer in a Corosync-based
+      cluster. Pacemaker daemons use CPG to communicate with their counterparts
+      on other nodes.
+
+    container
+      This can mean either a container in the usual sense (whether as a
+      standalone resource or as part of a bundle), or as the container resource
+      meta-attribute (which does not necessarily reference a container in the
+      usual sense).
+
+    dangling migration
+      Live migration of a resource consists of a **migrate_to** action on the
+      source node, followed by a **migrate_from** on the target node, followed
+      by a **stop** on the source node. If the **migrate_to** and
+      **migrate_from** have completed successfully, but the **stop** has not
+      yet been done, the migration is considered to be *dangling*.
+
+    dependent
+      In colocation constraints, this refers to the resource located relative
+      to the :term:`primary` resource. Do not use *rh* or *right-hand* for this
+      purpose.
+
+    IPC
+      Inter-process communication. In Pacemaker, clients send requests to
+      daemons using libqb IPC.
+
+    message
+      This can refer to log messages, custom messages defined for a
+      **pcmk_output_t** object, or XML messages sent via :term:`CPG` or
+      :term:`IPC`.
+
+    metadata
+      In the context of options and resource agents, this refers to OCF-style
+      metadata. Do not use a hyphen except when referring to the OCF-defined
+      action name *meta-data*.
+
+    primary
+      In colocation constraints, this refers to the resource that the
+      :term:`dependent` resource is located relative to. Do not use *lh* or
+      *left-hand* for this purpose.
+
+    primitive
+      The fundamental resource type in Pacemaker. Do not use *native* for this
+      purpose.
+
+    score
+      An integer value constrained between **-PCMK_SCORE_INFINITY** and
+      **+PCMK_SCORE_INFINITY**. Certain strings (such as
+      **PCMK_VALUE_INFINITY**) parse as particular score values. Do not use
+      *weight* for this purpose.
+
+    self-fencing
+      When a node is chosen to execute its own fencing. Do not use *suicide*
+      for this purpose.
diff --git a/doc/sphinx/Pacemaker_Development/index.rst b/doc/sphinx/Pacemaker_Development/index.rst
index cbe1499..a3f624f 100644
--- a/doc/sphinx/Pacemaker_Development/index.rst
+++ b/doc/sphinx/Pacemaker_Development/index.rst
@@ -20,11 +20,13 @@ Table of Contents
 
    faq
    general
+   documentation
    python
    c
    components
    helpers
    evolution
+   glossary
 
 Index
 -----