Adding upstream version 2.1.7.upstream/2.1.7

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

6 files changed, 225 insertions, 365 deletions

diff --git a/doc/sphinx/Pacemaker_Administration/administrative.rst b/doc/sphinx/Pacemaker_Administration/administrative.rst
new file mode 100644
index 0000000..7c8b346
--- /dev/null
+++ b/doc/sphinx/Pacemaker_Administration/administrative.rst
@@ -0,0 +1,150 @@
+.. index::
+   single: administrative mode
+
+Administrative Modes
+--------------------
+
+Intrusive administration can be performed on a Pacemaker cluster without
+causing resource failures, recovery, and fencing, by putting the cluster or a
+subset of it into an administrative mode.
+
+Pacemaker supports several administrative modes:
+
+* Maintenance mode for the entire cluster, specific nodes, or specific
+  resources
+* Unmanaged resources
+* Disabled configuration items
+* Standby mode for specific nodes
+
+Rules may be used to automatically set any of these modes for specific times or
+other conditions.
+
+
+.. index::
+   pair: administrative mode; maintenance mode
+
+.. _maintenance_mode:
+
+Maintenance Mode
+################
+
+In maintenance mode, the cluster will not start or stop resources. Recurring
+monitors for affected resources will be paused, except those specifying
+``role`` as ``Stopped``.
+
+To put a specific resource into maintenance mode, set the resource's
+``maintenance`` meta-attribute to ``true``.
+
+To put all active resources on a specific node into maintenance mode, set the
+node's ``maintenance`` node attribute to ``true``. When enabled, this overrides
+resource-specific maintenance mode.
+
+.. warning::
+
+   Restarting Pacemaker on a node that is in single-node maintenance mode will
+   likely lead to undesirable effects. If ``maintenance`` is set as a transient
+   attribute, it will be erased when Pacemaker is stopped, which will
+   immediately take the node out of maintenance mode and likely get it fenced.
+   If set as a permanent attribute, any resources active on the node will have
+   their local history erased when Pacemaker is restarted, so the cluster will
+   no longer consider them running on the node and thus will consider them
+   managed again, allowing them to be started elsewhere.
+
+To put all resources in the cluster into maintenance mode, set the
+``maintenance-mode`` cluster option to ``true``. When enabled, this overrides
+node- or resource- specific maintenance mode.
+
+Maintenance mode, at any level, overrides other administrative modes.
+
+
+.. index::
+   pair: administrative mode; unmanaged resources
+
+.. _unmanaged_resources:
+
+Unmanaged Resources
+###################
+
+An unmanaged resource will not be started or stopped by the cluster. A resource
+may become unmanaged in several ways:
+
+* The administrator may set the ``is-managed`` resource meta-attribute to
+  ``false`` (whether for a specific resource, or all resources without an
+  explicit setting via ``rsc_defaults``)
+* :ref:`Maintenance mode <maintenance_mode>` causes affected resources to
+  become unmanaged (and overrides any ``is-managed`` setting)
+* Certain types of failure cause affected resources to become unmanaged. These
+  include:
+
+  * Failed stop operations when the ``stonith-enabled`` cluster property is set
+    to ``false``
+  * Failure of an operation that has ``on-fail`` set to ``block``
+  * A resource detected as incorrectly active on more than one node when its
+    ``multiple-active`` meta-attribute is set to ``block``
+  * A resource constrained by a revoked ``rsc_ticket`` with ``loss-policy`` set
+    to ``freeze``
+  * Resources with ``requires`` set (or defaulting) to anything other than
+    ``nothing`` in a partition that loses quorum when the ``no-quorum-policy``
+    cluster option is set to ``freeze``
+
+Recurring actions are not affected by unmanaging a resource.
+
+.. warning::
+
+   Manually starting an unmanaged resource on a different node is strongly
+   discouraged. It will at least cause the cluster to consider the resource
+   failed, and may require the resource's ``target-role`` to be set to
+   ``Stopped`` then ``Started`` in order for recovery to succeed.
+
+
+.. index::
+   pair: administrative mode; disabled configuration
+
+.. _disabled_configuration:
+
+Disabled Configuration
+######################
+
+Some configuration elements disable particular behaviors:
+
+* The ``stonith-enabled`` cluster option, when set to ``false``, disables node
+  fencing. This is highly discouraged, as it can lead to data unavailability,
+  loss, or corruption.
+
+* The ``stop-all-resources`` cluster option, when set to ``true``, causes all
+  resources to be stopped.
+
+* Certain elements support an ``enabled`` meta-attribute, which if set to
+  ``false``, causes the cluster to act as if the specific element is not
+  configured. These include ``op``, ``alert`` *(since 2.1.6)*, and
+  ``recipient`` *(since 2.1.6)*. ``enabled`` may be set for specific ``op``
+  elements, or all operations without an explicit setting via ``op_defaults``.
+
+
+.. index::
+   pair: administrative mode; standby
+
+.. _standby:
+
+Standby Mode
+############
+
+When a node is put into standby, all resources will be moved away from the
+node, and all recurring operations will be stopped on the node, except those
+specifying ``role`` as ``Stopped`` (which will be newly initiated if
+appropriate).
+
+A node may be put into standby mode by setting its ``standby`` node attribute
+to ``true``. The attribute may be queried and set using the ``crm_standby``
+tool.
+
+
+.. index::
+   pair: administrative mode; rules
+
+Rules
+#####
+
+Rules may be used to set administrative mode options automatically according to
+various criteria such as date and time. See the "Rules" chapter of the
+*Pacemaker Explained* document for details.
diff --git a/doc/sphinx/Pacemaker_Administration/alerts.rst b/doc/sphinx/Pacemaker_Administration/alerts.rst
index c0f54c6..42efc8d 100644
--- a/doc/sphinx/Pacemaker_Administration/alerts.rst
+++ b/doc/sphinx/Pacemaker_Administration/alerts.rst
@@ -287,7 +287,7 @@ Special concerns when writing alert agents:
   this into consideration, for example by queueing resource-intensive actions
   into some other instance, instead of directly executing them.
    
-* Alert agents are run as the ``hacluster`` user, which has a minimal set
+* Alert agents are run as the |CRM_DAEMON_USER| user, which has a minimal set
   of permissions. If an agent requires additional privileges, it is
   recommended to configure ``sudo`` to allow the agent to run the necessary
   commands as another user with the appropriate privileges.
@@ -297,7 +297,7 @@ Special concerns when writing alert agents:
   user-configured ``timestamp-format``), ``CRM_alert_recipient,`` and all
   instance attributes. Mostly this is needed simply to protect against
   configuration errors, but if some user can modify the CIB without having
-  ``hacluster``-level access to the cluster nodes, it is a potential security
+  |CRM_DAEMON_USER| access to the cluster nodes, it is a potential security
   concern as well, to avoid the possibility of code injection.
    
 .. note:: **ocf:pacemaker:ClusterMon compatibility**
@@ -308,4 +308,4 @@ Special concerns when writing alert agents:
    passed to alert agents are available prepended with ``CRM_notify_``
    as well as ``CRM_alert_``. One break in compatibility is that ``ClusterMon``
    ran external scripts as the ``root`` user, while alert agents are run as the
-   ``hacluster`` user.
+   |CRM_DAEMON_USER| user.
diff --git a/doc/sphinx/Pacemaker_Administration/configuring.rst b/doc/sphinx/Pacemaker_Administration/configuring.rst
index 415dd81..295c96a 100644
--- a/doc/sphinx/Pacemaker_Administration/configuring.rst
+++ b/doc/sphinx/Pacemaker_Administration/configuring.rst
@@ -189,48 +189,53 @@ cluster even if the machine itself is not in the same cluster. To do this, one
 simply sets up a number of environment variables and runs the same commands as
 when working on a cluster node.
 
-.. table:: **Environment Variables Used to Connect to Remote Instances of the CIB**
-
-   +----------------------+-----------+------------------------------------------------+
-   | Environment Variable | Default   | Description                                    |
-   +======================+===========+================================================+
-   | CIB_user             | $USER     | .. index::                                     |
-   |                      |           |    single: CIB_user                            |
-   |                      |           |    single: environment variable; CIB_user      |
-   |                      |           |                                                |
-   |                      |           | The user to connect as. Needs to be            |
-   |                      |           | part of the ``haclient`` group on              |
-   |                      |           | the target host.                               |
-   +----------------------+-----------+------------------------------------------------+
-   | CIB_passwd           |           | .. index::                                     |
-   |                      |           |    single: CIB_passwd                          |
-   |                      |           |    single: environment variable; CIB_passwd    |
-   |                      |           |                                                |
-   |                      |           | The user's password. Read from the             |
-   |                      |           | command line if unset.                         |
-   +----------------------+-----------+------------------------------------------------+
-   | CIB_server           | localhost | .. index::                                     |
-   |                      |           |    single: CIB_server                          |
-   |                      |           |    single: environment variable; CIB_server    |
-   |                      |           |                                                |
-   |                      |           | The host to contact                            |
-   +----------------------+-----------+------------------------------------------------+
-   | CIB_port             |           | .. index::                                     |
-   |                      |           |    single: CIB_port                            |
-   |                      |           |    single: environment variable; CIB_port      |
-   |                      |           |                                                |
-   |                      |           | The port on which to contact the server;       |
-   |                      |           | required.                                      |
-   +----------------------+-----------+------------------------------------------------+
-   | CIB_encrypted        | TRUE      | .. index::                                     |
-   |                      |           |    single: CIB_encrypted                       |
-   |                      |           |    single: environment variable; CIB_encrypted |
-   |                      |           |                                                |
-   |                      |           | Whether to encrypt network traffic             |
-   +----------------------+-----------+------------------------------------------------+
+.. list-table:: **Environment Variables Used to Connect to Remote Instances of the CIB**
+   :class: longtable
+   :widths: 2 2 5
+   :header-rows: 1
+
+   * - Environment Variable
+     - Default
+     - Description
+   * - .. index::
+          single: CIB_user
+          single: environment variable; CIB_user
+
+       CIB_user
+     - |CRM_DAEMON_USER_RAW|
+     - The user to connect as. Needs to be part of the |CRM_DAEMON_GROUP| group
+       on the target host.
+   * - .. index::
+          single: CIB_passwd
+          single: environment variable; CIB_passwd
+
+       CIB_passwd
+     -
+     - The user's password. Read from the command line if unset.
+   * - .. index::
+          single: CIB_server
+          single: environment variable; CIB_server
+
+       CIB_server
+     - localhost
+     - The host to contact
+   * - .. index::
+          single: CIB_port
+          single: environment variable; CIB_port
+
+       CIB_port
+     -
+     - The port on which to contact the server; required
+   * - .. index::
+          single: CIB_encrypted
+          single: environment variable; CIB_encrypted
+
+       CIB_encrypted
+     - true
+     - Whether to encrypt network traffic
 
 So, if **c001n01** is an active cluster node and is listening on port 1234
-for connections, and **someuser** is a member of the **haclient** group,
+for connections, and **someuser** is a member of the |CRM_DAEMON_GROUP| group,
 then the following would prompt for **someuser**'s password and return
 the cluster's current configuration:
 
@@ -243,27 +248,9 @@ For security reasons, the cluster does not listen for remote connections by
 default.  If you wish to allow remote access, you need to set the
 ``remote-tls-port`` (encrypted) or ``remote-clear-port`` (unencrypted) CIB
 properties (i.e., those kept in the ``cib`` tag, like ``num_updates`` and
-``epoch``).
-
-.. table:: **Extra top-level CIB properties for remote access**
-
-   +----------------------+-----------+------------------------------------------------------+
-   | CIB Property         | Default   | Description                                          |
-   +======================+===========+======================================================+
-   | remote-tls-port      |           | .. index::                                           |
-   |                      |           |    single: remote-tls-port                           |
-   |                      |           |    single: CIB property; remote-tls-port             |
-   |                      |           |                                                      |
-   |                      |           | Listen for encrypted remote connections              |
-   |                      |           | on this port.                                        |
-   +----------------------+-----------+------------------------------------------------------+
-   | remote-clear-port    |           | .. index::                                           |
-   |                      |           |    single: remote-clear-port                         |
-   |                      |           |    single: CIB property; remote-clear-port           |
-   |                      |           |                                                      |
-   |                      |           | Listen for plaintext remote connections              |
-   |                      |           | on this port.                                        |
-   +----------------------+-----------+------------------------------------------------------+
+``epoch``). Encrypted communication is keyless, which makes it subject to
+man-in-the-middle attacks, and thus either option should be used only on
+protected networks.
 
 .. important::
 
diff --git a/doc/sphinx/Pacemaker_Administration/index.rst b/doc/sphinx/Pacemaker_Administration/index.rst
index 327ad31..af89380 100644
--- a/doc/sphinx/Pacemaker_Administration/index.rst
+++ b/doc/sphinx/Pacemaker_Administration/index.rst
@@ -22,6 +22,8 @@ Table of Contents
    cluster
    configuring
    tools
+   administrative
+   moving
    troubleshooting
    upgrading
    alerts
diff --git a/doc/sphinx/Pacemaker_Explained/advanced-options.rst b/doc/sphinx/Pacemaker_Administration/moving.rst
index 20ab79e..3d6a92a 100644
--- a/doc/sphinx/Pacemaker_Explained/advanced-options.rst
+++ b/doc/sphinx/Pacemaker_Administration/moving.rst
@@ -1,171 +1,11 @@
-Advanced Configuration
-----------------------
-
-.. index::
-   single: start-delay; operation attribute
-   single: interval-origin; operation attribute
-   single: interval; interval-origin
-   single: operation; interval-origin
-   single: operation; start-delay
-
-Specifying When Recurring Actions are Performed
-###############################################
-
-By default, recurring actions are scheduled relative to when the resource
-started. In some cases, you might prefer that a recurring action start relative
-to a specific date and time. For example, you might schedule an in-depth
-monitor to run once every 24 hours, and want it to run outside business hours.
-
-To do this, set the operation's ``interval-origin``. The cluster uses this point
-to calculate the correct ``start-delay`` such that the operation will occur
-at ``interval-origin`` plus a multiple of the operation interval.
-
-For example, if the recurring operation's interval is 24h, its
-``interval-origin`` is set to 02:00, and it is currently 14:32, then the
-cluster would initiate the operation after 11 hours and 28 minutes.
-
-The value specified for ``interval`` and ``interval-origin`` can be any
-date/time conforming to the
-`ISO8601 standard <https://en.wikipedia.org/wiki/ISO_8601>`_. By way of
-example, to specify an operation that would run on the first Monday of
-2021 and every Monday after that, you would add:
-
-.. topic:: Example recurring action that runs relative to base date/time
-
-   .. code-block:: xml
-
-      <op id="intensive-monitor" name="monitor" interval="P7D" interval-origin="2021-W01-1"/>
-
-.. index::
-   single: resource; failure recovery
-   single: operation; failure recovery
-
-.. _failure-handling:
-
-Handling Resource Failure
-#########################
-
-By default, Pacemaker will attempt to recover failed resources by restarting
-them. However, failure recovery is highly configurable.
-
-.. index::
-   single: resource; failure count
-   single: operation; failure count
-
-Failure Counts
-______________
-
-Pacemaker tracks resource failures for each combination of node, resource, and
-operation (start, stop, monitor, etc.).
-
-You can query the fail count for a particular node, resource, and/or operation
-using the ``crm_failcount`` command. For example, to see how many times the
-10-second monitor for ``myrsc`` has failed on ``node1``, run:
-
-.. code-block:: none
-
-   # crm_failcount --query -r myrsc -N node1 -n monitor -I 10s
-
-If you omit the node, ``crm_failcount`` will use the local node. If you omit
-the operation and interval, ``crm_failcount`` will display the sum of the fail
-counts for all operations on the resource.
-
-You can use ``crm_resource --cleanup`` or ``crm_failcount --delete`` to clear
-fail counts. For example, to clear the above monitor failures, run:
-
-.. code-block:: none
-
-   # crm_resource --cleanup -r myrsc -N node1 -n monitor -I 10s
-
-If you omit the resource, ``crm_resource --cleanup`` will clear failures for
-all resources. If you omit the node, it will clear failures on all nodes. If
-you omit the operation and interval, it will clear the failures for all
-operations on the resource.
-
-.. note::
-
-   Even when cleaning up only a single operation, all failed operations will
-   disappear from the status display. This allows us to trigger a re-check of
-   the resource's current status.
-
-Higher-level tools may provide other commands for querying and clearing
-fail counts.
-
-The ``crm_mon`` tool shows the current cluster status, including any failed
-operations. To see the current fail counts for any failed resources, call
-``crm_mon`` with the ``--failcounts`` option. This shows the fail counts per
-resource (that is, the sum of any operation fail counts for the resource).
-
-.. index::
-   single: migration-threshold; resource meta-attribute
-   single: resource; migration-threshold
-
-Failure Response
-________________
-
-Normally, if a running resource fails, pacemaker will try to stop it and start
-it again. Pacemaker will choose the best location to start it each time, which
-may be the same node that it failed on.
-
-However, if a resource fails repeatedly, it is possible that there is an
-underlying problem on that node, and you might desire trying a different node
-in such a case. Pacemaker allows you to set your preference via the
-``migration-threshold`` resource meta-attribute. [#]_
-
-If you define ``migration-threshold`` to *N* for a resource, it will be banned
-from the original node after *N* failures there.
-
-.. note::
-
-   The ``migration-threshold`` is per *resource*, even though fail counts are
-   tracked per *operation*. The operation fail counts are added together
-   to compare against the ``migration-threshold``.
-
-By default, fail counts remain until manually cleared by an administrator
-using ``crm_resource --cleanup`` or ``crm_failcount --delete`` (hopefully after
-first fixing the failure's cause). It is possible to have fail counts expire
-automatically by setting the ``failure-timeout`` resource meta-attribute.
-
-.. important::
-
-   A successful operation does not clear past failures. If a recurring monitor
-   operation fails once, succeeds many times, then fails again days later, its
-   fail count is 2. Fail counts are cleared only by manual intervention or
-   failure timeout.
-
-For example, setting ``migration-threshold`` to 2 and ``failure-timeout`` to
-``60s`` would cause the resource to move to a new node after 2 failures, and
-allow it to move back (depending on stickiness and constraint scores) after one
-minute.
-
-.. note::
-
-   ``failure-timeout`` is measured since the most recent failure. That is, older
-   failures do not individually time out and lower the fail count. Instead, all
-   failures are timed out simultaneously (and the fail count is reset to 0) if
-   there is no new failure for the timeout period.
-
-There are two exceptions to the migration threshold: when a resource either
-fails to start or fails to stop.
-
-If the cluster property ``start-failure-is-fatal`` is set to ``true`` (which is
-the default), start failures cause the fail count to be set to ``INFINITY`` and
-thus always cause the resource to move immediately.
-
-Stop failures are slightly different and crucial.  If a resource fails to stop
-and fencing is enabled, then the cluster will fence the node in order to be
-able to start the resource elsewhere.  If fencing is disabled, then the cluster
-has no way to continue and will not try to start the resource elsewhere, but
-will try to stop it again after any failure timeout or clearing.
+Moving Resources
+----------------
 
 .. index::
    single: resource; move
 
-Moving Resources
-################
-
 Moving Resources Manually
-_________________________
+#########################
 
 There are primarily two occasions when you would want to move a resource from
 its current location: when the whole node is under maintenance, and when a
@@ -176,7 +16,7 @@ single resource needs to be moved.
    single: node; standby mode
 
 Standby Mode
-~~~~~~~~~~~~
+____________
 
 Since everything eventually comes down to a score, you could create constraints
 for every resource to prevent them from running on one node. While Pacemaker
@@ -215,7 +55,7 @@ A cluster node in standby mode will not run resources, but still contributes to
 quorum, and may fence or be fenced by nodes.
 
 Moving One Resource
-~~~~~~~~~~~~~~~~~~~
+___________________
 
 When only one resource is required to move, we could do this by creating
 location constraints.  However, once again we provide a user-friendly shortcut
@@ -281,9 +121,10 @@ constraint will prevent the resource from running on that node until
 cluster node is no longer available!
 
 In some cases, such as when ``resource-stickiness`` is set to ``INFINITY``, it
-is possible that you will end up with the problem described in
-:ref:`node-score-equal`. The tool can detect some of these cases and deals with
-them by creating both positive and negative constraints. For example:
+is possible that you will end up with nodes with the same score, forcing the
+cluster to choose one (which may not be the one you want). The tool can detect
+some of these cases and deals with them by creating both positive and negative
+constraints. For example:
 
 .. code-block:: xml
 
@@ -293,7 +134,7 @@ them by creating both positive and negative constraints. For example:
 which has the same long-term consequences as discussed earlier.
 
 Moving Resources Due to Connectivity Changes
-____________________________________________
+############################################
 
 You can configure the cluster to move resources when external connectivity is
 lost in two steps.
@@ -303,7 +144,7 @@ lost in two steps.
    single: ping resource
 
 Tell Pacemaker to Monitor Connectivity
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+______________________________________
 
 First, add an ``ocf:pacemaker:ping`` resource to the cluster. The ``ping``
 resource uses the system utility of the same name to a test whether a list of
@@ -372,12 +213,12 @@ with a description of the most interesting parameters.
    deal with the connectivity status that ``ocf:pacemaker:ping`` is recording.
 
 Tell Pacemaker How to Interpret the Connectivity Data
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_____________________________________________________
 
 .. important::
 
-   Before attempting the following, make sure you understand
-   :ref:`rules`.
+   Before attempting the following, make sure you understand rules. See the
+   "Rules" chapter of the *Pacemaker Explained* document for details.
 
 There are a number of ways to use the connectivity data.
 
@@ -462,125 +303,3 @@ nodes provided they have connectivity to at least three (again assuming that
             <expression id="ping-prefer" attribute="pingd" operation="defined"/>
          </rule>
       </rsc_location>
-
-
-.. _live-migration:
-
-Migrating Resources
-___________________
-
-Normally, when the cluster needs to move a resource, it fully restarts the
-resource (that is, it stops the resource on the current node and starts it on
-the new node).
-
-However, some types of resources, such as many virtual machines, are able to
-move to another location without loss of state (often referred to as live
-migration or hot migration). In pacemaker, this is called resource migration.
-Pacemaker can be configured to migrate a resource when moving it, rather than
-restarting it.
-
-Not all resources are able to migrate; see the
-:ref:`migration checklist <migration_checklist>` below. Even those that can,
-won't do so in all situations. Conceptually, there are two requirements from
-which the other prerequisites follow:
-
-* The resource must be active and healthy at the old location; and
-* everything required for the resource to run must be available on both the old
-  and new locations.
-
-The cluster is able to accommodate both *push* and *pull* migration models by
-requiring the resource agent to support two special actions: ``migrate_to``
-(performed on the current location) and ``migrate_from`` (performed on the
-destination).
-
-In push migration, the process on the current location transfers the resource
-to the new location where is it later activated. In this scenario, most of the
-work would be done in the ``migrate_to`` action and, if anything, the
-activation would occur during ``migrate_from``.
-
-Conversely for pull, the ``migrate_to`` action is practically empty and
-``migrate_from`` does most of the work, extracting the relevant resource state
-from the old location and activating it.
-
-There is no wrong or right way for a resource agent to implement migration, as
-long as it works.
-
-.. _migration_checklist:
-
-.. topic:: Migration Checklist
-
-   * The resource may not be a clone.
-   * The resource agent standard must be OCF.
-   * The resource must not be in a failed or degraded state.
-   * The resource agent must support ``migrate_to`` and ``migrate_from``
-     actions, and advertise them in its meta-data.
-   * The resource must have the ``allow-migrate`` meta-attribute set to
-     ``true`` (which is not the default).
-
-If an otherwise migratable resource depends on another resource via an ordering
-constraint, there are special situations in which it will be restarted rather
-than migrated.
-
-For example, if the resource depends on a clone, and at the time the resource
-needs to be moved, the clone has instances that are stopping and instances that
-are starting, then the resource will be restarted. The scheduler is not yet
-able to model this situation correctly and so takes the safer (if less optimal)
-path.
-
-Also, if a migratable resource depends on a non-migratable resource, and both
-need to be moved, the migratable resource will be restarted.
-
-
-.. index::
-   single: reload
-   single: reload-agent
-
-Reloading an Agent After a Definition Change
-############################################
-
-The cluster automatically detects changes to the configuration of active
-resources. The cluster's normal response is to stop the service (using the old
-definition) and start it again (with the new definition). This works, but some
-resource agents are smarter and can be told to use a new set of options without
-restarting.
-
-To take advantage of this capability, the resource agent must:
-
-* Implement the ``reload-agent`` action. What it should do depends completely
-  on your application!
-
-  .. note::
-
-     Resource agents may also implement a ``reload`` action to make the managed
-     service reload its own *native* configuration. This is different from
-     ``reload-agent``, which makes effective changes in the resource's
-     *Pacemaker* configuration (specifically, the values of the agent's
-     reloadable parameters).
-
-* Advertise the ``reload-agent`` operation in the ``actions`` section of its
-  meta-data.
-
-* Set the ``reloadable`` attribute to 1 in the ``parameters`` section of
-  its meta-data for any parameters eligible to be reloaded after a change.
-
-Once these requirements are satisfied, the cluster will automatically know to
-reload the resource (instead of restarting) when a reloadable parameter
-changes.
-
-.. note::
-
-   Metadata will not be re-read unless the resource needs to be started. If you
-   edit the agent of an already active resource to set a parameter reloadable,
-   the resource may restart the first time the parameter value changes.
-
-.. note::
-
-   If both a reloadable and non-reloadable parameter are changed
-   simultaneously, the resource will be restarted.
-
-.. rubric:: Footnotes
-
-.. [#] The naming of this option was perhaps unfortunate as it is easily
-       confused with live migration, the process of moving a resource from one
-       node to another without stopping it.  Xen virtual guests are the most
-       common example of resources that can be migrated in this manner.
diff --git a/doc/sphinx/Pacemaker_Administration/pcs-crmsh.rst b/doc/sphinx/Pacemaker_Administration/pcs-crmsh.rst
index 61ab4e6..3eda60a 100644
--- a/doc/sphinx/Pacemaker_Administration/pcs-crmsh.rst
+++ b/doc/sphinx/Pacemaker_Administration/pcs-crmsh.rst
@@ -118,14 +118,11 @@ Manage Resources
 .. topic:: Create a Resource
 
    .. code-block:: none
-
-      crmsh # crm configure primitive ClusterIP ocf:heartbeat:IPaddr2 \
-              params ip=192.168.122.120 cidr_netmask=24 \
-              op monitor interval=30s 
+      crmsh # crm configure primitive ClusterIP IPaddr2 params ip=192.168.122.120 cidr_netmask=24
       pcs   # pcs resource create ClusterIP IPaddr2 ip=192.168.122.120 cidr_netmask=24
 
-pcs determines the standard and provider (``ocf:heartbeat``) automatically
-since ``IPaddr2`` is unique, and automatically creates operations (including
+Both crmsh and pcs determine the standard and provider (``ocf:heartbeat``) automatically
+since ``IPaddr2`` is unique, and automatically create operations (including
 monitor) based on the agent's meta-data.
 
 .. topic:: Show Configuration of All Resources
@@ -270,6 +267,10 @@ edited and verified before committing to the live configuration:
       crmsh    # crm configure ms WebDataClone WebData \
                  meta master-max=1 master-node-max=1 \
                  clone-max=2 clone-node-max=1 notify=true
+      crmsh    # crm configure clone WebDataClone WebData \
+                 meta promotable=true \
+                 promoted-max=1 promoted-node-max=1 \
+                 clone-max=2 clone-node-max=1 notify=true
       pcs-0.9  # pcs resource master WebDataClone WebData \
                  master-max=1 master-node-max=1 \
                  clone-max=2 clone-node-max=1 notify=true
@@ -277,6 +278,7 @@ edited and verified before committing to the live configuration:
                  promoted-max=1 promoted-node-max=1 \
                  clone-max=2 clone-node-max=1 notify=true
 
+crmsh supports both ways ('configure ms' is deprecated) to configure promotable clone since crmsh 4.4.0.
 pcs will generate the clone name automatically if it is omitted from the
 command line.