summaryrefslogtreecommitdiffstats
path: root/doc/sphinx/Pacemaker_Administration/agents.rst
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--doc/sphinx/Pacemaker_Administration/agents.rst1247
1 files changed, 993 insertions, 254 deletions
diff --git a/doc/sphinx/Pacemaker_Administration/agents.rst b/doc/sphinx/Pacemaker_Administration/agents.rst
index e5b17e2..34bea60 100644
--- a/doc/sphinx/Pacemaker_Administration/agents.rst
+++ b/doc/sphinx/Pacemaker_Administration/agents.rst
@@ -53,123 +53,143 @@ _______
All OCF resource agents are required to implement the following actions.
-.. table:: **Required Actions for OCF Agents**
-
- +--------------+-------------+------------------------------------------------+
- | Action | Description | Instructions |
- +==============+=============+================================================+
- | start | Start the | .. index:: |
- | | resource | single: OCF resource agent; start |
- | | | single: start action |
- | | | |
- | | | Return 0 on success and an appropriate |
- | | | error code otherwise. Must not report |
- | | | success until the resource is fully |
- | | | active. |
- +--------------+-------------+------------------------------------------------+
- | stop | Stop the | .. index:: |
- | | resource | single: OCF resource agent; stop |
- | | | single: stop action |
- | | | |
- | | | Return 0 on success and an appropriate |
- | | | error code otherwise. Must not report |
- | | | success until the resource is fully |
- | | | stopped. |
- +--------------+-------------+------------------------------------------------+
- | monitor | Check the | .. index:: |
- | | resource's | single: OCF resource agent; monitor |
- | | state | single: monitor action |
- | | | |
- | | | Exit 0 if the resource is running, 7 |
- | | | if it is stopped, and any other OCF |
- | | | exit code if it is failed. NOTE: The |
- | | | monitor script should test the state |
- | | | of the resource on the local machine |
- | | | only. |
- +--------------+-------------+------------------------------------------------+
- | meta-data | Describe | .. index:: |
- | | the | single: OCF resource agent; meta-data |
- | | resource | single: meta-data action |
- | | | |
- | | | Provide information about this |
- | | | resource in the XML format defined by |
- | | | the OCF standard. Exit with 0. NOTE: |
- | | | This is *not* required to be performed |
- | | | as root. |
- +--------------+-------------+------------------------------------------------+
+.. list-table:: **Required Actions for OCF Agents**
+ :class: longtable
+ :widths: 1 4 3
+ :header-rows: 1
+
+ * - Action
+ - Description
+ - Instructions
+ * - .. _start_action:
+
+ .. index::
+ single: OCF resource agent; start
+ single: start action
+
+ start
+ - Start the resource
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` on success and an appropriate
+ error code otherwise. Must not report success until the resource is fully
+ active.
+ * - .. _stop_action:
+
+ .. index::
+ single: OCF resource agent; stop
+ single: stop action
+
+ stop
+ - Stop the resource
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` on success and an appropriate
+ error code otherwise. Must not report success until the resource is fully
+ stopped.
+ * - .. _monitor_action:
+
+ .. index::
+ single: OCF resource agent; monitor
+ single: monitor action
+
+ monitor
+ - Check the resource's state
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` if the resource is running,
+ :ref:`OCF_NOT_RUNNING <OCF_NOT_RUNNING>` if it is stopped, and any other
+ :ref:`OCF exit code <ocf_return_codes>` if it is failed. **Note:** The
+ monitor action should test the state of the resource on the local machine
+ only.
+ * - .. _meta_data_action:
+
+ .. index::
+ single: OCF resource agent; meta-data
+ single: meta-data action
+
+ meta-data
+ - Describe the resource
+ - Provide information about this resource in the XML format defined by the
+ OCF standard. Return :ref:`OCF_SUCCESS <OCF_SUCCESS>`. **Note:** This is
+ *not* required to be performed as root.
OCF resource agents may optionally implement additional actions. Some are used
only with advanced resource types such as clones.
-.. table:: **Optional Actions for OCF Resource Agents**
-
- +--------------+-------------+------------------------------------------------+
- | Action | Description | Instructions |
- +==============+=============+================================================+
- | validate-all | This should | .. index:: |
- | | validate | single: OCF resource agent; validate-all |
- | | the | single: validate-all action |
- | | instance | |
- | | parameters | Return 0 if parameters are valid, 2 if |
- | | provided. | not valid, and 6 if resource is not |
- | | | configured. |
- +--------------+-------------+------------------------------------------------+
- | promote | Bring the | .. index:: |
- | | local | single: OCF resource agent; promote |
- | | instance of | single: promote action |
- | | a promotable| |
- | | clone | Return 0 on success |
- | | resource to | |
- | | the promoted| |
- | | role. | |
- +--------------+-------------+------------------------------------------------+
- | demote | Bring the | .. index:: |
- | | local | single: OCF resource agent; demote |
- | | instance of | single: demote action |
- | | a promotable| |
- | | clone | Return 0 on success |
- | | resource to | |
- | | the | |
- | | unpromoted | |
- | | role. | |
- +--------------+-------------+------------------------------------------------+
- | notify | Used by the | .. index:: |
- | | cluster to | single: OCF resource agent; notify |
- | | send | single: notify action |
- | | the agent | |
- | | pre- and | Must not fail. Must exit with 0 |
- | | post- | |
- | | notification| |
- | | events | |
- | | telling the | |
- | | resource | |
- | | what has | |
- | | happened and| |
- | | will happen.| |
- +--------------+-------------+------------------------------------------------+
- | reload | Reload the | .. index:: |
- | | service's | single: OCF resource agent; reload |
- | | own | single: reload action |
- | | config. | |
- | | | Not used by Pacemaker |
- +--------------+-------------+------------------------------------------------+
- | reload-agent | Make | .. index:: |
- | | effective | single: OCF resource agent; reload-agent |
- | | any changes | single: reload-agent action |
- | | in instance | |
- | | parameters | This is used when the agent can handle a |
- | | marked as | change in some of its parameters more |
- | | reloadable | efficiently than stopping and starting the |
- | | in the | resource. |
- | | agent's | |
- | | meta-data. | |
- +--------------+-------------+------------------------------------------------+
- | recover | Restart the | .. index:: |
- | | service. | single: OCF resource agent; recover |
- | | | single: recover action |
- | | | |
- | | | Not used by Pacemaker |
- +--------------+-------------+------------------------------------------------+
+.. list-table:: **Optional Actions for OCF Resource Agents**
+ :class: longtable:
+ :widths: 1 4 3
+ :header-rows: 1
+
+ * - Action
+ - Description
+ - Instructions
+ * - .. _validate_all_action:
+
+ .. index::
+ single: OCF resource agent; validate-all
+ single: validate-all action
+
+ validate-all
+ - Validate the instance parameters provided.
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` if parameters are valid,
+ :ref:`OCF_ERR_ARGS <OCF_ERR_ARGS>` if not valid, and
+ :ref:`OCF_ERR_CONFIGURED <OCF_ERR_CONFIGURED>` if resource is not
+ configured.
+ * - .. _promote_action:
+
+ .. index::
+ single: OCF resource agent; promote
+ single: promote action
+
+ promote
+ - Bring the local instance of a promotable clone resource to the promoted
+ role.
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` on success.
+ * - .. _demote_action:
+
+ .. index::
+ single: OCF resource agent; demote
+ single: demote action
+
+ demote
+ - Bring the local instance of a promotable clone resource to the unpromoted
+ role.
+ - Return :ref:`OCF_SUCCESS <OCF_SUCCESS>` on success.
+ * - .. _notify_action:
+
+ .. index::
+ single: OCF resource agent; notify
+ single: notify action
+
+ notify
+ - Used by the cluster to send the agent pre- and post-notification events
+ telling the resource what has happened and what will happen.
+ - Must not fail. Must return :ref:`OCF_SUCCESS <OCF_SUCCESS>`.
+ * - .. _reload_action:
+
+ .. index::
+ single: OCF resource agent; reload
+ single: reload action
+
+ reload
+ - Reload the service's own configuration.
+ - Not used by Pacemaker.
+ * - .. _reload_agent_action:
+
+ .. index::
+ single: OCF resource agent; reload-agent
+ single: reload-agent action
+
+ reload-agent
+ - Make effective any changes in instance parameters marked as reloadable in
+ the agent's meta-data.
+ - This is used when the agent can handle a change in some of its parameters
+ more efficiently than stopping and starting the resource.
+ * - .. _recover_action:
+
+ .. index::
+ single: OCF resource agent; recover
+ single: recover action
+
+ recover
+ - Restart the service.
+ - Not used by Pacemaker.
.. important::
@@ -180,159 +200,214 @@ only with advanced resource types such as clones.
.. index::
single: OCF resource agent; return code
-How are OCF Return Codes Interpreted?
+How Are OCF Return Codes Interpreted?
_____________________________________
-The first thing the cluster does is to check the return code against
-the expected result. If the result does not match the expected value,
-then the operation is considered to have failed, and recovery action is
-initiated.
+The first thing the cluster does is to check the return code against the
+expected result. If the result does not match the expected value, then the
+operation is considered to have failed, and recovery action is initiated.
There are three types of failure recovery:
-.. table:: **Types of recovery performed by the cluster**
-
- +-------+--------------------------------------------+--------------------------------------+
- | Type | Description | Action Taken by the Cluster |
- +=======+============================================+======================================+
- | soft | .. index:: | Restart the resource or move it to a |
- | | single: OCF resource agent; soft error | new location |
- | | | |
- | | A transient error occurred | |
- +-------+--------------------------------------------+--------------------------------------+
- | hard | .. index:: | Move the resource elsewhere and |
- | | single: OCF resource agent; hard error | prevent it from being retried on the |
- | | | current node |
- | | A non-transient error that | |
- | | may be specific to the | |
- | | current node | |
- +-------+--------------------------------------------+--------------------------------------+
- | fatal | .. index:: | Stop the resource and prevent it |
- | | single: OCF resource agent; fatal error | from being started on any cluster |
- | | | node |
- | | A non-transient error that | |
- | | will be common to all | |
- | | cluster nodes (e.g. a bad | |
- | | configuration was specified) | |
- +-------+--------------------------------------------+--------------------------------------+
+.. list-table:: **Types of Recovery Performed by the Cluster**
+ :class: longtable
+ :widths: 1 5 5
+ :header-rows: 1
+
+ * - Type
+ - Description
+ - Action Taken by the Cluster
+ * - .. _soft_error:
+
+ .. index::
+ single: OCF resource agent; soft error
+
+ soft
+ - A transient error
+ - Restart the resource or move it to a new location
+ * - .. _hard_error:
+
+ .. index::
+ single: OCF resource agent; hard error
+
+ hard
+ - A non-transient error that may be specific to the current node
+ - Move the resource elsewhere and prevent it from being retried on the
+ current node
+ * - .. _fatal_error:
+
+ .. index::
+ single: OCF resource agent; fatal error
+
+ fatal
+ - A non-transient error that will be common to all cluster nodes (for
+ example, a bad configuration was specified)
+ - Stop the resource and prevent it from being started on any cluster node
.. _ocf_return_codes:
OCF Return Codes
________________
-The following table outlines the different OCF return codes and the type of
+The following table outlines the various OCF return codes and the type of
recovery the cluster will initiate when a failure code is received. Although
-counterintuitive, even actions that return 0 (aka. ``OCF_SUCCESS``) can be
-considered to have failed, if 0 was not the expected return value.
-
-.. table:: **OCF Exit Codes and their Recovery Types**
-
- +-------+-----------------------+---------------------------------------------------+----------+
- | Exit | OCF Alias | Description | Recovery |
- | Code | | | |
- +=======+=======================+===================================================+==========+
- | 0 | OCF_SUCCESS | .. index:: | soft |
- | | | single: OCF_SUCCESS | |
- | | | single: OCF return code; OCF_SUCCESS | |
- | | | pair: OCF return code; 0 | |
- | | | | |
- | | | Success. The command completed successfully. | |
- | | | This is the expected result for all start, | |
- | | | stop, promote and demote commands. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 1 | OCF_ERR_GENERIC | .. index:: | soft |
- | | | single: OCF_ERR_GENERIC | |
- | | | single: OCF return code; OCF_ERR_GENERIC | |
- | | | pair: OCF return code; 1 | |
- | | | | |
- | | | Generic "there was a problem" error code. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 2 | OCF_ERR_ARGS | .. index:: | hard |
- | | | single: OCF_ERR_ARGS | |
- | | | single: OCF return code; OCF_ERR_ARGS | |
- | | | pair: OCF return code; 2 | |
- | | | | |
- | | | The resource's parameter values are not valid on | |
- | | | this machine (for example, a value refers to a | |
- | | | file not found on the local host). | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 3 | OCF_ERR_UNIMPLEMENTED | .. index:: | hard |
- | | | single: OCF_ERR_UNIMPLEMENTED | |
- | | | single: OCF return code; OCF_ERR_UNIMPLEMENTED | |
- | | | pair: OCF return code; 3 | |
- | | | | |
- | | | The requested action is not implemented. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 4 | OCF_ERR_PERM | .. index:: | hard |
- | | | single: OCF_ERR_PERM | |
- | | | single: OCF return code; OCF_ERR_PERM | |
- | | | pair: OCF return code; 4 | |
- | | | | |
- | | | The resource agent does not have | |
- | | | sufficient privileges to complete the task. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 5 | OCF_ERR_INSTALLED | .. index:: | hard |
- | | | single: OCF_ERR_INSTALLED | |
- | | | single: OCF return code; OCF_ERR_INSTALLED | |
- | | | pair: OCF return code; 5 | |
- | | | | |
- | | | The tools required by the resource are | |
- | | | not installed on this machine. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 6 | OCF_ERR_CONFIGURED | .. index:: | fatal |
- | | | single: OCF_ERR_CONFIGURED | |
- | | | single: OCF return code; OCF_ERR_CONFIGURED | |
- | | | pair: OCF return code; 6 | |
- | | | | |
- | | | The resource's parameter values are inherently | |
- | | | invalid (for example, a required parameter was | |
- | | | not given). | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 7 | OCF_NOT_RUNNING | .. index:: | N/A |
- | | | single: OCF_NOT_RUNNING | |
- | | | single: OCF return code; OCF_NOT_RUNNING | |
- | | | pair: OCF return code; 7 | |
- | | | | |
- | | | The resource is safely stopped. This should only | |
- | | | be returned by monitor actions, not stop actions. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 8 | OCF_RUNNING_PROMOTED | .. index:: | soft |
- | | | single: OCF_RUNNING_PROMOTED | |
- | | | single: OCF return code; OCF_RUNNING_PROMOTED | |
- | | | pair: OCF return code; 8 | |
- | | | | |
- | | | The resource is running in the promoted role. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 9 | OCF_FAILED_PROMOTED | .. index:: | soft |
- | | | single: OCF_FAILED_PROMOTED | |
- | | | single: OCF return code; OCF_FAILED_PROMOTED | |
- | | | pair: OCF return code; 9 | |
- | | | | |
- | | | The resource is (or might be) in the promoted | |
- | | | role but has failed. The resource will be | |
- | | | demoted, stopped and then started (and possibly | |
- | | | promoted) again. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 190 | OCF_DEGRADED | .. index:: | none |
- | | | single: OCF_DEGRADED | |
- | | | single: OCF return code; OCF_DEGRADED | |
- | | | pair: OCF return code; 190 | |
- | | | | |
- | | | The resource is properly active, but in such a | |
- | | | condition that future failures are more likely. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | 191 | OCF_DEGRADED_PROMOTED | .. index:: | none |
- | | | single: OCF_DEGRADED_PROMOTED | |
- | | | single: OCF return code; OCF_DEGRADED_PROMOTED | |
- | | | pair: OCF return code; 191 | |
- | | | | |
- | | | The resource is properly active in the promoted | |
- | | | role, but in such a condition that future | |
- | | | failures are more likely. | |
- +-------+-----------------------+---------------------------------------------------+----------+
- | other | *none* | Custom error code. | soft |
- +-------+-----------------------+---------------------------------------------------+----------+
+counterintuitive, even actions that return ``OCF_SUCCESS`` can be considered to
+have failed, if ``OCF_SUCCESS`` was not the expected return value.
+
+.. list-table:: **OCF Exit Codes and Their Recovery Types**
+ :class: longtable
+ :widths: 1 3 6 2
+ :header-rows: 1
+
+ * - Exit Code
+ - OCF Alias
+ - Description
+ - Recovery
+ * - .. _OCF_SUCCESS:
+
+ .. index::
+ single: OCF_SUCCESS
+ single: OCF return code; OCF_SUCCESS
+ pair: OCF return code; 0
+
+ 0
+ - OCF_SUCCESS
+ - Success. The command completed successfully. This is the expected result
+ for all start, stop, promote, and demote actions.
+ - :ref:`soft <soft_error>`
+ * - .. _OCF_ERR_GENERIC:
+
+ .. index::
+ single: OCF_ERR_GENERIC
+ single: OCF return code; OCF_ERR_GENERIC
+ pair: OCF return code; 1
+
+ 1
+ - OCF_ERR_GENERIC
+ - Generic "there was a problem" error code.
+ - :ref:`hard <hard_error>`
+ * - .. _OCF_ERR_ARGS:
+
+ .. index::
+ single: OCF_ERR_ARGS
+ single: OCF return code; OCF_ERR_ARGS
+ pair: OCF return code; 2
+
+ 2
+ - OCF_ERR_ARGS
+ - The resource's parameter values are not valid on this machine (for
+ example, a value refers to a file not found on the local host).
+ - :ref:`hard <hard_error>`
+ * - .. _OCF_ERR_UNIMPLEMENTED:
+
+ .. index::
+ single: OCF_ERR_UNIMPLEMENTED
+ single: OCF return code; OCF_ERR_UNIMPLEMENTED
+ pair: OCF return code; 3
+
+ 3
+ - OCF_ERR_UNIMPLEMENTED
+ - The requested action is not implemented.
+ - :ref:`hard <hard_error>`
+ * - .. _OCF_ERR_PERM:
+
+ .. index::
+ single: OCF_ERR_PERM
+ single: OCF return code; OCF_ERR_PERM
+ pair: OCF return code; 4
+
+ 4
+ - OCF_ERR_PERM
+ - The resource agent does not have sufficient privileges to complete the
+ task.
+ - :ref:`hard <hard_error>`
+ * - .. _OCF_ERR_INSTALLED:
+
+ .. index::
+ single: OCF_ERR_INSTALLED
+ single: OCF return code; OCF_ERR_INSTALLED
+ pair: OCF return code; 5
+
+ 5
+ - OCF_ERR_INSTALLED
+ - The tools required by the resource are not installed on this machine.
+ - :ref:`hard <hard_error>`
+ * - .. _OCF_ERR_CONFIGURED:
+
+ .. index::
+ single: OCF_ERR_CONFIGURED
+ single: OCF return code; OCF_ERR_CONFIGURED
+ pair: OCF return code; 6
+
+ 6
+ - OCF_ERR_CONFIGURED
+ - The resource's parameter values are inherently invalid (for example, a
+ required parameter was not given).
+ - :ref:`fatal <fatal_error>`
+ * - .. _OCF_NOT_RUNNING:
+
+ .. index::
+ single: OCF_NOT_RUNNING
+ single: OCF return code; OCF_NOT_RUNNING
+ pair: OCF return code; 7
+
+ 7
+ - OCF_NOT_RUNNING
+ - The resource is safely stopped. This should only be returned by monitor
+ actions, not stop actions.
+ - N/A
+ * - .. _OCF_RUNNING_PROMOTED:
+
+ .. index::
+ single: OCF_RUNNING_PROMOTED
+ single: OCF return code; OCF_RUNNING_PROMOTED
+ pair: OCF return code; 8
+
+ 8
+ - OCF_RUNNING_PROMOTED
+ - The resource is running in the promoted role.
+ - :ref:`soft <soft_error>`
+ * - .. _OCF_FAILED_PROMOTED:
+
+ .. index::
+ single: OCF_FAILED_PROMOTED
+ single: OCF return code; OCF_FAILED_PROMOTED
+ pair: OCF return code; 9
+
+ 9
+ - OCF_FAILED_PROMOTED
+ - The resource is (or might be) in the promoted role but has failed. The
+ resource will be demoted, stopped, and then started (and possibly
+ promoted) again.
+ - :ref:`soft <soft_error>`
+ * - .. _OCF_DEGRADED:
+
+ .. index::
+ single: OCF_DEGRADED
+ single: OCF return code; OCF_DEGRADED
+ pair: OCF return code; 190
+
+ 190
+ - OCF_DEGRADED
+ - The resource is properly active, but in such a condition that future
+ failures are more likely.
+ - none
+ * - .. _OCF_DEGRADED_PROMOTED:
+
+ .. index::
+ single: OCF_DEGRADED_PROMOTED
+ single: OCF return code; OCF_DEGRADED_PROMOTED
+ pair: OCF return code; 191
+
+ 191
+ - OCF_DEGRADED_PROMOTED
+ - The resource is properly active in the promoted role, but in such a
+ condition that future failures are more likely.
+ - none
+ * - other
+ - *none*
+ - Custom error code.
+ - soft
Exceptions to the recovery handling described above:
@@ -347,6 +422,670 @@ Exceptions to the recovery handling described above:
if they had returned success, but status output will indicate that the
resource is degraded.
+.. _ocf_env_vars:
+
+Environment Variables
+_____________________
+
+Pacemaker sets certain environment variables when it executes an OCF resource
+agent. Agents can check these variables to get information about resource
+parameters or the execution environment.
+
+**Note:** Pacemaker may set other environment variables for its own purposes.
+They may be present in the agent's environment, but Pacemaker is not providing
+them for the agent's use, and so the agent should not rely on any variables not
+listed in the table below.
+
+.. list-table:: **OCF Environment Variables**
+ :class: longtable
+ :widths: 1 6
+ :header-rows: 1
+
+ * - Environment Variable
+ - Description
+ * - .. _OCF_CHECK_LEVEL:
+
+ .. index::
+ single: OCF_CHECK_LEVEL
+ single: environment variable; OCF_CHECK_LEVEL
+
+ OCF_CHECK_LEVEL
+ - Requested intensity level of checks in ``monitor`` and ``validate-all``
+ actions. Usually set as an operation attribute; see Pacemaker Explained
+ for an example.
+ * - .. _OCF_EXIT_REASON_PREFIX:
+
+ .. index::
+ single: OCF_EXIT_REASON_PREFIX
+ single: environment variable; OCF_EXIT_REASON_PREFIX
+
+ OCF_EXIT_REASON_PREFIX
+ - Prefix for printing fatal error messages from the resource agent.
+ * - .. _OCF_RA_VERSION_MAJOR:
+
+ .. index::
+ single: OCF_RA_VERSION_MAJOR
+ single: environment variable; OCF_RA_VERSION_MAJOR
+
+ OCF_RA_VERSION_MAJOR
+ - Major version number of the OCF Resource Agent API. If the script does
+ not support this revision, it should report an error.
+ See the `OCF specification <http://standards.clusterlabs.org>`_ for an
+ explanation of the versioning scheme used. The version number is split
+ into two numbers for ease of use in shell scripts. These two may be used
+ by the agent to determine whether it is run under an OCF-compliant
+ resource manager.
+ * - .. _OCF_RA_VERSION_MINOR:
+
+ .. index::
+ single: OCF_RA_VERSION_MINOR
+ single: environment variable; OCF_RA_VERSION_MINOR
+
+ OCF_RA_VERSION_MINOR
+ - Minor version number of the OCF Resource Agent API. See
+ :ref:`OCF_RA_VERSION_MAJOR <OCF_RA_VERSION_MAJOR>` for more details.
+ * - .. _OCF_RESKEY_crm_feature_set:
+
+ .. index::
+ single: OCF_RESKEY_crm_feature_set
+ single: environment variable; OCF_RESKEY_crm_feature_set
+
+ OCF_RESKEY_crm_feature_set
+ - ``crm_feature_set`` on the DC (or on the local node, if the agent is run
+ by ``crm_resource``).
+ * - .. _OCF_RESKEY_CRM_meta_interval:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_interval
+ single: environment variable; OCF_RESKEY_CRM_meta_interval
+
+ OCF_RESKEY_CRM_meta_interval
+ - Interval (in milliseconds) of the current operation.
+ * - .. _OCF_RESKEY_CRM_meta_name:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_name
+ single: environment variable; OCF_RESKEY_CRM_meta_name
+
+ OCF_RESKEY_CRM_meta_name
+ - Name of the current operation.
+ * - .. _OCF_RESKEY_CRM_meta_notify:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_notify_*
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_*
+
+ OCF_RESKEY_CRM_meta_notify_*
+ - See :ref:`Clone Notifications <clone_notifications>`.
+ * - .. _OCF_RESKEY_CRM_meta_on_node:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_on_node
+ single: environment variable; OCF_RESKEY_CRM_meta_on_node
+
+ OCF_RESKEY_CRM_meta_on_node
+ - Name of the node where the current operation is running.
+ * - .. _OCF_RESKEY_CRM_meta_on_node_uuid:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_on_node_uuid
+ single: environment variable; OCF_RESKEY_CRM_meta_on_node_uuid
+
+ OCF_RESKEY_CRM_meta_on_node_uuid
+ - Cluster-layer ID of the node where the current operation is running (or
+ node name for Pacemaker Remote nodes).
+ * - .. _OCF_RESKEY_CRM_meta_physical_host:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_physical_host
+ single: environment variable; OCF_RESKEY_CRM_meta_physical_host
+
+ OCF_RESKEY_CRM_meta_physical_host
+ - If the node where the current operation is running is a guest node, the
+ host on which the container is running.
+ * - .. _OCF_RESKEY_CRM_meta_timeout:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_timeout
+ single: environment variable; OCF_RESKEY_CRM_meta_timeout
+
+ OCF_RESKEY_CRM_meta_timeout
+ - Timeout (in milliseconds) of the current operation.
+ * - .. _OCF_RESKEY_CRM_meta:
+
+ .. index::
+ single: OCF_RESKEY_CRM_meta_*
+ single: environment variable; OCF_RESKEY_CRM_meta_*
+
+ OCF_RESKEY_CRM_meta_*
+ - Each of a resource's meta-attributes is converted to an environment
+ variable prefixed with "OCF_RESKEY_CRM_meta\_". See Pacemaker Explained
+ for some meta-attributes that have special meaning to Pacemaker.
+ * - .. _OCF_RESKEY:
+
+ .. index::
+ single: OCF_RESKEY_*
+ single: environment variable; OCF_RESKEY_*
+
+ OCF_RESKEY_*
+ - Each of a resource's instance parameters is converted to an environment
+ variable prefixed with "OCF_RESKEY\_".
+ * - .. _OCF_RESOURCE_INSTANCE:
+
+ .. index::
+ single: OCF_RESOURCE_INSTANCE
+ single: environment variable; OCF_RESOURCE_INSTANCE
+
+ OCF_RESOURCE_INSTANCE
+ - The name of the resource instance.
+ * - .. _OCF_RESOURCE_PROVIDER:
+
+ .. index::
+ single: OCF_RESOURCE_PROVIDER
+ single: environment variable; OCF_RESOURCE_PROVIDER
+
+ OCF_RESOURCE_PROVIDER
+ - The name of the resource agent provider.
+ * - .. _OCF_RESOURCE_TYPE:
+
+ .. index::
+ single: OCF_RESOURCE_TYPE
+ single: environment variable; OCF_RESOURCE_TYPE
+
+ OCF_RESOURCE_TYPE
+ - The name of the resource type.
+ * - .. _OCF_ROOT:
+
+ .. index::
+ single: OCF_ROOT
+ single: environment variable; OCF_ROOT
+
+ OCF_ROOT
+ - The root of the OCF directory hierarchy.
+ * - .. _OCF_TRACE_FILE:
+
+ .. index::
+ single: OCF_TRACE_FILE
+ single: environment variable; OCF_TRACE_FILE
+
+ OCF_TRACE_FILE
+ - The absolute path or file descriptor to write trace output to, if
+ ``OCF_TRACE_RA`` is set to true. Pacemaker sets this only to
+ ``/dev/stderr`` and only when running a resource agent via
+ ``crm_resource``.
+ * - .. _OCF_TRACE_RA:
+
+ .. index::
+ single: OCF_TRACE_RA
+ single: environment variable; OCF_TRACE_RA
+
+ OCF_TRACE_RA
+ - If set to true, enable tracing of the resource agent. Trace output is
+ written to ``OCF_TRACE_FILE`` if set; otherwise, it's written to a file
+ in ``OCF_RESKEY_trace_dir`` if set or in a default directory if not.
+ Pacemaker sets this to true only when running a resource agent via
+ ``crm_resource`` with one or more ``-V`` flags.
+ * - .. _PCMK_DEBUGLOG:
+ .. _HA_DEBUGLOG:
+
+ .. index::
+ single: PCMK_DEBUGLOG
+ single: environment variable; PCMK_DEBUGLOG
+ single: HA_DEBUGLOG
+ single: environment variable; HA_DEBUGLOG
+
+ PCMK_DEBUGLOG (and HA_DEBUGLOG)
+ - Where to write resource agent debug logs. Pacemaker sets this to
+ ``PCMK_logfile`` if set to a value other than ``none`` and if debugging
+ is enabled for the executor.
+ * - .. _PCMK_LOGFACILITY:
+ .. _HA_LOGFACILITY:
+
+ .. index::
+ single: PCMK_LOGFACILITY
+ single: environment variable; PCMK_LOGFACILITY
+ single: HA_LOGFACILITY
+ single: environment variable; HA_LOGFACILITY
+
+ PCMK_LOGFACILITY (and HA_LOGFACILITY)
+ - Syslog facility for resource agent logs. Pacemaker sets this to
+ ``PCMK_logfacility`` if set to a value other than ``none`` or
+ ``/dev/null``.
+ * - .. _PCMK_LOGFILE:
+ .. _HA_LOGFILE::
+
+ .. index::
+ single: PCMK_LOGFILE:
+ single: environment variable; PCMK_LOGFILE:
+ single: HA_LOGFILE:
+ single: environment variable; HA_LOGFILE:
+
+ PCMK_LOGFILE (and HA_LOGFILE)
+ - Where to write resource agent logs. Pacemaker sets this to
+ ``PCMK_logfile`` if set to a value other than ``none``.
+ * - .. _PCMK_service:
+
+ .. index::
+ single: PCMK_service
+ single: environment variable; PCMK_service
+
+ PCMK_service
+ - The name of the Pacemaker subsystem or command-line tool that's executing
+ the resource agent. Specific values are subject to change; useful mainly
+ for logging.
+
+Clone Resource Agent Requirements
+_________________________________
+
+Any resource can be used as an anonymous clone, as it requires no additional
+support from the resource agent. Whether it makes sense to do so depends on your
+resource and its resource agent.
+
+Resource Agent Requirements for Globally Unique Clones
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Globally unique clones require additional support in the resource agent. In
+particular, it must respond with ``OCF_SUCCESS`` only if the node has that exact
+instance active. All other probes for instances of the clone should result in
+``OCF_NOT_RUNNING`` (or one of the other OCF error codes if they are failed).
+
+Individual instances of a clone are identified by appending a colon and a
+numerical offset (for example, ``apache:2``).
+
+A resource agent can find out how many copies there are by examining the
+``OCF_RESKEY_CRM_meta_clone_max`` environment variable and which instance it is
+by examining ``OCF_RESKEY_CRM_meta_clone``.
+
+The resource agent must not make any assumptions (based on
+``OCF_RESKEY_CRM_meta_clone``) about which numerical instances are active. In
+particular, the list of active copies is not always an unbroken sequence, nor
+does it always start at 0.
+
+Resource Agent Requirements for Promotable Clones
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Promotable clone resources require two extra actions, ``demote`` and ``promote``,
+which are responsible for changing the state of the resource. Like ``start`` and
+``stop``, they should return ``OCF_SUCCESS`` if they completed successfully or a
+relevant error code if they did not.
+
+The states can mean whatever you wish, but when the resource is started, it must
+begin in the unpromoted role. From there, the cluster will decide which
+instances to promote.
+
+In addition to the clone requirements for monitor actions, agents must also
+*accurately* report which state they are in. The cluster relies on the agent to
+report its status (including role) accurately and does not indicate to the agent
+what role it currently believes it to be in.
+
+.. list-table:: **Role Implications of OCF Return Codes**
+ :class: longtable
+ :widths: 1 3
+ :header-rows: 1
+
+ * - Monitor Return Code
+ - Description
+ * - :ref:`OCF_NOT_RUNNING <OCF_NOT_RUNNING>`
+ - .. index::
+ single: OCF_NOT_RUNNING
+ single: OCF return code; OCF_NOT_RUNNING
+
+ Stopped
+ * - :ref:`OCF_SUCCESS <OCF_SUCCESS>`
+ - .. index::
+ single: OCF_SUCCESS
+ single: OCF return code; OCF_SUCCESS
+
+ Running (Unpromoted)
+ * - :ref:`OCF_RUNNING_PROMOTED <OCF_RUNNING_PROMOTED>`
+ - .. index::
+ single: OCF_RUNNING_PROMOTED
+ single: OCF return code; OCF_RUNNING_PROMOTED
+
+ Running (Promoted)
+ * - :ref:`OCF_FAILED_PROMOTED <OCF_FAILED_PROMOTED>`
+ - .. index::
+ single: OCF_FAILED_PROMOTED
+ single: OCF return code; OCF_FAILED_PROMOTED
+
+ Failed (Promoted)
+ * - Other
+ - Failed (Unpromoted)
+
+.. _clone_notifications:
+
+Clone Notifications
+~~~~~~~~~~~~~~~~~~~
+
+If the clone has the ``notify`` meta-attribute set to ``true`` and the resource
+agent supports the ``notify`` action, Pacemaker will call the action when
+appropriate, passing a number of extra variables. These variables, when combined
+with additional context, can be used to calculate the current state of the
+cluster and what is about to happen to it.
+
+.. index::
+ single: clone; environment variables
+ single: notify; environment variables
+
+.. list-table:: **Environment Variables Supplied with Clone Notify Actions**
+ :class: longtable
+ :widths: 1 1
+ :header-rows: 1
+
+ * - Variable
+ - Description
+ * - .. _OCF_RESKEY_CRM_meta_notify_type:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_type
+ single: OCF_RESKEY_CRM_meta_notify_type
+
+ OCF_RESKEY_CRM_meta_notify_type
+ - Allowed values: ``pre``, ``post``
+ * - .. _OCF_RESKEY_CRM_meta_notify_operation:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_operation
+ single: OCF_RESKEY_CRM_meta_notify_operation
+
+ OCF_RESKEY_CRM_meta_notify_operation
+ - Allowed values: ``start``, ``stop``
+ * - .. _OCF_RESKEY_CRM_meta_notify_start_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_start_resource
+ single: OCF_RESKEY_CRM_meta_notify_start_resource
+
+ OCF_RESKEY_CRM_meta_notify_start_resource
+ - Resources to be started
+ * - .. _OCF_RESKEY_CRM_meta_notify_stop_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_stop_resource
+ single: OCF_RESKEY_CRM_meta_notify_stop_resource
+
+ OCF_RESKEY_CRM_meta_notify_stop_resource
+ - Resources to be stopped
+ * - .. _OCF_RESKEY_CRM_meta_notify_active_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_active_resource
+ single: OCF_RESKEY_CRM_meta_notify_active_resource
+
+ OCF_RESKEY_CRM_meta_notify_active_resource
+ - Resources that are running
+ * - .. _OCF_RESKEY_CRM_meta_notify_inactive_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_inactive_resource
+ single: OCF_RESKEY_CRM_meta_notify_inactive_resource
+
+ OCF_RESKEY_CRM_meta_notify_inactive_resource
+ - Resources that are not running
+ * - .. _OCF_RESKEY_CRM_meta_notify_start_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_start_uname
+ single: OCF_RESKEY_CRM_meta_notify_start_uname
+
+ OCF_RESKEY_CRM_meta_notify_start_uname
+ - Nodes on which resources will be started
+ * - .. _OCF_RESKEY_CRM_meta_notify_stop_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_stop_uname
+ single: OCF_RESKEY_CRM_meta_notify_stop_uname
+
+ OCF_RESKEY_CRM_meta_notify_stop_uname
+ - Nodes on which resources will be stopped
+ * - .. _OCF_RESKEY_CRM_meta_notify_active_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_active_uname
+ single: OCF_RESKEY_CRM_meta_notify_active_uname
+
+ OCF_RESKEY_CRM_meta_notify_active_uname
+ - Nodes on which resources are running
+
+The variables come in pairs, such as
+``OCF_RESKEY_CRM_meta_notify_start_resource`` and
+``OCF_RESKEY_CRM_meta_notify_start_uname``, and should be treated as an array of
+whitespace-separated elements.
+
+``OCF_RESKEY_CRM_meta_notify_inactive_resource`` is an exception, as the
+matching ``uname`` variable does not exist since inactive resources are not
+running on any node.
+
+Thus, in order to indicate that ``clone:0`` will be started on ``sles-1``,
+``clone:2`` will be started on ``sles-3``, and ``clone:3`` will be started
+on ``sles-2``, the cluster would set:
+
+.. topic:: Notification Variables
+
+ .. code-block:: none
+
+ OCF_RESKEY_CRM_meta_notify_start_resource="clone:0 clone:2 clone:3"
+ OCF_RESKEY_CRM_meta_notify_start_uname="sles-1 sles-3 sles-2"
+
+.. note::
+
+ Pacemaker will log but otherwise ignore failures of notify actions.
+
+Interpretation of Notification Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**Pre-notification (stop):**
+
+* Active resources: ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+* Inactive resources: ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+**Post-notification (stop) / Pre-notification (start):**
+
+* Active resources
+ * ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Inactive resources
+ * ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources that were started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources that were stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+**Post-notification (start):**
+
+* Active resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Inactive resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources that were started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources that were stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+Extra Notifications for Promotable Clones
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. index::
+ single: clone; environment variables
+ single: promotable; environment variables
+
+.. list-table:: **Extra Environment Variables Supplied for Promotable Clones**
+ :class: longtable
+ :widths: 1 1
+ :header-rows: 1
+
+ * - Variable
+ - Description
+ * - .. _OCF_RESKEY_CRM_meta_notify_promoted_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_promoted_resource
+ single: OCF_RESKEY_CRM_meta_notify_promoted_resource
+
+ OCF_RESKEY_CRM_meta_notify_promoted_resource
+ - Resources that are running in the promoted role
+ * - .. _OCF_RESKEY_CRM_meta_notify_unpromoted_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_unpromoted_resource
+ single: OCF_RESKEY_CRM_meta_notify_unpromoted_resource
+
+ OCF_RESKEY_CRM_meta_notify_unpromoted_resource
+ - Resources that are running in the unpromoted role
+ * - .. _OCF_RESKEY_CRM_meta_notify_promote_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_promote_resource
+ single: OCF_RESKEY_CRM_meta_notify_promote_resource
+
+ OCF_RESKEY_CRM_meta_notify_promote_resource
+ - Resources to be promoted
+ * - .. _OCF_RESKEY_CRM_meta_notify_demote_resource:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_demote_resource
+ single: OCF_RESKEY_CRM_meta_notify_demote_resource
+
+ OCF_RESKEY_CRM_meta_notify_demote_resource
+ - Resources to be demoted
+ * - .. _OCF_RESKEY_CRM_meta_notify_promote_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_promote_uname
+ single: OCF_RESKEY_CRM_meta_notify_promote_uname
+
+ OCF_RESKEY_CRM_meta_notify_promote_uname
+ - Nodes on which resources will be promoted
+ * - .. _OCF_RESKEY_CRM_meta_notify_demote_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_demote_uname
+ single: OCF_RESKEY_CRM_meta_notify_demote_uname
+
+ OCF_RESKEY_CRM_meta_notify_demote_uname
+ - Nodes on which resources will be demoted
+ * - .. _OCF_RESKEY_CRM_meta_notify_promoted_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_promoted_uname
+ single: OCF_RESKEY_CRM_meta_notify_promoted_uname
+
+ OCF_RESKEY_CRM_meta_notify_promoted_uname
+ - Nodes on which resources are running in the promoted role
+ * - .. _OCF_RESKEY_CRM_meta_notify_unpromoted_uname:
+
+ .. index::
+ single: environment variable; OCF_RESKEY_CRM_meta_notify_unpromoted_uname
+ single: OCF_RESKEY_CRM_meta_notify_unpromoted_uname
+
+ OCF_RESKEY_CRM_meta_notify_unpromoted_uname
+ - Nodes on which resources are running in the unpromoted role
+
+Interpretation of Promotable Notification Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**Pre-notification (demote):**
+
+* Active resources: ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+* Promoted resources: ``$OCF_RESKEY_CRM_meta_notify_promoted_resource``
+* Unpromoted resources: ``$OCF_RESKEY_CRM_meta_notify_unpromoted_resource``
+* Inactive resources: ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources to be demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+**Post-notification (demote) / Pre-notification (stop):**
+
+* Active resources: ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+* Promoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_promoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Unpromoted resources: ``$OCF_RESKEY_CRM_meta_notify_unpromoted_resource``
+* Inactive resources: ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources to be demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources that were demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+
+**Post-notification (stop) / Pre-notification (start)**
+
+* Active resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Promoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_promoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Unpromoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_unpromoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Inactive resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources to be demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources that were demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources that were stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+**Post-notification (start) / Pre-notification (promote)**
+
+* Active resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Promoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_promoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Unpromoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_unpromoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Inactive resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources to be demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources that were started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources that were demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources that were stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
+**Post-notification (promote)**
+
+* Active resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_active_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Promoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_promoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Unpromoted resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_unpromoted_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Inactive resources:
+ * ``$OCF_RESKEY_CRM_meta_notify_inactive_resource``
+ * plus ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+ * minus ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources to be promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources to be demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources to be stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+* Resources that were started: ``$OCF_RESKEY_CRM_meta_notify_start_resource``
+* Resources that were promoted: ``$OCF_RESKEY_CRM_meta_notify_promote_resource``
+* Resources that were demoted: ``$OCF_RESKEY_CRM_meta_notify_demote_resource``
+* Resources that were stopped: ``$OCF_RESKEY_CRM_meta_notify_stop_resource``
+
.. index::
single: resource agent; LSB