diff options
Diffstat (limited to '')
| -rw-r--r-- | doc/website-v1/scripts.adoc | 660 |
1 files changed, 660 insertions, 0 deletions
diff --git a/doc/website-v1/scripts.adoc b/doc/website-v1/scripts.adoc new file mode 100644 index 0000000..7742729 --- /dev/null +++ b/doc/website-v1/scripts.adoc @@ -0,0 +1,660 @@ += Cluster Scripts = +:source-highlighter: pygments + +.Version information +NOTE: This section applies to `crmsh 2.2+` only. + +== Introduction == + +A big part of the configuration and management of a cluster is +collecting information about all cluster nodes and deploying changes +to those nodes. Often, just performing the same procedure on all nodes +will encounter problems, due to subtle differences in the +configuration. + +For example, when configuring a cluster for the first time, the +software needs to be installed and configured on all nodes before the +cluster software can be launched and configured using `crmsh`. This +process is cumbersome and error-prone, and the goal is for scripts to +make this process easier. + +Another important function of scripts is collecting information and +reporting potential issues with the cluster. For example, software +versions may differ between nodes, causing byzantine errors or random +failure. `crmsh` comes packaged with a `health` script which will +detect and warn about many of these types of problems. + +There are many tools for managing a collection of nodes, and scripts +are not intended to replace these tools. Rather, they provide an +integrated way to perform tasks across the cluster that would +otherwise be tedious, repetitive and error-prone. The scripts +functionality in the crm shell is mainly inspired by Ansible, a +light-weight and efficient configuration management tool. + +Scripts are implemented using the python `parallax` package which +provides a thin wrapper on top of SSH. This allows the scripts to +function through the usual SSH channels used for system maintenance, +requiring no additional software to be installed or maintained. + +For many scripts that only configure cluster resources or only perform +changes on the local machine, the use of SSH is not necessary. These +scripts can be used even if there is no way for `crmsh` to reach the +other nodes other than through the cluster configuration. + +NOTE: The scripts functionality in `crmsh` has been greatly expanded +and improved in `crmsh` 2.2. Many new scripts have been added, and in +addition the scripts are now used as the backend for the wizards +functionality in HAWK, the HA web interface. For more information, see +https://github.com/ClusterLabs/hawk. + +== Usage == + +Scripts are available through the `cluster` sub-level in the crm +shell. Some scripts have custom commands linked to them for +convenience, such as the `init`, `add` and `remove` commands available +in the `cluster` sublevel, for creating new clusters, introducing new +nodes into the cluster and for removing nodes from a running cluster. + +Other scripts can be accessed through the `script` sub-level. + +=== Common Parameters === + +Which parameters a script accepts varies from script to +script. However, there is a set of parameters that are common to all +scripts. These parameters can be passed to any script. + +`nodes`:: + List of nodes to execute the script for +`dry_run`:: + If set, simulate execution only + (default: no) +`action`:: + If set, only execute a single action (index, as returned by verify) +`statefile`:: + When single-stepping, the state is saved in the given file +`user`:: + Run script as the given user +`sudo`:: + If set, crm will prompt for a sudo password and use sudo when appropriate + (default: no) +`port`:: + Port to connect on +`timeout`:: + Execution timeout in seconds + (default: 600) + +=== List available scripts === + +To list the available scripts, use the following command: + +......... +# crm script +list +......... + +The available scripts are listed along with a short +description. Optionally, the arguments +all+ or +names+ can be +used. Without the +all+ flag, some scripts that are used by `crmsh` to +implement certain commands are hidden from view. With the +names+ +flag, only a plain list of script names is printed. + +=== Script description === + +To get more details about a script, run the `show` command. For +example, to get more information about what the `virtual-ip` script does +and what parameters it accepts, use the following command: + +......... +# crm script +show virtual-ip +......... + +`show` will print a longer description of the script, along with a +list of parameters divided into _steps_. Each script is divided into a +series of steps which are performed in order. Some steps may not +accept any parameters, but for those that do, the available parameters +are listed here. + +By default, only a basic subset of the available parameters is printed +in order to make the scripts easier to use. By passing `all` to the +`show` command, the advanced parameters are also shown. In addition, +there is a list of common parameters + +`show` will print a longer explanation for the script, along with +a list of parameters, each parameter having a description, a note +saying if it is an optional or required parameter, and if optional, +what the default value is. + +=== Verifying parameters === + +Since a script potentially performs a series of actions and may fail +for various reasons at any point, it is advisable to review the +actions that a script will perform before actually running it. To do +this, the `verify` command can be used. + +Pass the parameters that you would pass to `run`, and `verify` will +check that the parameter values are OK, as well as print the sequence +of steps that will be performed given the particular parameter values +given. + +The following is an example showing how to verify the creation of a +Virtual IP resource, using the `virtual-ip` script: + +.......... +# crm script +verify virtual-ip id=my-virtual-ip ip=192.168.0.10 +.......... + +`crmsh` will print something similar to the following output: + +........... +1. Configure cluster resources + + primitive my-virtual-ip ocf:heartbeat:IPaddr2 + ip="192.168.0.10" + op start timeout="20" op stop timeout="20" + op monitor interval="10" timeout="20" +........... + +In this particular case, there is only a single step, and that step +configures a primitive resource. Other scripts may configure multiple +resources and constraints, or may perform multiple steps in sequence. + +=== Running a script === + +To run a script, all required parameters and any optional parameters +that should have values other than the default should be provided as +`key=value` pairs on the command line. + +The following example shows how to create a Virtual IP resource using +the `virtual-ip` script: + +........ +# crm script +run virtual-ip id=my-virtual-ip ip=192.168.0.10 +........ + +==== Single-stepping a script ==== + +It is possible to run a script action-by-action, with manual intervention +between actions. First of all, list the actions to perform given a +certain set of parameter values: + +........ +crm script verify health +........ + +To execute a single action, two things need to be provided: + +1. The index of the action to execute (printed by `verify`) +2. a file in which `crmsh` stores the state of execution. + +Note that it is entirely possible to run actions out-of-order, however +this is unlikely to work in practice since actions often rely on the +outcome of previous actions. + +The following command will execute the first action of the `health` +script and store the output in a temporary file named `health.json`: + +........ +crm script run health action=1 statefile='health.json' +........ + +The statefile contains the script parameters and the output of +previous steps, encoded as `json` data. + +To continue executing the next action in sequence, enter the next +action index: + +........ +crm script run health action=2 statefile='health.json' +........ + +Note that the `dry_run` flag that can be used to do partial execution +of scripts is not taken into consideration when single-stepping +through a script. + +== Creating a script == + +This section will describe how to create a new script, where to put +the script to allow `crmsh` to find it, and how to test that the +script works as intended. + +=== How scripts work, in detail === + +NOTE: The implementation of cluster scripts was revised between +`crmsh` 2.0 and `crmsh` 2.2. This section describes the revised +cluster script format. The old format is still accepted by `crmsh`. + +A cluster script consists of four main sections: + +. The name and description of the script. +. Any other scripts or agents included by this script, and any parameter value overrides to those provided by the included script. +. A set of parameters accepted by the script itself, in addition to those accepted by any scripts or agents included in the script. +. A sequence of actions which the script will perform. + +When the script runs, the actions defined in `main.yml` as described +below are executed one at a time. Each action prescribes a +modification that is applied to the cluster. Some actions work by +calling out to scripts on each of the cluster nodes, and others apply +only on the local node from which the script was executed. + +=== Actions === + +Scripts perform actions that are classified into a few basic +types. Each action is performed by calling out to a shell script, +but the arguments and location of that script varies depending on the +type. + +Here are the types of script actions that can be performed: + +cib:: + * Applies a new CIB configuration to the cluster + +install:: + * Ensures that the given list of packages is installed on all + cluster nodes using the system package manager. + +service:: + * Manages system services using the system init tools. The argument + should be a space-separated list of <service>:<state> pairs. + +call:: + * Run a shell command as specified in the action, either on the + local node on or all nodes. + +copy:: + * Installs a file on the cluster nodes. + * Using a configuration template, install a file on the cluster + nodes. + +crm:: + * Runs the given command using the `crm` shell. This can be used to + start and stop resources, for example. + +collect:: + * Runs on all cluster nodes + * Gathers information about the nodes, both general information and + information specific to the script. + +validate:: + * Runs on the local node + * Validate parameter values and node state based on collected + information. Can modify default values and report issues that + would prevent the script from applying successfully. + +apply:: + * Runs on all or any cluster nodes + * Applies changes, returning information about the applied changes + to the local node. + +apply_local:: + * Runs on the local node + * Applies changes to the cluster, where an action taken on a single + node affect the entire cluster. This includes updating the CIB in + Pacemaker, and also reloading the configuration for Corosync. + +report:: + * Runs on the local node + * This is similar to the _apply_local_ action, with the difference + that the output of a Report action is not interpreted as JSON data + to be passed to the next action. Instead, the output is printed to + the screen. + +==== When expressions ==== + +Actions can be made conditional on the value of script parameters using +the +when:+ expression. This expression has two basic forms. + +The first form is in the form of the name of a script parameter. For +example, given a boolean script parameter named +install+, an action +can be made conditional on that parameter being true using the syntax ++when: install+. + +The second form is a more complex expression. All parameters are +interpreted as either a string value or None if no value was provided. +These can be compared to string literals using python-style +comparators. For example, an action can be conditional on the string +parameter +mode+ having the value +"advanced"+ using the following +syntax: +when: mode == "advanced"+. + +=== Basic structure === + +The crm shell looks for scripts in two primary locations: Included +scripts are installed in the system-wide shared folder, usually +`/usr/share/crmsh/scripts/`. Local and custom scripts are loaded from +the user-local XDG_CONFIG folder, usually found at +`~/.local/crm/scripts/`. These locations may differ depending on how +the crm shell was installed and which system is used, but these are +the locations used on most distributions. + +To create a new script, make a new folder in the user-local scripts +folder and give it a unique name. In this example, we will call our +new script `check-uptime`. + +........ +mkdir -p ~/.local/crm/scripts/check-uptime +........ + +In this directory, create a file called `main.yml`. This is a YAML +document which describes the script, which parameters it requires, and +what actions it will perform. + +YAML is a human-readable markup language which is designed to be easy +to read and modify, while at the same time be compatible with JSON. To +learn more, see http:://yaml.org/[yaml.org]. + +Here is an example `main.yml` file which wraps the resource agent +`ocf:heartbeat:IPaddr2`. + +[source,yaml] +---- +# The version must be exactly 2.2, and must always be +# specified in the script. If the version is missing or +# is less than 2.2, the script is assumed to be a legacy +# script (specified in the format used before crmsh 2.2). +version: 2.2 +shortdesc: Virtual IP +category: Basic +include: + - agent: ocf:heartbeat:IPaddr2 + name: virtual-ip + parameters: + - name: id + type: resource + required: true + - name: ip + type: ip_address + required: true + - name: cidr_netmask + type: integer + required: false + - name: broadcast + type: ip_address + required: false + ops: | + op start timeout="20" op stop timeout="20" + op monitor interval="10" timeout="20" +actions: + - include: virtual-ip +---- + +For a bigger example, here is the `apache` agent which includes +multiple optional steps, the optional installation of packages, +defines multiple cluster resources and potentially calls bash commands +on each of the cluster nodes. + +[source,yaml] +---- +# Copyright (C) 2009 Dejan Muhamedagic +# Copyright (C) 2015 Kristoffer Gronlund +# +# License: GNU General Public License (GPL) +version: 2.2 +category: Server +shortdesc: Apache Webserver +longdesc: | + Configure a resource group containing a virtual IP address and + an instance of the Apache web server. + + You can optionally configure a Filesystem resource which will be + mounted before the web server is started. + + You can also optionally configure a database resource which will + be started before the web server but after mounting the optional + filesystem. +include: + - agent: ocf:heartbeat:apache + name: apache + longdesc: | + The Apache configuration file specified here must be available via the + same path on all cluster nodes, and Apache must be configured with + mod_status enabled. If in doubt, try running Apache manually via + its init script first, and ensure http://localhost:80/server-status is + accessible. + ops: | + op start timeout="40" + op stop timeout="60" + op monitor interval="10" timeout="20" + - script: virtual-ip + shortdesc: The IP address configured here will start before the Apache instance. + parameters: + - name: id + value: "{{id}}-vip" + - script: filesystem + shortdesc: Optional filesystem mounted before the web server is started. + required: false + - script: database + shortdesc: Optional database started before the web server is started. + required: false +parameters: + - name: install + type: boolean + shortdesc: Install and configure apache + value: false +actions: + - install: + - apache2 + shortdesc: Install the apache package + when: install + - service: + - apache: disable + shortdesc: Let cluster manage apache + when: install + - call: a2enmod status; true + shortdesc: Enable status module + when: install + - include: filesystem + - include: database + - include: virtual-ip + - include: apache + - cib: | + group g-{{id}} + {{filesystem:id}} + {{database:id}} + {{virtual-ip:id}} + {{id}} +---- + +The language for referring to parameter values in `cib` actions is +described below. + +=== Command arguments === + +The actions that accept a command as argument must not refer to +commands written in python. They can be plain bash scripts or any +other executable script as long as the nodes have the necessary +dependencies installed. However, see below why implementing scripts in +Python is easier. + +Actions report their progress either by returning JSON on standard +output, or by returning a non-zero return value and printing an error +message to standard error. + +Any JSON returned by an action will be available to the following +steps in the script. When the script executes, it does so in a +temporary folder created for that purpose. In that folder is a file +named `script.input`, containing a JSON array with the output produced +by previous steps. + +The first element in the array (the zeroth element, to be precise) is +a dict containing the parameter values. + +The following elements are dicts with the hostname of each node as key +and the output of the action generated by that node as value. + +In most cases, only local actions (`validate` and `apply_local`) will +use the information in previous steps, but scripts are not limited in +what they can do. + +With this knowledge, we can implement `fetch.py` and `report.py`. + +`fetch.py`: + +[source,python] +---- +#!/usr/bin/python3 +import crm_script as crm +try: + uptime = open('/proc/uptime').read().split()[0] + crm.exit_ok(uptime) +except Exception as e: + crm.exit_fail("Couldn't open /proc/uptime: %s" % (e)) +---- + +`report.py`: + +[source,python] +---- +#!/usr/bin/python3 +import crm_script as crm +show_all = crm.is_true(crm.param('show_all')) +uptimes = list(crm.output(1).items()) +max_uptime = '', 0 +for host, uptime in uptimes: + if float(uptime) > max_uptime[1]: + max_uptime = host, float(uptime) +if show_all: + print("Uptimes: %s" % (', '.join("%s: %s" % v for v in uptimes))) +print("Longest uptime is %s seconds on host %s" % (max_uptime[1], max_uptime[0])) +---- + +See below for more details on the helper library `crm_script`. + +Save the scripts as executable files in the same directory as the +`main.yml` file. + +Before running the script, it is possible to verify that the files are +in a valid format and in the right location. Run the following +command: + +........ +crm script verify check-uptime +........ + +If the verification is successful, try executing the script with the +following command: + +........ +crm script run check-uptime +........ + +Example output: + +[source,bash] +---- +# crm script run check-uptime +INFO: Check uptime of nodes +INFO: Nodes: ha-three, ha-one +OK: Fetch uptimes +OK: Report uptime +Longest uptime is 161054.04 seconds on host ha-one +---- + +To see if the `show_all` parameter works as intended, run the +following: + +........ +crm script run check-uptime show_all=yes +........ + +Example output: + +[source,bash] +---- +# crm script run check-uptime show_all=yes +INFO: Check uptime of nodes +INFO: Nodes: ha-three, ha-one +OK: Fetch uptimes +OK: Report uptime +Uptimes: ha-one: 161069.83, ha-three: 159950.38 +Longest uptime is 161069.83 seconds on host ha-one +---- + +=== Remote permissions === + +Some scripts may require super-user access to remote or local +nodes. It is recommended that this is handled through SSH certificates +and agents, to facilitate password-less access to nodes. + +=== Running scripts without a cluster === + +All cluster scripts can optionally take a `nodes` argument, which +determines the nodes that the script will run on. This node list is +not limited to nodes already in the cluster. It is even possible to +execute cluster scripts before a cluster is set up, such as the +`health` and `init` scripts used by the `cluster` sub-level. + +........ +crm script run health nodes=example1,example2 +........ + +The list of nodes can be comma- or space-separated, but if the list +contains spaces, the whole argument will have to be quoted: + +........ +crm script run health nodes="example1 example2" +........ + +=== Running in validate mode === + +It may be desirable to do a dry-run of a script, to see if any +problems are present that would make the script fail before trying to +apply it. To do this, add the argument `dry_run=yes` to the invocation: + +......... +crm script run health dry_run=yes +......... + +The script execution will stop at the first `apply` action. Note that +non-modifying steps that happen after the first `apply` action will +not be performed in a dry run. + +=== Helper library === + +When the script data is copied to each node, a small helper library is +also passed along with the script. This library can be found in +`utils/crm_script.py` in the source repository. This library helps +with producing output in the correct format, parsing the +`script.input` data provided to scripts, and more. + +.`crm_script` API +`host()`:: + Returns hostname of current node +`get_input()`:: + Returns the input data list. The first element in the list + is a dict of the script parameters. The rest are the output + from previous steps. +`parameters()`:: + Returns the script parameters as a dict. +`param(name)`:: + Returns the value of the named script parameter. +`output(step_idx)`:: + Returns the output of the given step, with the first step being step 1. +`exit_ok(data)`:: + Exits the step returning `data` as output. +`exit_fail(msg)`:: + Exits the step returning `msg` as error message. +`is_true(value)`:: + Converts a truth value from string to boolean. +`call(cmd, shell=False)`:: + Perform a system call. Returns `(rc, stdout, stderr)`. + +=== The handles language === + +CIB configurations and commands can refer to the value of parameters +in the text of the action. This is done using a custom language, +similar to handlebars. + +The language accepts the following constructions: + +............ +{{name}} = Inserts the value of the parameter <name> +{{script:name}} = Inserts the value of the parameter <name> from the + included script named <script>. +{{#name}} ... {{/name}} = Inserts the text between the mustasches when + name is truthy. +{{^name}} ... {{/name}} = Inserts the text between the mustasches when + name is falsy. +............ + |