diff options
Diffstat (limited to '')
| -rw-r--r-- | doc/website-v1/man-3.adoc | 5309 |
1 files changed, 5309 insertions, 0 deletions
diff --git a/doc/website-v1/man-3.adoc b/doc/website-v1/man-3.adoc new file mode 100644 index 0000000..e4411cc --- /dev/null +++ b/doc/website-v1/man-3.adoc @@ -0,0 +1,5309 @@ +:man source: crm +:man version: 2.3.0 +:man manual: crmsh documentation + +crm(8) +====== + +NAME +---- +crm - Pacemaker command line interface for configuration and management + + +SYNOPSIS +-------- +*crm* [OPTIONS] [SUBCOMMAND ARGS...] + + +[[topics_Description,Program description]] +DESCRIPTION +----------- +The `crm` shell is a command-line based cluster configuration and +management tool. Its goal is to assist as much as possible with the +configuration and maintenance of Pacemaker-based High Availability +clusters. + +For more information on Pacemaker itself, see http://clusterlabs.org/. + +`crm` works both as a command-line tool to be called directly from the +system shell, and as an interactive shell with extensive tab +completion and help. + +The primary focus of the `crm` shell is to provide a simplified and +consistent interface to Pacemaker, but it also provides tools for +managing the creation and configuration of High Availability clusters +from scratch. To learn more about this aspect of `crm`, see the +`cluster` section below. + +The `crm` shell can be used to manage every aspect of configuring and +maintaining a cluster. It provides a simplified line-based syntax on +top of the XML configuration format used by Pacemaker, commands for +starting and stopping resources, tools for exploring the history of a +cluster including log scraping and a set of cluster scripts useful for +automating the setup and installation of services on the cluster +nodes. + +The `crm` shell is line oriented: every command must start and finish +on the same line. It is possible to use a continuation character (+\+) +to write one command in two or more lines. The continuation character +is commonly used when displaying configurations. + +[[topics_CommandLine,Command line options]] +OPTIONS +------- +*-f, --file*='FILE':: + Load commands from the given file. If a dash +-+ is used in place + of a file name, `crm` will read commands from the shell standard + input (`stdin`). + +*-c, --cib*='CIB':: + Start the session using the given shadow CIB file. + Equivalent to +cib use <CIB>+. + +*-D, --display=*'OUTPUT_TYPE':: + Choose one of the output options: +plain+, +color-always+, +color+, + or +uppercase+. The default is +color+ if the terminal emulation + supports colors. Otherwise, +plain+ is used. + +*-F, --force*:: + Make `crm` proceed with applying changes where it would normally + ask the user to confirm before proceeding. This option is mainly + useful in scripts, and should be used with care. + +*-w, --wait*:: + Make `crm` wait for the cluster transition to finish (for the + changes to take effect) after each processed line. + +*-H, --history*='DIR|FILE|SESSION':: + A directory or file containing a cluster report to load + into the `history` commands, or the name of a previously + saved history session. + +*-h, --help*:: + Print help page. + +*--version*:: + Print crmsh version and build information (Mercurial Hg changeset + hash). + +*-d, --debug*:: + Print verbose debugging information. + +*-R, --regression-tests*:: + Enables extra verbose trace logging used by the regression + tests. Logs all external calls made by crmsh. + +*--scriptdir*='DIR':: + Extra directory where crm looks for cluster scripts, or a list of + directories separated by semi-colons (e.g. +/dir1;/dir2;etc.+). + +*-o, --opt*='OPTION=VALUE':: + Set crmsh option temporarily. If the options are saved using + +options save+ then the value passed here will also be saved. + Multiple options can be set by using +-o+ multiple times. + +[[topics_Introduction,Introduction]] +== Introduction + +This section of the user guide covers general topics about the user +interface and describes some of the features of `crmsh` in detail. + +[[topics_Introduction_Interface,User interface]] +=== User interface + +The main purpose of `crmsh` is to provide a simple yet powerful +interface to the cluster stack. There are two main modes of operation +with the user interface of `crmsh`: + +* Command line (single-shot) use - Use `crm` as a regular UNIX command + from your usual shell. `crm` has full bash completion built in, so + using it in this manner should be as comfortable and familiar as + using any other command-line tool. + +* Interactive mode - By calling `crm` without arguments, or by calling + it with only a sublevel as argument, `crm` enters the interactive + mode. In this mode, it acts as its own command shell, which + remembers which sublevel you are currently in and allows for rapid + and convenient execution of multiple commands within the same + sublevel. This mode also has full tab completion, as well as + built-in interactive help and syntax highlighting. + +Here are a few examples of using `crm` both as a command-line tool and +as an interactive shell: + +.Command line (one-shot) use: +........ +# crm resource stop www_app +........ + +.Interactive use: +........ +# crm +crm(live)# resource +crm(live)resource# unmanage tetris_1 +crm(live)resource# up +crm(live)# node standby node4 +........ + +.Cluster configuration: +........ +# crm configure<<EOF + # + # resources + # + primitive disk0 iscsi \ + params portal=192.168.2.108:3260 target=iqn.2008-07.com.suse:disk0 + primitive fs0 Filesystem \ + params device=/dev/disk/by-label/disk0 directory=/disk0 fstype=ext3 + primitive internal_ip IPaddr params ip=192.168.1.101 + primitive apache apache \ + params configfile=/disk0/etc/apache2/site0.conf + primitive apcfence stonith:apcsmart \ + params ttydev=/dev/ttyS0 hostlist="node1 node2" \ + op start timeout=60s + primitive pingd pingd \ + params name=pingd dampen=5s multiplier=100 host_list="r1 r2" + # + # monitor apache and the UPS + # + monitor apache 60s:30s + monitor apcfence 120m:60s + # + # cluster layout + # + group internal_www \ + disk0 fs0 internal_ip apache + clone fence apcfence \ + meta globally-unique=false clone-max=2 clone-node-max=1 + clone conn pingd \ + meta globally-unique=false clone-max=2 clone-node-max=1 + location node_pref internal_www \ + rule 50: #uname eq node1 \ + rule pingd: defined pingd + # + # cluster properties + # + property stonith-enabled=true + commit +EOF +........ + +The `crm` interface is hierarchical, with commands organized into +separate levels by functionality. To list the available levels and +commands, either execute +help <level>+, or, if at the top level of +the shell, simply typing `help` will provide an overview of all +available levels and commands. + +The +(live)+ string in the `crm` prompt signifies that the current CIB +in use is the cluster live configuration. It is also possible to +work with so-called <<topics_Features_Shadows,shadow CIBs>>. These are separate, inactive +configurations stored in files, that can be applied and thereby +replace the live configuration at any time. + +[[topics_Introduction_Completion,Tab completion]] +=== Tab completion + +The `crm` makes extensive use of tab completion. The completion +is both static (i.e. for `crm` commands) and dynamic. The latter +takes into account the current status of the cluster or +information from installed resource agents. Sometimes, completion +may also be used to get short help on resource parameters. Here +are a few examples: + +............... +crm(live)resource# <TAB><TAB> +bye failcount move restart unmigrate +cd help param show unmove +cleanup list promote start up +demote manage quit status utilization +end meta refresh stop +exit migrate reprobe unmanage + +crm(live)configure# primitive fence-1 <TAB><TAB> +heartbeat: lsb: ocf: stonith: + +crm(live)configure# primitive fence-1 stonith:<TAB><TAB> +apcmaster external/ippower9258 fence_legacy +apcmastersnmp external/kdumpcheck ibmhmc +apcsmart external/libvirt ipmilan + +crm(live)configure# primitive fence-1 stonith:ipmilan params <TAB><TAB> +auth= hostname= ipaddr= login= password= port= priv= + +crm(live)configure# primitive fence-1 stonith:ipmilan params auth=<TAB><TAB> +auth* (string) + The authorization type of the IPMI session ("none", "straight", "md2", or "md5") +............... + +`crmsh` also comes with bash completion usable directly from the +system shell. This should be installed automatically with the command +itself. + +[[topics_Introduction_Shorthand,Shorthand syntax]] +=== Shorthand syntax + +When using the `crm` shell to manage clusters, you will end up typing +a lot of commands many times over. Clear command names like ++configure+ help in understanding and learning to use the cluster +shell, but is easy to misspell and is tedious to type repeatedly. The +interactive mode and tab completion both help with this, but the `crm` +shell also has the ability to understand a variety of shorthand +aliases for all of the commands. + +For example, instead of typing `crm status`, you can type `crm st` or +`crm stat`. Instead of `crm configure` you can type `crm cfg` or even +`crm cf`. `crm resource` can be shorted as `crm rsc`, and so on. + +The exact list of accepted aliases is too long to print in full, but +experimentation and typos should help in discovering more of them. + +[[topics_Features,Features]] +== Features + +The feature set of crmsh covers a wide range of functionality, and +understanding how and when to use the various features of the shell +can be difficult. This section of the guide describes some of the +features and use cases of `crmsh` in more depth. The intention is to +provide a deeper understanding of these features, but also to serve as +a guide to using them. + +[[topics_Features_Shadows,Shadow CIB usage]] +=== Shadow CIB usage + +A Shadow CIB is a normal cluster configuration stored in a file. +They may be manipulated in much the same way as the _live_ CIB, with +the key difference that changes to a shadow CIB have no effect on the +actual cluster resources. An administrator may choose to apply any of +them to the cluster, thus replacing the running configuration with the +one found in the shadow CIB. + +The `crm` prompt always contains the name of the configuration which +is currently in use, or the string _live_ if using the live cluster +configuration. + +When editing the configuration in the `configure` level, no changes +are actually applied until the `commit` command is executed. It is +possible to start editing a configuration as usual, but instead of +committing the changes to the active CIB, save them to a shadow CIB. + +The following example `configure` session demonstrates how this can be +done: +............... +crm(live)configure# cib new test-2 +INFO: test-2 shadow CIB created +crm(test-2)configure# commit +............... + +[[topics_Features_Checks,Configuration semantic checks]] +=== Configuration semantic checks + +Resource definitions may be checked against the meta-data +provided with the resource agents. These checks are currently +carried out: + +- are required parameters set +- existence of defined parameters +- timeout values for operations + +The parameter checks are obvious and need no further explanation. +Failures in these checks are treated as configuration errors. + +The timeouts for operations should be at least as long as those +recommended in the meta-data. Too short timeout values are a +common mistake in cluster configurations and, even worse, they +often slip through if cluster testing was not thorough. Though +operation timeouts issues are treated as warnings, make sure that +the timeouts are usable in your environment. Note also that the +values given are just _advisory minimum_---your resources may +require longer timeouts. + +User may tune the frequency of checks and the treatment of errors +by the <<cmdhelp_options_check-frequency,`check-frequency`>> and +<<cmdhelp_options_check-mode,`check-mode`>> preferences. + +Note that if the +check-frequency+ is set to +always+ and the ++check-mode+ to +strict+, errors are not tolerated and such +configuration cannot be saved. + +[[topics_Features_Templates,Configuration templates]] +=== Configuration templates + +.Deprecation note +**************************** +Configuration templates have been deprecated in favor of the more +capable `cluster scripts`. To learn how to use cluster scripts, see +the dedicated documentation on the `crmsh` website at +http://crmsh.github.io/, or in the <<cmdhelp_script,Script section>>. +**************************** + +Configuration templates are ready made configurations created by +cluster experts. They are designed in such a way so that users +may generate valid cluster configurations with minimum effort. +If you are new to Pacemaker, templates may be the best way to +start. + +We will show here how to create a simple yet functional Apache +configuration: +............... +# crm configure +crm(live)configure# template +crm(live)configure template# list templates +apache filesystem virtual-ip +crm(live)configure template# new web <TAB><TAB> +apache filesystem virtual-ip +crm(live)configure template# new web apache +INFO: pulling in template apache +INFO: pulling in template virtual-ip +crm(live)configure template# list +web2-d web2 vip2 web3 vip web +............... + +We enter the `template` level from `configure`. Use the `list` +command to show templates available on the system. The `new` +command creates a configuration from the +apache+ template. You +can use tab completion to pick templates. Note that the apache +template depends on a virtual IP address which is automatically +pulled along. The `list` command shows the just created +web+ +configuration, among other configurations (I hope that you, +unlike me, will use more sensible and descriptive names). + +The `show` command, which displays the resulting configuration, +may be used to get an idea about the minimum required changes +which have to be done. All +ERROR+ messages show the line numbers +in which the respective parameters are to be defined: +............... +crm(live)configure template# show +ERROR: 23: required parameter ip not set +ERROR: 61: required parameter id not set +ERROR: 65: required parameter configfile not set +crm(live)configure template# edit +............... + +The `edit` command invokes the preferred text editor with the ++web+ configuration. At the top of the file, the user is advised +how to make changes. A good template should require from the user +to specify only parameters. For example, the +web+ configuration +we created above has the following required and optional +parameters (all parameter lines start with +%%+): +............... +$ grep -n ^%% ~/.crmconf/web +23:%% ip +31:%% netmask +35:%% lvs_support +61:%% id +65:%% configfile +71:%% options +76:%% envfiles +............... + +These lines are the only ones that should be modified. Simply +append the parameter value at the end of the line. For instance, +after editing this template, the result could look like this (we +used tabs instead of spaces to make the values stand out): +............... +$ grep -n ^%% ~/.crmconf/web +23:%% ip 192.168.1.101 +31:%% netmask +35:%% lvs_support +61:%% id websvc +65:%% configfile /etc/apache2/httpd.conf +71:%% options +76:%% envfiles +............... + +As you can see, the parameter line format is very simple: +............... +%% <name> <value> +............... + +After editing the file, use `show` again to display the +configuration: +............... +crm(live)configure template# show +primitive virtual-ip IPaddr \ + params ip=192.168.1.101 +primitive apache apache \ + params configfile="/etc/apache2/httpd.conf" +monitor apache 120s:60s +group websvc \ + apache virtual-ip +............... + +The target resource of the apache template is a group which we +named +websvc+ in this sample session. + +This configuration looks exactly as you could type it at the +`configure` level. The point of templates is to save you some +typing. It is important, however, to understand the configuration +produced. + +Finally, the configuration may be applied to the current +crm configuration (note how the configuration changed slightly, +though it is still equivalent, after being digested at the +`configure` level): +............... +crm(live)configure template# apply +crm(live)configure template# cd .. +crm(live)configure# show +node xen-b +node xen-c +primitive apache apache \ + params configfile="/etc/apache2/httpd.conf" \ + op monitor interval=120s timeout=60s +primitive virtual-ip IPaddr \ + params ip=192.168.1.101 +group websvc apache virtual-ip +............... + +Note that this still does not commit the configuration to the CIB +which is used in the shell, either the running one (+live+) or +some shadow CIB. For that you still need to execute the `commit` +command. + +To complete our example, we should also define the preferred node +to run the service: + +............... +crm(live)configure# location websvc-pref websvc 100: xen-b +............... + +If you are not happy with some resource names which are provided +by default, you can rename them now: + +............... +crm(live)configure# rename virtual-ip intranet-ip +crm(live)configure# show +node xen-b +node xen-c +primitive apache apache \ + params configfile="/etc/apache2/httpd.conf" \ + op monitor interval=120s timeout=60s +primitive intranet-ip IPaddr \ + params ip=192.168.1.101 +group websvc apache intranet-ip +location websvc-pref websvc 100: xen-b +............... + +To summarize, working with templates typically consists of the +following steps: + +- `new`: create a new configuration from templates +- `edit`: define parameters, at least the required ones +- `show`: see if the configuration is valid +- `apply`: apply the configuration to the `configure` level + +[[topics_Features_Testing,Resource testing]] +=== Resource testing + +The amount of detail in a cluster makes all configurations prone +to errors. By far the largest number of issues in a cluster is +due to bad resource configuration. The shell can help quickly +diagnose such problems. And considerably reduce your keyboard +wear. + +Let's say that we entered the following configuration: +............... +node xen-b +node xen-c +node xen-d +primitive fencer stonith:external/libvirt \ + params hypervisor_uri="qemu+tcp://10.2.13.1/system" \ + hostlist="xen-b xen-c xen-d" \ + op monitor interval=2h +primitive svc Xinetd \ + params service=systat \ + op monitor interval=30s +primitive intranet-ip IPaddr2 \ + params ip=10.2.13.100 \ + op monitor interval=30s +primitive apache apache \ + params configfile="/etc/apache2/httpd.conf" \ + op monitor interval=120s timeout=60s +group websvc apache intranet-ip +location websvc-pref websvc 100: xen-b +............... + +Before typing `commit` to submit the configuration to the cib we +can make sure that all resources are usable on all nodes: +............... +crm(live)configure# rsctest websvc svc fencer +............... + +It is important that resources being tested are not running on +any nodes. Otherwise, the `rsctest` command will refuse to do +anything. Of course, if the current configuration resides in a +CIB shadow, then a `commit` is irrelevant. The point being that +resources are not running on any node. + +.Note on stopping all resources +**************************** +Alternatively to not committing a configuration, it is also +possible to tell Pacemaker not to start any resources: + +............... +crm(live)configure# property stop-all-resources=yes +............... +Almost none---resources of class stonith are still started. But +shell is not as strict when it comes to stonith resources. +**************************** + +Order of resources is significant insofar that a resource depends +on all resources to its left. In most configurations, it's +probably practical to test resources in several runs, based on +their dependencies. + +Apart from groups, `crm` does not interpret constraints and +therefore knows nothing about resource dependencies. It also +doesn't know if a resource can run on a node at all in case of an +asymmetric cluster. It is up to the user to specify a list of +eligible nodes if a resource is not meant to run on every node. + +[[topics_Features_Security,Access Control Lists (ACL)]] +=== Access Control Lists (ACL) + +.Note on ACLs in Pacemaker 1.1.12 +**************************** +The support for ACLs has been revised in Pacemaker version 1.1.12 and +up. Depending on which version you are using, the information in this +section may no longer be accurate. Look for the `acl_target` +configuration element for more details on the new syntax. +**************************** + +By default, the users from the +haclient+ group have full access +to the cluster (or, more precisely, to the CIB). Access control +lists allow for finer access control to the cluster. + +Access control lists consist of an ordered set of access rules. +Each rule allows read or write access or denies access +completely. Rules are typically combined to produce a specific +role. Then, users may be assigned a role. + +For instance, this is a role which defines a set of rules +allowing management of a single resource: + +............... +role bigdb_admin \ + write meta:bigdb:target-role \ + write meta:bigdb:is-managed \ + write location:bigdb \ + read ref:bigdb +............... + +The first two rules allow modifying the +target-role+ and ++is-managed+ meta attributes which effectively enables users in +this role to stop/start and manage/unmanage the resource. The +constraints write access rule allows moving the resource around. +Finally, the user is granted read access to the resource +definition. + +For proper operation of all Pacemaker programs, it is advisable +to add the following role to all users: + +............... +role read_all \ + read cib +............... + +For finer grained read access try with the rules listed in the +following role: + +............... +role basic_read \ + read node attribute:uname \ + read node attribute:type \ + read property \ + read status +............... + +It is however possible that some Pacemaker programs (e.g. +`ptest`) may not function correctly if the whole CIB is not +readable. + +Some of the ACL rules in the examples above are expanded by the +shell to XPath specifications. For instance, ++meta:bigdb:target-role+ expands to: + +........ +//primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role'] +........ + +You can see the expansion by showing XML: + +............... +crm(live) configure# show xml bigdb_admin +... +<acls> + <acl_role id="bigdb_admin"> + <write id="bigdb_admin-write" + xpath="//primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role']"/> +............... + +Many different XPath expressions can have equal meaning. For +instance, the following two are equal, but only the first one is +going to be recognized as shortcut: + +............... +//primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role'] +//resources/primitive[@id='bigdb']/meta_attributes/nvpair[@name='target-role'] +............... + +XPath is a powerful language, but you should try to keep your ACL +xpaths simple and the builtin shortcuts should be used whenever +possible. + +[[topics_Features_Resourcesets,Syntax: Resource sets]] +=== Syntax: Resource sets + +Using resource sets can be a bit confusing unless one knows the +details of the implementation in Pacemaker as well as how to interpret +the syntax provided by `crmsh`. + +Three different types of resource sets are provided by `crmsh`, and +each one implies different values for the two resource set attributes, ++sequential+ and +require-all+. + ++sequential+:: + If false, the resources in the set do not depend on each other + internally. Setting +sequential+ to +true+ implies a strict order of + dependency within the set. + ++require-all+:: + If false, only one resource in the set is required to fulfil the + requirements of the set. The set of A, B and C with +require-all+ + set to +false+ is be read as "A OR B OR C" when its dependencies + are resolved. + +The three types of resource sets modify the attributes in the +following way: + +1. Implicit sets (no brackets). +sequential=true+, +require-all=true+ +2. Parenthesis set (+(+ ... +)+). +sequential=false+, +require-all=true+ +3. Bracket set (+[+ ... +]+). +sequential=false+, +require-all=false+ + +To create a set with the properties +sequential=true+ and ++require-all=false+, explicitly set +sequential+ in a bracketed set, ++[ A B C sequential=true ]+. + +To create multiple sets with both +sequential+ and +require-all+ set to +true, explicitly set +sequential+ in a parenthesis set: ++A B ( C D sequential=true )+. + +[[topics_Features_AttributeListReferences,Syntax: Attribute list references]] +=== Syntax: Attribute list references + +Attribute lists are used to set attributes and parameters for +resources, constraints and property definitions. For example, to set +the virtual IP used by an +IPAddr2+ resource the attribute +ip+ can be +set in an attribute list for that resource. + +Attribute lists can have identifiers that name them, and other +resources can reuse the same attribute list by referring to that name +using an +$id-ref+. For example, the following statement defines a +simple dummy resource with an attribute list which sets the parameter ++state+ to the value 1 and sets the identifier for the attribute list +to +on-state+: + +.............. +primitive dummy-1 Dummy params $id=on-state state=1 +.............. + +To refer to this attribute list from a different resource, refer to +the +on-state+ name using an id-ref: + +.............. +primitive dummy-2 Dummy params $id-ref=on-state +.............. + +The resource +dummy-2+ will now also have the parameter +state+ set to the value 1. + +[[topics_Features_AttributeReferences,Syntax: Attribute references]] +=== Syntax: Attribute references + +In some cases, referencing complete attribute lists is too +coarse-grained, for example if two different parameters with different +names should have the same value set. Instead of having to copy the +value in multiple places, it is possible to create references to +individual attributes in attribute lists. + +To name an attribute in order to be able to refer to it later, prefix +the attribute name with a +$+ character (as seen above with the +special names +$id+ and +$id-ref+: + +............ +primitive dummy-1 Dummy params $state=1 +............ + +The identifier +state+ can now be used to refer to this attribute from other +primitives, using the +@<id>+ syntax: + +............ +primitive dummy-2 Dummy params @state +............ + +In some cases, using the attribute name as the identifier doesn't work +due to name clashes. In this case, the syntax +$<id>:<name>=<value>+ +can be used to give the attribute a different identifier: + +............ +primitive dummy-1 params $dummy-state-on:state=1 +primitive dummy-2 params @dummy-state-on +............ + +There is also the possibility that two resources both use the same +attribute value but with different names. For example, a web server +may have a parameter +server_ip+ for setting the IP address where it +listens for incoming requests, and a virtual IP resource may have a +parameter called +ip+ which sets the IP address it creates. To +configure these two resources with an IP without repeating the value, +the reference can be given a name using the syntax +@<id>:<name>+. + +Example: +............ +primitive virtual-ip IPaddr2 params $vip:ip=192.168.1.100 +primitive webserver apache params @vip:server_ip +............ + +[[topics_Syntax_RuleExpressions,Syntax: Rule expressions]] +=== Syntax: Rule expressions + +Many of the configuration commands in `crmsh` now support the use of +_rule expressions_, which can influence what attributes apply to a +resource or under which conditions a constraint is applied, depending +on changing conditions like date, time, the value of attributes and +more. + +Here is an example of a simple rule expression used to apply a +a different resource parameter on the node named `node1`: + +.............. +primitive my_resource Special \ + params 2: rule #uname eq node1 interface=eth1 \ + params 1: interface=eth0 +.............. + +This primitive resource has two lists of parameters with descending +priority. The parameter list with the highest priority is applied +first, but only if the rule expressions for that parameter list all +apply. In this case, the rule `#uname eq node1` limits the parameter +list so that it is only applied on `node1`. + +Note that rule expressions are not terminated and are immediately +followed by the data to which the rule is applied. In this case, the +name-value pair `interface=eth1`. + +Rule expressions can contain multiple expressions connected using the +boolean operator `or` and `and`. The full syntax for rule expressions +is listed below. + +.............. +rules :: + rule [id_spec] [$role=<role>] <score>: <expression> + [rule [id_spec] [$role=<role>] <score>: <expression> ...] + +id_spec :: $id=<id> | $id-ref=<id> +score :: <number> | <attribute> | [-]inf +expression :: <simple_exp> [<bool_op> <simple_exp> ...] +bool_op :: or | and +simple_exp :: <attribute> [type:]<binary_op> <value> + | <unary_op> <attribute> + | date <date_expr> +type :: <string> | <version> | <number> +binary_op :: lt | gt | lte | gte | eq | ne +unary_op :: defined | not_defined + +date_expr :: lt <end> + | gt <start> + | in start=<start> end=<end> + | in start=<start> <duration> + | spec <date_spec> +duration|date_spec :: + hours=<value> + | monthdays=<value> + | weekdays=<value> + | yearsdays=<value> + | months=<value> + | weeks=<value> + | years=<value> + | weekyears=<value> + | moon=<value> +.............. + +[[topics_Reference,Command reference]] +== Command reference + +The commands are structured to be compatible with the shell command +line. Sometimes, the underlying Pacemaker grammar uses characters that +have special meaning in bash, that will need to be quoted. This +includes the hash or pound sign (`#`), single and double quotes, and +any significant whitespace. + +Whitespace is also significant when assigning values, meaning that ++key=value+ is different from +key = value+. + +Commands can be referenced using short-hand as long as the short-hand +is unique. This can be either a prefix of the command name or a prefix +string of characters found in the name. + +For example, +status+ can be abbreviated as +st+ or +su+, and ++configure+ as +conf+ or +cfg+. + +The syntax for the commands is given below in an informal, BNF-like +grammar. + +* `<value>` denotes a string. +* `[value]` means that the construct is optional. +* The ellipsis (`...`) signifies that the previous construct may be + repeated. +* `first|second` means either first or second. +* The rest are literals (strings, `:`, `=`). + +[[cmdhelp_root_status,Cluster status]] +=== `status` + +Show cluster status. The status is displayed by `crm_mon`. Supply +additional arguments for more information or different format. +See `crm_mon(8)` for more details. + +Example: +............... +status +status simple +status full +............... + +Usage: +............... +status [<option> ...] + +option :: full + | bynode + | inactive + | ops + | timing + | failcounts + | verbose + | quiet + | html + | xml + | simple + | tickets + | noheaders + | detail + | brief +............... + +[[cmdhelp_root_verify,Verify cluster status]] +=== `verify` + +Performs basic checks for the cluster configuration and +current status, reporting potential issues. + +See `crm_verify(8)` and `crm_simulate(8)` for more details. + +Example: +............... +verify +verify scores +............... + +Usage: +............... +verify [scores] +............... + + +[[cmdhelp_cluster,Cluster setup and management]] +=== `cluster` - Cluster setup and management + +Whole-cluster configuration management with High Availability +awareness. + +The commands on the cluster level allows configuration and +modification of the underlying cluster infrastructure, and also +supplies tools to do whole-cluster systems management. + +These commands enable easy installation and maintenance of a HA +cluster, by providing support for package installation, configuration +of the cluster messaging layer, file system setup and more. + +[[cmdhelp_cluster_add,Add a new node to the cluster]] +==== `add` + +Add a new node to the cluster. The new node will be +configured as a cluster member. + +Options: + +*-y, --yes*:: + Answer "yes" to all prompts (use with caution) + +Usage: +............... +add [options] [<node> ...] +............... + +[[cmdhelp_cluster_copy,Copy file to other cluster nodes]] +==== `copy` + +Copy file to other cluster nodes. + +Copies the given file to all other nodes unless given a +list of nodes to copy to as argument. + +Usage: +............... +copy <filename> [nodes ...] +............... + +Example: +............... +copy /etc/motd +............... + +[[cmdhelp_cluster_diff,Diff file across cluster]] +==== `diff` + +Displays the difference, if any, between a given file +on different nodes. If the second argument is `--checksum`, +a checksum of the file will be calculated and displayed for +each node. + +Usage: +............... +diff <file> [--checksum] [nodes...] +............... + +Example: +............... +diff /etc/crm/crm.conf node2 +diff /etc/resolv.conf --checksum +............... + +[[cmdhelp_cluster_geo_init,Configure cluster as geo cluster]] +==== `geo-init` + +Create a new geo cluster with the current cluster as the +first member. Pass the complete geo cluster topology as +arguments to this command, and then use `geo-join` and +`geo-init-arbitrator` to add the remaining members to +the geo cluster. + +Options: + +*-q, --quiet*:: + Be quiet (don't describe what's happening, just do it) + +*-y, --yes*:: + Answer "yes" to all prompts (use with caution) + +*--arbitrator=IP*:: + IP address of geo cluster arbitrator + +*--clusters=DESC*:: + Cluster description (see details below) + +*--tickets=LIST*:: + Tickets to create (space-separated) + + +Cluster Description: + +This is a map of cluster names to IP addresses. +Each IP address will be configured as a virtual IP +representing that cluster in the geo cluster +configuration. + +Example with two clusters named paris and amsterdam: + +............ + --clusters "paris=192.168.10.10 amsterdam=192.168.10.11" +............ + +Name clusters using the +--name+ parameter to `init`. + +Usage: +............... +geo-init [options] +............... + + +[[cmdhelp_cluster_geo_init_arbitrator,Initialize node as geo cluster arbitrator]] +==== `geo-init-arbitrator` + +Configure the current node as a geo arbitrator. The command +requires an existing geo cluster or geo arbitrator from which +to get the geo cluster configuration. + +Options: + +*--clusters=DESC*:: + Cluster description (see +geo-init+ for details) + +*-c IP, --cluster-node=IP*:: + IP address of an already-configured geo cluster + +Usage: +............... +geo-init-arbitrator [options] +............... + + +[[cmdhelp_cluster_geo_join,Join cluster to existing geo cluster]] +==== `geo-join` + +This command should be run from one of the nodes in a cluster +which is currently not a member of a geo cluster. The geo +cluster configuration will be fetched from the provided node, +and the cluster will be added to the geo cluster. + +Note that each cluster in a geo cluster needs to have a unique +name set. The cluster name can be set using the `--name` argument +to `init`, or by configuring corosync with the cluster name in +an existing cluster. + +Options: + +*-c IP, --cluster-node=IP*:: + IP address of an already-configured geo cluster or arbitrator + +Usage: +............... +geo-join [options] +............... + + +[[cmdhelp_cluster_health,Cluster health check]] +==== `health` + +Runs a larger set of tests and queries on all nodes in the cluster to +verify the general system health and detect potential problems. + +Usage: +............... +health +............... + +[[cmdhelp_cluster_init,Initializes a new HA cluster]] +==== `init` + +Initialize a cluster from scratch. This command configures +a complete cluster, and can also add additional cluster +nodes to the initial one-node cluster using the `--nodes` +option. + +Options: + +*-q, --quiet*:: + Be quiet (don't describe what's happening, just do it) + +*-y, --yes*:: + Answer "yes" to all prompts (use with caution, this + is destructive, especially during the "storage" stage) + +*-t TEMPLATE, --template=TEMPLATE**:: + Optionally configure cluster with template "name" + (currently only "ocfs2" is valid here) + +*-n NAME, --name=NAME*:: + Set the name of the configured cluster. + +*-N NODES, --nodes=NODES*:: + Additional nodes to add to the created cluster. May + include the current node, which will always be the + initial cluster node. + +*-w WATCHDOG, --watchdog=WATCHDOG*:: + Use the given watchdog device. + +Network configuration: + +Options for configuring the network and messaging layer. + +*-i IF, --interface=IF*:: + Bind to IP address on interface IF + +*-u, --unicast*:: + Configure corosync to communicate over unicast (UDP), + and not multicast. Default is multicast unless an + environment where multicast cannot be used is + detected. + +*-A IP, --admin-ip=IP*:: + Configure IP address as an administration virtual IP + +Storage configuration: + +Options for configuring shared storage. + +*-p DEVICE, --partition-device=DEVICE*:: + Partition this shared storage device (only used in + "storage" stage) + +*-s DEVICE, --sbd-device=DEVICE*:: + Block device to use for SBD fencing + +*-o DEVICE, --ocfs2-device=DEVICE*:: + Block device to use for OCFS2 (only used in "vgfs" + stage) + + +Stage can be one of: + +*ssh*:: + Create SSH keys for passwordless SSH between cluster nodes + +*csync2*:: + Configure csync2 + +*corosync*:: + Configure corosync + +*storage*:: + Partition shared storage (ocfs2 template only) + +*sbd*:: + Configure SBD (requires -s <dev>) + +*cluster*:: + Bring the cluster online + +*vgfs*:: + Create volume group and filesystem (ocfs2 template only, requires `-o <dev>`) + +*admin*:: + Create administration virtual IP (optional) + +[NOTE] +============ +- If stage is not specified, the script will run through each stage + in sequence, with prompts for required information. +- If using the ocfs2 template, the storage stage will partition a block + device into two pieces, one for SBD, the remainder for OCFS2. This is + good for testing and demonstration, but not ideal for production. + To use storage you have already configured, pass -s and -o to specify + the block devices for SBD and OCFS2, and the automatic partitioning + will be skipped. +============ + +Usage: +............... +init [options] [STAGE] +............... + + +[[cmdhelp_cluster_join,Join existing cluster]] +==== `join` + +Join the current node to an existing cluster. The +current node cannot be a member of a cluster already. +Pass any node in the existing cluster as the argument +to the `-c` option. + +Options: + +*-q, --quiet*:: + Be quiet (don't describe what's happening, just do it) + +*-y, --yes*:: + Answer "yes" to all prompts (use with caution) + +*-w WATCHDOG, --watchdog=WATCHDOG*:: + Use the given watchdog device + +Network configuration: + +Options for configuring the network and messaging layer. + + +*-c HOST, --cluster-node=HOST*:: + IP address or hostname of existing cluster node + +*-i IF, --interface=IF*:: + Bind to IP address on interface IF + + +Stage can be one of: + +*ssh*:: + Obtain SSH keys from existing cluster node (requires -c <host>) + +*csync2*:: + Configure csync2 (requires -c <host>) + +*ssh_merge*:: + Merge root's SSH known_hosts across all nodes (csync2 must + already be configured). + +*cluster*:: + Start the cluster on this node + +If stage is not specified, each stage will be invoked in sequence. + +Usage: +............... +join [options] [STAGE] +............... + + +[[cmdhelp_cluster_remove,Remove node(s) from the cluster]] +==== `remove` + +Remove one or more nodes from the cluster. + +This command can remove the last node in the cluster, +thus effectively removing the whole cluster. To remove +the last node, pass `--force` argument to `crm` or set +the `config.core.force` option. + +Options: + +*-q, --quiet*:: + Be quiet (don't describe what's happening, just do it) + +*-y, --yes*:: + Answer "yes" to all prompts (use with caution) + +*-c HOST, --cluster-node=HOST*:: + IP address or hostname of cluster node which will be + removed from the cluster + +Usage: +............... +remove [options] [<node> ...] +............... + + +[[cmdhelp_cluster_run,Execute an arbitrary command on all nodes]] +==== `run` + +This command takes a shell statement as argument, executes that +statement on all nodes in the cluster, and reports the result. + +Usage: +............... +run <command> +............... + +Example: +............... +run "cat /proc/uptime" +............... + +[[cmdhelp_cluster_start,Start cluster services]] +==== `start` + +Starts the cluster-related system services on this node. + +Usage: +......... +start +......... + +[[cmdhelp_cluster_status,Cluster status check]] +==== `status` + +Reports the status for the cluster messaging layer on the local +node. + +Usage: +............... +status +............... + +[[cmdhelp_cluster_stop,Stop cluster services]] +==== `stop` + +Stops the cluster-related system services on this node. + +Usage: +......... +stop +......... + +[[cmdhelp_cluster_wait_for_startup,Wait for cluster to start]] +==== `wait_for_startup` + +Mostly useful in scripts or automated workflows, this command will +attempt to connect to the local cluster node repeatedly. The command +will keep trying until the cluster node responds, or the `timeout` +elapses. The timeout can be changed by supplying a value in seconds as +an argument. + +Usage: +........ +wait_for_startup +........ + +[[cmdhelp_script,Cluster script management]] +=== `script` - Cluster script management + +A big part of the configuration and management of a cluster is +collecting information about all cluster nodes and deploying changes +to those nodes. Often, just performing the same procedure on all nodes +will encounter problems, due to subtle differences in the +configuration. + +For example, when configuring a cluster for the first time, the +software needs to be installed and configured on all nodes before the +cluster software can be launched and configured using `crmsh`. This +process is cumbersome and error-prone, and the goal is for scripts to +make this process easier. + +Scripts are implemented using the python `parallax` package which +provides a thin wrapper on top of SSH. This allows the scripts to +function through the usual SSH channels used for system maintenance, +requiring no additional software to be installed or maintained. + +[[cmdhelp_script_json,JSON API for cluster scripts]] +==== `json` + +This command provides a JSON API for the cluster scripts, intended for +use in user interface tools that want to interact with the cluster via +scripts. + +The command takes a single argument, which should be a JSON array with +the first member identifying the command to perform. + +The output is line-based: Commands that return multiple results will +return them line-by-line, ending with a terminator value: "end". + +When providing parameter values to this command, they should be +provided as nested objects, so +virtual-ip:ip=192.168.0.5+ on the +command line becomes the JSON object ++{"virtual-ip":{"ip":"192.168.0.5"}}+. + +API: +........ +["list"] +=> [{name, shortdesc, category}] + +["show", <name>] +=> [{name, shortdesc, longdesc, category, <<steps>>}] + +<<steps>> := [{name, shortdesc], longdesc, required, parameters, steps}] + +<<params>> := [{name, shortdesc, longdesc, required, unique, advanced, + type, value, example}] + +["verify", <name>, <<values>>] +=> [{shortdesc, longdesc, text, nodes}] + +["run", <name>, <<values>>] +=> [{shortdesc, rc, output|error}] +........ + + +[[cmdhelp_script_list,List available scripts]] +==== `list` + +Lists the available scripts, sorted by category. Scripts that have the +special `Script` category are hidden by default, since they are mainly +used by other scripts or commands. To also show these, pass `all` as +argument. + +To get a flat list of script names, not sorted by category, pass +`names` as an extra argument. + +Usage: +............ +list [all] [names] +............ + +Example: +............ +list +list all names +............ + +[[cmdhelp_script_run,Run the script]] +==== `run` + +Given a list of parameter values, this command will execute the +actions specified by the cluster script. The format for the parameter +values is the same as for the `verify` command. + +Can optionally take at least two parameters: +* `nodes=<nodes>`: List of nodes that the script runs over +* `dry_run=yes|no`: If set, the script will not perform any modifications. + +Additional parameters may be available depending on the script. + +Use the `show` command to see what parameters are available. + +Usage: +............. +run <script> [args...] +............. + +Example: +............. +run apache install=true +run sbd id=sbd-1 node=node1 sbd_device=/dev/disk/by-uuid/F00D-CAFE +............. + +[[cmdhelp_script_show,Describe the script]] +==== `show` + +Prints a description and short summary of the script, with +descriptions of the accepted parameters. + +Advanced parameters are hidden by default. To show the complete list +of parameters accepted by the script, pass `all` as argument. + +Usage: +............ +show <script> [all] +............ + +Example: +............ +show virtual-ip +............ + +[[cmdhelp_script_verify,Verify the script]] +==== `verify` + +Checks the given parameter values, and returns a list +of actions that will be executed when running the script +if provided the same list of parameter values. + +Usage: +............ +verify <script> [args...] +............ + +Example: +............ +verify sbd id=sbd-1 node=node1 sbd_device=/dev/disk/by-uuid/F00D-CAFE +............ + +[[cmdhelp_corosync,Corosync management]] +=== `corosync` - Corosync management + +Corosync is the underlying messaging layer for most HA clusters. +This level provides commands for editing and managing the corosync +configuration. + +[[cmdhelp_corosync_add-node,Add a corosync node]] +==== `add-node` + +Adds a node to the corosync configuration. This is used with the `udpu` +type configuration in corosync. + +A nodeid for the added node is generated automatically. + +Note that this command assumes that only a single ring is used, and +sets only the address for ring0. + +Usage: +......... +add-node <addr> [name] +......... + +[[cmdhelp_corosync_del-node,Remove a corosync node]] +==== `del-node` + +Removes a node from the corosync configuration. The argument given is +the `ring0_addr` address set in the configuration file. + +Usage: +......... +del-node <addr> +......... + +[[cmdhelp_corosync_diff,Diffs the corosync configuration]] +==== `diff` + +Diffs the corosync configurations on different nodes. If no nodes are +given as arguments, the corosync configurations on all nodes in the +cluster are compared. + +`diff` takes an option argument `--checksum`, to display a checksum +for each file instead of calculating a diff. + +Usage: +......... +diff [--checksum] [node...] +......... + +[[cmdhelp_corosync_edit,Edit the corosync configuration]] +==== `edit` + +Opens the Corosync configuration file in an editor. + +Usage: +......... +edit +......... + +[[cmdhelp_corosync_get,Get a corosync configuration value]] +==== `get` + +Returns the value configured in `corosync.conf`, which is not +necessarily the value used in the running configuration. See `reload` +for telling corosync about configuration changes. + +The argument is the complete dot-separated path to the value. + +If there are multiple values configured with the same path, the +command returns all values for that path. For example, to get all +configured `ring0_addr` values, use this command: + +Example: +........ +get nodelist.node.ring0_addr +........ + +[[cmdhelp_corosync_log,Show the corosync log file]] +==== `log` + +Opens the log file specified in the corosync configuration file. If no +log file is configured, this command returns an error. + +The pager used can be configured either using the PAGER +environment variable or in `crm.conf`. + +Usage: +......... +log +......... + +[[cmdhelp_corosync_pull,Pulls the corosync configuration]] +==== `pull` + +Gets the corosync configuration from another node and copies +it to this node. + +Usage: +......... +pull <node> +......... + +[[cmdhelp_corosync_push,Push the corosync configuration]] +==== `push` + +Pushes the corosync configuration file on this node to +the list of nodes provided. If no target nodes are given, +the configuration is pushed to all other nodes in the cluster. + +It is recommended to use `csync2` to distribute the cluster +configuration files rather than relying on this command. + +Usage: +......... +push [node] ... +......... + +Example: +......... +push node-2 node-3 +......... + +[[cmdhelp_corosync_reload,Reload the corosync configuration]] +==== `reload` + +Tells all instances of corosync in this cluster to reload +`corosync.conf`. + +After pushing a new configuration to all cluster nodes, call this +command to make corosync use the new configuration. + +Usage: +......... +reload +......... + +[[cmdhelp_corosync_set,Set a corosync configuration value]] +==== `set` + +Sets the value identified by the given path. If the value does not +exist in the configuration file, it will be added. However, if the +section containing the value does not exist, the command will fail. + +Usage: +......... +set quorum.expected_votes 2 +......... + +[[cmdhelp_corosync_show,Display the corosync configuration]] +==== `show` + +Displays the corosync configuration on the current node. + +......... +show +......... + +[[cmdhelp_corosync_status,Display the corosync status]] +==== `status` + +Displays the status of Corosync, including the votequorum state. + +Usage: +......... +status +......... + +[[cmdhelp_cib,CIB shadow management]] +=== `cib` - CIB shadow management + +This level is for management of shadow CIBs. It is available both +at the top level and the `configure` level. + +All the commands are implemented using `cib_shadow(8)` and the +`CIB_shadow` environment variable. The user prompt always +includes the name of the currently active shadow or the live CIB. + +[[cmdhelp_cib_cibstatus,CIB status management and editing]] +==== `cibstatus` + +Enter edit and manage the CIB status section level. See the +<<cmdhelp_cibstatus,CIB status management section>>. + +[[cmdhelp_cib_commit,copy a shadow CIB to the cluster]] +==== `commit` + +Apply a shadow CIB to the cluster. If the shadow name is omitted +then the current shadow CIB is applied. + +Temporary shadow CIBs are removed automatically on commit. + +Usage: +............... +commit [<cib>] +............... + +[[cmdhelp_cib_delete,delete a shadow CIB]] +==== `delete` + +Delete an existing shadow CIB. + +Usage: +............... +delete <cib> +............... + +[[cmdhelp_cib_diff,diff between the shadow CIB and the live CIB]] +==== `diff` + +Print differences between the current cluster configuration and +the active shadow CIB. + +Usage: +............... +diff +............... + +[[cmdhelp_cib_import,import a CIB or PE input file to a shadow]] +==== `import` + +At times it may be useful to create a shadow file from the +existing CIB. The CIB may be specified as file or as a PE input +file number. The shell will look up files in the local directory +first and then in the PE directory (typically `/var/lib/pengine`). +Once the CIB file is found, it is copied to a shadow and this +shadow is immediately available for use at both `configure` and +`cibstatus` levels. + +If the shadow name is omitted then the target shadow is named +after the input CIB file. + +Note that there are often more than one PE input file, so you may +need to specify the full name. + +Usage: +............... +import {<file>|<number>} [<shadow>] +............... +Examples: +............... +import pe-warn-2222 +import 2289 issue2 +............... + +[[cmdhelp_cib_list,list all shadow CIBs]] +==== `list` + +List existing shadow CIBs. + +Usage: +............... +list +............... + +[[cmdhelp_cib_new,create a new shadow CIB]] +==== `new` + +Create a new shadow CIB. The live cluster configuration and +status is copied to the shadow CIB. + +If the name of the shadow is omitted, we create a temporary CIB +shadow. It is useful if multiple level sessions are desired +without affecting the cluster. A temporary CIB shadow is short +lived and will be removed either on `commit` or on program exit. +Note that if the temporary shadow is not committed all changes in +the temporary shadow are lost. + +Specify `withstatus` if you want to edit the status section of +the shadow CIB (see the <<cmdhelp_cibstatus,cibstatus section>>). +Add `force` to force overwriting the existing shadow CIB. + +To start with an empty configuration that is not copied from the live +CIB, specify the `empty` keyword. (This also allows a shadow CIB to be +created in case no cluster is running.) + +Usage: +............... +new [<cib>] [withstatus] [force] [empty] +............... + +[[cmdhelp_cib_reset,copy live cib to a shadow CIB]] +==== `reset` + +Copy the current cluster configuration into the shadow CIB. + +Usage: +............... +reset <cib> +............... + +[[cmdhelp_cib_use,change working CIB]] +==== `use` + +Choose a CIB source. If you want to edit the status from the +shadow CIB specify `withstatus` (see <<cmdhelp_cibstatus,`cibstatus`>>). +Leave out the CIB name to switch to the running CIB. + +Usage: +............... +use [<cib>] [withstatus] +............... + +[[cmdhelp_ra,Resource Agents (RA) lists and documentation]] +=== `ra` - Resource Agents (RA) lists and documentation + +This level contains commands which show various information about +the installed resource agents. It is available both at the top +level and at the `configure` level. + +[[cmdhelp_ra_classes,list classes and providers]] +==== `classes` + +Print all resource agents' classes and, where appropriate, a list +of available providers. + +Usage: +............... +classes +............... + +[[cmdhelp_ra_info,show meta data for a RA]] +==== `info` (`meta`) + +Show the meta-data of a resource agent type. This is where users +can find information on how to use a resource agent. It is also +possible to get information from some programs: `pengine`, +`crmd`, `cib`, and `stonithd`. Just specify the program name +instead of an RA. + +Usage: +............... +info [<class>:[<provider>:]]<type> +info <type> <class> [<provider>] (obsolete) +............... +Example: +............... +info apache +info ocf:pacemaker:Dummy +info stonith:ipmilan +info pengine +............... + +[[cmdhelp_ra_list,list RA for a class (and provider)]] +==== `list` + +List available resource agents for the given class. If the class +is `ocf`, supply a provider to get agents which are available +only from that provider. + +Usage: +............... +list <class> [<provider>] +............... +Example: +............... +list ocf pacemaker +............... + +[[cmdhelp_ra_providers,show providers for a RA and a class]] +==== `providers` + +List providers for a resource agent type. The class parameter +defaults to `ocf`. + +Usage: +............... +providers <type> [<class>] +............... +Example: +............... +providers apache +............... + +[[cmdhelp_ra_validate,validate parameters for RA]] +==== `validate` + +If the resource agent supports the `validate-all` action, this calls +the action with the given parameters, printing any warnings or errors +reported by the agent. + +Usage: +................ +validate <agent> [<key>=<value> ...] +................ + +[[cmdhelp_resource,Resource management]] +=== `resource` - Resource management + +At this level resources may be managed. + +All (or almost all) commands are implemented with the CRM tools +such as `crm_resource(8)`. + +[[cmdhelp_resource_ban,ban a resource from a node]] +==== `ban` + +Ban a resource from running on a certain node. If no node is given +as argument, the resource is banned from the current location. + +See `move` for details on other arguments. + +Usage: +............... +ban <rsc> [<node>] [<lifetime>] [force] +............... + +[[cmdhelp_resource_cleanup,cleanup resource status]] +==== `cleanup` + +Cleanup resource status. Typically done after the resource has +temporarily failed. If a node is omitted, cleanup on all nodes. +If there are many nodes, the command may take a while. + ++(Pacemaker 1.1.14)+ Pass force to cleanup the resource itself, +otherwise the cleanup command will apply to the parent resource (if +any). + +Usage: +............... +cleanup <rsc> [<node>] [force] +............... + +[[cmdhelp_resource_clear,Clear any relocation constraint]] +==== `clear` (`unmove`, `unmigrate`, `unban`) + +Remove any relocation constraint created by +the `move`, `migrate` or `ban` command. + +Usage: +............... +clear <rsc> +unmigrate <rsc> +unban <rsc> +............... + +[[cmdhelp_resource_constraints,Show constraints affecting a resource]] +==== `constraints` + +Display the location and colocation constraints affecting the +resource. + +Usage: +................ +constraints <rsc> +................ + +[[cmdhelp_resource_demote,demote a master-slave resource]] +==== `demote` + +Demote a master-slave resource using the `target-role` +attribute. + +Usage: +............... +demote <rsc> +............... + +[[cmdhelp_resource_failcount,manage failcounts]] +==== `failcount` + +Show/edit/delete the failcount of a resource. + +Usage: +............... +failcount <rsc> set <node> <value> +failcount <rsc> delete <node> +failcount <rsc> show <node> +............... +Example: +............... +failcount fs_0 delete node2 +............... + +[[cmdhelp_resource_locate,show the location of resources]] +==== `locate` + +Show the current location of one or more resources. + +Usage: +............... +locate [<rsc> ...] +............... + +[[cmdhelp_resource_maintenance,Enable/disable per-resource maintenance mode]] +==== `maintenance` + +Enables or disables the per-resource maintenance mode. When this mode +is enabled, no monitor operations will be triggered for the resource. +`maintenance` attribute conflicts with the `is-managed`. When setting +the `maintenance` attribute, the user is proposed to remove the +`is-managed` attribute if it exists. + +Usage: +.................. +maintenance <resource> [on|off|true|false] +.................. + +Example: +.................. +maintenance rsc1 +maintenance rsc2 off +.................. + +[[cmdhelp_resource_manage,put a resource into managed mode]] +==== `manage` + +Manage a resource using the `is-managed` attribute. If there +are multiple meta attributes sets, the attribute is set in all of +them. If the resource is a clone, all `is-managed` attributes are +removed from the children resources. +`is-managed` attribute conflicts with the `maintenance`. When setting +the `is-managed` attribute, the user is proposed to remove the +`maintenance` attribute if it exists. + +For details on group management see <<cmdhelp_options_manage-children,`options manage-children`>>. + +Usage: +............... +manage <rsc> +............... + +[[cmdhelp_resource_meta,manage a meta attribute]] +==== `meta` + +Show/edit/delete a meta attribute of a resource. Currently, all +meta attributes of a resource may be managed with other commands +such as `resource stop`. + +Usage: +............... +meta <rsc> set <attr> <value> +meta <rsc> delete <attr> +meta <rsc> show <attr> +............... +Example: +............... +meta ip_0 set target-role stopped +............... + +[[cmdhelp_resource_move,Move a resource to another node]] +==== `move` (`migrate`) + +Move a resource away from its current location. + +If the destination node is left out, the resource is migrated by +creating a constraint which prevents it from running on the current +node. For this type of constraint to be created, the +force+ argument +is required. + +A lifetime may be given for the constraint. Once it expires, the +location constraint will no longer be active. + +Usage: +............... +move <rsc> [<node>] [<lifetime>] [force] +............... + +[[cmdhelp_resource_operations,Show active resource operations]] +==== `operations` + +Show active operations, optionally filtered by resource and node. + +Usage: +................ +operations [<rsc>] [<node>] +................ + +[[cmdhelp_resource_param,manage a parameter of a resource]] +==== `param` + +Show/edit/delete a parameter of a resource. + +Usage: +............... +param <rsc> set <param> <value> +param <rsc> delete <param> +param <rsc> show <param> +............... +Example: +............... +param ip_0 show ip +............... + +[[cmdhelp_resource_promote,promote a master-slave resource]] +==== `promote` + +Promote a master-slave resource using the `target-role` +attribute. + +Usage: +............... +promote <rsc> +............... + +[[cmdhelp_resource_refresh,refresh CIB from the LRM status]] +==== `refresh` + +Refresh CIB from the LRM status. + +.Note +**************************** +`refresh` has been deprecated and is now +an alias for `cleanup`. +**************************** + +Usage: +............... +refresh [<node>] +............... + +[[cmdhelp_resource_reprobe,probe for resources not started by the CRM]] +==== `reprobe` + +Probe for resources not started by the CRM. + +.Note +**************************** +`reprobe` has been deprecated and is now +an alias for `cleanup`. +**************************** + +Usage: +............... +reprobe [<node>] +............... + +[[cmdhelp_resource_restart,restart resources]] +==== `restart` + +Restart one or more resources. This is essentially a shortcut for +resource stop followed by a start. The shell is first going to wait +for the stop to finish, that is for all resources to really stop, and +only then to order the start action. Due to this command +entailing a whole set of operations, informational messages are +printed to let the user see some progress. + +For details on group management see +<<cmdhelp_options_manage-children,`options manage-children`>>. + +Usage: +............... +restart <rsc> [<rsc> ...] +............... +Example: +............... +# crm resource restart g_webserver +INFO: ordering g_webserver to stop +waiting for stop to finish .... done +INFO: ordering g_webserver to start +# +............... + +[[cmdhelp_resource_scores,Display resource scores]] +==== `scores` + +Display the allocation scores for all resources. + +Usage: +................ +scores +................ + +[[cmdhelp_resource_secret,manage sensitive parameters]] +==== `secret` + +Sensitive parameters can be kept in local files rather than CIB +in order to prevent accidental data exposure. Use the `secret` +command to manage such parameters. `stash` and `unstash` move the +value from the CIB and back to the CIB respectively. The `set` +subcommand sets the parameter to the provided value. `delete` +removes the parameter completely. `show` displays the value of +the parameter from the local file. Use `check` to verify if the +local file content is valid. + +Usage: +............... +secret <rsc> set <param> <value> +secret <rsc> stash <param> +secret <rsc> unstash <param> +secret <rsc> delete <param> +secret <rsc> show <param> +secret <rsc> check <param> +............... +Example: +............... +secret fence_1 show password +secret fence_1 stash password +secret fence_1 set password secret_value +............... + +[[cmdhelp_resource_start,start resources]] +==== `start` + +Start one or more resources by setting the `target-role` attribute. If +there are multiple meta attributes sets, the attribute is set in all +of them. If the resource is a clone, all `target-role` attributes are +removed from the children resources. + +For details on group management see +<<cmdhelp_options_manage-children,`options manage-children`>>. + +Usage: +............... +start <rsc> [<rsc> ...] +............... + +[[cmdhelp_resource_status,show status of resources]] +==== `status` (`show`, `list`) + +Print resource status. More than one resource can be shown at once. If +the resource parameter is left out, the status of all resources is +printed. + +Usage: +............... +status [<rsc> ...] +............... + +[[cmdhelp_resource_stop,stop resources]] +==== `stop` + +Stop one or more resources using the `target-role` attribute. If there +are multiple meta attributes sets, the attribute is set in all of +them. If the resource is a clone, all `target-role` attributes are +removed from the children resources. + +For details on group management see +<<cmdhelp_options_manage-children,`options manage-children`>>. + +Usage: +............... +stop <rsc> [<rsc> ...] +............... + +[[cmdhelp_resource_trace,start RA tracing]] +==== `trace` + +Start tracing RA for the given operation. The trace files are +stored in `$HA_VARLIB/trace_ra`. If the operation to be traced is +monitor, note that the number of trace files can grow very +quickly. + +If no operation name is given, crmsh will attempt to trace all +operations for the RA. This includes any configured operations, start +and stop as well as promote/demote for multistate resources. + +To trace the probe operation which exists for all resources, either +set a trace for `monitor` with interval `0`, or use `probe` as the +operation name. + +Usage: +............... +trace <rsc> [<op> [<interval>] ] +............... +Example: +............... +trace fs start +trace webserver +trace webserver probe +trace fs monitor 0 +............... + +[[cmdhelp_resource_unmanage,put a resource into unmanaged mode]] +==== `unmanage` + +Unmanage a resource using the `is-managed` attribute. If there +are multiple meta attributes sets, the attribute is set in all of +them. If the resource is a clone, all `is-managed` attributes are +removed from the children resources. + +For details on group management see <<cmdhelp_options_manage-children,`options manage-children`>>. + +Usage: +............... +unmanage <rsc> +............... + +[[cmdhelp_resource_untrace,stop RA tracing]] +==== `untrace` + +Stop tracing RA for the given operation. If no operation name is +given, crmsh will attempt to stop tracing all operations in resource. + +Usage: +............... +untrace <rsc> [<op> [<interval>] ] +............... +Example: +............... +untrace fs start +untrace webserver +............... + +[[cmdhelp_resource_utilization,manage a utilization attribute]] +==== `utilization` + +Show/edit/delete a utilization attribute of a resource. These +attributes describe hardware requirements. By setting the +`placement-strategy` cluster property appropriately, it is +possible then to distribute resources based on resource +requirements and node size. See also <<cmdhelp_node_utilization,node utilization attributes>>. + +Usage: +............... +utilization <rsc> set <attr> <value> +utilization <rsc> delete <attr> +utilization <rsc> show <attr> +............... +Example: +............... +utilization xen1 set memory 4096 +............... + +[[cmdhelp_node,Node management]] +=== `node` - Node management + +Node management and status commands. + +[[cmdhelp_node_attribute,manage attributes]] +==== `attribute` + +Edit node attributes. This kind of attribute should refer to +relatively static properties, such as memory size. + +Usage: +............... +attribute <node> set <attr> <value> +attribute <node> delete <attr> +attribute <node> show <attr> +............... +Example: +............... +attribute node_1 set memory_size 4096 +............... + +[[cmdhelp_node_clearstate,Clear node state]] +==== `clearstate` + +Resets and clears the state of the specified node. This node is +afterwards assumed clean and offline. This command can be used to +manually confirm that a node has been fenced (e.g., powered off). + +Be careful! This can cause data corruption if you confirm that a node is +down that is, in fact, not cleanly down - the cluster will proceed as if +the fence had succeeded, possibly starting resources multiple times. + +Usage: +............... +clearstate <node> +............... + +[[cmdhelp_node_delete,delete node]] +==== `delete` + +Delete a node. This command will remove the node from the CIB +and, in case the cluster stack is running, use the appropriate +program (`crm_node` or `hb_delnode`) to remove the node from the +membership. + +If the node is still listed as active and a member of our +partition we refuse to remove it. With the global force option +(`-F`) we will try to delete the node anyway. + +Usage: +............... +delete <node> +............... + +[[cmdhelp_node_fence,fence node]] +==== `fence` + +Make CRM fence a node. This functionality depends on stonith +resources capable of fencing the specified node. No such stonith +resources, no fencing will happen. + +Usage: +............... +fence <node> +............... + +[[cmdhelp_node_maintenance,put node into maintenance mode]] +==== `maintenance` + +Set the node status to maintenance. This is equivalent to the +cluster-wide `maintenance-mode` property but puts just one node +into the maintenance mode. If there are maintenaned resources on +the node, the user will be proposed to remove the maintenance +property from them. + +The node parameter defaults to the node where the command is run. + +Usage: +............... +maintenance [<node>] +............... + +[[cmdhelp_node_online,set node online]] +==== `online` + +Set a node to online status. + +The node parameter defaults to the node where the command is run. + +Usage: +............... +online [<node>] +............... + +[[cmdhelp_node_ready,put node into ready mode]] +==== `ready` + +Set the node's maintenance status to `off`. The node should be +now again fully operational and capable of running resource +operations. + +The node parameter defaults to the node where the command is run. + +Usage: +............... +ready [<node>] +............... + +[[cmdhelp_node_server,show node hostname or server address]] +==== `server` + +Remote nodes may have a configured server address which should +be used when contacting the node. This command prints the +server address if configured, else the node name. + +If no parameter is given, the addresses or names for all nodes +are printed. + +Usage: +............... +server [<node> ...] +............... + +[[cmdhelp_node_show,show node]] +==== `show` + +Show a node definition. If the node parameter is omitted then all +nodes are shown. + +Usage: +............... +show [<node>] +............... + +[[cmdhelp_node_standby,put node into standby]] +==== `standby` + +Set a node to standby status. The node parameter defaults to the +node where the command is run. + +Additionally, you may specify a lifetime for the standby---if set to +`reboot`, the node will be back online once it reboots. `forever` will +keep the node in standby after reboot. The life time defaults to +`forever`. + +Usage: +............... +standby [<node>] [<lifetime>] + +lifetime :: reboot | forever +............... + +Example: +............... +standby bob reboot +............... + + +[[cmdhelp_node_status,show nodes' status as XML]] +==== `status` + +Show nodes' status as XML. If the node parameter is omitted then +all nodes are shown. + +Usage: +............... +status [<node>] +............... + +[[cmdhelp_node_status-attr,manage status attributes]] +==== `status-attr` + +Edit node attributes which are in the CIB status section, i.e. +attributes which hold properties of a more volatile nature. One +typical example is attribute generated by the `pingd` utility. + +Usage: +............... +status-attr <node> set <attr> <value> +status-attr <node> delete <attr> +status-attr <node> show <attr> +............... +Example: +............... +status-attr node_1 show pingd +............... + +[[cmdhelp_node_utilization,manage utilization attributes]] +==== `utilization` + +Edit node utilization attributes. These attributes describe +hardware characteristics as integer numbers such as memory size +or the number of CPUs. By setting the `placement-strategy` +cluster property appropriately, it is possible then to distribute +resources based on resource requirements and node size. See also +<<cmdhelp_resource_utilization,resource utilization attributes>>. + +Usage: +............... +utilization <node> set <attr> <value> +utilization <node> delete <attr> +utilization <node> show <attr> +............... +Examples: +............... +utilization node_1 set memory 16384 +utilization node_1 show cpu +............... + +[[cmdhelp_site,GEO clustering site support]] +=== `site` - GEO clustering site support + +A cluster may consist of two or more subclusters in different and +distant locations. This set of commands supports such setups. + +[[cmdhelp_site_ticket,manage site tickets]] +==== `ticket` + +Tickets are cluster-wide attributes. They can be managed at the +site where this command is executed. + +It is then possible to constrain resources depending on the +ticket availability (see the <<cmdhelp_configure_rsc_ticket,`rsc_ticket`>> command +for more details). + +Usage: +............... +ticket {grant|revoke|standby|activate|show|time|delete} <ticket> +............... +Example: +............... +ticket grant ticket1 +............... + +[[cmdhelp_options,User preferences]] +=== `options` - User preferences + +The user may set various options for the crm shell itself. + +[[cmdhelp_options_add-quotes,add quotes around parameters containing spaces]] +==== `add-quotes` + +The shell (as in `/bin/sh`) parser strips quotes from the command +line. This may sometimes make it really difficult to type values +which contain white space. One typical example is the configure +filter command. The crm shell will supply extra quotes around +arguments which contain white space. The default is `yes`. + +.Note on quotes use +**************************** +Adding quotes around arguments automatically has been introduced +with version 1.2.2 and it is technically a regression. Being a +regression is the only reason the `add-quotes` option exists. If +you have custom shell scripts which would break, just set the +`add-quotes` option to `no`. + +For instance, with adding quotes enabled, it is possible to do +the following: +............... +# crm configure primitive d1 Dummy \ + meta description="some description here" +# crm configure filter 'sed "s/hostlist=./&node-c /"' fencing +............... +**************************** + +[[cmdhelp_options_check-frequency,when to perform semantic check]] +==== `check-frequency` + +Semantic check of the CIB or elements modified or created may be +done on every configuration change (`always`), when verifying +(`on-verify`) or `never`. It is by default set to `always`. +Experts may want to change the setting to `on-verify`. + +The checks require that resource agents are present. If they are +not installed at the configuration time set this preference to +`never`. + +See <<topics_Features_Checks,Configuration semantic checks>> for more details. + +[[cmdhelp_options_check-mode,how to treat semantic errors]] +==== `check-mode` + +Semantic check of the CIB or elements modified or created may be +done in the `strict` mode or in the `relaxed` mode. In the former +certain problems are treated as configuration errors. In the +`relaxed` mode all are treated as warnings. The default is `strict`. + +See <<topics_Features_Checks,Configuration semantic checks>> for more details. + +[[cmdhelp_options_colorscheme,set colors for output]] +==== `colorscheme` + +With `output` set to `color`, a comma separated list of colors +from this option are used to emphasize: + +- keywords +- object ids +- attribute names +- attribute values +- scores +- resource references + +`crm` can show colors only if there is curses support for python +installed (usually provided by the `python-curses` package). The +colors are whatever is available in your terminal. Use `normal` +if you want to keep the default foreground color. + +This user preference defaults to +`yellow,normal,cyan,red,green,magenta` which is good for +terminals with dark background. You may want to change the color +scheme and save it in the preferences file for other color +setups. + +Example: +............... +colorscheme yellow,normal,blue,red,green,magenta +............... + +[[cmdhelp_options_editor,set preferred editor program]] +==== `editor` + +The `edit` command invokes an editor. Use this to specify your +preferred editor program. If not set, it will default to either +the value of the `EDITOR` environment variable or to one of the +standard UNIX editors (`vi`,`emacs`,`nano`). + +Usage: +............... +editor program +............... +Example: +............... +editor vim +............... + +[[cmdhelp_options_manage-children,how to handle children resource attributes]] +==== `manage-children` + +Some resource management commands, such as `resource stop`, when +the target resource is a group, may not always produce desired +result. Each element, group and the primitive members, can have a +meta attribute and those attributes may end up with conflicting +values. Consider the following construct: +............... +crm(live)# configure show svc fs virtual-ip +primitive fs Filesystem \ + params device="/dev/drbd0" directory="/srv/nfs" fstype=ext3 \ + op monitor interval=10s \ + meta target-role=Started +primitive virtual-ip IPaddr2 \ + params ip=10.2.13.110 iflabel=1 \ + op monitor interval=10s \ + op start interval=0 \ + meta target-role=Started +group svc fs virtual-ip \ + meta target-role=Stopped +............... + +Even though the element +svc+ should be stopped, the group is +actually running because all its members have the +target-role+ +set to +Started+: +............... +crm(live)# resource show svc +resource svc is running on: xen-f +............... + +Hence, if the user invokes +resource stop svc+ the intention is +not clear. This preference gives the user an opportunity to +better control what happens if attributes of group members have +values which are in conflict with the same attribute of the group +itself. + +Possible values are +ask+ (the default), +always+, and +never+. +If set to +always+, the crm shell removes all children attributes +which have values different from the parent. If set to +never+, +all children attributes are left intact. Finally, if set to ++ask+, the user will be asked for each member what is to be done. + +[[cmdhelp_options_output,set output type]] +==== `output` + +`crm` can adorn configurations in two ways: in color (similar to +for instance the `ls --color` command) and by showing keywords in +upper case. Possible values are `plain`, `color-always`, `color`, +and 'uppercase'. It is possible to combine `uppercase` with one +of the color values in order to get an upper case xmass tree. Just +set this option to `color,uppercase` or `color-always,uppercase`. +In case you need color codes in pipes, `color-always` forces color +codes even in case the terminal is not a tty (just like `ls +--color=always`). + +[[cmdhelp_options_pager,set preferred pager program]] +==== `pager` + +The `view` command displays text through a pager. Use this to +specify your preferred pager program. If not set, it will default +to either the value of the `PAGER` environment variable or to one +of the standard UNIX system pagers (`less`,`more`,`pg`). + +[[cmdhelp_options_reset,reset user preferences to factory defaults]] +==== `reset` + +This command resets all user options to the defaults. If used as +a single-shot command, the rc file (+$HOME/.config/crm/rc+) is +reset to the defaults too. + +[[cmdhelp_options_save,save the user preferences to the rc file]] +==== `save` + +Save current settings to the rc file (+$HOME/.config/crm/rc+). On +further `crm` runs, the rc file is automatically read and parsed. + +[[cmdhelp_options_set,Set the value of a given option]] +==== `set` + +Sets the value of an option. Takes the fully qualified +name of the option as argument, as displayed by +show all+. + +The modified option value is stored in the user-local +configuration file, usually found in +~/.config/crm/crm.conf+. + +Usage: +........ +set <option> <value> +........ + +Example: +........ +set color.warn "magenta bold" +set editor nano +........ + +[[cmdhelp_options_show,show current user preference]] +==== `show` + +Display all current settings. + +Given an option name as argument, `show` will display only the value +of that argument. + +Given +all+ as argument, `show` displays all available user options. + +Usage: +........ +show [all|<option>] +........ + +Example: +........ +show +show skill-level +show all +........ + +[[cmdhelp_options_skill-level,set skill level]] +==== `skill-level` + +Based on the skill-level setting, the user is allowed to use only +a subset of commands. There are three levels: operator, +administrator, and expert. The operator level allows only +commands at the `resource` and `node` levels, but not editing +or deleting resources. The administrator may do that and may also +configure the cluster at the `configure` level and manage the +shadow CIBs. The expert may do all. + +Usage: +............... +skill-level <level> + +level :: operator | administrator | expert +............... + +.Note on security +**************************** +The `skill-level` option is advisory only. There is nothing +stopping any users change their skill level (see +<<topics_Features_Security,Access Control Lists (ACL)>> on how to enforce +access control). +**************************** + +[[cmdhelp_options_sort-elements,sort CIB elements]] +==== `sort-elements` + +`crm` by default sorts CIB elements. If you want them appear in +the order they were created, set this option to `no`. + +Usage: +............... +sort-elements {yes|no} +............... +Example: +............... +sort-elements no +............... + +[[cmdhelp_options_user,set the cluster user]] +==== `user` + +Sufficient privileges are necessary in order to manage a +cluster: programs such as `crm_verify` or `crm_resource` and, +ultimately, `cibadmin` have to be run either as `root` or as the +CRM owner user (typically `hacluster`). You don't have to worry +about that if you run `crm` as `root`. A more secure way is to +run the program with your usual privileges, set this option to +the appropriate user (such as `hacluster`), and setup the +`sudoers` file. + +Usage: +............... +user system-user +............... +Example: +............... +user hacluster +............... + +[[cmdhelp_options_wait,synchronous operation]] +==== `wait` + +In normal operation, `crm` runs a command and gets back +immediately to process other commands or get input from the user. +With this option set to `yes` it will wait for the started +transition to finish. In interactive mode dots are printed to +indicate progress. + +Usage: +............... +wait {yes|no} +............... +Example: +............... +wait yes +............... + +[[cmdhelp_configure,CIB configuration]] +=== `configure` - CIB configuration + +This level enables all CIB object definition commands. + +The configuration may be logically divided into four parts: +nodes, resources, constraints, and (cluster) properties and +attributes. Each of these commands support one or more basic CIB +objects. + +Nodes and attributes describing nodes are managed using the +`node` command. + +Commands for resources are: + +- `primitive` +- `monitor` +- `group` +- `clone` +- `ms`/`master` (master-slave) + +In order to streamline large configurations, it is possible to +define a template which can later be referenced in primitives: + +- `rsc_template` + +In that case the primitive inherits all attributes defined in the +template. + +There are three types of constraints: + +- `location` +- `colocation` +- `order` + +It is possible to define fencing order (stonith resource +priorities): + +- `fencing_topology` + +Finally, there are the cluster properties, resource meta +attributes defaults, and operations defaults. All are just a set +of attributes. These attributes are managed by the following +commands: + +- `property` +- `rsc_defaults` +- `op_defaults` + +In addition to the cluster configuration, the Access Control +Lists (ACL) can be setup to allow access to parts of the CIB for +users other than +root+ and +hacluster+. The following commands +manage ACL: + +- `user` +- `role` + +In Pacemaker 1.1.12 and up, this command replaces the `user` command +for handling ACLs: + +- `acl_target` + +The changes are applied to the current CIB only on ending the +configuration session or using the `commit` command. + +Comments start with +#+ in the first line. The comments are tied +to the element which follows. If the element moves, its comments +will follow. + +[[cmdhelp_configure_acl_target,Define target access rights]] +==== `acl_target` + +Defines an ACL target. + +Usage: +................ +acl_target <tid> [<role> ...] +................ +Example: +................ +acl_target joe resource_admin constraint_editor +................ + +[[cmdhelp_configure_alert,Event-driven alerts]] +==== `alert` + +.Version note +**************************** +This feature is only available +in Pacemaker 1.1.15+. +**************************** + +Event-driven alerts enables calling scripts whenever interesting +events occur in the cluster (nodes joining or leaving, resources +starting or stopping, etc.). + +The +path+ is an arbitrary file path to an alert script. Existing +external scripts used with ClusterMon resources can be used as alert +scripts, since the interface is compatible. + +Each alert may have a number of receipients configured. These will be +passed to the script as arguments. The first recipient will also be +passed as the +CRM_alert_recipient+ environment variable, for +compatibility with existing scripts that only support one recipient. + +The available meta attributes are +timeout+ (default 30s) and ++timestamp-format+ (default `"%H:%M:%S.%06N"`). + +Some configurations may require each recipient to be delimited by +brackets, to avoid ambiguity. In the example +alert-2+ below, the meta +attribute for `timeout` is defined after the recipient, so the +brackets are used to ensure that the meta attribute is set for the +alert and not just the recipient. This can be avoided by setting any +alert attributes before defining the recipients. + +Usage: +............... +alert <id> <path> \ + [attributes <nvpair> ...] \ + [meta <nvpair> ...] \ + [to [{] <recipient> + [attributes <nvpair> ...] \ + [meta <nvpair> ...] [}] \ + ...] +............... + +Example: +............... +alert alert-1 /srv/pacemaker/pcmk_alert_sample.sh \ + to /var/log/cluster-alerts.log + +alert alert-2 /srv/pacemaker/example_alert.sh \ + meta timeout=60s \ + to { /var/log/cluster-alerts.log } +............... + +[[cmdhelp_configure_cib,CIB shadow management]] +==== `cib` + +This level is for management of shadow CIBs. It is available at +the `configure` level to enable saving intermediate changes to a +shadow CIB instead of to the live cluster. This short excerpt +shows how: +............... +crm(live)configure# cib new test-2 +INFO: test-2 shadow CIB created +crm(test-2)configure# commit +............... +Note how the current CIB in the prompt changed from +live+ to ++test-2+ after issuing the `cib new` command. See also the +<<cmdhelp_cib,CIB shadow management>> for more information. + +[[cmdhelp_configure_cibstatus,CIB status management and editing]] +==== `cibstatus` + +Enter edit and manage the CIB status section level. See the +<<cmdhelp_cibstatus,CIB status management section>>. + +[[cmdhelp_configure_clone,define a clone]] +==== `clone` + +The `clone` command creates a resource clone. It may contain a +single primitive resource or one group of resources. + +Usage: +............... +clone <name> <rsc> + [description=<description>] + [meta <attr_list>] + [params <attr_list>] + +attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id> +............... +Example: +............... +clone cl_fence apc_1 \ + meta clone-node-max=1 globally-unique=false +............... + +[[cmdhelp_configure_colocation,colocate resources]] +==== `colocation` (`collocation`) + +This constraint expresses the placement relation between two +or more resources. If there are more than two resources, then the +constraint is called a resource set. + +The score is used to indicate the priority of the constraint. A +positive score indicates that the resources should run on the same +node. A negative score that they should not run on the same +node. Values of positive or negative +infinity+ indicate a mandatory +constraint. + +In the two resource form, the cluster will place +<with-rsc>+ first, +and then decide where to put the +<rsc>+ resource. + +Collocation resource sets have an extra attribute (+sequential+) +to allow for sets of resources which don't depend on each other +in terms of state. The shell syntax for such sets is to put +resources in parentheses. + +Sets cannot be nested. + +The optional +node-attribute+ can be used to colocate resources on a +set of nodes and not necessarily on the same node. For example, by +setting a node attribute +color+ on all nodes and setting the ++node-attribute+ value to +color+ as well, the colocated resources +will be placed on any node that has the same color. + +For more details on how to configure resource sets, see +<<topics_Features_Resourcesets,`Syntax: Resource sets`>>. + +Usage: +............... +colocation <id> <score>: <rsc>[:<role>] <with-rsc>[:<role>] + [node-attribute=<node_attr>] + +colocation <id> <score>: <resource_sets> + [node-attribute=<node_attr>] + +resource_sets :: <resource_set> [<resource_set> ...] + +resource_set :: ["("|"["] <rsc>[:<role>] [<rsc>[:<role>] ...] \ + [<attributes>] [")"|"]"] + +attributes :: [require-all=(true|false)] [sequential=(true|false)] + +............... +Example: +............... +colocation never_put_apache_with_dummy -inf: apache dummy +colocation c1 inf: A ( B C ) +............... + +[[cmdhelp_configure_commit,commit the changes to the CIB]] +==== `commit` + +Commit the current configuration to the CIB in use. As noted +elsewhere, commands in a configure session don't have immediate +effect on the CIB. All changes are applied at one point in time, +either using `commit` or when the user leaves the configure +level. In case the CIB in use changed in the meantime, presumably +by somebody else, the crm shell will refuse to apply the changes. + +If you know that it's fine to still apply them, add +force+ to the +command line. + +To disable CIB patching and apply the changes by replacing the CIB +completely, add +replace+ to the command line. Note that this can lead +to previous changes being overwritten if some other process +concurrently modifies the CIB. + +Usage: +............... +commit [force] [replace] +............... + +[[cmdhelp_configure_default-timeouts,set timeouts for operations to minimums from the meta-data]] +==== `default-timeouts` + +This command takes the timeouts from the actions section of the +resource agent meta-data and sets them for the operations of the +primitive. + +Usage: +............... +default-timeouts <id> [<id>...] +............... + +.Note on `default-timeouts` +**************************** +The use of this command is discouraged in favor of manually +determining the best timeouts required for the particular +configuration. Relying on the resource agent to supply appropriate +timeouts can cause the resource to fail at the worst possible moment. + +Appropriate timeouts for resource actions are context-sensitive, and +should be carefully considered with the whole configuration in mind. +**************************** + +[[cmdhelp_configure_delete,delete CIB objects]] +==== `delete` + +Delete one or more objects. If an object to be deleted belongs to +a container object, such as a group, and it is the only resource +in that container, then the container is deleted as well. Any +related constraints are removed as well. + +If the object is a started resource, it will not be deleted unless the ++--force+ flag is passed to the command, or the +force+ option is set. + +Usage: +............... +delete [--force] <id> [<id>...] +............... + +[[cmdhelp_configure_edit,edit CIB objects]] +==== `edit` + +This command invokes the editor with the object description. As +with the `show` command, the user may choose to edit all objects +or a set of objects. + +If the user insists, he or she may edit the XML edition of the +object. If you do that, don't modify any id attributes. + +Usage: +............... +edit [xml] [<id> ...] +edit [xml] changed +............... + +.Note on renaming element ids +**************************** +The edit command sometimes cannot properly handle modifying +element ids. In particular for elements which belong to group or +ms resources. Group and ms resources themselves also cannot be +renamed. Please use the `rename` command instead. +**************************** + +[[cmdhelp_configure_erase,erase the CIB]] +==== `erase` + +The `erase` clears all configuration. Apart from nodes. To remove +nodes, you have to specify an additional keyword `nodes`. + +Note that removing nodes from the live cluster may have some +strange/interesting/unwelcome effects. + +Usage: +............... +erase [nodes] +............... + +[[cmdhelp_configure_fencing_topology,node fencing order]] +==== `fencing_topology` + +If multiple fencing (stonith) devices are available capable of +fencing a node, their order may be specified by +fencing_topology+. +The order is specified per node. + +Stonith resources can be separated by +,+ in which case all of +them need to succeed. If they fail, the next stonith resource (or +set of resources) is used. In other words, use comma to separate +resources which all need to succeed and whitespace for serial +order. It is not allowed to use whitespace around comma. + +If the node is left out, the order is used for all nodes. +That should reduce the configuration size in some stonith setups. + +From Pacemaker version 1.1.14, it is possible to use a node attribute +as the +target+ in a fencing topology. The syntax for this usage is +described below. + +From Pacemaker version 1.1.14, it is also possible to use regular +expression patterns as the +target+ in a fencing topology. The configured +fencing sequence then applies to all devices matching the pattern. + +Usage: +............... +fencing_topology <stonith_resources> [<stonith_resources> ...] +fencing_topology <fencing_order> [<fencing_order> ...] + +fencing_order :: <target> <stonith_resources> [<stonith_resources> ...] + +stonith_resources :: <rsc>[,<rsc>...] +target :: <node>: | attr:<node-attribute>=<value> | pattern:<pattern> +............... +Example: +............... +# Only kill the power if poison-pill fails +fencing_topology poison-pill power + +# As above for node-a, but a different strategy for node-b +fencing_topology \ + node-a: poison-pill power \ + node-b: ipmi serial + +# Fencing anything on rack 1 requires fencing via both APC 1 and 2, +# to defeat the redundancy provided by two separate UPS units. +fencing_topology attr:rack=1 apc01,apc02 + +# Fencing for all machines named green.* is done using the pear +# fencing device first, while all machines named red.* are fenced +# using the apple fencing device first. +fencing_topology \ + pattern:green.* pear apple \ + pattern:red.* apple pear +............... + +[[cmdhelp_configure_filter,filter CIB objects]] +==== `filter` + +This command filters the given CIB elements through an external +program. The program should accept input on `stdin` and send +output to `stdout` (the standard UNIX filter conventions). As +with the `show` command, the user may choose to filter all or +just a subset of elements. + +It is possible to filter the XML representation of objects, but +probably not as useful as the configuration language. The +presentation is somewhat different from what would be displayed +by the `show` command---each element is shown on a single line, +i.e. there are no backslashes and no other embelishments. + +Don't forget to put quotes around the filter if it contains +spaces. + +Usage: +............... +filter <prog> [xml] [<id> ...] +filter <prog> [xml] changed +............... +Examples: +............... +filter "sed '/^primitive/s/target-role=[^ ]*//'" +# crm configure filter "sed '/^primitive/s/target-role=[^ ]*//'" +crm configure <<END + filter "sed '/threshold=\"1\"/s/=\"1\"/=\"0\"/g'" +END +............... + +.Note on quotation marks +************************** +Filter commands which feature a blend of quotation marks can be +difficult to get right, especially when used directly from bash, since +bash does its own quotation parsing. In these cases, it can be easier +to supply the filter command as standard input. See the last example +above. +************************** + +[[cmdhelp_configure_get_property,Get property value]] +==== `get-property` + +Show the value of the given property. If the value is not set, the +command will print the default value for the property, if known. + +If no property name is passed to the command, the list of known +cluster properties is printed. + +If the property is set multiple times, for example using multiple +property sets with different rule expressions, the output of this +command is undefined. + +Pass the argument +-t+ or +--true+ to `get-property` to translate +the argument value into +true+ or +false+. If the value is not +set, the command will print +false+. + +Usage: +............... +get-property [-t|--true] [<name>] +............... + +Example: +............... +get-property stonith-enabled +get-property -t maintenance-mode +............... + +[[cmdhelp_configure_graph,generate a directed graph]] +==== `graph` + +Create a graphviz graphical layout from the current cluster +configuration. + +Currently, only `dot` (directed graph) is supported. It is +essentially a visualization of resource ordering. + +The graph may be saved to a file which can be used as source for +various graphviz tools (by default it is displayed in the user's +X11 session). Optionally, by specifying the format, one can also +produce an image instead. + +For more or different graphviz attributes, it is possible to save +the default set of attributes to an ini file. If this file exists +it will always override the builtin settings. The +exportsettings+ +subcommand also prints the location of the ini file. + +Usage: +............... +graph [<gtype> [<file> [<img_format>]]] +graph exportsettings + +gtype :: dot +img_format :: `dot` output format (see the +-T+ option) +............... +Example: +............... +graph dot +graph dot clu1.conf.dot +graph dot clu1.conf.svg svg +............... + +[[cmdhelp_configure_group,define a group]] +==== `group` + +The `group` command creates a group of resources. This can be useful +when resources depend on other resources and require that those +resources start in order on the same node. A common use of resource +groups is to ensure that a server and a virtual IP are located +together, and that the virtual IP is started before the server. + +Grouped resources are started in the order they appear in the group, +and stopped in the reverse order. If a resource in the group cannot +run anywhere, resources following it in the group will not start. + +`group` can be passed the "container" meta attribute, to indicate that +it is to be used to group VM resources monitored using Nagios. The +resource referred to by the container attribute must be of type +`ocf:heartbeat:Xen`, `ocf:heartbeat:VirtualDomain` or `ocf:heartbeat:lxc`. + +Usage: +............... +group <name> <rsc> [<rsc>...] + [description=<description>] + [meta attr_list] + [params attr_list] + +attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id> +............... +Example: +............... +group internal_www disk0 fs0 internal_ip apache \ + meta target_role=stopped + +group vm-and-services vm vm-sshd meta container="vm" +............... + +[[cmdhelp_configure_load,import the CIB from a file]] +==== `load` + +Load a part of configuration (or all of it) from a local file or +a network URL. The +replace+ method replaces the current +configuration with the one from the source. The +update+ method +tries to import the contents into the current configuration. The ++push+ method imports the contents into the current configuration +and removes any lines that are not present in the given +configuration. +The file may be a CLI file or an XML file. + +If the URL is `-`, the configuration is read from standard input. + +Usage: +............... +load [xml] <method> URL + +method :: replace | update | push +............... +Example: +............... +load xml update myfirstcib.xml +load xml replace http://storage.big.com/cibs/bigcib.xml +load xml push smallcib.xml +............... + +[[cmdhelp_configure_location,a location preference]] +==== `location` + +`location` defines the preference of nodes for the given +resource. The location constraints consist of one or more rules +which specify a score to be awarded if the rule matches. + +The resource referenced by the location constraint can be one of the +following: + +* Plain resource reference: +location loc1 webserver 100: node1+ +* Resource set in curly brackets: +location loc1 { virtual-ip webserver } 100: node1+ +* Tag containing resource ids: +location loc1 tag1 100: node1+ +* Resource pattern: +location loc1 /web.*/ 100: node1+ + +The +resource-discovery+ attribute allows probes to be selectively +enabled or disabled per resource and node. + +The syntax for resource sets is described in detail for +<<cmdhelp_configure_colocation,`colocation`>>. + +For more details on how to configure resource sets, see +<<topics_Features_Resourcesets,`Syntax: Resource sets`>>. + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +location <id> <rsc> [<attributes>] {<node_pref>|<rules>} + +rsc :: /<rsc-pattern>/ + | { resource_sets } + | <rsc> + +attributes :: role=<role> | resource-discovery=always|never|exclusive + +node_pref :: <score>: <node> + +rules :: + rule [id_spec] [$role=<role>] <score>: <expression> + [rule [id_spec] [$role=<role>] <score>: <expression> ...] + +id_spec :: $id=<id> | $id-ref=<id> +score :: <number> | <attribute> | [-]inf +expression :: <simple_exp> [<bool_op> <simple_exp> ...] +bool_op :: or | and +simple_exp :: <attribute> [type:]<binary_op> <value> + | <unary_op> <attribute> + | date <date_expr> +type :: string | version | number +binary_op :: lt | gt | lte | gte | eq | ne +unary_op :: defined | not_defined + +date_expr :: lt <end> + | gt <start> + | in start=<start> end=<end> + | in start=<start> <duration> + | spec <date_spec> +duration|date_spec :: + hours=<value> + | monthdays=<value> + | weekdays=<value> + | yearsdays=<value> + | months=<value> + | weeks=<value> + | years=<value> + | weekyears=<value> + | moon=<value> +............... +Examples: +............... +location conn_1 internal_www 100: node1 + +location conn_1 internal_www \ + rule 50: #uname eq node1 \ + rule pingd: defined pingd + +location conn_2 dummy_float \ + rule -inf: not_defined pingd or pingd number:lte 0 + +# never probe for rsc1 on node1 +location no-probe rsc1 resource-discovery=never -inf: node1 +............... + +[[cmdhelp_configure_modgroup,modify group]] +==== `modgroup` + +Add or remove primitives in a group. The `add` subcommand appends +the new group member by default. Should it go elsewhere, there +are `after` and `before` clauses. + +Usage: +............... +modgroup <id> add <id> [after <id>|before <id>] +modgroup <id> remove <id> +............... +Examples: +............... +modgroup share1 add storage2 before share1-fs +............... + +[[cmdhelp_configure_monitor,add monitor operation to a primitive]] +==== `monitor` + +Monitor is by far the most common operation. It is possible to +add it without editing the whole resource. Also, long primitive +definitions may be a bit uncluttered. In order to make this +command as concise as possible, less common operation attributes +are not available. If you need them, then use the `op` part of +the `primitive` command. + +Usage: +............... +monitor <rsc>[:<role>] <interval>[:<timeout>] +............... +Example: +............... +monitor apcfence 60m:60s +............... + +Note that after executing the command, the monitor operation may +be shown as part of the primitive definition. + +[[cmdhelp_configure_ms,define a master-slave resource]] +==== `ms` (`master`) + +The `ms` command creates a master/slave resource type. It may contain a +single primitive resource or one group of resources. + +Usage: +............... +ms <name> <rsc> + [description=<description>] + [meta attr_list] + [params attr_list] + +attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id> +............... +Example: +............... +ms disk1 drbd1 \ + meta notify=true globally-unique=false +............... + +.Note on `id-ref` usage +**************************** +Instance or meta attributes (`params` and `meta`) may contain +a reference to another set of attributes. In that case, no other +attributes are allowed. Since attribute sets' ids, though they do +exist, are not shown in the `crm`, it is also possible to +reference an object instead of an attribute set. `crm` will +automatically replace such a reference with the right id: + +............... +crm(live)configure# primitive a2 www-2 meta $id-ref=a1 +crm(live)configure# show a2 +primitive a2 apache \ + meta $id-ref=a1-meta_attributes + [...] +............... +It is advisable to give meaningful names to attribute sets which +are going to be referenced. +**************************** + +[[cmdhelp_configure_node,define a cluster node]] +==== `node` + +The node command describes a cluster node. Nodes in the CIB are +commonly created automatically by the CRM. Hence, you should not +need to deal with nodes unless you also want to define node +attributes. Note that it is also possible to manage node +attributes at the `node` level. + +Usage: +............... +node [$id=<id>] <uname>[:<type>] + [description=<description>] + [attributes [$id=<id>] [<score>:] [rule...] + <param>=<value> [<param>=<value>...]] | $id-ref=<ref> + [utilization [$id=<id>] [<score>:] [rule...] + <param>=<value> [<param>=<value>...]] | $id-ref=<ref> + +type :: normal | member | ping | remote +............... +Example: +............... +node node1 +node big_node attributes memory=64 +............... + +[[cmdhelp_configure_op_defaults,set resource operations defaults]] +==== `op_defaults` + +Set defaults for the operations meta attributes. + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +op_defaults [$id=<set_id>] [rule ...] <option>=<value> [<option>=<value> ...] +............... +Example: +............... +op_defaults record-pending=true +............... + +[[cmdhelp_configure_order,order resources]] +==== `order` + +This constraint expresses the order of actions on two resources +or more resources. If there are more than two resources, then the +constraint is called a resource set. + +Ordered resource sets have an extra attribute to allow for sets +of resources whose actions may run in parallel. The shell syntax +for such sets is to put resources in parentheses. + +If the subsequent resource can start or promote after any one of the +resources in a set has done, enclose the set in brackets (+[+ and +]+). + +Sets cannot be nested. + +Three strings are reserved to specify a kind of order constraint: ++Mandatory+, +Optional+, and +Serialize+. It is preferred to use +one of these settings instead of score. Previous versions mapped +scores +0+ and +inf+ to keywords +advisory+ and +mandatory+. +That is still valid but deprecated. + +For more details on how to configure resource sets, see +<<topics_Features_Resourcesets,`Syntax: Resource sets`>>. + +Usage: +............... +order <id> [{kind|<score>}:] first then [symmetrical=<bool>] + +order <id> [{kind|<score>}:] resource_sets [symmetrical=<bool>] + +kind :: Mandatory | Optional | Serialize + +first :: <rsc>[:<action>] + +then :: <rsc>[:<action>] + +resource_sets :: resource_set [resource_set ...] + +resource_set :: ["["|"("] <rsc>[:<action>] [<rsc>[:<action>] ...] \ + [attributes] ["]"|")"] + +attributes :: [require-all=(true|false)] [sequential=(true|false)] + +............... +Example: +............... +order o-1 Mandatory: apache:start ip_1 +order o-2 Serialize: A ( B C ) +order o-3 inf: [ A B ] C +order o-4 first-resource then-resource +............... + +[[cmdhelp_configure_primitive,define a resource]] +==== `primitive` + +The primitive command describes a resource. It may be referenced +only once in group, clone, or master-slave objects. If it's not +referenced, then it is placed as a single resource in the CIB. + +Operations may be specified anonymously, as a group or by reference: + +* "Anonymous", as a list of +op+ specifications. Use this + method if you don't need to reference the set of operations + elsewhere. This is the most common way to define operations. + +* If reusing operation sets is desired, use the +operations+ keyword + along with an id to give the operations set a name. Use the + +operations+ keyword and an id-ref value set to the id of another + operations set, to apply the same set of operations to this + primitive. + +Operation attributes which are not recognized are saved as +instance attributes of that operation. A typical example is ++OCF_CHECK_LEVEL+. + +For multistate resources, roles are specified as +role=<role>+. + +A template may be defined for resources which are of the same +type and which share most of the configuration. See +<<cmdhelp_configure_rsc_template,`rsc_template`>> for more information. + +Attributes containing time values, such as the +interval+ attribute on +operations, are configured either as a plain number, which is +interpreted as a time in seconds, or using one of the following +suffixes: + +* +s+, +sec+ - time in seconds (same as no suffix) +* +ms+, +msec+ - time in milliseconds +* +us+, +usec+ - time in microseconds +* +m+, +min+ - time in minutes +* +h+, +hr+ - time in hours + +Usage: +............... +primitive <rsc> {[<class>:[<provider>:]]<type>|@<template>} + [description=<description>] + [[params] attr_list] + [meta attr_list] + [utilization attr_list] + [operations id_spec] + [op op_type [<attribute>=<value>...] ...] + +attr_list :: [$id=<id>] [<score>:] [rule...] + <attr>=<val> [<attr>=<val>...]] | $id-ref=<id> +id_spec :: $id=<id> | $id-ref=<id> +op_type :: start | stop | monitor +............... +Example: +............... +primitive apcfence stonith:apcsmart \ + params ttydev=/dev/ttyS0 hostlist="node1 node2" \ + op start timeout=60s \ + op monitor interval=30m timeout=60s + +primitive www8 apache \ + configfile=/etc/apache/www8.conf \ + operations $id-ref=apache_ops + +primitive db0 mysql \ + params config=/etc/mysql/db0.conf \ + op monitor interval=60s \ + op monitor interval=300s OCF_CHECK_LEVEL=10 + +primitive r0 ocf:linbit:drbd \ + params drbd_resource=r0 \ + op monitor role=Master interval=60s \ + op monitor role=Slave interval=300s + +primitive xen0 @vm_scheme1 xmfile=/etc/xen/vm/xen0 + +primitive mySpecialRsc Special \ + params 3: rule #uname eq node1 interface=eth1 \ + params 2: rule #uname eq node2 interface=eth2 port=8888 \ + params 1: interface=eth0 port=9999 + +............... + +[[cmdhelp_configure_property,set a cluster property]] +==== `property` + +Set cluster configuration properties. To list the +available cluster configuration properties, use the +<<cmdhelp_ra_info,`ra info`>> command with +pengine+, +crmd+, ++cib+ and +stonithd+ as arguments. +When setting the +maintenance-mode+ property, it will +inform the user if there are nodes or resources that +have the +maintenance+ property. + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +property [<set_id>:] [rule ...] <option>=<value> [<option>=<value> ...] +............... +Example: +............... +property stonith-enabled=true +property rule date spec years=2014 stonith-enabled=false +............... + +[[cmdhelp_configure_ptest,show cluster actions if changes were committed]] +==== `ptest` (`simulate`) + +Show PE (Policy Engine) motions using `ptest(8)` or +`crm_simulate(8)`. + +A CIB is constructed using the current user edited configuration +and the status from the running CIB. The resulting CIB is run +through `ptest` (or `crm_simulate`) to show changes which would +happen if the configuration is committed. + +The status section may be loaded from another source and modified +using the <<cmdhelp_cibstatus,`cibstatus`>> level commands. In that case, the +`ptest` command will issue a message informing the user that the +Policy Engine graph is not calculated based on the current status +section and therefore won't show what would happen to the +running but some imaginary cluster. + +If you have graphviz installed and X11 session, `dotty(1)` is run +to display the changes graphically. + +Add a string of +v+ characters to increase verbosity. `ptest` +can also show allocation scores. +utilization+ turns on +information about the remaining capacity of nodes. With the ++actions+ option, `ptest` will print all resource actions. + +The `ptest` program has been replaced by `crm_simulate` in newer +Pacemaker versions. In some installations both could be +installed. Use `simulate` to enfore using `crm_simulate`. + +Usage: +............... +ptest [nograph] [v...] [scores] [actions] [utilization] +............... +Examples: +............... +ptest scores +ptest vvvvv +simulate actions +............... + +[[cmdhelp_configure_refresh,refresh from CIB]] +==== `refresh` + +Refresh the internal structures from the CIB. All changes made +during this session are lost. + +Usage: +............... +refresh +............... + +[[cmdhelp_configure_rename,rename a CIB object]] +==== `rename` + +Rename an object. It is recommended to use this command to rename +a resource, because it will take care of updating all related +constraints and a parent resource. Changing ids with the edit +command won't have the same effect. + +If you want to rename a resource, it must be in the stopped state. + +Usage: +............... +rename <old_id> <new_id> +............... + +[[cmdhelp_configure_role,define role access rights]] +==== `role` + +An ACL role is a set of rules which describe access rights to +CIB. Rules consist of an access right +read+, +write+, or +deny+ +and a specification denoting part of the configuration to which +the access right applies. The specification can be an XPath or a +combination of tag and id references. If an attribute is +appended, then the specification applies only to that attribute +of the matching element. + +There is a number of shortcuts for XPath specifications. The ++meta+, +params+, and +utilization+ shortcuts reference resource +meta attributes, parameters, and utilization respectively. The +`location` may be used to specify location constraints most of +the time to allow resource `move` and `unmove` commands. The +`property` references cluster properties. The `node` allows +reading node attributes. +nodeattr+ and +nodeutil+ reference node +attributes and node capacity (utilization). The `status` shortcut +references the whole status section of the CIB. Read access to +status is necessary for various monitoring tools such as +`crm_mon(8)` (aka `crm status`). + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +role <role-id> rule [rule ...] + +rule :: acl-right cib-spec [attribute:<attribute>] + +acl-right :: read | write | deny + +cib-spec :: xpath-spec | tag-ref-spec +xpath-spec :: xpath:<xpath> | shortcut +tag-ref-spec :: tag:<tag> | ref:<id> | tag:<tag> ref:<id> + +shortcut :: meta:<rsc>[:<attr>] + params:<rsc>[:<attr>] + utilization:<rsc> + location:<rsc> + property[:<attr>] + node[:<node>] + nodeattr[:<attr>] + nodeutil[:<node>] + status +............... +Example: +............... +role app1_admin \ + write meta:app1:target-role \ + write meta:app1:is-managed \ + write location:app1 \ + read ref:app1 +............... + +[[cmdhelp_configure_rsc_defaults,set resource defaults]] +==== `rsc_defaults` + +Set defaults for the resource meta attributes. + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +rsc_defaults [<set_id>:] [rule ...] <option>=<value> [<option>=<value> ...] +............... +Example: +............... +rsc_defaults failure-timeout=3m +............... + +[[cmdhelp_configure_rsc_template,define a resource template]] +==== `rsc_template` + +The `rsc_template` command creates a resource template. It may be +referenced in primitives. It is used to reduce large +configurations with many similar resources. + +Usage: +............... +rsc_template <name> [<class>:[<provider>:]]<type> + [description=<description>] + [params attr_list] + [meta attr_list] + [utilization attr_list] + [operations id_spec] + [op op_type [<attribute>=<value>...] ...] + +attr_list :: [$id=<id>] <attr>=<val> [<attr>=<val>...] | $id-ref=<id> +id_spec :: $id=<id> | $id-ref=<id> +op_type :: start | stop | monitor +............... +Example: +............... +rsc_template public_vm Xen \ + op start timeout=300s \ + op stop timeout=300s \ + op monitor interval=30s timeout=60s \ + op migrate_from timeout=600s \ + op migrate_to timeout=600s +primitive xen0 @public_vm \ + params xmfile=/etc/xen/xen0 +primitive xen1 @public_vm \ + params xmfile=/etc/xen/xen1 +............... + +[[cmdhelp_configure_rsc_ticket,resources ticket dependency]] +==== `rsc_ticket` + +This constraint expresses dependency of resources on cluster-wide +attributes, also known as tickets. Tickets are mainly used in +geo-clusters, which consist of multiple sites. A ticket may be +granted to a site, thus allowing resources to run there. + +The +loss-policy+ attribute specifies what happens to the +resource (or resources) if the ticket is revoked. The default is +either +stop+ or +demote+ depending on whether a resource is +multi-state. + +See also the <<cmdhelp_site_ticket,`site`>> set of commands. + +Usage: +............... +rsc_ticket <id> <ticket_id>: <rsc>[:<role>] [<rsc>[:<role>] ...] + [loss-policy=<loss_policy_action>] + +loss_policy_action :: stop | demote | fence | freeze +............... +Example: +............... +rsc_ticket ticket-A_public-ip ticket-A: public-ip +rsc_ticket ticket-A_bigdb ticket-A: bigdb loss-policy=fence +rsc_ticket ticket-B_storage ticket-B: drbd-a:Master drbd-b:Master +............... + + +[[cmdhelp_configure_rsctest,test resources as currently configured]] +==== `rsctest` + +Test resources with current resource configuration. If no nodes +are specified, tests are run on all known nodes. + +The order of resources is significant: it is assumed that later +resources depend on earlier ones. + +If a resource is multi-state, it is assumed that the role on +which later resources depend is master. + +Tests are run sequentially to prevent running the same resource +on two or more nodes. Tests are carried out only if none of the +specified nodes currently run any of the specified resources. +However, it won't verify whether resources run on the other +nodes. + +Superuser privileges are obviously required: either run this as +root or setup the `sudoers` file appropriately. + +Note that resource testing may take some time. + +Usage: +............... +rsctest <rsc_id> [<rsc_id> ...] [<node_id> ...] +............... +Examples: +............... +rsctest my_ip websvc +rsctest websvc nodeB +............... + +[[cmdhelp_configure_save,save the CIB to a file]] +==== `save` + +Save the current configuration to a file. Optionally, as XML. Use ++-+ instead of file name to write the output to `stdout`. + +The `save` command accepts the same selection arguments as the `show` +command. See the <<cmdhelp_configure_show,help section>> for `show` +for more details. + +Usage: +............... +save [xml] [<id> | type:<type | tag:<tag> | + related:<obj> | changed ...] <file> +............... +Example: +............... +save myfirstcib.txt +save web-server server-config.txt +............... + +[[cmdhelp_configure_schema,set or display current CIB RNG schema]] +==== `schema` + +CIB's content is validated by a RNG schema. Pacemaker supports +several, depending on version. At least the following schemas are +accepted by `crmsh`: + +* +pacemaker-1.0+ +* +pacemaker-1.1+ +* +pacemaker-1.2+ +* +pacemaker-1.3+ +* +pacemaker-2.0+ + +Use this command to display or switch to another RNG schema. + +Usage: +............... +schema [<schema>] +............... +Example: +............... +schema pacemaker-1.1 +............... + +[[cmdhelp_configure_set,set an attribute value]] +==== `set` + +Set the value of a configured attribute. The attribute must +have a value configured previously, and can be an agent +parameter, meta attribute or utilization value. + +The first argument to the command is a path to an attribute. +This is a dot-separated sequence beginning with the name of +the resource, and ending with the name of the attribute to +set. + +Usage: +............... +set <path> <value> +............... +Examples: +............... +set vip1.ip 192.168.20.5 +set vm-a.force_stop 1 +............... + +[[cmdhelp_configure_show,display CIB objects]] +==== `show` + +The `show` command displays CIB objects. Without any argument, it +displays all objects in the CIB, but the set of objects displayed by +`show` can be limited to only objects with the given IDs or by using +one or more of the special prefixes described below. + +The XML representation for the objects can be displayed by passing ++xml+ as the first argument. + +To show one or more specific objects, pass the object IDs as +arguments. + +To show all objects of a certain type, use the +type:+ prefix. + +To show all objects in a tag, use the +tag:+ prefix. + +To show all constraints related to a primitive, use the +related:+ prefix. + +To show all modified objects, pass the argument +changed+. + +The prefixes can be used together on a single command line. For +example, to show both the tag itself and the objects tagged by it the +following combination can be used: +show tag:my-tag my-tag+. + +To refine a selection of objects using multiple modifiers, the keywords ++and+ and +or+ can be used. For example, to select all primitives tagged ++foo+, the following combination can be used: ++show type:primitive and tag:foo+. + +To hide values when displaying the configuration, use the ++obscure:<glob>+ argument. This can be useful when sending the +configuration over a public channel, to avoid exposing potentially +sensitive information. The +<glob>+ argument is a bash-style pattern +matching attribute keys. + +Usage: +............... +show [xml] [<id> + | changed + | type:<type> + | tag:<id> + | related:<obj> + | obscure:<glob> + ...] + +type :: node | primitive | group | clone | ms | rsc_template + | location | colocation | order + | rsc_ticket + | property | rsc_defaults | op_defaults + | fencing_topology + | role | user | acl_target + | tag +............... + +Example: +............... +show webapp +show type:primitive +show xml tag:db tag:fs +show related:webapp +show type:primitive obscure:passwd +............... + +[[cmdhelp_configure_tag,Define resource tags]] +==== `tag` + +Define a resource tag. A tag is an id referring to one or more +resources, without implying any constraints between the tagged +resources. This can be useful for grouping conceptually related +resources. + +Usage: +............... +tag <tag-name>: <rsc> [<rsc> ...] +tag <tag-name> <rsc> [<rsc> ...] +............... +Example: +............... +tag web: p-webserver p-vip +tag ips server-vip admin-vip +............... + +[[cmdhelp_configure_template,edit and import a configuration from a template]] +==== `template` + +The specified template is loaded into the editor. It's up to the +user to make a good CRM configuration out of it. See also the +<<cmdhelp_template,template section>>. + +Usage: +............... +template [xml] url +............... +Example: +............... +template two-apaches.txt +............... + +[[cmdhelp_configure_upgrade,upgrade the CIB]] +==== `upgrade` + +Attempts to upgrade the CIB to validate with the current +version. Commonly, this is required if the error +`CIB not supported` occurs. It typically means that the +active CIB version is coming from an older release. + +As a safety precaution, the force argument is required if the ++validation-with+ attribute is set to anything other than ++0.6+. Thus in most cases, it is required. + +Usage: +............... +upgrade [force] +............... + +Example: +............... +upgrade force +............... + +[[cmdhelp_configure_user,define user access rights]] +==== `user` + +Users which normally cannot view or manage cluster configuration +can be allowed access to parts of the CIB. The access is defined +by a set of +read+, +write+, and +deny+ rules as in role +definitions or by referencing roles. The latter is considered +best practice. + +For more information on rule expressions, see +<<topics_Syntax_RuleExpressions,Syntax: Rule expressions>>. + +Usage: +............... +user <uid> {roles|rules} + +roles :: role:<role-ref> [role:<role-ref> ...] +rules :: rule [rule ...] +............... +Example: +............... +user joe \ + role:app1_admin \ + role:read_all +............... + +[[cmdhelp_configure_validate_all,call agent validate-all for resource]] +==== `validate-all` + +Call the `validate-all` action for the resource, if possible. + +Limitations: + +* The resource agent must implement the `validate-all` action. +* The current user must be root. +* The primitive resource must not use nvpair references. + +Usage: +............... +validate-all <rsc> +............... + + +[[cmdhelp_configure_verify,verify the CIB with crm_verify]] +==== `verify` + +Verify the contents of the CIB which would be committed. + +Usage: +............... +verify +............... + +[[cmdhelp_configure_xml,raw xml]] +==== `xml` + +Even though we promissed no xml, it may happen, but hopefully +very very seldom, that an element from the CIB cannot be rendered +in the configuration language. In that case, the element will be +shown as raw xml, prefixed by this command. That element can then +be edited like any other. If the shell finds out that after the +change it can digest it, then it is going to be converted into +the normal configuration language. Otherwise, there is no need to +use `xml` for configuration. + +Usage: +............... +xml <xml> +............... + +[[cmdhelp_template,edit and import a configuration from a template]] +=== `template` - Import configuration from templates + +User may be assisted in the cluster configuration by templates +prepared in advance. Templates consist of a typical ready +configuration which may be edited to suit particular user needs. + +This command enters a template level where additional commands +for configuration/template management are available. + +[[cmdhelp_template_apply,process and apply the current configuration to the current CIB]] +==== `apply` + +Copy the current or given configuration to the current CIB. By +default, the CIB is replaced, unless the method is set to +"update". + +Usage: +............... +apply [<method>] [<config>] + +method :: replace | update +............... + +[[cmdhelp_template_delete,delete a configuration]] +==== `delete` + +Remove a configuration. The loaded (active) configuration may be +removed by force. + +Usage: +............... +delete <config> [force] +............... + +[[cmdhelp_template_edit,edit a configuration]] +==== `edit` + +Edit current or given configuration using your favourite editor. + +Usage: +............... +edit [<config>] +............... + +[[cmdhelp_template_list,list configurations/templates]] +==== `list` + +When called with no argument, lists existing templates and +configurations. + +Given the argument +templates+, lists the available templates. + +Given the argument +configs+, lists the available configurations. + +Usage: +............... +list [templates|configs] +............... + +[[cmdhelp_template_load,load a configuration]] +==== `load` + +Load an existing configuration. Further `edit`, `show`, and +`apply` commands will refer to this configuration. + +Usage: +............... +load <config> +............... + +[[cmdhelp_template_new,create a new configuration from templates]] +==== `new` + +Create a new configuration from one or more templates. Note that +configurations and templates are kept in different places, so it +is possible to have a configuration name equal a template name. + +If you already know which parameters are required, you can set +them directly on the command line. + +The parameter name +id+ is set by default to the name of the +configuration. + +If no parameters are being set and you don't want a particular name +for your configuration, you can call this command with a template name +as the only parameter. A unique configuration name based on the +template name will be generated. + +Usage: +............... +new [<config>] <template> [<template> ...] [params name=value ...] +............... + +Example: +............... +new vip virtual-ip +new bigfs ocfs2 params device=/dev/sdx8 directory=/bigfs +new apache +............... + +[[cmdhelp_template_show,show the processed configuration]] +==== `show` + +Process the current or given configuration and display the result. + +Usage: +............... +show [<config>] +............... + +[[cmdhelp_cibstatus,CIB status management and editing]] +=== `cibstatus` - CIB status management and editing + +The `status` section of the CIB keeps the current status of nodes +and resources. It is modified _only_ on events, i.e. when some +resource operation is run or node status changes. For obvious +reasons, the CRM has no user interface with which it is possible +to affect the status section. From the user's point of view, the +status section is essentially a read-only part of the CIB. The +current status is never even written to disk, though it is +available in the PE (Policy Engine) input files which represent +the history of cluster motions. The current status may be read +using the +cibadmin -Q+ command. + +It may sometimes be of interest to see how status changes would +affect the Policy Engine. The set of `cibstatus` level commands +allow the user to load status sections from various sources and +then insert or modify resource operations or change nodes' state. + +The effect of those changes may then be observed by running the +<<cmdhelp_configure_ptest,`ptest`>> command at the `configure` level +or `simulate` and `run` commands at this level. The `ptest` +runs with the user edited CIB whereas the latter two commands +run with the CIB which was loaded along with the status section. + +The `simulate` and `run` commands as well as all status +modification commands are implemented using `crm_simulate(8)`. + +[[cmdhelp_cibstatus_load,load the CIB status section]] +==== `load` + +Load a status section from a file, a shadow CIB, or the running +cluster. By default, the current (+live+) status section is +modified. Note that if the +live+ status section is modified it +is not going to be updated if the cluster status changes, because +that would overwrite the user changes. To make `crm` drop changes +and resume use of the running cluster status, run +load live+. + +All CIB shadow configurations contain the status section which is +a snapshot of the status section taken at the time the shadow was +created. Obviously, this status section doesn't have much to do +with the running cluster status, unless the shadow CIB has just +been created. Therefore, the `ptest` command by default uses the +running cluster status section. + +Usage: +............... +load {<file>|shadow:<cib>|live} +............... +Example: +............... +load bug-12299.xml +load shadow:test1 +............... + +[[cmdhelp_cibstatus_node,change node status]] +==== `node` + +Change the node status. It is possible to throw a node out of +the cluster, make it a member, or set its state to unclean. + ++online+:: Set the +node_state+ `crmd` attribute to +online+ +and the +expected+ and +join+ attributes to +member+. The effect +is that the node becomes a cluster member. + ++offline+:: Set the +node_state+ `crmd` attribute to +offline+ +and the +expected+ attribute to empty. This makes the node +cleanly removed from the cluster. + ++unclean+:: Set the +node_state+ `crmd` attribute to +offline+ +and the +expected+ attribute to +member+. In this case the node +has unexpectedly disappeared. + +Usage: +............... +node <node> {online|offline|unclean} +............... +Example: +............... +node xen-b unclean +............... + +[[cmdhelp_cibstatus_op,edit outcome of a resource operation]] +==== `op` + +Edit the outcome of a resource operation. This way you can +tell CRM that it ran an operation and that the resource agent +returned certain exit code. It is also possible to change the +operation's status. In case the operation status is set to +something other than +done+, the exit code is effectively +ignored. + +Usage: +............... +op <operation> <resource> <exit_code> [<op_status>] [<node>] + +operation :: probe | monitor[:<n>] | start | stop | + promote | demote | notify | migrate_to | migrate_from +exit_code :: <rc> | success | generic | args | + unimplemented | perm | installed | configured | not_running | + master | failed_master +op_status :: pending | done | cancelled | timeout | notsupported | error + +n :: the monitor interval in seconds; if omitted, the first + recurring operation is referenced +rc :: numeric exit code in range 0..9 +............... +Example: +............... +op start d1 xen-b generic +op start d1 xen-b 1 +op monitor d1 xen-b not_running +op stop d1 xen-b 0 timeout +............... + +[[cmdhelp_cibstatus_origin,display origin of the CIB status section]] +==== `origin` + +Show the origin of the status section currently in use. This +essentially shows the latest `load` argument. + +Usage: +............... +origin +............... + +[[cmdhelp_cibstatus_quorum,set the quorum]] +==== `quorum` + +Set the quorum value. + +Usage: +............... +quorum <bool> +............... +Example: +............... +quorum false +............... + +[[cmdhelp_cibstatus_run,run policy engine]] +==== `run` + +Run the policy engine with the edited status section. + +Add a string of +v+ characters to increase verbosity. Specify ++scores+ to see allocation scores also. +utilization+ turns on +information about the remaining capacity of nodes. + +If you have graphviz installed and X11 session, `dotty(1)` is run +to display the changes graphically. + +Usage: +............... +run [nograph] [v...] [scores] [utilization] +............... +Example: +............... +run +............... + +[[cmdhelp_cibstatus_save,save the CIB status section]] +==== `save` + +The current internal status section with whatever modifications +were performed can be saved to a file or shadow CIB. + +If the file exists and contains a complete CIB, only the status +section is going to be replaced and the rest of the CIB will +remain intact. Otherwise, the current user edited configuration +is saved along with the status section. + +Note that all modifications are saved in the source file as soon +as they are run. + +Usage: +............... +save [<file>|shadow:<cib>] +............... +Example: +............... +save bug-12299.xml +............... + +[[cmdhelp_cibstatus_show,show CIB status section]] +==== `show` + +Show the current status section in the XML format. Brace yourself +for some unreadable output. Add +changed+ option to get a human +readable output of all changes. + +Usage: +............... +show [changed] +............... + +[[cmdhelp_cibstatus_simulate,simulate cluster transition]] +==== `simulate` + +Run the policy engine with the edited status section and simulate +the transition. + +Add a string of +v+ characters to increase verbosity. Specify ++scores+ to see allocation scores also. +utilization+ turns on +information about the remaining capacity of nodes. + +If you have graphviz installed and X11 session, `dotty(1)` is run +to display the changes graphically. + +Usage: +............... +simulate [nograph] [v...] [scores] [utilization] +............... +Example: +............... +simulate +............... + +[[cmdhelp_cibstatus_ticket,manage tickets]] +==== `ticket` + +Modify the ticket status. Tickets can be granted and revoked. +Granted tickets could be activated or put in standby. + +Usage: +............... +ticket <ticket> {grant|revoke|activate|standby} +............... +Example: +............... +ticket ticketA grant +............... + +[[cmdhelp_assist,Configuration assistant]] +=== `assist` - Configuration assistant + +The `assist` sublevel is a collection of helper +commands that create or modify resources and +constraints, to simplify the creation of certain +configurations. + +For more information on individual commands, see +the help text for those commands. + +[[cmdhelp_assist_template,Create template for primitives]] +==== `template` + +This command takes a list of primitives as argument, and creates a new +`rsc_template` for these primitives. It can only do this if the +primitives do not already share a template and are of the same type. + +Usage: +........ +template primitive-1 primitive-2 primitive-3 +........ + +[[cmdhelp_assist_weak-bond,Create a weak bond between resources]] +==== `weak-bond` + +A colocation between a group of resources says that the resources +should be located together, but it also means that those resources are +dependent on each other. If one of the resources fails, the others +will be restarted. + +If this is not desired, it is possible to circumvent: By placing the +resources in a non-sequential set and colocating the set with a dummy +resource which is not monitored, the resources will be placed together +but will have no further dependency on each other. + +This command creates both the constraint and the dummy resource needed +for such a colocation. + +Usage: +........ +weak-bond resource-1 resource-2 +........ + +[[cmdhelp_maintenance,Maintenance mode commands]] +=== `maintenance` - Maintenance mode commands + +Maintenance mode commands are commands that manipulate resources +directly without going through the cluster infrastructure. Therefore, +it is essential to ensure that the cluster does not attempt to monitor +or manipulate the resources while these commands are being executed. + +To ensure this, these commands require that maintenance mode is set +either for the particular resource, or for the whole cluster. + +[[cmdhelp_maintenance_action,Invoke a resource action]] +==== `action` + +Invokes the given action for the resource. This is +done directly via the resource agent, so the command must +be issued while the cluster or the resource is in +maintenance mode. + +Unless the action is `start` or `monitor`, the action must be invoked +on the same node as where the resource is running. If the resource is +running on multiple nodes, the command will fail. + +To use SSH for executing resource actions on multiple nodes, append +`ssh` after the action name. This requires SSH access to be configured +between the nodes and the parallax python package to be installed. + +Usage: +............... +action <rsc> <action> +action <rsc> <action> ssh +............... +Example: +............... +action webserver reload +action webserver monitor ssh +............... + +[[cmdhelp_maintenance_off,Disable maintenance mode]] +==== `off` + +Disables maintenances mode, either for the whole cluster +or for the given resource. + +Usage: +............... +off +off <rsc> +............... +Example: +............... +off rsc1 +............... + +[[cmdhelp_maintenance_on,Enable maintenance mode]] +==== `on` + +Enables maintenances mode, either for the whole cluster +or for the given resource. + +Usage: +............... +on +on <rsc> +............... +Example: +............... +on rsc1 +............... + +[[cmdhelp_history,Cluster history]] +=== `history` - Cluster history + +Examining Pacemaker's history is a particularly involved task. The +number of subsystems to be considered, the complexity of the +configuration, and the set of various information sources, most of +which are not exactly human readable, keep analyzing resource or node +problems accessible to only the most knowledgeable. Or, depending on +the point of view, to the most persistent. The following set of +commands has been devised in hope to make cluster history more +accessible. + +Of course, looking at _all_ history could be time consuming regardless +of how good the tools at hand are. Therefore, one should first say +which period he or she wants to analyze. If not otherwise specified, +the last hour is considered. Logs and other relevant information is +collected using `crm report`. Since this process takes some time and +we always need fresh logs, information is refreshed in a much faster +way using the python parallax module. If +python-parallax+ is not +found on the system, examining a live cluster is still possible -- +though not as comfortable. + +Apart from examining a live cluster, events may be retrieved from a +report generated by `crm report` (see also the +-H+ option). In that +case we assume that the period stretching the whole report needs to be +investigated. Of course, it is still possible to further reduce the +time range. + +If you have discovered an issue that you want to show someone else, +you can use the `session pack` command to save the current session as +a tarball, similar to those generated by `crm report`. + +In order to minimize the size of the tarball, and to make it easier +for others to find the interesting events, it is recommended to limit +the time frame which the saved session covers. This can be done using +the `timeframe` command (example below). + +It is also possible to name the saved session using the `session save` +command. + +Example: +............... +crm(live)history# limit "Jul 18 12:00" "Jul 18 12:30" +crm(live)history# session save strange_restart +crm(live)history# session pack +Report saved in .../strange_restart.tar.bz2 +crm(live)history# +............... + +[[cmdhelp_history_detail,set the level of detail shown]] +==== `detail` + +How much detail to show from the logs. Valid detail levels are either +`0` or `1`, where `1` is the highest detail level. The default detail +level is `0`. + +Usage: +............... +detail <detail_level> + +detail_level :: small integer (defaults to 0) +............... +Example: +............... +detail 1 +............... + +[[cmdhelp_history_diff,cluster states/transitions difference]] +==== `diff` + +A transition represents a change in cluster configuration or +state. Use `diff` to see what has changed between two +transitions. + +If you want to specify the current cluster configuration and +status, use the string +live+. + +Normally, the first transition specified should be the one which +is older, but we are not going to enforce that. + +Note that a single configuration update may result in more than +one transition. + +Usage: +............... +diff <pe> <pe> [status] [html] + +pe :: <number>|<index>|<file>|live +............... +Examples: +............... +diff 2066 2067 +diff pe-input-2080.bz2 live status +............... + +[[cmdhelp_history_events,Show events in log]] +==== `events` + +By analysing the log output and looking for particular +patterns, the `events` command helps sifting through +the logs to find when particular events like resources +changing state or node failure may have occurred. + +This can be used to generate a combined list of events +from all nodes. + +Usage: +............... +events +............... + +Example: +............... +events +............... + +[[cmdhelp_history_exclude,exclude log messages]] +==== `exclude` + +If a log is infested with irrelevant messages, those messages may +be excluded by specifying a regular expression. The regular +expressions used are Python extended. This command is additive. +To drop all regular expressions, use +exclude clear+. Run +`exclude` only to see the current list of regular expressions. +Excludes are saved along with the history sessions. + +Usage: +............... +exclude [<regex>|clear] +............... +Example: +............... +exclude kernel.*ocfs2 +............... + +[[cmdhelp_history_graph,generate a directed graph from the PE file]] +==== `graph` + +Create a graphviz graphical layout from the PE file (the +transition). Every transition contains the cluster configuration +which was active at the time. See also <<cmdhelp_configure_graph,generate a directed graph +from configuration>>. + +Usage: +............... +graph <pe> [<gtype> [<file> [<img_format>]]] + +gtype :: dot +img_format :: `dot` output format (see the +-T+ option) +............... +Example: +............... +graph -1 +graph 322 dot clu1.conf.dot +graph 322 dot clu1.conf.svg svg +............... + +[[cmdhelp_history_info,Cluster information summary]] +==== `info` + +The `info` command provides a summary of the information source, which +can be either a live cluster snapshot or a previously generated +report. + +Usage: +............... +info +............... +Example: +............... +info +............... + +[[cmdhelp_history_latest,show latest news from the cluster]] +==== `latest` + +The `latest` command shows a bit of recent history, more +precisely whatever happened since the last cluster change (the +latest transition). If the transition is running, the shell will +first wait until it finishes. + +Usage: +............... +latest +............... +Example: +............... +latest +............... + +[[cmdhelp_history_limit,limit timeframe to be examined]] +==== `limit` (`timeframe`) + +This command can be used to modify the time span to examine. All +history commands look at events within a certain time span. + +For the `live` source, the default time span is the _last hour_. + +There is no time span limit for the `hb_report` source. + +The time period is parsed by the `dateutil` python module. It +covers a wide range of date formats. For instance: + +- 3:00 (today at 3am) +- 15:00 (today at 3pm) +- 2010/9/1 2pm (September 1st 2010 at 2pm) + +For more examples of valid time/date statements, please refer to the +`python-dateutil` documentation: + +- https://dateutil.readthedocs.org/[dateutil.readthedocs.org] + +If the dateutil module is not available, then the time is parsed using +strptime and only the kind as printed by `date(1)` is allowed: + +- Tue Sep 15 20:46:27 CEST 2010 + +Usage: +............... +limit [<from_time>] [<to_time>] +............... +Examples: +............... +limit 10:15 +limit 15h22m 16h +limit "Sun 5 20:46" "Sun 5 22:00" +............... + +[[cmdhelp_history_log,log content]] +==== `log` + +Show messages logged on one or more nodes. Leaving out a node +name produces combined logs of all nodes. Messages are sorted by +time and, if the terminal emulations supports it, displayed in +different colours depending on the node to allow for easier +reading. + +The sorting key is the timestamp as written by syslog which +normally has the maximum resolution of one second. Obviously, +messages generated by events which share the same timestamp may +not be sorted in the same way as they happened. Such close events +may actually happen fairly often. + +Usage: +............... +log [<node> [<node> ...] ] +............... +Example: +............... +log node-a +............... + +[[cmdhelp_history_node,node events]] +==== `node` + +Show important events that happened on a node. Important events +are node lost and join, standby and online, and fence. Use either +node names or extended regular expressions. + +Usage: +............... +node <node> [<node> ...] +............... +Example: +............... +node node1 +............... + +[[cmdhelp_history_peinputs,list or get PE input files]] +==== `peinputs` + +Every event in the cluster results in generating one or more +Policy Engine (PE) files. These files describe future motions of +resources. The files are listed as full paths in the current +report directory. Add +v+ to also see the creation time stamps. + +Usage: +............... +peinputs [{<range>|<number>} ...] [v] + +range :: <n1>:<n2> +............... +Example: +............... +peinputs +peinputs 440:444 446 +peinputs v +............... + +[[cmdhelp_history_refresh,refresh live report]] +==== `refresh` + +This command makes sense only for the +live+ source and makes +`crm` collect the latest logs and other relevant information from +the logs. If you want to make a completely new report, specify ++force+. + +Usage: +............... +refresh [force] +............... + +[[cmdhelp_history_resource,resource events]] +==== `resource` + +Show actions and any failures that happened on all specified +resources on all nodes. Normally, one gives resource names as +arguments, but it is also possible to use extended regular +expressions. Note that neither groups nor clones or master/slave +names are ever logged. The resource command is going to expand +all of these appropriately, so that clone instances or resources +which are part of a group are shown. + +Usage: +............... +resource <rsc> [<rsc> ...] +............... +Example: +............... +resource bigdb public_ip +resource my_.*_db2 +resource ping_clone +............... + +[[cmdhelp_history_session,manage history sessions]] +==== `session` + +Sometimes you may want to get back to examining a particular +history period or bug report. In order to make that easier, the +current settings can be saved and later retrieved. + +If the current history being examined is coming from a live +cluster the logs, PE inputs, and other files are saved too, +because they may disappear from nodes. For the existing reports +coming from `hb_report`, only the directory location is saved +(not to waste space). + +A history session may also be packed into a tarball which can +then be sent to support. + +Leave out subcommand to see the current session. + +Usage: +............... +session [{save|load|delete} <name> | pack [<name>] | update | list] +............... +Examples: +............... +session save bnc966622 +session load rsclost-2 +session list +............... + +[[cmdhelp_history_setnodes,set the list of cluster nodes]] +==== `setnodes` + +In case the host this program runs on is not part of the cluster, +it is necessary to set the list of nodes. + +Usage: +............... +setnodes node <node> [<node> ...] +............... +Example: +............... +setnodes node_a node_b +............... + +[[cmdhelp_history_show,show status or configuration of the PE input file]] +==== `show` + +Every transition is saved as a PE file. Use this command to +render that PE file either as configuration or status. The +configuration output is the same as `crm configure show`. + +Usage: +............... +show <pe> [status] + +pe :: <number>|<index>|<file>|live +............... +Examples: +............... +show 2066 +show pe-input-2080.bz2 status +............... + +[[cmdhelp_history_source,set source to be examined]] +==== `source` + +Events to be examined can come from the current cluster or from a +`hb_report` report. This command sets the source. `source live` +sets source to the running cluster and system logs. If no source +is specified, the current source information is printed. + +In case a report source is specified as a file reference, the file +is going to be unpacked in place where it resides. This directory +is not removed on exit. + +Usage: +............... +source [<dir>|<file>|live] +............... +Examples: +............... +source live +source /tmp/customer_case_22.tar.bz2 +source /tmp/customer_case_22 +source +............... + +[[cmdhelp_history_transition,show transition]] +==== `transition` + +This command will print actions planned by the PE and run +graphviz (`dotty`) to display a graphical representation of the +transition. Of course, for the latter an X11 session is required. +This command invokes `ptest(8)` in background. + +The +showdot+ subcommand runs graphviz (`dotty`) to display a +graphical representation of the +.dot+ file which has been +included in the report. Essentially, it shows the calculation +produced by `pengine` which is installed on the node where the +report was produced. In optimal case this output should not +differ from the one produced by the locally installed `pengine`. + +The `log` subcommand shows the full log for the duration of the +transition. + +A transition can also be saved to a CIB shadow for further +analysis or use with `cib` or `configure` commands (use the +`save` subcommand). The shadow file name defaults to the name of +the PE input file. + +If the PE input file number is not provided, it defaults to the +last one, i.e. the last transition. The last transition can also +be referenced with number 0. If the number is negative, then the +corresponding transition relative to the last one is chosen. + +If there are warning and error PE input files or different nodes +were the DC in the observed timeframe, it may happen that PE +input file numbers collide. In that case provide some unique part +of the path to the file. + +After the `ptest` output, logs about events that happened during +the transition are printed. + +The `tags` subcommand scans the logs for the transition and return a +list of key events during that transition. For example, the tag ++error+ will be returned if there are any errors logged during the +transition. + +Usage: +............... +transition [<number>|<index>|<file>] [nograph] [v...] [scores] [actions] [utilization] +transition showdot [<number>|<index>|<file>] +transition log [<number>|<index>|<file>] +transition save [<number>|<index>|<file> [name]] +transition tags [<number>|<index>|<file>] +............... +Examples: +............... +transition +transition 444 +transition -1 +transition pe-error-3.bz2 +transition node-a/pengine/pe-input-2.bz2 +transition showdot 444 +transition log +transition save 0 enigma-22 +............... + +[[cmdhelp_history_transitions,List transitions]] +==== `transitions` + +A transition represents a change in cluster configuration or +state. This command lists the transitions in the current timeframe. + +Usage: +............... +transitions +............... +Example: +............... +transitions +............... + + +[[cmdhelp_history_wdiff,cluster states/transitions difference]] +==== `wdiff` + +A transition represents a change in cluster configuration or +state. Use `wdiff` to see what has changed between two +transitions as word differences on a line-by-line basis. + +If you want to specify the current cluster configuration and +status, use the string +live+. + +Normally, the first transition specified should be the one which +is older, but we are not going to enforce that. + +Note that a single configuration update may result in more than +one transition. + +Usage: +............... +wdiff <pe> <pe> [status] + +pe :: <number>|<index>|<file>|live +............... +Examples: +............... +wdiff 2066 2067 +wdiff pe-input-2080.bz2 live status +............... + +[[cmdhelp_root_report,Create cluster status report]] +=== `report` + +Interface to a tool for creating a cluster report. A report is an +archive containing log files, configuration files, system information +and other relevant data for a given time period. This is a useful tool +for collecting data to attach to bug reports, or for detecting the +root cause of errors resulting in resource failover, for example. + +See `crmsh_hb_report(8)` for more details on arguments, +or call `crm report -h` + +Usage: +............... +report -f {time|"cts:"testnum} [-t time] [-u user] [-l file] + [-n nodes] [-E files] [-p patt] [-L patt] [-e prog] + [-MSDZAVsvhd] [dest] +............... + +Examples: +............... +report -f 2pm report_1 +report -f "2007/9/5 12:30" -t "2007/9/5 14:00" report_2 +report -f 1:00 -t 3:00 -l /var/log/cluster/ha-debug report_3 +report -f "09sep07 2:00" -u hbadmin report_4 +report -f 18:00 -p "usern.*" -p "admin.*" report_5 +report -f cts:133 ctstest_133 +............... + +=== `end` (`cd`, `up`) + +The `end` command ends the current level and the user moves to +the parent level. This command is available everywhere. + +Usage: +............... +end +............... + +=== `help` + +The `help` command prints help for the current level or for the +specified topic (command). This command is available everywhere. + +Usage: +............... +help [<topic>] +............... + +=== `quit` (`exit`, `bye`) + +Leave the program. + +BUGS +---- +Even though all sensible configurations (and most of those that +are not) are going to be supported by the crm shell, I suspect +that it may still happen that certain XML constructs may confuse +the tool. When that happens, please file a bug report. + +The crm shell will not try to update the objects it does not +understand. Of course, it is always possible to edit such objects +in the XML format. + +AUTHORS +------- +Dejan Muhamedagic, <dejan@suse.de> +Kristoffer Gronlund <kgronlund@suse.com> +and many OTHERS + +SEE ALSO +-------- +crm_resource(8), crm_attribute(8), crm_mon(8), cib_shadow(8), +ptest(8), dotty(1), crm_simulate(8), cibadmin(8) + + +COPYING +------- +Copyright \(C) 2008-2013 Dejan Muhamedagic. +Copyright \(C) 2013 Kristoffer Gronlund. + +Free use of this software is granted under the terms of the GNU General Public License (GPL). + +////////////////////// + vim:ts=4:sw=4:expandtab: +////////////////////// |