diff options
Diffstat (limited to 'doc/dev/config.rst')
| -rw-r--r-- | doc/dev/config.rst | 283 |
1 files changed, 283 insertions, 0 deletions
diff --git a/doc/dev/config.rst b/doc/dev/config.rst new file mode 100644 index 000000000..9cb20aee7 --- /dev/null +++ b/doc/dev/config.rst @@ -0,0 +1,283 @@ +================================= + Configuration Management System +================================= + +The configuration management system exists to provide every daemon with the +proper configuration information. The configuration can be viewed as a set of +key-value pairs. + +How can the configuration be set? Well, there are several sources: + + - the ceph configuration file, usually named ceph.conf + - command line arguments:: + + --debug-ms=1 + --debug-monc=10 + + etc. + - arguments injected at runtime using ``injectargs`` or ``config set`` + + +The Configuration File +====================== + +Most configuration settings originate in the Ceph configuration file. + +How do we find the configuration file? Well, in order, we check: + + - the default locations + - the environment variable ``CEPH_CONF`` + - the command line argument ``-c`` + +Each stanza of the configuration file describes the key-value pairs that will be in +effect for a particular subset of the daemons. The "global" stanza applies to +everything. The "mon", "osd", and "mds" stanzas specify settings to take effect +for all monitors, all OSDs, and all mds servers, respectively. A stanza of the +form ``mon.$name``, ``osd.$name``, or ``mds.$name`` gives settings for the monitor, OSD, or +MDS of that name, respectively. Configuration values that appear later in the +file win over earlier ones. + +A sample configuration file can be found in src/sample.ceph.conf. + + +Metavariables +============= + +The configuration system allows any configuration value to be +substituted into another value using the ``$varname`` syntax, similar +to how bash shell expansion works. + +A few additional special metavariables are also defined: + + - $host: expands to the current hostname + - $type: expands to one of "mds", "osd", "mon", or "client" + - $id: expands to the daemon identifier. For ``osd.0``, this would be ``0``; for ``mds.a``, it would be ``a``; for ``client.admin``, it would be ``admin``. + - $num: same as $id + - $name: expands to $type.$id + + +Reading configuration values +==================================================== + +There are two ways for Ceph code to get configuration values. One way is to +read it directly from a variable named ``g_conf``, or equivalently, +``g_ceph_ctx->_conf``. The other is to register an observer that will be called +every time the relevant configuration values change. This observer will be +called soon after the initial configuration is read, and every time after that +when one of the relevant values changes. Each observer tracks a set of keys +and is invoked only when one of the relevant keys changes. + +The interface to implement is found in ``common/config_obs.h``. + +The observer method should be preferred in new code because + + - It is more flexible, allowing the code to do whatever reinitialization needs + to be done to implement the new configuration value. + - It is the only way to create a std::string configuration variable that can + be changed by injectargs. + - Even for int-valued configuration options, changing the values in one thread + while another thread is reading them can lead to subtle and + impossible-to-diagnose bugs. + +For these reasons, reading directly from ``g_conf`` should be considered deprecated +and not done in new code. Do not ever alter ``g_conf``. + +Changing configuration values +==================================================== + +Configuration values can be changed by calling ``g_conf()->set_val``. After changing +the configuration, you should call ``g_conf()->apply_changes`` to re-run all the +affected configuration observers. For convenience, you can call +``g_conf()->set_val_or_die`` to make a configuration change which you think should +never fail. + +``injectargs``, ``parse_argv``, and ``parse_env`` are three other functions which modify +the configuration. Just like with set_val, you should call apply_changes after +calling these functions to make sure your changes get applied. + + +Defining config options +======================= + +Config options are defined in ``common/options/*.yaml.in``. The options are categorized +by their consumers. If an option is only used by ceph-osd, it should go to +``osd.yaml.in``. All the ``.yaml.in`` files are translated into ``.cc`` and ``.h`` files +at build time by ``y2c.py``. + +Each option is represented using a YAML mapping (dictionary). A typical option looks like + +.. code-block:: yaml + + - name: public_addr + type: addr + level: basic + desc: public-facing address to bind to + long_desc: The IP address for the public (front-side) network. + Set for each daemon. + services: + - mon + - mds + - osd + - mgr + flags: + - startup + with_legacy: true + +In which, following keys are allowed: + +level +----- + +The ``level`` property of an option is an indicator for the probability the +option is adjusted by an operator or a developer: + +.. describe:: basic + + for basic config options that a normal operator is likely to adjust. + +.. describe:: advanced + + for options that an operator *can* adjust, but should not touch unless they + understand what they are doing. Adjusting advanced options poorly can lead to + problems (performance or even data loss) if done incorrectly. + +.. describe:: dev + + for options in place for use by developers only, either for testing purposes, + or to describe constants that no user should adjust but we prefer not to compile + into the code. + +``desc``, ``long_desc`` and ``fmt_desc`` +---------------------------------------- + +.. describe:: desc + + Short description of the option. Sentence fragment. e.g. + + .. code-block:: yaml + + desc: Default checksum algorithm to use + +.. describe:: long_desc + + The long description is complete sentences, perhaps even multiple + paragraphs, and may include other detailed information or notes. e.g. + + .. code-block:: yaml + + long_desc: crc32c, xxhash32, and xxhash64 are available. The _16 and _8 variants use + only a subset of the bits for more compact (but less reliable) checksumming. + +.. describe:: fmt_desc + + The description formatted using reStructuredText. This property is + only used by the ``confval`` directive to render an option in the + document. e.g.: + + .. code-block:: yaml + + fmt_desc: The interval for "deep" scrubbing (fully reading all data). The + ``osd_scrub_load_threshold`` does not affect this setting. + +Default values +-------------- + +There is a default value for every config option. In some cases, there may +also be a *daemon default* that only applies to code that declares itself +as a daemon (in this case, the regular default only applies to non-daemons). Like: + +.. code-block:: yaml + + default: crc32c + +Some literal postfixes are allowed when options with type of ``float``, ``size`` +and ``secs``, like: + +.. code-block:: yaml + + - name: mon_scrub_interval + type: secs + default: 1_day + - name: osd_journal_size + type: size + default: 5_K + +For better readability, it is encouraged to use these literal postfixes when +adding or updating the default value for an option. + +Service +------- + +Service is a component name, like "common", "osd", "rgw", "mds", etc. It may +be a list of components, like: + +.. code-block:: yaml + + services: + - mon + - mds + - osd + - mgr + +For example, the rocksdb options affect both the osd and mon. If an option is put +into a service specific ``.yaml.in`` file, the corresponding service is added to +its ``services`` property automatically. For instance, ``osd_scrub_begin_hour`` +option is located in ``osd.yaml.in``, even its ``services`` is not specified +explicitly in this file, this property still contains ``osd``. + +Tags +---- + +Tags identify options across services that relate in some way. For example: + +network + options affecting network configuration +mkfs + options that only matter at mkfs time + +Like: + +.. code-block:: yaml + + tags: + - network + +Enums +----- + +For options with a defined set of allowed values: + +.. code-block:: yaml + + enum_values: + - none + - crc32c + - crc32c_16 + - crc32c_8 + - xxhash32 + - xxhash64 + +Flags +----- + +.. describe:: runtime + + the value can be updated at runtime + +.. describe:: no_mon_update + + Daemons/clients do not pull this value from the monitor config database. We + disallow setting this option via ``ceph config set ...``. This option should + be configured via ``ceph.conf`` or via the command line. + +.. describe:: startup + + option takes effect only during daemon startup + +.. describe:: cluster_create + + option only affects cluster creation + +.. describe:: create + + option only affects daemon creation |