Adding upstream version 18.2.2.upstream/18.2.2

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 283 insertions, 0 deletions

diff --git a/doc/dev/config.rst b/doc/dev/config.rst
new file mode 100644
index 000000000..9cb20aee7
--- /dev/null
+++ b/doc/dev/config.rst
@@ -0,0 +1,283 @@
+=================================
+ Configuration Management System
+=================================
+
+The configuration management system exists to provide every daemon with the
+proper configuration information. The configuration can be viewed as a set of
+key-value pairs.
+
+How can the configuration be set? Well, there are several sources:
+
+ - the ceph configuration file, usually named ceph.conf
+ - command line arguments::
+
+    --debug-ms=1
+    --debug-monc=10
+
+   etc.
+ - arguments injected at runtime using ``injectargs`` or ``config set``
+
+
+The Configuration File
+======================
+
+Most configuration settings originate in the Ceph configuration file.
+
+How do we find the configuration file? Well, in order, we check:
+
+ - the default locations
+ - the environment variable ``CEPH_CONF``
+ - the command line argument ``-c``
+
+Each stanza of the configuration file describes the key-value pairs that will be in
+effect for a particular subset of the daemons. The "global" stanza applies to
+everything. The "mon", "osd", and "mds" stanzas specify settings to take effect
+for all monitors, all OSDs, and all mds servers, respectively.  A stanza of the
+form ``mon.$name``, ``osd.$name``, or ``mds.$name`` gives settings for the monitor, OSD, or
+MDS of that name, respectively. Configuration values that appear later in the
+file win over earlier ones.
+
+A sample configuration file can be found in src/sample.ceph.conf.
+
+
+Metavariables
+=============
+
+The configuration system allows any configuration value to be
+substituted into another value using the ``$varname`` syntax, similar
+to how bash shell expansion works.
+
+A few additional special metavariables are also defined:
+
+ - $host: expands to the current hostname
+ - $type: expands to one of "mds", "osd", "mon", or "client"
+ - $id: expands to the daemon identifier. For ``osd.0``, this would be ``0``; for ``mds.a``, it would be ``a``; for ``client.admin``, it would be ``admin``.
+ - $num: same as $id
+ - $name: expands to $type.$id
+
+
+Reading configuration values
+====================================================
+
+There are two ways for Ceph code to get configuration values. One way is to
+read it directly from a variable named ``g_conf``, or equivalently,
+``g_ceph_ctx->_conf``. The other is to register an observer that will be called
+every time the relevant configuration values change. This observer will be
+called soon after the initial configuration is read, and every time after that
+when one of the relevant values changes. Each observer tracks a set of keys
+and is invoked only when one of the relevant keys changes.
+
+The interface to implement is found in ``common/config_obs.h``.
+
+The observer method should be preferred in new code because
+
+ - It is more flexible, allowing the code to do whatever reinitialization needs
+   to be done to implement the new configuration value.
+ - It is the only way to create a std::string configuration variable that can
+   be changed by injectargs.
+ - Even for int-valued configuration options, changing the values in one thread
+   while another thread is reading them can lead to subtle and
+   impossible-to-diagnose bugs.
+
+For these reasons, reading directly from ``g_conf`` should be considered deprecated
+and not done in new code.  Do not ever alter ``g_conf``.
+
+Changing configuration values
+====================================================
+
+Configuration values can be changed by calling ``g_conf()->set_val``. After changing
+the configuration, you should call ``g_conf()->apply_changes`` to re-run all the
+affected configuration observers. For convenience, you can call
+``g_conf()->set_val_or_die`` to make a configuration change which you think should
+never fail.
+
+``injectargs``, ``parse_argv``, and ``parse_env`` are three other functions which modify
+the configuration. Just like with set_val, you should call apply_changes after
+calling these functions to make sure your changes get applied.
+
+
+Defining config options
+=======================
+
+Config options are defined in ``common/options/*.yaml.in``. The options are categorized
+by their consumers. If an option is only used by ceph-osd, it should go to
+``osd.yaml.in``. All the ``.yaml.in`` files are translated into ``.cc`` and ``.h`` files
+at build time by ``y2c.py``.
+
+Each option is represented using a YAML mapping (dictionary). A typical option looks like
+
+.. code-block:: yaml
+
+   - name: public_addr
+     type: addr
+     level: basic
+     desc: public-facing address to bind to
+     long_desc: The IP address for the public (front-side) network.
+       Set for each daemon.
+     services:
+     - mon
+     - mds
+     - osd
+     - mgr
+     flags:
+     - startup
+     with_legacy: true
+
+In which, following keys are allowed:
+
+level
+-----
+
+The ``level`` property of an option is an indicator for the probability the
+option is adjusted by an operator or a developer:
+
+.. describe:: basic
+
+   for basic config options that a normal operator is likely to adjust.
+
+.. describe:: advanced
+
+   for options that an operator *can* adjust, but should not touch unless they
+   understand what they are doing. Adjusting advanced options poorly can lead to
+   problems (performance or even data loss) if done incorrectly.
+
+.. describe:: dev
+
+   for options in place for use by developers only, either for testing purposes,
+   or to describe constants that no user should adjust but we prefer not to compile
+   into the code.
+
+``desc``, ``long_desc`` and ``fmt_desc``
+----------------------------------------
+
+.. describe:: desc
+
+   Short description of the option. Sentence fragment. e.g.
+
+   .. code-block:: yaml
+
+      desc: Default checksum algorithm to use
+
+.. describe:: long_desc
+
+   The long description is complete sentences, perhaps even multiple
+   paragraphs, and may include other detailed information or notes. e.g.
+
+   .. code-block:: yaml
+
+      long_desc: crc32c, xxhash32, and xxhash64 are available.  The _16 and _8 variants use
+        only a subset of the bits for more compact (but less reliable) checksumming.
+
+.. describe:: fmt_desc
+
+   The description formatted using reStructuredText. This property is
+   only used by the ``confval`` directive to render an option in the
+   document. e.g.:
+
+   .. code-block:: yaml
+
+      fmt_desc: The interval for "deep" scrubbing (fully reading all data). The
+        ``osd_scrub_load_threshold`` does not affect this setting.
+
+Default values
+--------------
+
+There is a default value for every config option. In some cases, there may
+also be a *daemon default* that only applies to code that declares itself
+as a daemon (in this case, the regular default only applies to non-daemons). Like:
+
+.. code-block:: yaml
+
+   default: crc32c
+
+Some literal postfixes are allowed when options with type of ``float``, ``size``
+and ``secs``, like:
+
+.. code-block:: yaml
+
+   - name: mon_scrub_interval
+     type: secs
+     default: 1_day
+   - name: osd_journal_size
+     type: size
+     default: 5_K
+
+For better readability, it is encouraged to use these literal postfixes when
+adding or updating the default value for an option.
+
+Service
+-------
+
+Service is a component name, like "common", "osd", "rgw", "mds", etc. It may
+be a list of components, like:
+
+.. code-block:: yaml
+
+   services:
+   - mon
+   - mds
+   - osd
+   - mgr
+
+For example, the rocksdb options affect both the osd and mon. If an option is put
+into a service specific ``.yaml.in`` file, the corresponding service is added to
+its ``services`` property automatically. For instance, ``osd_scrub_begin_hour``
+option is located in ``osd.yaml.in``, even its ``services`` is not specified
+explicitly in this file, this property still contains ``osd``.
+
+Tags
+----
+
+Tags identify options across services that relate in some way. For example:
+
+network
+  options affecting network configuration
+mkfs
+  options that only matter at mkfs time
+
+Like:
+
+.. code-block:: yaml
+
+   tags:
+   - network
+
+Enums
+-----
+
+For options with a defined set of allowed values:
+
+.. code-block:: yaml
+
+   enum_values:
+   - none
+   - crc32c
+   - crc32c_16
+   - crc32c_8
+   - xxhash32
+   - xxhash64
+
+Flags
+-----
+
+.. describe:: runtime
+
+   the value can be updated at runtime
+
+.. describe:: no_mon_update
+
+   Daemons/clients do not pull this value from the monitor config database.  We
+   disallow setting this option via ``ceph config set ...``.  This option should
+   be configured via ``ceph.conf`` or via the command line.
+
+.. describe:: startup
+
+   option takes effect only during daemon startup
+
+.. describe:: cluster_create
+
+   option only affects cluster creation
+
+.. describe:: create
+
+   option only affects daemon creation