summaryrefslogtreecommitdiffstats
path: root/doc/cephfs/administration.rst
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--doc/cephfs/administration.rst375
1 files changed, 375 insertions, 0 deletions
diff --git a/doc/cephfs/administration.rst b/doc/cephfs/administration.rst
new file mode 100644
index 000000000..966b45ee5
--- /dev/null
+++ b/doc/cephfs/administration.rst
@@ -0,0 +1,375 @@
+.. _cephfs-administration:
+
+CephFS Administrative commands
+==============================
+
+File Systems
+------------
+
+.. note:: The names of the file systems, metadata pools, and data pools can
+ only have characters in the set [a-zA-Z0-9\_-.].
+
+These commands operate on the CephFS file systems in your Ceph cluster.
+Note that by default only one file system is permitted: to enable
+creation of multiple file systems use ``ceph fs flag set enable_multiple true``.
+
+::
+
+ fs new <file system name> <metadata pool name> <data pool name>
+
+This command creates a new file system. The file system name and metadata pool
+name are self-explanatory. The specified data pool is the default data pool and
+cannot be changed once set. Each file system has its own set of MDS daemons
+assigned to ranks so ensure that you have sufficient standby daemons available
+to accommodate the new file system.
+
+::
+
+ fs ls
+
+List all file systems by name.
+
+::
+
+ fs dump [epoch]
+
+This dumps the FSMap at the given epoch (default: current) which includes all
+file system settings, MDS daemons and the ranks they hold, and the list of
+standby MDS daemons.
+
+
+::
+
+ fs rm <file system name> [--yes-i-really-mean-it]
+
+Destroy a CephFS file system. This wipes information about the state of the
+file system from the FSMap. The metadata pool and data pools are untouched and
+must be destroyed separately.
+
+::
+
+ fs get <file system name>
+
+Get information about the named file system, including settings and ranks. This
+is a subset of the same information from the ``fs dump`` command.
+
+::
+
+ fs set <file system name> <var> <val>
+
+Change a setting on a file system. These settings are specific to the named
+file system and do not affect other file systems.
+
+::
+
+ fs add_data_pool <file system name> <pool name/id>
+
+Add a data pool to the file system. This pool can be used for file layouts
+as an alternate location to store file data.
+
+::
+
+ fs rm_data_pool <file system name> <pool name/id>
+
+This command removes the specified pool from the list of data pools for the
+file system. If any files have layouts for the removed data pool, the file
+data will become unavailable. The default data pool (when creating the file
+system) cannot be removed.
+
+
+Settings
+--------
+
+::
+
+ fs set <fs name> max_file_size <size in bytes>
+
+CephFS has a configurable maximum file size, and it's 1TB by default.
+You may wish to set this limit higher if you expect to store large files
+in CephFS. It is a 64-bit field.
+
+Setting ``max_file_size`` to 0 does not disable the limit. It would
+simply limit clients to only creating empty files.
+
+
+Maximum file sizes and performance
+----------------------------------
+
+CephFS enforces the maximum file size limit at the point of appending to
+files or setting their size. It does not affect how anything is stored.
+
+When users create a file of an enormous size (without necessarily
+writing any data to it), some operations (such as deletes) cause the MDS
+to have to do a large number of operations to check if any of the RADOS
+objects within the range that could exist (according to the file size)
+really existed.
+
+The ``max_file_size`` setting prevents users from creating files that
+appear to be eg. exabytes in size, causing load on the MDS as it tries
+to enumerate the objects during operations like stats or deletes.
+
+
+Taking the cluster down
+-----------------------
+
+Taking a CephFS cluster down is done by setting the down flag:
+
+::
+
+ fs set <fs_name> down true
+
+To bring the cluster back online:
+
+::
+
+ fs set <fs_name> down false
+
+This will also restore the previous value of max_mds. MDS daemons are brought
+down in a way such that journals are flushed to the metadata pool and all
+client I/O is stopped.
+
+
+Taking the cluster down rapidly for deletion or disaster recovery
+-----------------------------------------------------------------
+
+To allow rapidly deleting a file system (for testing) or to quickly bring the
+file system and MDS daemons down, use the ``fs fail`` command:
+
+::
+
+ fs fail <fs_name>
+
+This command sets a file system flag to prevent standbys from
+activating on the file system (the ``joinable`` flag).
+
+This process can also be done manually by doing the following:
+
+::
+
+ fs set <fs_name> joinable false
+
+Then the operator can fail all of the ranks which causes the MDS daemons to
+respawn as standbys. The file system will be left in a degraded state.
+
+::
+
+ # For all ranks, 0-N:
+ mds fail <fs_name>:<n>
+
+Once all ranks are inactive, the file system may also be deleted or left in
+this state for other purposes (perhaps disaster recovery).
+
+To bring the cluster back up, simply set the joinable flag:
+
+::
+
+ fs set <fs_name> joinable true
+
+
+Daemons
+-------
+
+Most commands manipulating MDSs take a ``<role>`` argument which can take one
+of three forms:
+
+::
+
+ <fs_name>:<rank>
+ <fs_id>:<rank>
+ <rank>
+
+Commands to manipulate MDS daemons:
+
+::
+
+ mds fail <gid/name/role>
+
+Mark an MDS daemon as failed. This is equivalent to what the cluster
+would do if an MDS daemon had failed to send a message to the mon
+for ``mds_beacon_grace`` second. If the daemon was active and a suitable
+standby is available, using ``mds fail`` will force a failover to the standby.
+
+If the MDS daemon was in reality still running, then using ``mds fail``
+will cause the daemon to restart. If it was active and a standby was
+available, then the "failed" daemon will return as a standby.
+
+
+::
+
+ tell mds.<daemon name> command ...
+
+Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all
+daemons. Use ``ceph tell mds.* help`` to learn available commands.
+
+::
+
+ mds metadata <gid/name/role>
+
+Get metadata about the given MDS known to the Monitors.
+
+::
+
+ mds repaired <role>
+
+Mark the file system rank as repaired. Unlike the name suggests, this command
+does not change a MDS; it manipulates the file system rank which has been
+marked damaged.
+
+
+Required Client Features
+------------------------
+
+It is sometimes desirable to set features that clients must support to talk to
+CephFS. Clients without those features may disrupt other clients or behave in
+surprising ways. Or, you may want to require newer features to prevent older
+and possibly buggy clients from connecting.
+
+Commands to manipulate required client features of a file system:
+
+::
+
+ fs required_client_features <fs name> add reply_encoding
+ fs required_client_features <fs name> rm reply_encoding
+
+To list all CephFS features
+
+::
+
+ fs feature ls
+
+Clients that are missing newly added features will be evicted automatically.
+
+Here are the current CephFS features and first release they came out:
+
++------------------+--------------+-----------------+
+| Feature | Ceph release | Upstream Kernel |
++==================+==============+=================+
+| jewel | jewel | 4.5 |
++------------------+--------------+-----------------+
+| kraken | kraken | 4.13 |
++------------------+--------------+-----------------+
+| luminous | luminous | 4.13 |
++------------------+--------------+-----------------+
+| mimic | mimic | 4.19 |
++------------------+--------------+-----------------+
+| reply_encoding | nautilus | 5.1 |
++------------------+--------------+-----------------+
+| reclaim_client | nautilus | N/A |
++------------------+--------------+-----------------+
+| lazy_caps_wanted | nautilus | 5.1 |
++------------------+--------------+-----------------+
+| multi_reconnect | nautilus | 5.1 |
++------------------+--------------+-----------------+
+| deleg_ino | octopus | 5.6 |
++------------------+--------------+-----------------+
+| metric_collect | pacific | N/A |
++------------------+--------------+-----------------+
+| alternate_name | pacific | PLANNED |
++------------------+--------------+-----------------+
+
+CephFS Feature Descriptions
+
+
+::
+
+ reply_encoding
+
+MDS encodes request reply in extensible format if client supports this feature.
+
+
+::
+
+ reclaim_client
+
+MDS allows new client to reclaim another (dead) client's states. This feature
+is used by NFS-Ganesha.
+
+
+::
+
+ lazy_caps_wanted
+
+When a stale client resumes, if the client supports this feature, mds only needs
+to re-issue caps that are explicitly wanted.
+
+
+::
+
+ multi_reconnect
+
+When mds failover, client sends reconnect messages to mds, to reestablish cache
+states. If MDS supports this feature, client can split large reconnect message
+into multiple ones.
+
+
+::
+
+ deleg_ino
+
+MDS delegate inode numbers to client if client supports this feature. Having
+delegated inode numbers is a prerequisite for client to do async file creation.
+
+
+::
+
+ metric_collect
+
+Clients can send performance metric to MDS if MDS support this feature.
+
+::
+
+ alternate_name
+
+Clients can set and understand "alternate names" for directory entries. This is
+to be used for encrypted file name support.
+
+
+Global settings
+---------------
+
+
+::
+
+ fs flag set <flag name> <flag val> [<confirmation string>]
+
+Sets a global CephFS flag (i.e. not specific to a particular file system).
+Currently, the only flag setting is 'enable_multiple' which allows having
+multiple CephFS file systems.
+
+Some flags require you to confirm your intentions with "--yes-i-really-mean-it"
+or a similar string they will prompt you with. Consider these actions carefully
+before proceeding; they are placed on especially dangerous activities.
+
+.. _advanced-cephfs-admin-settings:
+
+Advanced
+--------
+
+These commands are not required in normal operation, and exist
+for use in exceptional circumstances. Incorrect use of these
+commands may cause serious problems, such as an inaccessible
+file system.
+
+::
+
+ mds rmfailed
+
+This removes a rank from the failed set.
+
+::
+
+ fs reset <file system name>
+
+This command resets the file system state to defaults, except for the name and
+pools. Non-zero ranks are saved in the stopped set.
+
+
+::
+
+ fs new <file system name> <metadata pool name> <data pool name> --fscid <fscid> --force
+
+This command creates a file system with a specific **fscid** (file system cluster ID).
+You may want to do this when an application expects the file system's ID to be
+stable after it has been recovered, e.g., after monitor databases are lost and
+rebuilt. Consequently, file system IDs don't always keep increasing with newer
+file systems.