Adding upstream version 18.2.2.upstream/18.2.2

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 182 insertions, 0 deletions

diff --git a/doc/cephfs/capabilities.rst b/doc/cephfs/capabilities.rst
new file mode 100644
index 000000000..79dfd3ace
--- /dev/null
+++ b/doc/cephfs/capabilities.rst
@@ -0,0 +1,182 @@
+======================
+Capabilities in CephFS
+======================
+When a client wants to operate on an inode, it will query the MDS in various
+ways, which will then grant the client a set of **capabilities**. This
+grants the client permissions to operate on the inode in various ways. One
+of the major differences from other network file systems (e.g NFS or SMB) is
+that the capabilities granted are quite granular, and it's possible that
+multiple clients can hold different capabilities on the same inodes.
+
+Types of Capabilities
+---------------------
+There are several "generic" capability bits. These denote what sort of ability
+the capability grants.
+
+::
+
+        /* generic cap bits */
+        #define CEPH_CAP_GSHARED     1  /* (metadata) client can read (s) */
+        #define CEPH_CAP_GEXCL       2  /* (metadata) client can read and update (x) */
+        #define CEPH_CAP_GCACHE      4  /* (file) client can cache reads (c) */
+        #define CEPH_CAP_GRD         8  /* (file) client can read (r) */
+        #define CEPH_CAP_GWR        16  /* (file) client can write (w) */
+        #define CEPH_CAP_GBUFFER    32  /* (file) client can buffer writes (b) */
+        #define CEPH_CAP_GWREXTEND  64  /* (file) client can extend EOF (a) */
+        #define CEPH_CAP_GLAZYIO   128  /* (file) client can perform lazy io (l) */
+
+These are then shifted by a particular number of bits. These denote a part of
+the inode's data or metadata on which the capability is being granted:
+
+::
+
+        /* per-lock shift */
+        #define CEPH_CAP_SAUTH      2 /* A */
+        #define CEPH_CAP_SLINK      4 /* L */
+        #define CEPH_CAP_SXATTR     6 /* X */
+        #define CEPH_CAP_SFILE      8 /* F */
+
+Only certain generic cap types are ever granted for some of those "shifts",
+however. In particular, only the FILE shift ever has more than the first two
+bits.
+
+::
+
+        | AUTH | LINK | XATTR | FILE
+        2      4      6       8
+
+From the above, we get a number of constants, that are generated by taking
+each bit value and shifting to the correct bit in the word:
+
+::
+
+        #define CEPH_CAP_AUTH_SHARED  (CEPH_CAP_GSHARED  << CEPH_CAP_SAUTH)
+
+These bits can then be or'ed together to make a bitmask denoting a set of
+capabilities.
+
+There is one exception:
+
+::
+
+        #define CEPH_CAP_PIN  1  /* no specific capabilities beyond the pin */
+
+The "pin" just pins the inode into memory, without granting any other caps.
+
+Graphically:
+
+::
+
+    +---+---+---+---+---+---+---+---+
+    | p | _ |As   x |Ls   x |Xs   x |
+    +---+---+---+---+---+---+---+---+
+    |Fs   x   c   r   w   b   a   l |
+    +---+---+---+---+---+---+---+---+
+
+The second bit is currently unused.
+
+Abilities granted by each cap
+-----------------------------
+While that is how capabilities are granted (and communicated), the important
+bit is what they actually allow the client to do:
+
+* **PIN**: This just pins the inode into memory. This is sufficient to allow
+  the client to get to the inode number, as well as other immutable things like
+  major or minor numbers in a device inode, or symlink contents.
+
+* **AUTH**: This grants the ability to get to the authentication-related metadata.
+  In particular, the owner, group and mode. Note that doing a full permission
+  check may require getting at ACLs as well, which are stored in xattrs.
+
+* **LINK**: The link count of the inode.
+
+* **XATTR**: Ability to access or manipulate xattrs. Note that since ACLs are
+  stored in xattrs, it's also sometimes necessary to access them when checking
+  permissions.
+
+* **FILE**: This is the big one. This allows the client to access and manipulate
+  file data. It also covers certain metadata relating to file data -- the
+  size, mtime, atime and ctime, in particular.
+
+Shorthand
+---------
+Note that the client logging can also present a compact representation of the
+capabilities. For example:
+
+::
+
+        pAsLsXsFs
+
+The 'p' represents the pin. Each capital letter corresponds to the shift
+values, and the lowercase letters after each shift are for the actual
+capabilities granted in each shift.
+
+The relation between the lock states and the capabilities
+---------------------------------------------------------
+In MDS there are four different locks for each inode, they are simplelock,
+scatterlock, filelock and locallock. Each lock has several different lock
+states, and the MDS will issue capabilities to clients based on the lock
+state.
+
+In each state the MDS Locker will always try to issue all the capabilities to the
+clients allowed, even some capabilities are not needed or wanted by the clients,
+as pre-issuing capabilities could reduce latency in some cases.
+
+If there is only one client, usually it will be the loner client for all the inodes.
+While in multiple clients case, the MDS will try to calculate a loner client out for
+each inode depending on the capabilities the clients (needed | wanted), but usually
+it will fail. The loner client will always get all the capabilities.
+
+The filelock will control files' partial metadatas' and the file contents' access
+permissions. The metadatas include **mtime**, **atime**, **size**, etc.
+
+* **Fs**: Once a client has it, all other clients are denied **Fw**.
+
+* **Fx**: Only the loner client is allowed this capability. Once the lock state
+  transitions to LOCK_EXCL, the loner client is granted this along with all other
+  file capabilities except the **Fl**.
+
+* **Fr**: Once a client has it, the **Fb** capability will be already revoked from
+  all the other clients.
+
+  If clients only request to read the file, the lock state will be transferred
+  to LOCK_SYNC stable state directly. All the clients can be granted **Fscrl**
+  capabilities from the auth MDS and **Fscr** capabilities from the replica MDSes.
+
+  If multiple clients read from and write to the same file, then the lock state
+  will be transferred to LOCK_MIX stable state finally and all the clients could
+  have the **Frwl** capabilities from the auth MDS, and the **Fr** from the replica
+  MDSes. The **Fcb** capabilities won't be granted to all the clients and the
+  clients will do sync read/write.
+
+* **Fw**: If there is no loner client and once a client have this capability, the
+  **Fsxcb** capabilities won't be granted to other clients.
+
+  If multiple clients read from and write to the same file, then the lock state
+  will be transferred to LOCK_MIX stable state finally and all the clients could
+  have the **Frwl** capabilities from the auth MDS, and the **Fr** from the replica
+  MDSes. The **Fcb** capabilities won't be granted to all the clients and the
+  clients will do sync read/write.
+
+* **Fc**: This capability means the clients could cache file read and should be
+  issued together with **Fr** capability and only in this use case will it make
+  sense.
+
+  While actually in some stable or interim transitional states they tend to keep
+  the **Fc** allowed even the **Fr** capability isn't granted as this can avoid
+  forcing clients to drop full caches, for example on a simple file size extension
+  or truncating use case.
+
+* **Fb**: This capability means the clients could buffer file write and should be
+  issued together with **Fw** capability and only in this use case will it make
+  sense.
+
+  While actually in some stable or interim transitional states they tend to keep
+  the **Fc** allowed even the **Fw** capability isn't granted as this can avoid
+  forcing clients to drop dirty buffers, for example on a simple file size extension
+  or truncating use case.
+
+* **Fl**: This capability means the clients could perform lazy io. LazyIO relaxes
+  POSIX semantics. Buffered reads/writes are allowed even when a file is opened by
+  multiple applications on multiple clients. Applications are responsible for managing
+  cache coherency themselves.