path: root/doc/dev/cephx.rst

=====
Cephx
=====

.. _cephx:

Intro
-----

The protocol design looks a lot like kerberos.  The authorizer "KDC"
role is served by the monitor, who has a database of shared secrets
for each entity.  Clients and non-monitor daemons all start by
authenticating with the monitor to obtain tickets, mostly referred to
in the code as authorizers.  These tickets provide both
*authentication* and *authorization* in that they include a
description of the *capabilities* for the entity, a concise structured
description of what actions are allowed, that can be interpreted and
enforced by the service daemons.

Other references
----------------

- A write-up from 2012 on cephx as it existed at that time by Peter
  Reiher: :ref:`cephx_2012_peter`

Terms
-----

- *monitor(s)*: central authorization authority
- *service*: the set of all daemons of a particular type (e.g., all
  OSDs, all MDSs)
- *client*: an entity or principal that is accessing the service
- *entity name*: the string identifier for a principal
  (e.g. client.admin, osd.123)
- *ticket*: a bit of data that cryptographically asserts identify and
  authorization

- *principal*: a client or daemon, identified by a unique entity_name,
  that shares a secret with the monitor.
- *principal_secret*: principal secret, a shared secret (16 bytes)
  known by the principal and the monitor
- *mon_secret*: monitor secret, a shared secret known by all monitors
- *service_secret*: a rotating secret known by all members of a
  service class (e.g., all OSDs)

- *auth ticket*: a ticket proving identity to the monitors
- *service ticket*: a ticket proving identify and authorization to a
  service

  
Terminology
-----------

``{foo, bar}^secret`` denotes encryption by secret.


Context
-------

The authentication messages described here are specific to the cephx
auth implementation.  The messages are transferred by the Messenger
protocol or by MAuth messages, depending on the version of the
messenger protocol.  See also :ref:`msgr2-protocol`.

An initial (messenger) handshake negotiates an authentication method
to be used (cephx vs none or krb or whatever) and an assertion of what
entity the client or daemon is attempting to authenticate as.

Phase I: obtaining auth ticket
------------------------------

The cephx exchange begins with the monitor knowing who the client
claims to be, and an initial cephx message from the monitor to the
client/principal.::

  a->p : 
    CephxServerChallenge {
      u64 server_challenge     # random (by server)
    }

The client responds by adding its own challenge, and calculating a
value derived from both challenges and its shared key
principal_secret.::

  p->a :
    CephxRequestHeader {
      u16 CEPHX_GET_AUTH_SESSION_KEY
    }
    CephXAuthenticate {
      u8 2                     # 2 means nautilus+
      u64 client_challenge     # random (by client)
      u64 key = {client_challenge ^ server_challenge}^principal_secret   # (roughly)
      blob old_ticket          # old ticket, if we are reconnecting or renewing
      u32 other_keys           # bit mask of service keys we want
    }

Prior to nautilus,::

    CephXAuthenticate {
      u8 1                     # 2 means nautilus+
      u64 client_challenge     # random (by client)
      u64 key = {client_challenge + server_challenge}^principal_secret   # (roughly)
      blob old_ticket          # old ticket, if we are reconnecting or renewing
    }

The monitor looks up principal_secret in database, and verifies the
key is correct.  If old_ticket is present, verify it is valid, and we
can reuse the same global_id.  (Otherwise, a new global_id is assigned
by the monitor.)::

  a->p :
    CephxReplyHeader {
      u16 CEPHX_GET_AUTH_SESSION_KEY
      s32 result (0)
    }
    u8 encoding_version = 1
    u32 num_tickets ( = 1)
    ticket_info           # (N = 1)

plus (for Nautilus and later)::

    u32 connection_secret_len      # in bytes
    connection_secret^session_key
    u32 other_keys_len             # bytes of other keys (encoded)
    other_keys {
      u8 encoding_version = 1
      u32 num_tickets
      service_ticket_info * N      # for each service ticket
    }

where::

    ticket_info {
      u32 service_id       # CEPH_ENTITY_TYPE_AUTH
      u8 msg_version (1)
      {CephXServiceTicket service_ticket}^principal_secret
      {CephxTicketBlob ticket_blob}^existing session_key   # if we are renewing a ticket,
      CephxTicketBlob ticket_blob                          # otherwise
    }

    service_ticket_info {
      u32 service_id       # CEPH_ENTITY_TYPE_{OSD,MDS,MGR}
      u8 msg_version (1)
      {CephXServiceTicket service_ticket}^principal_secret
      CephxTicketBlob ticket_blob
    }

    CephxServiceTicket {
      CryptoKey session_key      # freshly generated (even if old_ticket is present)
      utime_t expiration         # now + auth_mon_ticket_ttl
    }

    CephxTicketBlob {
      u64 secret_id             # which service ticket encrypted this; -1 == monsecret, otherwise service's rotating key id
      {CephXServiceTicketInfo ticket}^mon_secret
    }

    CephxServiceTicketInfo {
      CryptoKey session_key     # same session_key as above
      AuthTicket ticket
    }

    AuthTicket {
      EntityName name           # client's identity, as proven by its possession of principal_secret
      u64 global_id             # newly assigned, or from old_ticket
      utime_t created, renew_after, expires
      AuthCapsInfo       # what client is allowed to do
      u32 flags = 0      # unused
    }

So: for each ticket, principal gets a part that it decrypts with its
secret to get the session_key (CephxServiceTicket).  And the
CephxTicketBlob is opaque (secured by the mon secret) but can be used
later to prove who we are and what we can do (see CephxAuthorizer
below).

For Nautilus+, we also include the service tickets.

The client can infer that the monitor is authentic because it can
decrypt the service_ticket with its secret (i.e., the server has its
secret key).


Phase II: Obtaining service tickets (pre-nautilus)
--------------------------------------------------

Now the client needs the keys used to talk to non-monitors (osd, mds,
mgr).::

  p->a :
    CephxRequestHeader {
      u16 CEPHX_GET_PRINCIPAL_SESSION_KEY
    }
    CephxAuthorizer authorizer      
    CephxServiceTicketRequest {
      u32 keys    # bitmask of CEPH_ENTITY_TYPE_NAME (MGR, OSD, MDS, etc)
    }

where::

    CephxAuthorizer {
      u8 AUTH_MODE_AUTHORIZER (1)
      u64 global_id
      u32 service_id    # CEPH_ENTITY_TYPE_*
      CephxTicketBlob auth_ticket
      {CephxAuthorize msg}^session_key
    }

    CephxAuthorize msg {
      u8 2
      u64 nonce                         # random from client
      bool have_challenge = false       # not used here
      u64 server_challenge_plus_one = 0 # not used here
    }

The monitor validates the authorizer by decrypting the auth_ticket
with ``mon_secret`` and confirming that it says this principal is who
they say they are in the CephxAuthorizer fields.  Note that the nonce
random bytes aren't used here (the field exists for Phase III below).

Assuming all is well, the authorizer can generate service tickets
based on the CEPH_ENTITY_TYPE_* bits in the ``keys`` bitmask.

The response looks like::

    CephxResponseHeader {
      u16 CEPHX_GET_PRINCIPAL_SESSION_KEY
      s32 result (= 0)
    }
    u8 encoding_version = 1
    u32 num_tickets
    ticket_info * N
  
Where, as above,::

    ticket_info {
      u32 service_id      # CEPH_ENTITY_TYPE_{OSD,MGR,MDS}
      u8 msg_version (1)
      {CephXServiceTicket service_ticket}^principal_secret
      CephxTicketBlob ticket_blob
    }

    CephxServiceTicket {
      CryptoKey session_key
      utime_t expiration
    }

    CephxTicketBlob {
      u64 secret_id       # which version of the (rotating) service ticket encrypted this
      {CephXServiceTicketInfo ticket}^rotating_service_secret
    }

    CephxServiceTicketInfo {
      CryptoKey session_key
      AuthTicket ticket
    }

    AuthTicket {
      EntityName name
      u64 global_id
      utime_t created, renew_after, expires
      AuthCapsInfo       # what you are allowed to do
      u32 flags = 0      # unused
    }

This concludes the authentication exchange with the monitor.  The
client or daemon now has tickets to talk to the mon and all other
daemons of interest.


Phase III: Opening a connection to a service
--------------------------------------------

When a connection is opened, an "authorizer" payload is sent::

  p->s :
    CephxAuthorizer {
      u8 AUTH_MODE_AUTHORIZER (1)
      u64 global_id
      u32 service_id    # CEPH_ENTITY_TYPE_*
      CephxTicketBlob auth_ticket
      {CephxAuthorize msg}^session_key
    }

    CephxAuthorize msg {
      u8 2
      u64 nonce               # random from client
      bool have_challenge = false
      u64 server_challenge_plus_one = 0
    }

Note that prior to the Luminous v12.2.6 or Mimic v13.2.2 releases, the
CephxAuthorize msg did not contain a challenge, and consisted only
of::

    CephxAuthorize msg {
      u8 1
      u64 nonce               # random from client
    }
    
The server will inspect the auth_ticket CephxTicketBlob (by decrypting
it with its current rotating service key).  If it is a pre-v12.2.6 or
pre-v13.2.2 client, the server immediately replies with::

  s->p :
    {CephxAuthorizeReply reply}^session_key

where::

    CephxAuthorizeReply {
      u64 nonce_plus_one
    }

Otherwise, the server will respond with a challenge (to prevent replay
attacks)::

  s->p :
    {CephxAuthorizeChallenge challenge}^session_key

where::

    CephxAuthorizeChallenge {
      u64 server_challenge        # random from server
    }

The client decrypts and updates its CephxAuthorize msg accordingly,
resending most of the same information as before::

  p->s :
    CephxAuthorizer {
      u8 AUTH_MODE_AUTHORIZER (1)
      u64 global_id
      u32 service_id    # CEPH_ENTITY_TYPE_*
      CephxTicketBlob auth_ticket
      {CephxAuthorize msg}^session_key
    }

where::

    CephxAuthorize msg {
      u8 2
      u64 nonce                        # (new) random from client
      bool have_challenge = true
      u64 server_challenge_plus_one    # server_challenge + 1
    }

The server validates the ticket as before, and then also verifies the
msg nonce has it's challenge + 1, confirming this is a live
authentication attempt (not a replay).

Finally, the server responds with a reply that proves its authenticity
to the client.  It also includes some entropy to use for encryption of
the session, if it is needed for the mode.::

  s->p :
    {CephxAuthorizeReply reply}^session_key

where::

    CephxAuthorizeReply {
      u64 nonce_plus_one
      u32 connection_secret_length
      connection secret
    }

Prior to nautilus, there is no connection secret::

    CephxAuthorizeReply {
      u64 nonce_plus_one
    }

The client decrypts and confirms that the server incremented nonce
properly and that this is thus a live authentication request and not a
replay.


Rotating service secrets
------------------------

Daemons make use of a rotating secret for their tickets instead of a
fixed secret in order to limit the severity of a compromised daemon.
If a daemon's secret key is compromised by an attacker, that daemon
and its key can be removed from the monitor's database, but the
attacker may also have obtained a copy of the service secret shared by
all daemons.  To mitigate this, service keys rotate periodically so
that after a period of time (auth_service_ticket_ttl) the key the
attacker obtained will no longer be valid.::

  p->a :
    CephxRequestHeader {
      u16 CEPHX_GET_ROTATING_KEY
    }

  a->p :
    CephxReplyHeader {
      u16 CEPHX_GET_ROTATING_KEY
      s32 result = 0
    }
    {CryptoKey service_key}^principal_secret

That is, the new rotating key is simply protected by the daemon's
rotating secret.

Note that, as an implementation detail, the services keep the current
key and the prior key on hand so that they can continue to validate
requests while the key is being rotated.