summaryrefslogtreecommitdiffstats
path: root/doc/mgr/dashboard.rst
diff options
context:
space:
mode:
Diffstat (limited to 'doc/mgr/dashboard.rst')
-rw-r--r--doc/mgr/dashboard.rst1655
1 files changed, 1655 insertions, 0 deletions
diff --git a/doc/mgr/dashboard.rst b/doc/mgr/dashboard.rst
new file mode 100644
index 000000000..696676aeb
--- /dev/null
+++ b/doc/mgr/dashboard.rst
@@ -0,0 +1,1655 @@
+.. _mgr-dashboard:
+
+Ceph Dashboard
+==============
+
+Overview
+--------
+
+The Ceph Dashboard is a built-in web-based Ceph management and monitoring
+application through which you can inspect and administer various aspects
+and resources within the cluster. It is implemented as a :ref:`ceph-manager-daemon` module.
+
+The original Ceph Dashboard that was shipped with Ceph Luminous started
+out as a simple read-only view into run-time information and performance
+data of Ceph clusters. It used a very simple architecture to achieve the
+original goal. However, there was growing demand for richer web-based
+management capabilities, to make it easier to administer Ceph for users that
+prefer a WebUI over the CLI.
+
+The new :term:`Ceph Dashboard` module adds web-based monitoring and
+administration to the Ceph Manager. The architecture and functionality of this new
+module are derived from
+and inspired by the `openATTIC Ceph management and monitoring tool
+<https://openattic.org/>`_. Development is actively driven by the
+openATTIC team at `SUSE <https://www.suse.com/>`_, with support from
+companies including `Red Hat <https://redhat.com/>`_ and members of the Ceph
+community.
+
+The dashboard module's backend code uses the CherryPy framework and implements
+a custom REST API. The WebUI implementation is based on
+Angular/TypeScript and includes both functionality from the original dashboard
+and new features originally developed for the standalone version
+of openATTIC. The Ceph Dashboard module is implemented as an
+application that provides a graphical representation of information and statistics
+through a web server hosted by ``ceph-mgr``.
+
+Feature Overview
+^^^^^^^^^^^^^^^^
+
+The dashboard provides the following features:
+
+* **Multi-User and Role Management**: The dashboard supports multiple user
+ accounts with different permissions (roles). User accounts and roles
+ can be managed via both the command line and the WebUI. The dashboard
+ supports various methods to enhance password security. Password
+ complexity rules may be configured, requiring users to change their password
+ after the first login or after a configurable time period. See
+ :ref:`dashboard-user-role-management` for details.
+* **Single Sign-On (SSO)**: The dashboard supports authentication
+ via an external identity provider using the SAML 2.0 protocol. See
+ :ref:`dashboard-sso-support` for details.
+* **SSL/TLS support**: All HTTP communication between the web browser and the
+ dashboard is secured via SSL. A self-signed certificate can be created with
+ a built-in command, but it's also possible to import custom certificates
+ signed and issued by a CA. See :ref:`dashboard-ssl-tls-support` for details.
+* **Auditing**: The dashboard backend can be configured to log all ``PUT``, ``POST``
+ and ``DELETE`` API requests in the Ceph audit log. See :ref:`dashboard-auditing`
+ for instructions on how to enable this feature.
+* **Internationalization (I18N)**: The language used for dashboard text can be
+ selected at run-time.
+
+The Ceph Dashboard offers the following monitoring and management capabilities:
+
+* **Overall cluster health**: Display performance and capacity metrics as well
+ as cluster status.
+* **Embedded Grafana Dashboards**: Ceph Dashboard
+ `Grafana`_ dashboards may be embedded in external applications and web pages
+ to surface information and performance metrics gathered by
+ the :ref:`mgr-prometheus` module. See
+ :ref:`dashboard-grafana` for details on how to configure this functionality.
+* **Cluster logs**: Display the latest updates to the cluster's event and
+ audit log files. Log entries can be filtered by priority, date or keyword.
+* **Hosts**: Display a list of all cluster hosts along with their
+ storage drives, which services are running, and which version of Ceph is
+ installed.
+* **Performance counters**: Display detailed service-specific statistics for
+ each running service.
+* **Monitors**: List all Mons, their quorum status, and open sessions.
+* **Monitoring**: Enable creation, re-creation, editing, and expiration of
+ Prometheus' silences, list the alerting configuration and all
+ configured and firing alerts. Show notifications for firing alerts.
+* **Configuration Editor**: Display all available configuration options,
+ their descriptions, types, default and currently set values. These may be edited as well.
+* **Pools**: List Ceph pools and their details (e.g. applications,
+ pg-autoscaling, placement groups, replication size, EC profile, CRUSH
+ rules, quotas etc.)
+* **OSDs**: List OSDs, their status and usage statistics as well as
+ detailed information like attributes (OSD map), metadata, performance
+ counters and usage histograms for read/write operations. Mark OSDs
+ up/down/out, purge and reweight OSDs, perform scrub operations, modify
+ various scrub-related configuration options, select profiles to
+ adjust the level of backfilling activity. List all drives associated with an
+ OSD. Set and change the device class of an OSD, display and sort OSDs by
+ device class. Deploy OSDs on new drives and hosts.
+* **Device management**: List all hosts known by the orchestrator. List all
+ drives attached to a host and their properties. Display drive
+ health predictions and SMART data. Blink enclosure LEDs.
+* **iSCSI**: List all hosts that run the TCMU runner service, display all
+ images and their performance characteristics (read/write ops, traffic).
+ Create, modify, and delete iSCSI targets (via ``ceph-iscsi``). Display the
+ iSCSI gateway status and info about active initiators.
+ See :ref:`dashboard-iscsi-management` for instructions on how to configure
+ this feature.
+* **RBD**: List all RBD images and their properties (size, objects, features).
+ Create, copy, modify and delete RBD images (incl. snapshots) and manage RBD
+ namespaces. Define various I/O or bandwidth limitation settings on a global,
+ per-pool or per-image level. Create, delete and rollback snapshots of selected
+ images, protect/unprotect these snapshots against modification. Copy or clone
+ snapshots, flatten cloned images.
+* **RBD mirroring**: Enable and configure RBD mirroring to a remote Ceph server.
+ List active daemons and their status, pools and RBD images including
+ sync progress.
+* **CephFS**: List active file system clients and associated pools,
+ including usage statistics. Evict active CephFS clients. Manage CephFS
+ quotas and snapshots. Browse a CephFS directory structure.
+* **Object Gateway**: List all active object gateways and their performance
+ counters. Display and manage (add/edit/delete) object gateway users and their
+ details (e.g. quotas) as well as the users' buckets and their details (e.g.
+ placement targets, owner, quotas, versioning, multi-factor authentication).
+ See :ref:`dashboard-enabling-object-gateway` for configuration instructions.
+* **NFS**: Manage NFS exports of CephFS file systems and RGW S3 buckets via NFS
+ Ganesha. See :ref:`dashboard-nfs-ganesha-management` for details on how to
+ enable this functionality.
+* **Ceph Manager Modules**: Enable and disable Ceph Manager modules, manage
+ module-specific configuration settings.
+
+Overview of the Dashboard Landing Page
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The landing page of Ceph Dashboard serves as the home page and features metrics
+such as the overall cluster status, performance, and capacity. It provides real-time
+updates on any changes in the cluster and allows quick access to other sections of the dashboard.
+
+.. image:: dashboard-landing-page.png
+
+
+.. note::
+ You can change the landing page to the previous version from:
+ ``Cluster >> Manager Modules >> Dashboard >> Edit``.
+ Editing the ``FEATURE_TOGGLE_DASHBOARD`` option will change the landing page, from one view to another.
+
+ Note that the previous version of the landing page will be disabled in future releases.
+
+.. _dashboard-landing-page-details:
+
+Details
+"""""""
+Provides an overview of the cluster configuration, displaying various critical aspects of the cluster.
+
+.. image:: details-card.png
+
+.. _dashboard-landing-page-status:
+
+Status
+""""""
+Provides a visual indication of cluster health, and displays cluster alerts grouped by severity.
+
+.. image:: status-card-open.png
+
+.. _dashboard-landing-page-capacity:
+
+Capacity
+""""""""
+* **Used**: Displays the used capacity out of the total physical capacity provided by storage nodes (OSDs)
+* **Warning**: Displays the `nearfull` threshold of the OSDs
+* **Danger**: Displays the `full` threshold of the OSDs
+
+.. image:: capacity-card.png
+
+.. _dashboard-landing-page-inventory:
+
+Inventory
+"""""""""
+An inventory for all assets within the cluster.
+Provides direct access to subpages of the dashboard from each item of this card.
+
+.. image:: inventory-card.png
+
+.. _dashboard-landing-page-performance:
+
+Cluster Utilization
+"""""""""""""""""""
+* **Used Capacity**: Total capacity used of the cluster. The maximum value of the chart is the maximum capacity of the cluster.
+* **IOPS (Input/Output Operations Per Second)**: Number of read and write operations.
+* **Latency**: Amount of time that it takes to process a read or a write request.
+* **Client Throughput**: Amount of data that clients read or write to the cluster.
+* **Recovery Throughput**: Amount of recovery data that clients read or write to the cluster.
+
+
+.. image:: cluster-utilization-card.png
+
+Supported Browsers
+^^^^^^^^^^^^^^^^^^
+
+Ceph Dashboard is primarily tested and developed using the following web
+browsers:
+
++---------------------------------------------------------------+---------------------------------------+
+| Browser | Versions |
++===============================================================+=======================================+
+| `Chrome <https://www.google.com/chrome/>`_ and | latest 2 major versions |
+| `Chromium <https://www.chromium.org/>`_ based browsers | |
++---------------------------------------------------------------+---------------------------------------+
+| `Firefox <https://www.mozilla.org/firefox/>`_ | latest 2 major versions |
++---------------------------------------------------------------+---------------------------------------+
+| `Firefox ESR <https://www.mozilla.org/firefox/enterprise/>`_ | latest major version |
++---------------------------------------------------------------+---------------------------------------+
+
+While Ceph Dashboard might work in older browsers, we cannot guarantee compatibility and
+recommend keeping your browser up to date.
+
+Enabling
+--------
+
+If you have installed ``ceph-mgr-dashboard`` from distribution packages, the
+package management system should take care of installing all required
+dependencies.
+
+If you're building Ceph from source and want to start the dashboard from your
+development environment, please see the files ``README.rst`` and ``HACKING.rst``
+in the source directory ``src/pybind/mgr/dashboard``.
+
+Within a running Ceph cluster, the Ceph Dashboard is enabled with:
+
+.. prompt:: bash $
+
+ ceph mgr module enable dashboard
+
+Configuration
+-------------
+
+.. _dashboard-ssl-tls-support:
+
+SSL/TLS Support
+^^^^^^^^^^^^^^^
+
+All HTTP connections to the dashboard are secured with SSL/TLS by default.
+
+To get the dashboard up and running quickly, you can generate and install a
+self-signed certificate:
+
+.. prompt:: bash $
+
+ ceph dashboard create-self-signed-cert
+
+Note that most web browsers will complain about self-signed certificates
+and require explicit confirmation before establishing a secure connection to the
+dashboard.
+
+To properly secure a deployment and to remove the warning, a
+certificate that is issued by a certificate authority (CA) should be used.
+
+For example, a key pair can be generated with a command similar to:
+
+.. prompt:: bash $
+
+ openssl req -new -nodes -x509 \
+ -subj "/O=IT/CN=ceph-mgr-dashboard" -days 3650 \
+ -keyout dashboard.key -out dashboard.crt -extensions v3_ca
+
+The ``dashboard.crt`` file should then be signed by a CA. Once that is done, you
+can enable it for Ceph manager instances by running the following commands:
+
+.. prompt:: bash $
+
+ ceph dashboard set-ssl-certificate -i dashboard.crt
+ ceph dashboard set-ssl-certificate-key -i dashboard.key
+
+If unique certificates are desired for each manager instance,
+the name of the instance can be included as follows (where ``$name`` is the name
+of the ``ceph-mgr`` instance, usually the hostname):
+
+.. prompt:: bash $
+
+ ceph dashboard set-ssl-certificate $name -i dashboard.crt
+ ceph dashboard set-ssl-certificate-key $name -i dashboard.key
+
+SSL can also be disabled by setting this configuration value:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/ssl false
+
+This might be useful if the dashboard will be running behind a proxy which does
+not support SSL for its upstream servers or other situations where SSL is not
+wanted or required. See :ref:`dashboard-proxy-configuration` for more details.
+
+.. warning::
+
+ Use caution when disabling SSL as usernames and passwords will be sent to the
+ dashboard unencrypted.
+
+
+.. note::
+
+ You must restart Ceph manager processes after changing the SSL
+ certificate and key. This can be accomplished by either running ``ceph mgr
+ fail mgr`` or by disabling and re-enabling the dashboard module (which also
+ triggers the manager to respawn itself):
+
+ .. prompt:: bash $
+
+ ceph mgr module disable dashboard
+ ceph mgr module enable dashboard
+
+.. _dashboard-host-name-and-port:
+
+Host Name and Port
+^^^^^^^^^^^^^^^^^^
+
+Like most web applications, the dashboard binds to a TCP/IP address and TCP port.
+
+By default, the ``ceph-mgr`` daemon hosting the dashboard (i.e., the currently
+active manager) will bind to TCP port 8443 or 8080 when SSL is disabled.
+
+If no specific address has been configured, the web app will bind to ``::``,
+which corresponds to all available IPv4 and IPv6 addresses.
+
+These defaults can be changed via the configuration key facility on a
+cluster-wide level (so they apply to all manager instances) as follows:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/server_addr $IP
+ ceph config set mgr mgr/dashboard/server_port $PORT
+ ceph config set mgr mgr/dashboard/ssl_server_port $PORT
+
+Since each ``ceph-mgr`` hosts its own instance of the dashboard, it may be
+necessary to configure them separately. The IP address and port for a specific
+manager instance can be changed with the following commands:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/$name/server_addr $IP
+ ceph config set mgr mgr/dashboard/$name/server_port $PORT
+ ceph config set mgr mgr/dashboard/$name/ssl_server_port $PORT
+
+Replace ``$name`` with the ID of the ceph-mgr instance hosting the dashboard.
+
+.. note::
+
+ The command ``ceph mgr services`` will show you all endpoints that are
+ currently configured. Look for the ``dashboard`` key to obtain the URL for
+ accessing the dashboard.
+
+Username and Password
+^^^^^^^^^^^^^^^^^^^^^
+
+In order to be able to log in, you need to create a user account and associate
+it with at least one role. We provide a set of predefined *system roles* that
+you can use. For more details please refer to the `User and Role Management`_
+section.
+
+To create a user with the administrator role you can use the following
+commands:
+
+.. prompt:: bash $
+
+ ceph dashboard ac-user-create <username> -i <file-containing-password> administrator
+
+Account Lock-out
+^^^^^^^^^^^^^^^^
+
+It disables a user account if a user repeatedly enters the wrong credentials
+for multiple times. It is enabled by default to prevent brute-force or dictionary
+attacks. The user can get or set the default number of lock-out attempts using
+these commands respectively:
+
+.. prompt:: bash $
+
+ ceph dashboard get-account-lockout-attempts
+ ceph dashboard set-account-lockout-attempts <value:int>
+
+.. warning::
+
+ This feature can be disabled by setting the default number of lock-out attempts to 0.
+ However, by disabling this feature, the account is more vulnerable to brute-force or
+ dictionary based attacks. This can be disabled by:
+
+ .. prompt:: bash $
+
+ ceph dashboard set-account-lockout-attempts 0
+
+Enable a Locked User
+^^^^^^^^^^^^^^^^^^^^
+
+If a user account is disabled as a result of multiple invalid login attempts, then
+it needs to be manually enabled by the administrator. This can be done by the following
+command:
+
+.. prompt:: bash $
+
+ ceph dashboard ac-user-enable <username>
+
+Accessing the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+You can now access the dashboard using your (JavaScript-enabled) web browser, by
+pointing it to any of the host names or IP addresses and the selected TCP port
+where a manager instance is running: e.g., ``http(s)://<$IP>:<$PORT>/``.
+
+The dashboard page displays and requests a previously defined username and
+password.
+
+.. _dashboard-enabling-object-gateway:
+
+Enabling the Object Gateway Management Frontend
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When RGW is deployed with cephadm, the RGW credentials used by the
+dashboard will be automatically configured. You can also manually force the
+credentials to be set up with:
+
+.. prompt:: bash $
+
+ ceph dashboard set-rgw-credentials
+
+This will create an RGW user with uid ``dashboard`` for each realm in
+the system.
+
+If you've configured a custom 'admin' resource in your RGW admin API, you should set it here also:
+
+.. prompt:: bash $
+
+ ceph dashboard set-rgw-api-admin-resource <admin_resource>
+
+If you are using a self-signed certificate in your Object Gateway setup,
+you should disable certificate verification in the dashboard to avoid refused
+connections, e.g. caused by certificates signed by unknown CA or not matching
+the host name:
+
+.. prompt:: bash $
+
+ ceph dashboard set-rgw-api-ssl-verify False
+
+If the Object Gateway takes too long to process requests and the dashboard runs
+into timeouts, you can set the timeout value to your needs:
+
+.. prompt:: bash $
+
+ ceph dashboard set-rest-requests-timeout <seconds>
+
+The default value is 45 seconds.
+
+.. _dashboard-iscsi-management:
+
+Enabling iSCSI Management
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The Ceph Dashboard can manage iSCSI targets using the REST API provided by the
+``rbd-target-api`` service of the :ref:`ceph-iscsi`. Please make sure that it is
+installed and enabled on the iSCSI gateways.
+
+.. note::
+
+ The iSCSI management functionality of Ceph Dashboard depends on the latest
+ version 3 of the `ceph-iscsi <https://github.com/ceph/ceph-iscsi>`_ project.
+ Make sure that your operating system provides the correct version, otherwise
+ the dashboard will not enable the management features.
+
+If the ``ceph-iscsi`` REST API is configured in HTTPS mode and its using a self-signed
+certificate, you need to configure the dashboard to avoid SSL certificate
+verification when accessing ceph-iscsi API.
+
+To disable API SSL verification run the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard set-iscsi-api-ssl-verification false
+
+The available iSCSI gateways must be defined using the following commands:
+
+.. prompt:: bash $
+
+ ceph dashboard iscsi-gateway-list
+ # Gateway URL format for a new gateway: <scheme>://<username>:<password>@<host>[:port]
+ ceph dashboard iscsi-gateway-add -i <file-containing-gateway-url> [<gateway_name>]
+ ceph dashboard iscsi-gateway-rm <gateway_name>
+
+
+.. _dashboard-grafana:
+
+Enabling the Embedding of Grafana Dashboards
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+`Grafana`_ pulls data from `Prometheus <https://prometheus.io/>`_. Although
+Grafana can use other data sources, the Grafana dashboards we provide contain
+queries that are specific to Prometheus. Our Grafana dashboards therefore
+require Prometheus as the data source. The Ceph :ref:`mgr-prometheus`
+module exports its data in the Prometheus exposition format. These Grafana
+dashboards rely on metric names from the Prometheus module and `Node exporter
+<https://prometheus.io/docs/guides/node-exporter/>`_. The Node exporter is a
+separate application that provides machine metrics.
+
+.. note::
+
+ Prometheus' security model presumes that untrusted users have access to the
+ Prometheus HTTP endpoint and logs. Untrusted users have access to all the
+ (meta)data Prometheus collects that is contained in the database, plus a
+ variety of operational and debugging information.
+
+ However, Prometheus' HTTP API is limited to read-only operations.
+ Configurations can *not* be changed using the API and secrets are not
+ exposed. Moreover, Prometheus has some built-in measures to mitigate the
+ impact of denial of service attacks.
+
+ Please see `Prometheus' Security model
+ <https://prometheus.io/docs/operating/security/>` for more detailed
+ information.
+
+Installation and Configuration using cephadm
+""""""""""""""""""""""""""""""""""""""""""""
+
+Grafana and Prometheus can be installed using :ref:`cephadm`. They will
+automatically be configured by ``cephadm``. Please see
+:ref:`mgr-cephadm-monitoring` documentation for more details on how to use
+``cephadm`` for installing and configuring Prometheus and Grafana.
+
+Manual Installation and Configuration
+"""""""""""""""""""""""""""""""""""""
+
+The following process describes how to configure Grafana and Prometheus
+manually. After you have installed Prometheus, Grafana, and the Node exporter
+on appropriate hosts, proceed with the following steps.
+
+#. Enable the Ceph Exporter which comes as Ceph Manager module by running:
+
+ .. prompt:: bash $
+
+ ceph mgr module enable prometheus
+
+ More details can be found in the documentation of the :ref:`mgr-prometheus`.
+
+#. Add the corresponding scrape configuration to Prometheus. This may look
+ like::
+
+ global:
+ scrape_interval: 5s
+
+ scrape_configs:
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['localhost:9090']
+ - job_name: 'ceph'
+ static_configs:
+ - targets: ['localhost:9283']
+ - job_name: 'node-exporter'
+ static_configs:
+ - targets: ['localhost:9100']
+
+ .. note::
+
+ Please note that in the above example, Prometheus is configured
+ to scrape data from itself (port 9090), the Ceph manager module
+ `prometheus` (port 9283), which exports Ceph internal data, and the Node
+ Exporter (port 9100), which provides OS and hardware metrics for each host.
+
+ Depending on your configuration, you may need to change the hostname in
+ or add additional configuration entries for the Node
+ Exporter. It is unlikely that you will need to change the default TCP ports.
+
+ Moreover, you don't *need* to have more than one target for Ceph specific
+ data, provided by the `prometheus` mgr module. But it is recommended to
+ configure Prometheus to scrape Ceph specific data from all existing Ceph
+ managers. This enables a built-in high availability mechanism, so that
+ services run on a manager host will be restarted automatically on a different
+ manager host if one Ceph Manager goes down.
+
+#. Add Prometheus as data source to Grafana `using the Grafana Web UI <https://grafana.com/docs/grafana/latest/features/datasources/add-a-data-source/>`_.
+
+ .. IMPORTANT::
+ The data source must be named "Dashboard1".
+
+#. Install the `vonage-status-panel and grafana-piechart-panel` plugins using:
+
+ .. prompt:: bash $
+
+ grafana-cli plugins install vonage-status-panel
+ grafana-cli plugins install grafana-piechart-panel
+
+#. Add Dashboards to Grafana:
+
+ Dashboards can be added to Grafana by importing dashboard JSON files.
+ Use the following command to download the JSON files:
+
+ .. prompt:: bash $
+
+ wget https://raw.githubusercontent.com/ceph/ceph/main/monitoring/ceph-mixin/dashboards_out/<Dashboard-name>.json
+
+ You can find various dashboard JSON files `here <https://github.com/ceph/ceph/tree/
+ main/monitoring/ceph-mixin/dashboards_out>`_.
+
+ For Example, for ceph-cluster overview you can use:
+
+ .. prompt:: bash $
+
+ wget https://raw.githubusercontent.com/ceph/ceph/main/monitoring/ceph-mixin/dashboards_out/ceph-cluster.json
+
+ You may also author your own dashboards.
+
+#. Configure anonymous mode in ``/etc/grafana/grafana.ini``::
+
+ [auth.anonymous]
+ enabled = true
+ org_name = Main Org.
+ org_role = Viewer
+
+ In newer versions of Grafana (starting with 6.2.0-beta1) a new setting named
+ ``allow_embedding`` has been introduced. This setting must be explicitly
+ set to ``true`` for the Grafana integration in Ceph Dashboard to work, as the
+ default is ``false``.
+
+ ::
+
+ [security]
+ allow_embedding = true
+
+Enabling RBD-Image monitoring
+"""""""""""""""""""""""""""""
+
+Monitoring of RBD images is disabled by default, as it can significantly impact
+performance. For more information please see :ref:`prometheus-rbd-io-statistics`.
+When disabled, the overview and details dashboards will be empty in Grafana and
+metrics will not be visible in Prometheus.
+
+Configuring Dashboard
+"""""""""""""""""""""
+
+After you have set up Grafana and Prometheus, you will need to configure the
+connection information that the Ceph Dashboard will use to access Grafana.
+
+You need to tell the dashboard on which URL the Grafana instance is
+running/deployed:
+
+.. prompt:: bash $
+
+ ceph dashboard set-grafana-api-url <grafana-server-url> # default: ''
+
+The format of url is : `<protocol>:<IP-address>:<port>`
+
+.. note::
+
+ The Ceph Dashboard embeds Grafana dashboards via ``iframe`` HTML elements.
+ If Grafana is configured without SSL/TLS support, most browsers will block the
+ embedding of insecure content if SSL support is
+ enabled for the dashboard (which is the default). If you
+ can't see the embedded Grafana dashboards after enabling them as outlined
+ above, check your browser's documentation on how to unblock mixed content.
+ Alternatively, consider enabling SSL/TLS support in Grafana.
+
+If you are using a self-signed certificate for Grafana,
+disable certificate verification in the dashboard to avoid refused connections,
+which can be a result of certificates signed by an unknown CA or that do not
+match the host name:
+
+.. prompt:: bash $
+
+ ceph dashboard set-grafana-api-ssl-verify False
+
+You can also access Grafana directly to monitor your cluster.
+
+.. note::
+
+ Ceph Dashboard configuration information can also be unset. For example, to
+ clear the Grafana API URL we configured above:
+
+ .. prompt:: bash $
+
+ ceph dashboard reset-grafana-api-url
+
+Alternative URL for Browsers
+""""""""""""""""""""""""""""
+
+The Ceph Dashboard backend requires the Grafana URL to be able to verify the
+existence of Grafana Dashboards before the frontend even loads them. Due to the
+nature of how Grafana is implemented in Ceph Dashboard, this means that two
+working connections are required in order to be able to see Grafana graphs in
+Ceph Dashboard:
+
+- The backend (Ceph Mgr module) needs to verify the existence of the requested
+ graph. If this request succeeds, it lets the frontend know that it can safely
+ access Grafana.
+- The frontend then requests the Grafana graphs directly from the user's
+ browser using an iframe. The Grafana instance is accessed directly without any
+ detour through Ceph Dashboard.
+
+Now, it might be the case that your environment makes it difficult for the
+user's browser to directly access the URL configured in Ceph Dashboard. To solve
+this issue, a separate URL can be configured which will solely be used to tell
+the frontend (the user's browser) which URL it should use to access Grafana.
+This setting won't ever be changed automatically, unlike the GRAFANA_API_URL
+which is set by :ref:`cephadm` (only if cephadm is used to deploy monitoring
+services).
+
+To change the URL that is returned to the frontend issue the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard set-grafana-frontend-api-url <grafana-server-url>
+
+If no value is set for that option, it will simply fall back to the value of the
+GRAFANA_API_URL option. If set, it will instruct the browser to use this URL to
+access Grafana.
+
+.. _dashboard-sso-support:
+
+Enabling Single Sign-On (SSO)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The Ceph Dashboard supports external authentication of users via the
+`SAML 2.0 <https://en.wikipedia.org/wiki/SAML_2.0>`_ protocol. You need to
+first create user accounts and associate them with desired roles, as
+authorization is performed by the Dashboard. However, the authentication
+process can be performed by an existing Identity Provider (IdP).
+
+.. note::
+
+ Ceph Dashboard SSO support relies on onelogin's
+ `python-saml <https://pypi.org/project/python-saml/>`_ library.
+ Please ensure that this library is installed on your system, either by using
+ your distribution's package management or via Python's `pip` installer.
+
+To configure SSO on Ceph Dashboard, you should use the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard sso setup saml2 <ceph_dashboard_base_url> <idp_metadata> {<idp_username_attribute>} {<idp_entity_id>} {<sp_x_509_cert>} {<sp_private_key>}
+
+Parameters:
+
+* **<ceph_dashboard_base_url>**: Base URL where Ceph Dashboard is accessible (e.g., `https://cephdashboard.local`)
+* **<idp_metadata>**: URL to remote (`http://`, `https://`) or local (`file://`) path or content of the IdP metadata XML (e.g., `https://myidp/metadata`, `file:///home/myuser/metadata.xml`).
+* **<idp_username_attribute>** *(optional)*: Attribute that should be used to get the username from the authentication response. Defaults to `uid`.
+* **<idp_entity_id>** *(optional)*: Use this when more than one entity id exists on the IdP metadata.
+* **<sp_x_509_cert> / <sp_private_key>** *(optional)*: File path of the certificate that should be used by Ceph Dashboard (Service Provider) for signing and encryption (these file paths should be accessible from the active ceph-mgr instance).
+
+.. note::
+
+ The issuer value of SAML requests will follow this pattern: **<ceph_dashboard_base_url>**/auth/saml2/metadata
+
+To display the current SAML 2.0 configuration, use the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard sso show saml2
+
+.. note::
+
+ For more information about `onelogin_settings`, please check the `onelogin documentation <https://github.com/onelogin/python-saml>`_.
+
+To disable SSO:
+
+.. prompt:: bash $
+
+ ceph dashboard sso disable
+
+To check if SSO is enabled:
+
+.. prompt:: bash $
+
+ ceph dashboard sso status
+
+To enable SSO:
+
+.. prompt:: bash $
+
+ ceph dashboard sso enable saml2
+
+.. _dashboard-alerting:
+
+Enabling Prometheus Alerting
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To use Prometheus for alerting you must define `alerting rules
+<https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules>`_.
+These are managed by the `Alertmanager
+<https://prometheus.io/docs/alerting/alertmanager>`_.
+If you are not yet using the Alertmanager, `install it
+<https://github.com/prometheus/alertmanager#install>`_ as it receives
+and manages alerts from Prometheus.
+
+Alertmanager capabilities can be consumed by the dashboard in three different
+ways:
+
+#. Use the notification receiver of the dashboard.
+
+#. Use the Prometheus Alertmanager API.
+
+#. Use both sources simultaneously.
+
+All three methods notify you about alerts. You won't be notified
+twice if you use both sources, but you need to consume at least the Alertmanager API
+in order to manage silences.
+
+1. Use the notification receiver of the dashboard
+
+ This allows you to get notifications as `configured
+ <https://prometheus.io/docs/alerting/configuration/>`_ from the Alertmanager.
+ You will get notified inside the dashboard once a notification is send out,
+ but you are not able to manage alerts.
+
+ Add the dashboard receiver and the new route to your Alertmanager
+ configuration. This should look like::
+
+ route:
+ receiver: 'ceph-dashboard'
+ ...
+ receivers:
+ - name: 'ceph-dashboard'
+ webhook_configs:
+ - url: '<url-to-dashboard>/api/prometheus_receiver'
+
+
+ Ensure that the Alertmanager considers your SSL certificate in terms
+ of the dashboard as valid. For more information about the correct
+ configuration checkout the `<http_config> documentation
+ <https://prometheus.io/docs/alerting/configuration/#%3Chttp_config%3E>`_.
+
+2. Use the API of Prometheus and the Alertmanager
+
+ This allows you to manage alerts and silences and will enable the "Active
+ Alerts", "All Alerts" as well as the "Silences" tabs in the "Monitoring"
+ section of the "Cluster" menu entry.
+
+ Alerts can be sorted by name, job, severity, state and start time.
+ Unfortunately it's not possible to know when an alert was sent out through a
+ notification by the Alertmanager based on your configuration, that's why the
+ dashboard will notify the user on any visible change to an alert and will
+ notify the changed alert.
+
+ Silences can be sorted by id, creator, status, start, updated and end time.
+ Silences can be created in various ways, it's also possible to expire them.
+
+ #. Create from scratch
+
+ #. Based on a selected alert
+
+ #. Recreate from expired silence
+
+ #. Update a silence (which will recreate and expire it (default Alertmanager behaviour))
+
+ To use it, specify the host and port of the Alertmanager server:
+
+ .. prompt:: bash $
+
+ ceph dashboard set-alertmanager-api-host <alertmanager-host:port> # default: ''
+
+ For example:
+
+ .. prompt:: bash $
+
+ ceph dashboard set-alertmanager-api-host 'http://localhost:9093'
+
+ To be able to see all configured alerts, you will need to configure the URL to
+ the Prometheus API. Using this API, the UI will also help you in verifying
+ that a new silence will match a corresponding alert.
+
+
+ .. prompt:: bash $
+
+ ceph dashboard set-prometheus-api-host <prometheus-host:port> # default: ''
+
+ For example:
+
+ .. prompt:: bash $
+
+ ceph dashboard set-prometheus-api-host 'http://localhost:9090'
+
+ After setting up the hosts, refresh your browser's dashboard window or tab.
+
+3. Use both methods
+
+ The behaviors of both methods are configured in a way that they
+ should not disturb each other, through annoying duplicated notifications
+ may pop up.
+
+If you are using a self-signed certificate in your Prometheus or your
+Alertmanager setup, you should disable certificate verification in the
+dashboard to avoid refused connections caused by certificates signed by
+an unknown CA or that do not match the host name.
+
+- For Prometheus:
+
+.. prompt:: bash $
+
+ ceph dashboard set-prometheus-api-ssl-verify False
+
+- For Alertmanager:
+
+.. prompt:: bash $
+
+ ceph dashboard set-alertmanager-api-ssl-verify False
+
+.. _dashboard-user-role-management:
+
+User and Role Management
+------------------------
+
+Password Policy
+^^^^^^^^^^^^^^^
+
+By default the password policy feature is enabled, which includes the
+following checks:
+
+- Is the password longer than N characters?
+- Are the old and new password the same?
+
+The password policy feature can be switched on or off completely:
+
+.. prompt:: bash $
+
+ ceph dashboard set-pwd-policy-enabled <true|false>
+
+The following individual checks can also be switched on or off:
+
+.. prompt:: bash $
+
+ ceph dashboard set-pwd-policy-check-length-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-oldpwd-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-username-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-exclusion-list-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-complexity-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-sequential-chars-enabled <true|false>
+ ceph dashboard set-pwd-policy-check-repetitive-chars-enabled <true|false>
+
+Additionally the following options are available to configure password
+policy.
+
+- Minimum password length (defaults to 8):
+
+.. prompt:: bash $
+
+ ceph dashboard set-pwd-policy-min-length <N>
+
+- Minimum password complexity (defaults to 10):
+
+ .. prompt:: bash $
+
+ ceph dashboard set-pwd-policy-min-complexity <N>
+
+ Password complexity is calculated by classifying each character in
+ the password. The complexity count starts by 0. A character is rated by
+ the following rules in the given order.
+
+ - Increase by 1 if the character is a digit.
+ - Increase by 1 if the character is a lower case ASCII character.
+ - Increase by 2 if the character is an upper case ASCII character.
+ - Increase by 3 if the character is a special character like ``!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~``.
+ - Increase by 5 if the character has not been classified by one of the previous rules.
+
+- A list of comma separated words that are not allowed to be used in a
+ password:
+
+ .. prompt:: bash $
+
+ ceph dashboard set-pwd-policy-exclusion-list <word>[,...]
+
+
+User Accounts
+^^^^^^^^^^^^^
+
+The Ceph Dashboard supports multiple user accounts. Each user account
+consists of a username, a password (stored in encrypted form using ``bcrypt``),
+an optional name, and an optional email address.
+
+If a new user is created via the Web UI, it is possible to set an option that the
+user must assign a new password when they log in for the first time.
+
+User accounts are stored in the monitors' configuration database, and are
+available to all ``ceph-mgr`` instances.
+
+We provide a set of CLI commands to manage user accounts:
+
+- *Show User(s)*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-show [<username>]
+
+- *Create User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-create [--enabled] [--force-password] [--pwd_update_required] <username> -i <file-containing-password> [<rolename>] [<name>] [<email>] [<pwd_expiration_date>]
+
+ To bypass password policy checks use the `force-password` option.
+ Add the option `pwd_update_required` so that a newly created user has
+ to change their password after the first login.
+
+- *Delete User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-delete <username>
+
+- *Change Password*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-set-password [--force-password] <username> -i <file-containing-password>
+
+- *Change Password Hash*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-set-password-hash <username> -i <file-containing-password-hash>
+
+ The hash must be a bcrypt hash and salt, e.g. ``$2b$12$Pt3Vq/rDt2y9glTPSV.VFegiLkQeIpddtkhoFetNApYmIJOY8gau2``.
+ This can be used to import users from an external database.
+
+- *Modify User (name, and email)*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-set-info <username> <name> <email>
+
+- *Disable User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-disable <username>
+
+- *Enable User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-enable <username>
+
+User Roles and Permissions
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+User accounts are associated with a set of roles that define which
+dashboard functionality can be accessed.
+
+The Dashboard functionality/modules are grouped within a *security scope*.
+Security scopes are predefined and static. The current available security
+scopes are:
+
+- **hosts**: includes all features related to the ``Hosts`` menu
+ entry.
+- **config-opt**: includes all features related to management of Ceph
+ configuration options.
+- **pool**: includes all features related to pool management.
+- **osd**: includes all features related to OSD management.
+- **monitor**: includes all features related to monitor management.
+- **rbd-image**: includes all features related to RBD image
+ management.
+- **rbd-mirroring**: includes all features related to RBD mirroring
+ management.
+- **iscsi**: includes all features related to iSCSI management.
+- **rgw**: includes all features related to RADOS Gateway (RGW) management.
+- **cephfs**: includes all features related to CephFS management.
+- **nfs-ganesha**: includes all features related to NFS Ganesha management.
+- **manager**: include all features related to Ceph Manager
+ management.
+- **log**: include all features related to Ceph logs management.
+- **grafana**: include all features related to Grafana proxy.
+- **prometheus**: include all features related to Prometheus alert management.
+- **dashboard-settings**: allows to change dashboard settings.
+
+A *role* specifies a set of mappings between a *security scope* and a set of
+*permissions*. There are four types of permissions:
+
+- **read**
+- **create**
+- **update**
+- **delete**
+
+See below for an example of a role specification, in the form of a Python dictionary::
+
+ # example of a role
+ {
+ 'role': 'my_new_role',
+ 'description': 'My new role',
+ 'scopes_permissions': {
+ 'pool': ['read', 'create'],
+ 'rbd-image': ['read', 'create', 'update', 'delete']
+ }
+ }
+
+The above role dictates that a user has *read* and *create* permissions for
+features related to pool management, and has full permissions for
+features related to RBD image management.
+
+The Dashboard provides a set of predefined roles that we call
+*system roles*, which can be used right away by a fresh Ceph Dashboard
+installation.
+
+The list of system roles are:
+
+- **administrator**: allows full permissions for all security scopes.
+- **read-only**: allows *read* permission for all security scopes except
+ dashboard settings.
+- **block-manager**: allows full permissions for *rbd-image*,
+ *rbd-mirroring*, and *iscsi* scopes.
+- **rgw-manager**: allows full permissions for the *rgw* scope
+- **cluster-manager**: allows full permissions for the *hosts*, *osd*,
+ *monitor*, *manager*, and *config-opt* scopes.
+- **pool-manager**: allows full permissions for the *pool* scope.
+- **cephfs-manager**: allows full permissions for the *cephfs* scope.
+
+The list of available roles can be retrieved with the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard ac-role-show [<rolename>]
+
+You can also use the CLI to create new roles. The available commands are the
+following:
+
+- *Create Role*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-role-create <rolename> [<description>]
+
+- *Delete Role*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-role-delete <rolename>
+
+- *Add Scope Permissions to Role*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-role-add-scope-perms <rolename> <scopename> <permission> [<permission>...]
+
+- *Delete Scope Permission from Role*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-role-del-scope-perms <rolename> <scopename>
+
+To assign roles to users, the following commands are available:
+
+- *Set User Roles*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-set-roles <username> <rolename> [<rolename>...]
+
+- *Add Roles To User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-add-roles <username> <rolename> [<rolename>...]
+
+- *Delete Roles from User*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-del-roles <username> <rolename> [<rolename>...]
+
+
+Example of User and Custom Role Creation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In this section we show a complete example of the commands that
+create a user account that can manage RBD images, view and create Ceph pools,
+and has read-only access to other scopes.
+
+1. *Create the user*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-create bob -i <file-containing-password>
+
+2. *Create role and specify scope permissions*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-role-create rbd/pool-manager
+ ceph dashboard ac-role-add-scope-perms rbd/pool-manager rbd-image read create update delete
+ ceph dashboard ac-role-add-scope-perms rbd/pool-manager pool read create
+
+3. *Associate roles to user*:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-set-roles bob rbd/pool-manager read-only
+
+.. _dashboard-proxy-configuration:
+
+Proxy Configuration
+-------------------
+
+In a Ceph cluster with multiple ``ceph-mgr`` instances, only the dashboard
+running on the currently active ``ceph-mgr`` daemon will serve incoming requests.
+Connections to the dashboard's TCP port on standby ``ceph-mgr`` instances
+will receive an HTTP redirect (303) to the active manager's dashboard URL.
+This enables you to point your browser to any ``ceph-mgr`` instance in
+order to access the dashboard.
+
+If you want to establish a fixed URL to reach the dashboard or if you don't want
+to allow direct connections to the manager nodes, you could set up a proxy that
+automatically forwards incoming requests to the active ``ceph-mgr``
+instance.
+
+Configuring a URL Prefix
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are accessing the dashboard via a reverse proxy,
+you may wish to service it under a URL prefix. To get the dashboard
+to use hyperlinks that include your prefix, you can set the
+``url_prefix`` setting:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/url_prefix $PREFIX
+
+so you can access the dashboard at ``http://$IP:$PORT/$PREFIX/``.
+
+Disable the redirection
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If the dashboard is behind a load-balancing proxy like `HAProxy <https://www.haproxy.org/>`_
+you might want to disable redirection to prevent situations in which
+internal (unresolvable) URLs are published to the frontend client. Use the
+following command to get the dashboard to respond with an HTTP error (500 by default)
+instead of redirecting to the active dashboard:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/standby_behaviour "error"
+
+To reset the setting to default redirection, use the following command:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/standby_behaviour "redirect"
+
+Configure the error status code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When redirection is disabled, you may want to customize the HTTP status
+code of standby dashboards. To do so you need to run the command:
+
+.. prompt:: bash $
+
+ ceph config set mgr mgr/dashboard/standby_error_status_code 503
+
+Resolve IP address to hostname before redirect
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The redirect from a standby to the active dashboard is done via the IP
+address. This is done because resolving IP addresses to hostnames can be error
+prone in containerized environments. It is also the reason why the option is
+disabled by default.
+However, in some situations it might be helpful to redirect via the hostname.
+For example if the configured TLS certificate matches only the hostnames. To
+activate the redirection via the hostname run the following command::
+
+ $ ceph config set mgr mgr/dashboard/redirect_resolve_ip_addr True
+
+You can disable it again by::
+
+ $ ceph config set mgr mgr/dashboard/redirect_resolve_ip_addr False
+
+HAProxy example configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Below you will find an example configuration for SSL/TLS passthrough using
+`HAProxy <https://www.haproxy.org/>`_.
+
+Please note that this configuration works under the following conditions.
+If the dashboard fails over, the front-end client might receive a HTTP redirect
+(303) response and will be redirected to an unresolvable host. This happens when
+failover occurs between two HAProxy health checks. In this situation the
+previously active dashboard node will now respond with a 303 which points to
+the new active node. To prevent that situation you should consider disabling
+redirection on standby nodes.
+
+::
+
+ defaults
+ log global
+ option log-health-checks
+ timeout connect 5s
+ timeout client 50s
+ timeout server 450s
+
+ frontend dashboard_front
+ mode http
+ bind *:80
+ option httplog
+ redirect scheme https code 301 if !{ ssl_fc }
+
+ frontend dashboard_front_ssl
+ mode tcp
+ bind *:443
+ option tcplog
+ default_backend dashboard_back_ssl
+
+ backend dashboard_back_ssl
+ mode tcp
+ option httpchk GET /
+ http-check expect status 200
+ server x <HOST>:<PORT> ssl check verify none
+ server y <HOST>:<PORT> ssl check verify none
+ server z <HOST>:<PORT> ssl check verify none
+
+.. _dashboard-auditing:
+
+Auditing API Requests
+---------------------
+
+The REST API can log PUT, POST and DELETE requests to the Ceph
+audit log. This feature is disabled by default, but can be enabled with the
+following command:
+
+.. prompt:: bash $
+
+ ceph dashboard set-audit-api-enabled <true|false>
+
+If enabled, the following parameters are logged per each request:
+
+* from - The origin of the request, e.g. https://[::1]:44410
+* path - The REST API path, e.g. /api/auth
+* method - e.g. PUT, POST or DELETE
+* user - The name of the user, otherwise 'None'
+
+The logging of the request payload (the arguments and their values) is enabled
+by default. Execute the following command to disable this behaviour:
+
+.. prompt:: bash $
+
+ ceph dashboard set-audit-api-log-payload <true|false>
+
+A log entry may look like this::
+
+ 2018-10-22 15:27:01.302514 mgr.x [INF] [DASHBOARD] from='https://[::ffff:127.0.0.1]:37022' path='/api/rgw/user/klaus' method='PUT' user='admin' params='{"max_buckets": "1000", "display_name": "Klaus Mustermann", "uid": "klaus", "suspended": "0", "email": "klaus.mustermann@ceph.com"}'
+
+.. _dashboard-nfs-ganesha-management:
+
+NFS-Ganesha Management
+----------------------
+
+The dashboard requires enabling the NFS module which will be used to manage
+NFS clusters and NFS exports. For more information check :ref:`mgr-nfs`.
+
+Plug-ins
+--------
+
+Plug-ins extend the functionality of the Ceph Dashboard in a modular
+and loosely coupled fashion.
+
+.. _Grafana: https://grafana.com/
+
+.. include:: dashboard_plugins/feature_toggles.inc.rst
+.. include:: dashboard_plugins/debug.inc.rst
+.. include:: dashboard_plugins/motd.inc.rst
+
+
+Troubleshooting the Dashboard
+-----------------------------
+
+Locating the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unsure of the location of the Ceph Dashboard, run the following command:
+
+.. prompt:: bash $
+
+ ceph mgr services | jq .dashboard
+
+::
+
+ "https://host:port"
+
+The command returns the URL where the Ceph Dashboard is located: ``https://<host>:<port>/``
+
+.. note::
+
+ Many Ceph tools return results in JSON format. We suggest that
+ you install the `jq <https://stedolan.github.io/jq>`_ command-line
+ utility to facilitate working with JSON data.
+
+
+Accessing the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unable to access the Ceph Dashboard, run the following
+commands:
+
+#. Verify the Ceph Dashboard module is enabled:
+
+ .. prompt:: bash $
+
+ ceph mgr module ls | jq .enabled_modules
+
+ Ensure the Ceph Dashboard module is listed in the return value of the
+ command. Example snipped output from the command above::
+
+ [
+ "dashboard",
+ "iostat",
+ "restful"
+ ]
+
+#. If it is not listed, activate the module with the following command:
+
+ .. prompt:: bash $
+
+ ceph mgr module enable dashboard
+
+#. Check the Ceph Dashboard and/or ``ceph-mgr`` log files for any errors.
+
+ * Check if ``ceph-mgr`` log messages are written to a file by:
+
+ .. prompt:: bash $
+
+ ceph config get mgr log_to_file
+
+ ::
+
+ true
+
+ * Get the location of the log file (it's ``/var/log/ceph/<cluster-name>-<daemon-name>.log``
+ by default):
+
+ .. prompt:: bash $
+
+ ceph config get mgr log_file
+
+ ::
+
+ /var/log/ceph/$cluster-$name.log
+
+#. Ensure the SSL/TSL support is configured properly:
+
+ * Check if the SSL/TSL support is enabled:
+
+ .. prompt:: bash $
+
+ ceph config get mgr mgr/dashboard/ssl
+
+ * If the command returns ``true``, verify a certificate exists by:
+
+ .. prompt:: bash $
+
+ ceph config-key get mgr/dashboard/crt
+
+ and:
+
+ .. prompt:: bash $
+
+ ceph config-key get mgr/dashboard/key
+
+ * If it doesn't return ``true``, run the following command to generate a self-signed
+ certificate or follow the instructions outlined in
+ :ref:`dashboard-ssl-tls-support`:
+
+ .. prompt:: bash $
+
+ ceph dashboard create-self-signed-cert
+
+
+Trouble Logging into the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unable to log into the Ceph Dashboard and you receive the following
+error, run through the procedural checks below:
+
+.. image:: ../images/dashboard/invalid-credentials.png
+ :align: center
+
+#. Check that your user credentials are correct. If you are seeing the
+ notification message above when trying to log into the Ceph Dashboard, it
+ is likely you are using the wrong credentials. Double check your username
+ and password, and ensure that your keyboard's caps lock is not enabled by accident.
+
+#. If your user credentials are correct, but you are experiencing the same
+ error, check that the user account exists:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-show <username>
+
+ This command returns your user data. If the user does not exist, it will
+ print::
+
+ Error ENOENT: User <username> does not exist
+
+#. Check if the user is enabled:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-show <username> | jq .enabled
+
+ ::
+
+ true
+
+ Check if ``enabled`` is set to ``true`` for your user. If not the user is
+ not enabled, run:
+
+ .. prompt:: bash $
+
+ ceph dashboard ac-user-enable <username>
+
+Please see :ref:`dashboard-user-role-management` for more information.
+
+
+A Dashboard Feature is Not Working
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When an error occurs on the backend, you will usually receive an error
+notification on the frontend. Run through the following scenarios to debug.
+
+#. Check the Ceph Dashboard and ``ceph-mgr`` logfile(s) for any errors. These can
+ found by searching for keywords, such as *500 Internal Server Error*,
+ followed by ``traceback``. The end of a traceback contains more details about
+ what exact error occurred.
+#. Check your web browser's JavaScript Console for any errors.
+
+
+Ceph Dashboard Logs
+^^^^^^^^^^^^^^^^^^^
+
+Dashboard Debug Flag
+""""""""""""""""""""
+
+With this flag enabled, error traceback is included in backend responses.
+
+To enable this flag via the Ceph Dashboard, navigate from *Cluster* to *Manager
+modules*. Select *Dashboard module* and click the edit button. Click the
+*debug* checkbox and update.
+
+To enable it via the CLI, run the following command:
+
+.. prompt:: bash $
+
+ ceph dashboard debug enable
+
+
+Setting Logging Level of Dashboard Module
+"""""""""""""""""""""""""""""""""""""""""
+
+Setting the logging level to debug makes the log more verbose and helpful for
+debugging.
+
+#. Increase the logging level of manager daemons:
+
+ .. prompt:: bash $
+
+ ceph tell mgr config set debug_mgr 20
+
+#. Adjust the logging level of the Ceph Dashboard module via the Dashboard or
+ CLI:
+
+ * Navigate from *Cluster* to *Manager modules*. Select *Dashboard module*
+ and click the edit button. Modify the ``log_level`` configuration.
+ * To adjust it via the CLI, run the following command:
+
+ .. prompt:: bash $
+
+ bin/ceph config set mgr mgr/dashboard/log_level debug
+
+3. High log levels can result in considerable log volume, which can
+easily fill up your filesystem. Set a calendar reminder for an hour, a day,
+or a week in the future to revert this temporary logging increase. This looks
+something like this:
+
+ .. prompt:: bash $
+
+ ceph config log
+
+ ::
+
+ ...
+ --- 11 --- 2020-11-07 11:11:11.960659 --- mgr.x/dashboard/log_level = debug ---
+ ...
+
+ .. prompt:: bash $
+
+ ceph config reset 11
+
+.. _centralized-logging:
+
+Enable Centralized Logging in Dashboard
+"""""""""""""""""""""""""""""""""""""""
+
+To learn more about centralized logging, see :ref:`cephadm-monitoring-centralized-logs`
+
+1. Create the Loki service on any particular host using "Create Services" option.
+
+2. Similarly create the Promtail service which will be by default deployed
+ on all the running hosts.
+
+3. To see debug-level messages as well as info-level events, run the following command via CLI:
+
+ .. prompt:: bash $
+
+ ceph config set mgr mgr/cephadm/log_to_cluster_level debug
+
+4. To enable logging to files, run the following commands via CLI:
+
+ .. prompt:: bash $
+
+ ceph config set global log_to_file true
+ ceph config set global mon_cluster_log_to_file true
+
+5. Click on the Daemon Logs tab under Cluster -> Logs.
+
+6. You can find some pre-defined labels there on clicking the Log browser button such as filename,
+ job etc that can help you query the logs at one go.
+
+7. You can query the logs with LogQL for advanced search and perform some
+ calculations as well - https://grafana.com/docs/loki/latest/logql/.
+
+
+Reporting issues from Dashboard
+"""""""""""""""""""""""""""""""
+
+Ceph-Dashboard provides two ways to create an issue in the Ceph Issue Tracker,
+either using the Ceph command line interface or by using the Ceph Dashboard
+user interface.
+
+To create an issue in the Ceph Issue Tracker, a user needs to have an account
+on the issue tracker. Under the ``my account`` tab in the Ceph Issue Tracker,
+the user can see their API access key. This key is used for authentication
+when creating a new issue. To store the Ceph API access key, in the CLI run:
+
+.. prompt:: bash $
+
+ ``ceph dashboard set-issue-tracker-api-key -i <file-containing-key>``
+
+Then on successful update, you can create an issue using:
+
+.. prompt:: bash $
+
+ ``ceph dashboard create issue <project> <tracker_type> <subject> <description>``
+
+The available projects to create an issue on are:
+#. dashboard
+#. block
+#. object
+#. file_system
+#. ceph_manager
+#. orchestrator
+#. ceph_volume
+#. core_ceph
+
+The available tracker types are:
+#. bug
+#. feature
+
+The subject and description are then set by the user.
+
+The user can also create an issue using the Dashboard user interface. The settings
+icon drop down menu on the top right of the navigation bar has the option to
+``Raise an issue``. On clicking it, a modal dialog opens that has the option to
+select the project and tracker from their respective drop down menus. The subject
+and multiline description are added by the user. The user can then submit the issue.