diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/01-About.md | 23 | ||||
| -rw-r--r-- | doc/02-Installation.md | 319 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/01-Amazon-Linux.md | 3 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/02-CentOS.md | 3 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/03-Debian.md | 3 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/04-RHEL.md | 3 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/05-SLES.md | 3 | ||||
| -rw-r--r-- | doc/02-Installation.md.d/06-Ubuntu.md | 3 | ||||
| -rw-r--r-- | doc/03-Configuration.md | 93 | ||||
| -rw-r--r-- | doc/04-Upgrading.md | 51 | ||||
| -rw-r--r-- | doc/05-Distributed-Setups.md | 85 | ||||
| -rw-r--r-- | doc/06-Migration.md | 102 | ||||
| -rw-r--r-- | doc/icingadb-migration.example.yml | 35 | ||||
| -rw-r--r-- | doc/images/icingadb-architecture.png | bin | 0 -> 563761 bytes | |||
| -rw-r--r-- | doc/images/icingadb-daemon.png | bin | 0 -> 527021 bytes | |||
| -rw-r--r-- | doc/images/icingadb-dashboard.png | bin | 0 -> 246028 bytes | |||
| -rw-r--r-- | doc/images/icingadb-database.png | bin | 0 -> 528639 bytes | |||
| -rw-r--r-- | doc/images/icingadb-envs.png | bin | 0 -> 660235 bytes | |||
| -rw-r--r-- | doc/images/icingadb-ha.png | bin | 0 -> 642860 bytes | |||
| -rw-r--r-- | doc/images/icingadb-web.png | bin | 0 -> 532529 bytes |
20 files changed, 726 insertions, 0 deletions
diff --git a/doc/01-About.md b/doc/01-About.md new file mode 100644 index 0000000..8211e57 --- /dev/null +++ b/doc/01-About.md @@ -0,0 +1,23 @@ +# Icinga DB + +Icinga DB is a set of components for publishing, synchronizing and +visualizing monitoring data in the Icinga ecosystem, consisting of: + +* The Icinga DB daemon, which synchronizes monitoring data between a Redis server and a database +* Icinga 2 with its [Icinga DB feature](https://icinga.com/docs/icinga-2/latest/doc/14-features/#icinga-db) enabled, + responsible for publishing the data to the Redis server, i.e. configuration and its runtime updates, check results, + state changes, downtimes, acknowledgements, notifications, and other events such as flapping +* And Icinga Web with the + [Icinga DB Web](https://icinga.com/docs/icinga-db-web) module enabled, + which connects to both Redis and the database to display and work with the most up-to-date data + + + +## Installation + +To install Icinga DB see [Installation](02-Installation.md). + +## License + +Icinga DB and the Icinga DB documentation are licensed under the terms of the +GNU General Public License Version 2. diff --git a/doc/02-Installation.md b/doc/02-Installation.md new file mode 100644 index 0000000..8e04a1f --- /dev/null +++ b/doc/02-Installation.md @@ -0,0 +1,319 @@ +<!-- {% if index %} --> +# Installing Icinga DB + +The recommended way to install Icinga DB is to use prebuilt packages for +all supported platforms from our official release repository. +Please follow the steps listed for your target operating system, +which guide you through setting up the repository and installing Icinga DB. + +To upgrade an existing Icinga DB installation to a newer version, +see the [Upgrading](04-Upgrading.md) documentation for the necessary steps. + + + +Before installing Icinga DB, make sure you have installed [Icinga 2](https://icinga.com/docs/icinga-2), +set up a Redis server, and enabled the `icingadb` feature. +The Icinga 2 installation documentation covers all the necessary steps. +Additionally, Icinga offers the `icingadb-redis` package for all supported operating systems, +which ships an up-to-date Redis server version and is pre-configured for the Icinga DB components. + +!!! tip + + Although Icinga DB can run anywhere in an Icinga environment, + we recommend to install it where the corresponding Icinga 2 node and Redis server is running to + keep latency between the components low. + +<!-- {% else %} --> +## Adding Icinga Package Repository + +The recommended way to install Icinga DB is to use prebuilt packages from our official release repository. + +!!! tip + + If you install Icinga DB on a node that has Icinga 2 or Icinga Web installed via packages, + proceed to [installing the Icinga DB package](#installing-icinga-db-web-package) as + the repository is already configured. + +Here's how to add the official release repository: + +<!-- {% if amazon_linux %} --> +<!-- {% if not icingaDocs %} --> +### Amazon Linux 2 Repository +<!-- {% endif %} --> +!!! info + + A paid repository subscription is required for Amazon Linux 2 repositories. Get more information on + [icinga.com/subscription](https://icinga.com/subscription). + + Don't forget to fill in the username and password section with appropriate credentials in the local .repo file. + +```bash +rpm --import https://packages.icinga.com/icinga.key +wget https://packages.icinga.com/subscription/amazon/ICINGA-release.repo -O /etc/yum.repos.d/ICINGA-release.repo +``` +<!-- {% endif %} --> + +<!-- {% if centos %} --> +<!-- {% if not icingaDocs %} --> +### CentOS Repository +<!-- {% endif %} --> +```bash +rpm --import https://packages.icinga.com/icinga.key +wget https://packages.icinga.com/centos/ICINGA-release.repo -O /etc/yum.repos.d/ICINGA-release.repo +``` +<!-- {% endif %} --> + +<!-- {% if debian %} --> +<!-- {% if not icingaDocs %} --> +### Debian Repository +<!-- {% endif %} --> + +```bash +apt update +apt -y install apt-transport-https wget gnupg + +wget -O - https://packages.icinga.com/icinga.key | apt-key add - + +DIST=$(awk -F"[)(]+" '/VERSION=/ {print $2}' /etc/os-release); \ + echo "deb https://packages.icinga.com/debian icinga-${DIST} main" > \ + /etc/apt/sources.list.d/${DIST}-icinga.list + echo "deb-src https://packages.icinga.com/debian icinga-${DIST} main" >> \ + /etc/apt/sources.list.d/${DIST}-icinga.list + +apt update +``` +<!-- {% endif %} --> + +<!-- {% if rhel %} --> +<!-- {% if not icingaDocs %} --> +### RHEL Repository +<!-- {% endif %} --> +!!! info + + A paid repository subscription is required for RHEL repositories. Get more information on + [icinga.com/subscription](https://icinga.com/subscription). + + Don't forget to fill in the username and password section with appropriate credentials in the local .repo file. + +```bash +rpm --import https://packages.icinga.com/icinga.key +wget https://packages.icinga.com/subscription/rhel/ICINGA-release.repo -O /etc/yum.repos.d/ICINGA-release.repo +``` +<!-- {% endif %} --> + +<!-- {% if sles %} --> +<!-- {% if not icingaDocs %} --> +### SLES Repository +<!-- {% endif %} --> +!!! info + + A paid repository subscription is required for SLES repositories. Get more information on + [icinga.com/subscription](https://icinga.com/subscription). + + Don't forget to fill in the username and password section with appropriate credentials in the local .repo file. + +```bash +rpm --import https://packages.icinga.com/icinga.key + +zypper ar https://packages.icinga.com/subscription/sles/ICINGA-release.repo +zypper ref +``` +<!-- {% endif %} --> + +<!-- {% if ubuntu %} --> +<!-- {% if not icingaDocs %} --> +### Ubuntu Repository +<!-- {% endif %} --> + +```bash +apt update +apt -y install apt-transport-https wget gnupg + +wget -O - https://packages.icinga.com/icinga.key | apt-key add - + +. /etc/os-release; if [ ! -z ${UBUNTU_CODENAME+x} ]; then DIST="${UBUNTU_CODENAME}"; else DIST="$(lsb_release -c| awk '{print $2}')"; fi; \ + echo "deb https://packages.icinga.com/ubuntu icinga-${DIST} main" > \ + /etc/apt/sources.list.d/${DIST}-icinga.list + echo "deb-src https://packages.icinga.com/ubuntu icinga-${DIST} main" >> \ + /etc/apt/sources.list.d/${DIST}-icinga.list + +apt update +``` +<!-- {% endif %} --> + +## Installing Icinga DB Package + +Use your distribution's package manager to install the `icingadb` package as follows: + +<!-- {% if amazon_linux %} --> +<!-- {% if not icingaDocs %} --> +#### Amazon Linux 2 +<!-- {% endif %} --> +```bash +yum install icingadb +``` +<!-- {% endif %} --> + +<!-- {% if centos %} --> +<!-- {% if not icingaDocs %} --> +#### CentOS +<!-- {% endif %} --> +!!! info + + Note that installing Icinga DB is only supported on CentOS 7 as CentOS 8 is EOL. + +```bash +yum install icingadb +``` +<!-- {% endif %} --> + +<!-- {% if debian or ubuntu %} --> +<!-- {% if not icingaDocs %} --> +#### Debian / Ubuntu +<!-- {% endif %} --> +```bash +apt install icingadb +``` +<!-- {% endif %} --> + +<!-- {% if rhel %} --> +#### RHEL 8 or Later + +```bash +dnf install icingadb +``` + +#### RHEL 7 + +```bash +yum install icingadb +``` +<!-- {% endif %} --> + +<!-- {% if sles %} --> +<!-- {% if not icingaDocs %} --> +#### SLES +<!-- {% endif %} --> +```bash +zypper install icingadb +``` +<!-- {% endif %} --> + +## Setting up the Database + +A MySQL (≥5.5), MariaDB (≥10.1), or PostgreSQL (≥9.6) database is required to run Icinga DB. +Please follow the steps listed for your target database, +which guide you through setting up the database and user and importing the schema. + + + +!!! info + + In high availability setups, all Icinga DB instances must write to the same database. + +### Setting up a MySQL or MariaDB Database + +If you use a version of MySQL < 5.7 or MariaDB < 10.2, the following server options must be set: + +``` +innodb_file_format=barracuda +innodb_file_per_table=1 +innodb_large_prefix=1 +``` + +Set up a MySQL database for Icinga DB: + +``` +# mysql -u root -p + +CREATE DATABASE icingadb; +CREATE USER 'icingadb'@'localhost' IDENTIFIED BY 'CHANGEME'; +GRANT ALL ON icingadb.* TO 'icingadb'@'localhost'; +``` + +After creating the database, import the Icinga DB schema using the following command: + +``` +mysql -u root -p icingadb </usr/share/icingadb/schema/mysql/schema.sql +``` + +### Setting up a PostgreSQL Database + +Set up a PostgreSQL database for Icinga DB: + +``` +# su -l postgres + +createuser -P icingadb +createdb -E UTF8 --locale en_US.UTF-8 -T template0 -O icingadb icingadb +psql icingadb <<<'CREATE EXTENSION IF NOT EXISTS citext;' +``` + +The `CREATE EXTENSION` command requires the `postgresql-contrib` package. + +Edit `pg_hba.conf`, insert the following before everything else: + +``` +local all icingadb md5 +host all icingadb 0.0.0.0/0 md5 +host all icingadb ::/0 md5 +``` + +To apply these changes, run `systemctl reload postgresql`. + +After creating the database, import the Icinga DB schema using the following command: + +``` +psql -U icingadb icingadb < /usr/share/icingadb/schema/pgsql/schema.sql +``` + +## Configuring Icinga DB + +Icinga DB installs its configuration file to `/etc/icingadb/config.yml`, +pre-populating most of the settings for a local setup. Before running Icinga DB, +adjust the Redis and database credentials and, if necessary, the connection configuration. +The configuration file explains general settings. +All available settings can be found under [Configuration](03-Configuration.md). + +## Running Icinga DB + +The `icingadb` package automatically installs the necessary systemd unit files to run Icinga DB. +Please run the following command to enable and start its service: + +```bash +systemctl enable --now icingadb +``` + +## Installing Icinga DB Web + +With Icinga 2, Redis, Icinga DB and the database fully set up, it is now time to install Icinga DB Web, +which connects to both Redis and the database to display and work with the monitoring data. + + + +The Icinga DB Web package is also included in the Icinga repository, and since it is already set up, +you have completed the instructions here and can proceed to +<!-- {% if amazon_linux %} --> +[installing Icinga DB Web on Amazon Linux](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/01-Amazon-Linux/#installing-icinga-db-web-package), +<!-- {% endif %} --> +<!-- {% if centos %} --> +[installing Icinga DB Web on CentOS](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/02-CentOS/#installing-icinga-db-web-package), +<!-- {% endif %} --> +<!-- {% if debian %} --> +[installing Icinga DB Web on Debian](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/03-Debian/#installing-icinga-db-web-package), +<!-- {% endif %} --> +<!-- {% if rhel %} --> +[installing Icinga DB Web on RHEL](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/04-RHEL/#installing-icinga-db-web-package), +<!-- {% endif %} --> +<!-- {% if sles %} --> +[installing Icinga DB Web on SLES](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/05-SLES/#installing-icinga-db-web-package), +<!-- {% endif %} --> +<!-- {% if ubuntu %} --> +[installing Icinga DB Web on Ubuntu](https://icinga.com/docs/icinga-db-web/latest/doc/02-Installation/06-Ubuntu/#installing-icinga-db-web-package), +<!-- {% endif %} --> +which will also guide you through the setup of the Icinga Web PHP framework, +which is required to run the Icinga DB web module. +Below is a preview of how the interface visualizes monitoring data and also supports dark and light mode: + + +<!-- {% endif %} --><!-- {# end else if index #} --> diff --git a/doc/02-Installation.md.d/01-Amazon-Linux.md b/doc/02-Installation.md.d/01-Amazon-Linux.md new file mode 100644 index 0000000..cb5e161 --- /dev/null +++ b/doc/02-Installation.md.d/01-Amazon-Linux.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on Amazon Linux +<!-- {% set amazon_linux = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/02-Installation.md.d/02-CentOS.md b/doc/02-Installation.md.d/02-CentOS.md new file mode 100644 index 0000000..343d640 --- /dev/null +++ b/doc/02-Installation.md.d/02-CentOS.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on CentOS +<!-- {% set centos = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/02-Installation.md.d/03-Debian.md b/doc/02-Installation.md.d/03-Debian.md new file mode 100644 index 0000000..2cea527 --- /dev/null +++ b/doc/02-Installation.md.d/03-Debian.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on Debian +<!-- {% set debian = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/02-Installation.md.d/04-RHEL.md b/doc/02-Installation.md.d/04-RHEL.md new file mode 100644 index 0000000..0dd059a --- /dev/null +++ b/doc/02-Installation.md.d/04-RHEL.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on RHEL +<!-- {% set rhel = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/02-Installation.md.d/05-SLES.md b/doc/02-Installation.md.d/05-SLES.md new file mode 100644 index 0000000..9af0e48 --- /dev/null +++ b/doc/02-Installation.md.d/05-SLES.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on SLES +<!-- {% set sles = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/02-Installation.md.d/06-Ubuntu.md b/doc/02-Installation.md.d/06-Ubuntu.md new file mode 100644 index 0000000..0ce05e0 --- /dev/null +++ b/doc/02-Installation.md.d/06-Ubuntu.md @@ -0,0 +1,3 @@ +# Installing Icinga DB on Ubuntu +<!-- {% set ubuntu = True %} --> +<!-- {% include "02-Installation.md" %} --> diff --git a/doc/03-Configuration.md b/doc/03-Configuration.md new file mode 100644 index 0000000..af92783 --- /dev/null +++ b/doc/03-Configuration.md @@ -0,0 +1,93 @@ +# Configuration + +The configuration is stored in `/etc/icingadb/config.yml`. +See [config.example.yml](../config.example.yml) for an example configuration. + +## Redis Configuration + +Connection configuration for the Redis server where Icinga 2 writes its configuration, state and history items. +This is the same connection as configured in the +[Icinga DB feature](https://icinga.com/docs/icinga-2/latest/doc/14-features/#icinga-db) of +the corresponding Icinga 2 node. High availability setups require a dedicated Redis server per Icinga 2 node and +therefore a dedicated Icinga DB instance that connects to it. + +| Option | Description | +|----------|------------------------------------------------------------------------------------------------------------------------------------| +| host | **Required.** Redis host or absolute Unix socket path. | +| port | **Optional.** Redis port. Defaults to `6380` since the Redis server provided by the `icingadb-redis` package listens on that port. | +| password | **Optional.** The password to use. | +| tls | **Optional.** Whether to use TLS. | +| cert | **Optional.** Path to TLS client certificate. | +| key | **Optional.** Path to TLS private key. | +| ca | **Optional.** Path to TLS CA certificate. | +| insecure | **Optional.** Whether not to verify the peer. | + +## Database Configuration + +Connection configuration for the database to which Icinga DB synchronizes monitoring data. +This is also the database used in +[Icinga DB Web](https://icinga.com/docs/icinga-db/latest/icinga-db-web/doc/01-About/) to view and work with the data. +In high availability setups, all Icinga DB instances must write to the same database. + +| Option | Description | +|----------|--------------------------------------------------------------------------------------------------------| +| type | **Optional.** Either `mysql` (default) or `pgsql`. | +| host | **Required.** Database host or absolute Unix socket path. | +| port | **Optional.** Database port. By default, the MySQL or PostgreSQL port, depending on the database type. | +| database | **Required.** Database name. | +| user | **Required.** Database username. | +| password | **Optional.** Database password. | +| tls | **Optional.** Whether to use TLS. | +| cert | **Optional.** Path to TLS client certificate. | +| key | **Optional.** Path to TLS private key. | +| ca | **Optional.** Path to TLS CA certificate. | +| insecure | **Optional.** Whether not to verify the peer. | + +## Logging Configuration + +Configuration of the logging component used by Icinga DB. + +| Option | Description | +|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| level | **Optional.** Specifies the default logging level. Can be set to `fatal`, `error`, `warn`, `info` or `debug`. Defaults to `info`. | +| output | **Optional.** Configures the logging output. Can be set to `console` (stderr) or `systemd-journald`. If not set, logs to systemd-journald when running under systemd, otherwise stderr. | +| interval | **Optional.** Interval for periodic logging defined as [duration string](#duration-string). Defaults to `"20s"`. | +| options | **Optional.** Map of component name to logging level in order to set a different logging level for each component instead of the default one. See [logging components](#logging-components) for details. | + +### Logging Components + +| Component | Description | +|-------------------|--------------------------------------------------------------------------------| +| config-sync | Config object synchronization between Redis and MySQL. | +| database | Database connection status and queries. | +| dump-signals | Dump signals received from Icinga. | +| heartbeat | Icinga heartbeats received through Redis. | +| high-availability | Manages responsibility of Icinga DB instances. | +| history-sync | Synchronization of history entries from Redis to MySQL. | +| overdue-sync | Calculation and synchronization of the overdue status of checkables. | +| redis | Redis connection status and queries. | +| retention | Deletes historical data that exceed their configured retention period. | +| runtime-updates | Runtime updates of config objects after the initial config synchronization. | +| telemetry | Reporting of Icinga DB status to Icinga 2 via Redis (for monitoring purposes). | + +## Retention + +By default, no historical data is deleted, which means that the longer the data is retained, +the more disk space is required to store it. History retention is an optional feature that allows to +limit the number of days that historical data is available for each history category. +There are separate options for the full history tables used to display history information in the web interface and +SLA tables which store the minimal information required for SLA reporting, +allowing to keep this information for longer with a smaller storage footprint. + +| Option | Description | +|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| history-days | **Optional.** Number of days to retain historical data for all history categories. Use `options` in order to enable retention only for specific categories or to override the retention days configured here. | +| sla-days | **Optional.** Number of days to retain historical data for SLA reporting. | +| options | **Optional.** Map of history category to number of days to retain its data. Available categories are `acknowledgement`, `comment`, `downtime`, `flapping`, `notification`, `sla` and `state`. | + +## Appendix + +### Duration String + +A duration string is a sequence of decimal numbers and a unit suffix, such as `"20s"`. +Valid units are `"ms"`, `"s"`, `"m"` and `"h"`. diff --git a/doc/04-Upgrading.md b/doc/04-Upgrading.md new file mode 100644 index 0000000..5a085e0 --- /dev/null +++ b/doc/04-Upgrading.md @@ -0,0 +1,51 @@ +# Upgrading Icinga DB + +Specific version upgrades are described below. Please note that version upgrades are incremental. +If you are upgrading across multiple versions, make sure to follow the steps for each of them. + +## Upgrading to Icinga DB v1.1.1 + +Please apply the `1.1.1.sql` upgrade script to your database. +For package installations, you can find this file at `/usr/share/icingadb/schema/mysql/upgrades/` or +`/usr/share/icingadb/schema/pgsql/upgrades/`, depending on your database type. + +Note that this upgrade will change the `history` table, which can take some time depending on the size of the table and +the performance of the database. While the upgrade is running, that table will be locked and can't be accessed. This +means that the existing history can't be viewed in Icinga Web and new history entries will be buffered in Redis. + +As the daemon checks the schema version, the recommended way to perform the upgrade is to stop the daemon, apply the +schema upgrade and then start the new daemon version. If you want to minimize downtime as much as possible, it is safe +to apply this schema upgrade while the Icinga DB v1.1.0 daemon is still running and then restart the daemon with the +new version. Please keep in mind that depending on the distribution, your package manager may automatically attempt to +restart the daemon when upgrading the package. + +## Upgrading to Icinga DB v1.0 + +**Requirements** + +* You need at least Icinga 2 version 2.13.4 to run Icinga DB v1.0.0. + +**Database Schema** + +* For MySQL databases, please apply the `1.0.0.sql` upgrade script. + For package installations, you can find this file at `/usr/share/icingadb/schema/mysql/upgrades/`. + +## Upgrading to Icinga DB RC2 + +Icinga DB RC2 is a complete rewrite compared to RC1. Because of this, a lot has changed in the Redis and database +schema, which is why they have to be deleted and recreated. The configuration file has changed from `icingadb.ini` +to `config.yml`. Instead of the INI format, we are now using YAML and have introduced more configuration options. We +have also changed the packages of `icingadb-redis`, which is why the Redis CLI commands are now prefixed with `icingadb` +instead of just `icinga`, i.e. the Redis CLI is now accessed via `icingadb-redis-cli`. + +Please follow the steps below to upgrade to Icinga DB RC2: + +1. Stop Icinga 2 and Icinga DB. +2. Flush your Redis instances using `icinga-redis-cli flushall` (note the `icinga` prefix as we did not + upgrade `icingadb-redis` yet) and stop them afterwards. +3. Upgrade Icinga 2 to version 2.13.2 or newer. +4. Remove the `icinga-redis` package where installed as it may conflict with `icingadb-redis`. +5. Install Icinga DB Redis (`icingadb-redis`) on your primary Icinga 2 nodes to version 6.2.6 or newer. +6. Upgrade Icinga DB to RC2. +7. Drop the Icinga DB MySQL database and recreate it using the provided schema. +8. Start Icinga DB Redis, Icinga 2 and Icinga DB. diff --git a/doc/05-Distributed-Setups.md b/doc/05-Distributed-Setups.md new file mode 100644 index 0000000..a324a65 --- /dev/null +++ b/doc/05-Distributed-Setups.md @@ -0,0 +1,85 @@ +# Distributed Setups + +## High Availability + +High availability functionality is built directly into Icinga DB and +can be deployed without additional third-party components. + + + +First, you need an Icinga 2 high availability setup with two master nodes, such as described +[here](https://icinga.com/docs/icinga-2/latest/doc/06-distributed-monitoring#high-availability-master-with-agents). + +Each of the master nodes must have the Icinga DB feature enabled and +have their own dedicated Redis server set up for it, so that each node writes the monitoring data separately. +The setup steps per node are no different from a single node setup and can be found in the +[Icinga 2 installation documentation](https://icinga.com/docs/icinga-2/latest/doc/02-installation). +Each Redis server will always have the complete data available as long as +its corresponding Icinga 2 master is running and writing to its Redis. +This is because the Icinga 2 master nodes synchronize their data and events with each other as long as +they are connected, +and each takes over the entire configuration in case the other node or their connection to each other fails. + +For each Redis server you need to set up its own dedicated Icinga DB instance that connects to it, +but the Icinga DB instances must write to the same database, which of course can be replicated or a cluster. +So the steps from the standard +[Icinga DB installation documentation](https://icinga.com/docs/icinga-db/latest/doc/02-installation) +can be followed. However, as mentioned, the database only needs to be set up once. + +All in all, an Icinga DB HA environment involves setting up two Icinga 2 master nodes, two Redis servers, +two Icinga DB instances and one database. + +Please read the note about the [environment ID](#environment-id), +which is common to all Icinga DB components and generated by Icinga 2's Icinga DB feature. + +There is only one Icinga DB instance at a time, +which is responsible for performing database operations in the following areas: + +* Synchronizing configuration, also across Icinga 2 restarts. +* Performing configuration runtime updates made via the Icinga 2 API. +* Updating recent host and service states. +* Flagging hosts and services that are overdue for updates. +* Deleting history items that have exceeded their configured retention time. + +However, both Icinga DB instances write all events relevant to the history of hosts and services to the database. +This way, no data is lost if one of the Icinga 2 masters is unavailable or if their connection to each other fails. + +Which Icinga DB instance is responsible is determined by a specific database operation that +can only be performed by one instance first. +In the case of concurrent operations, simply put, only one wins via a locking mechanism. +Of course, this is only true if the environment is healthy. +Icinga DB is not trying to be responsible if its corresponding Redis server is unavailable or +Icinga 2 is not writing data to Redis. +If Icinga 2 or Redis become unavailable for more than 60 seconds, +Icinga DB releases responsibility so the other instance can take over. + +## Multiple Environments + +Icinga DB supports synchronization of monitoring data from multiple different Icinga environments into +a single database. This allows Icinga DB Web to provide a centralized view of the data. +Although everything is prepared in Icinga DB, there is no full support in Icinga DB Web yet. +As soon as it is ready, the documentation will be adapted and the feature will be explained in more detail. + + + +## Environment ID + +!!! important + + Icinga 2 generates a unique environment ID from its CA certificate when it is first started with the + Icinga DB feature enabled. The ID is used in all Icinga DB components to separate data from + multiple different environments and is written to the file `/var/lib/icinga2/icingadb.env` by Icinga 2. + + It is strictly recommended not to change this ID afterwards, as all data would be resynchronized and + the old ones would remain in the database, resulting in duplicate data. As long as the file remains, + Icinga 2 will not regenerate the environment ID. This is also true if the CA is changed to avoid duplicate data. + Special care should be taken if you add or redeploy the master node(s) and + as a result or over time the CA has changed, which would result in a new environment ID. + If this is the case, make sure you copy the file between your Icinga 2 masters to get a consistent ID. + + For high-availability setups, it is a good idea to enable the Icinga DB feature on the secondary master after + you have successfully connected from/to the primary master so that the certificates are set up properly. + The secondary master will then generate the same environment ID since it is working with the same CA certificate. + + In any case make sure that the file `/var/lib/icinga2/icingadb.env` does not change over time and + is the same on all Icinga 2 master nodes per environment. diff --git a/doc/06-Migration.md b/doc/06-Migration.md new file mode 100644 index 0000000..5a87e41 --- /dev/null +++ b/doc/06-Migration.md @@ -0,0 +1,102 @@ +# Migration from IDO + +Migrating from the IDO feature to Icinga DB starts by setting up Icinga DB. To +do so, please follow the [installation instructions]. The Icinga DB feature can +be enabled in parallel to the IDO, allowing you to perform the migration while +the IDO is still running. + +After setting up Icinga DB, all Icinga objects and their current state should +already show up in Icinga DB Web as this information is synced from Icinga 2. +At this point, the old host and service history is missing in Icinga DB. If it +is desired to keep it, this information has to be migrated explicitly from the +old IDO database. To do so, follow the instructions below. + +## History + +To migrate history data from the [IDO] database, the Icinga DB Migration +commandline tool is provided. If you have installed Icinga DB from our +packages, it is automatically installed as well. + +### Preparing the Configuration + +Please take the [example configuration] as a starting point and copy it to the +host you will perform the migration on. The following sections will guide you +through how to adjust it for your needs. + +#### Environment ID + +Icinga DB allows writing multiple Icinga environments to the same database. +Thus, you have to tell the migration tool for which environment you want to +migrate the history. On each Icinga 2 node that has the Icinga DB feature +enabled, the environment ID is written to the file +`/var/lib/icinga2/icingadb.env`. Please use the contents of this file for the +`env` option in the section `icinga2`. + +#### Database Connection + +The migration tool needs to access both the IDO and the Icinga DB databases. +Please specify the connection details in the corresponding `ido` and `icingadb` +sections of the configuration. + +Both the IDO and Icinga DB support MySQL and PostgreSQL. You can migrate from +and to both types, including from one type to the other. + +#### Input Time Range + +The migration tool allows you to restrict the time range of the history events +to be migrated. This is controlled by the options `from` and `to` in the `ido` +section of the configuration. Both options can be set to Unix timestamps. + +It is recommended to set the `to` option to a cutoff time at which the history +in the Icinga DB database switches from migrated events to events written +directly by Icinga DB. If you kept running the IDO in parallel to Icinga DB and +do not do this, there will be duplicate events for the time both were running. + +You can query the time of the first history event written by Icinga DB by +running this query in its database: + +``` +SELECT MIN(event_time)/1000 FROM history; +``` + +In case you had trouble setting up Icinga DB or this is not the first time you +are setting up Icinga DB, please make sure to double-check this timestamp and +adjust it accordingly if it is not what you expect. + +!!! tip + + You can convert between Unix timestamps and a human-readable format using the `date` command: + + * Unix timestamp to readable date: `date -d @1667219820` + * Current date/time to Unix timestamp: `date +%s` + * Specific date/time to Unix timestamp: `date -d '2022-01-01 00:00:00' +%s` + * Relative date/time to Unix timestamp: `date -d '-1 year' +%s` + +Similarly, you can use `from` to limit how much old history gets migrated. + +### Cache Directory + +Choose a (not necessarily yet existing) directory for Icinga DB Migration's +internal cache. If either there isn't much to migrate or the migration +process won't be interrupted by a reboot (of the machine +Icinga DB migration/database runs on), `mktemp -d` is enough. + +### Run the Migration + +To start the actual migration, execute the following command: + +```shell +icingadb-migrate -c icingadb-migration.yml -t ~/icingadb-migration.cache +``` + +In case this command was interrupted, you can run it again. It will continue +where it left off and reuse the cache if it is still present. + +!!! tip + + If there is much to migrate, use e.g. tmux to + protect yourself against SSH connection losses. + +[installation instructions]: 02-Installation.md +[IDO]: https://icinga.com/docs/icinga-2/latest/doc/14-features/#ido-database-db-ido +[example configuration]: icingadb-migration.example.yml diff --git a/doc/icingadb-migration.example.yml b/doc/icingadb-migration.example.yml new file mode 100644 index 0000000..2acd883 --- /dev/null +++ b/doc/icingadb-migration.example.yml @@ -0,0 +1,35 @@ +icinga2: + # Content of /var/lib/icinga2/icingadb.env + env: "da39a3e..." + +# IDO database +ido: + type: mysql # or "pgsql" for PostgreSQL + host: 192.0.2.1 + #port: 3306 + database: icinga + user: icinga + password: CHANGEME + #tls: false + #cert: <Path to TLS client certificate> + #key: <Path to TLS private key> + #ca: <Path to TLS CA certificate> + #insecure: false + + # Input time range + #from: 0 + #to: 2147483647 + +# Icinga DB database +icingadb: + type: mysql # or "pgsql" for PostgreSQL + host: 2001:db8::1 + #port: 3306 + database: icingadb + user: icingadb + password: CHANGEME + #tls: false + #cert: <Path to TLS client certificate> + #key: <Path to TLS private key> + #ca: <Path to TLS CA certificate> + #insecure: false diff --git a/doc/images/icingadb-architecture.png b/doc/images/icingadb-architecture.png Binary files differnew file mode 100644 index 0000000..3d55ff7 --- /dev/null +++ b/doc/images/icingadb-architecture.png diff --git a/doc/images/icingadb-daemon.png b/doc/images/icingadb-daemon.png Binary files differnew file mode 100644 index 0000000..de3f4c7 --- /dev/null +++ b/doc/images/icingadb-daemon.png diff --git a/doc/images/icingadb-dashboard.png b/doc/images/icingadb-dashboard.png Binary files differnew file mode 100644 index 0000000..521eb39 --- /dev/null +++ b/doc/images/icingadb-dashboard.png diff --git a/doc/images/icingadb-database.png b/doc/images/icingadb-database.png Binary files differnew file mode 100644 index 0000000..c300095 --- /dev/null +++ b/doc/images/icingadb-database.png diff --git a/doc/images/icingadb-envs.png b/doc/images/icingadb-envs.png Binary files differnew file mode 100644 index 0000000..e9938d8 --- /dev/null +++ b/doc/images/icingadb-envs.png diff --git a/doc/images/icingadb-ha.png b/doc/images/icingadb-ha.png Binary files differnew file mode 100644 index 0000000..a86b6a0 --- /dev/null +++ b/doc/images/icingadb-ha.png diff --git a/doc/images/icingadb-web.png b/doc/images/icingadb-web.png Binary files differnew file mode 100644 index 0000000..05a3e31 --- /dev/null +++ b/doc/images/icingadb-web.png |
