From b485aab7e71c1625cfc27e0f92c9509f42378458 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 5 May 2024 13:19:16 +0200 Subject: Adding upstream version 1.45.3+dfsg. Signed-off-by: Daniel Baumann --- docs/maintenance/backup-restore.md | 66 ++++++++++++++ docs/maintenance/start-stop-restart.md | 153 +++++++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+) create mode 100644 docs/maintenance/backup-restore.md create mode 100644 docs/maintenance/start-stop-restart.md (limited to 'docs/maintenance') diff --git a/docs/maintenance/backup-restore.md b/docs/maintenance/backup-restore.md new file mode 100644 index 000000000..7e603d21c --- /dev/null +++ b/docs/maintenance/backup-restore.md @@ -0,0 +1,66 @@ +# Backing up a Netdata Agent + +## Introduction + +When preparing to backup a Netdata Agent it is worth considering that there are different kinds of data that you may wish to backup independently or all together: + +| Data type | Description | Location | +|---------------------|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------| +| Agent configuration | Files controlling configuration of the Netdata Agent | [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md) | +| Metrics | Database files | /var/cache/netdata | +| Identity | Claim token, API key and some other files | /var/lib/netdata | + + +## Scenarios + +### Backing up to restore data in case of a node failure + +In this standard scenario, you are backing up your Netdata Agent in case of a node failure or data corruption so that the metrics and the configuration can be recovered. The purpose is not to backup/restore the application itself. + +1. Verify that the directory paths in the table above contain the information you expect. + + > **Note** + > The specific paths may vary depending on installation method, Operating System, and whether it is a Docker/Kubernetes deployment. + +2. It is recommended that you [stop the Netdata Agent](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) when backing up the Metrics/database files. + Backing up the Agent configuration and Identity folders is straightforward as they should not be changing very frequently. + +3. Using a backup tool such as `tar` you will need to run the backup as _root_ or as the _netdata_ user to access all the files in the directories. + + ``` + sudo tar -cvpzf netdata_backup.tar.gz /etc/netdata/ /var/cache/netdata /var/lib/netdata + ``` + + Stopping the Netdata agent is typically necessary to back up the database files of the Netdata Agent. + +If you want to minimize the gap in metrics caused by stopping the Netdata Agent, consider implementing a backup job or script that follows this sequence: + +- Backup the Agent configuration Identity directories +- Stop the Netdata service +- Backup up the database files +- Restart the netdata agent. + +### Restoring Netdata + +1. Ensure that the Netdata agent is installed and is [stopped](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md#maintaining-a-netdata-agent-installation) + + If you plan to deploy the Agent and restore a backup on top of it, then you might find it helpful to use the [`--dont-start-it`](https://github.com/netdata/netdata/blob/master/packaging/installer/methods/kickstart.md#other-options) option upon installation. + + ``` + wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh && sh /tmp/netdata-kickstart.sh --dont-start-it + ``` + + > **Note** + > If you are going to restore the database files then you should first ensure that the Metrics directory is empty. + > + > ``` + > sudo rm -Rf /var/cache/netdata + > ``` + +2. Restore the backup from the archive + + ``` + sudo tar -xvpzf /path/to/netdata_backup.tar.gz -C / + ``` + +3. [Start the Netdata agent](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md#maintaining-a-netdata-agent-installation) diff --git a/docs/maintenance/start-stop-restart.md b/docs/maintenance/start-stop-restart.md new file mode 100644 index 000000000..2effe8863 --- /dev/null +++ b/docs/maintenance/start-stop-restart.md @@ -0,0 +1,153 @@ +# Start, stop, or restart the Netdata Agent + +When you install the Netdata Agent, the [daemon](https://github.com/netdata/netdata/blob/master/src/daemon/README.md) is +configured to start at boot and stop and restart/shutdown. + +You will most often need to _restart_ the Agent to load new or editing configuration files. +[Health configuration](#reload-health-configuration) files are the only exception, as they can be reloaded without restarting +the entire Agent. + +Stopping or restarting the Netdata Agent will cause gaps in stored metrics until the `netdata` process initiates +collectors and the database engine. + +## Using `systemctl`, `service`, or `init.d` + +This is the recommended way to start, stop, or restart the Netdata daemon. + +- To **start** Netdata, run `sudo systemctl start netdata`. +- To **stop** Netdata, run `sudo systemctl stop netdata`. +- To **restart** Netdata, run `sudo systemctl restart netdata`. + +If the above commands fail, or you know that you're using a non-systemd system, try using the `service` command: + +- **service**: `sudo service netdata start`, `sudo service netdata stop`, `sudo service netdata restart` + +## Using `netdata` + +Use the `netdata` command, typically located at `/usr/sbin/netdata`, to start the Netdata daemon. + +```bash +sudo netdata +``` + +If you start the daemon this way, close it with `sudo killall netdata`. + +## Using `netdatacli` + +The Netdata Agent also comes with a [CLI tool](https://github.com/netdata/netdata/blob/master/src/cli/README.md) capable of performing shutdowns. Start the Agent back up +using your preferred method listed above. + +```bash +sudo netdatacli shutdown-agent +``` + +## Netdata MSI installations + +Netdata provides an installer for Windows using WSL, on those installations by using a Windows terminal (e.g. the Command prompt or Windows Powershell) you can: + +- Start Netdata, by running `start-netdata` +- Stop Netdata, by running `stop-netdata` +- Restart Netdata, by running `restart-netdata` + +## Reload health configuration + +You do not need to restart the Netdata Agent between changes to health configuration files, such as specific health +entities. Instead, use [`netdatacli`](#using-netdatacli) and the `reload-health` option to prevent gaps in metrics +collection. + +```bash +sudo netdatacli reload-health +``` + +If `netdatacli` doesn't work on your system, send a `SIGUSR2` signal to the daemon, which reloads health configuration +without restarting the entire process. + +```bash +killall -USR2 netdata +``` + +## Force stop stalled or unresponsive `netdata` processes + +In rare cases, the Netdata Agent may stall or not properly close sockets, preventing a new process from starting. In +these cases, try the following three commands: + +```bash +sudo systemctl stop netdata +sudo killall netdata +ps aux| grep netdata +``` + +The output of `ps aux` should show no `netdata` or associated processes running. You can now start the Netdata Agent +again with `service netdata start`, or the appropriate method for your system. + +## Starting Netdata at boot + +In the `system` directory you can find scripts and configurations for the +various distros. + +### systemd + +The installer already installs `netdata.service` if it detects a systemd system. + +To install `netdata.service` by hand, run: + +```sh +# stop Netdata +killall netdata + +# copy netdata.service to systemd +cp system/netdata.service /etc/systemd/system/ + +# let systemd know there is a new service +systemctl daemon-reload + +# enable Netdata at boot +systemctl enable netdata + +# start Netdata +systemctl start netdata +``` + +### init.d + +In the system directory you can find `netdata-lsb`. Copy it to the proper place according to your distribution +documentation. For Ubuntu, this can be done via running the following commands as root. + +```sh +# copy the Netdata startup file to /etc/init.d +cp system/netdata-lsb /etc/init.d/netdata + +# make sure it is executable +chmod +x /etc/init.d/netdata + +# enable it +update-rc.d netdata defaults +``` + +### openrc (gentoo) + +In the `system` directory you can find `netdata-openrc`. Copy it to the proper +place according to your distribution documentation. + +### CentOS / Red Hat Enterprise Linux + +For older versions of RHEL/CentOS that don't have systemd, an init script is included in the system directory. This can +be installed by running the following commands as root. + +```sh +# copy the Netdata startup file to /etc/init.d +cp system/netdata-init-d /etc/init.d/netdata + +# make sure it is executable +chmod +x /etc/init.d/netdata + +# enable it +chkconfig --add netdata +``` + +_There have been some recent work on the init script, see PR +_ + +### other systems + +You can start Netdata by running it from `/etc/rc.local` or equivalent. -- cgit v1.2.3