diff options
Diffstat (limited to '')
24 files changed, 7109 insertions, 0 deletions
diff --git a/packaging/installer/README.md b/packaging/installer/README.md new file mode 100644 index 0000000..317ac63 --- /dev/null +++ b/packaging/installer/README.md @@ -0,0 +1,253 @@ +<!-- +title: "Installation guide" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/README.md +--> + +# Installation guide + +Netdata is a monitoring agent designed to run on all your systems: physical and virtual servers, containers, even +IoT/edge devices. Netdata runs on Linux, FreeBSD, macOS, Kubernetes, Docker, and all their derivatives. + +The best way to install Netdata is with our [**automatic one-line installation +script**](#automatic-one-line-installation-script), which works with all Linux distributions, or our [**.deb/rpm +packages**](/packaging/installer/methods/packages.md), which seamlessly install with your distribution's package +manager. + +If you want to install Netdata with Docker, on a Kubernetes cluster, or a different operating system, see [Have a +different operating system, or want to try another +method?](#have-a-different-operating-system-or-want-to-try-another-method) + +Some third parties, such as the packaging teams at various Linux distributions, distribute old, broken, or altered +packages. We recommend you install Netdata using one of the methods listed below to guarantee you get the latest +checksum-verified packages. + +Starting with v1.12, Netdata collects anonymous usage information by default and sends it to Google Analytics. Read +about the information collected, and learn how to-opt, on our [anonymous statistics](/docs/anonymous-statistics.md) +page. + +The usage statistics are _vital_ for us, as we use them to discover bugs and prioritize new features. We thank you for +_actively_ contributing to Netdata's future. + +## Automatic one-line installation script + +  + +This method is fully automatic on all Linux distributions, including Ubuntu, Debian, Fedora, CentOS, and others. + +To install Netdata from source, including all dependencies required to connect to Netdata Cloud, and get _automatic +nightly updates_, run the following as your normal user: + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) +``` + +To see more information about this installation script, including how to disable automatic updates, get nightly vs. +stable releases, or disable anonymous statistics, see the [`kickstart.sh` method +page](/packaging/installer/methods/kickstart.md). + +Scroll down for details about [automatic updates](#automatic-updates) or [nightly vs. stable +releases](#nightly-vs-stable-releases). + +### Post-installation + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +## Have a different operating system, or want to try another method? + +Netdata works on many different operating systems, each with a few possible installation methods. To see the full list +of approved methods for each operating system/version we support, see our [distribution +matrix](/packaging/DISTRIBUTIONS.md). + +Below, you can find a few additional installation methods, followed by separate instructions for a variety of unique +operating systems. + +### Alternative methods + +<div class="installer-grid" markdown="1"> + +[ Install +with .deb or .rpm packages](/packaging/installer/methods/packages.md) + +[ Install +with a pre-built static binary for 64-bit systems](/packaging/installer/methods/kickstart-64.md) + +[ Install +Netdata on Docker](/packaging/docker/README.md) + +[ Install +Netdata on a Kubernetes cluster](/packaging/installer/methods/kubernetes.md) + +[ +Install Netdata on cloud providers (GCP/AWS/Azure)](/packaging/installer/methods/cloud-providers.md) + +[ Install +Netdata on macOS](/packaging/installer/methods/macos.md) + +[ Install +Netdata on FreeBSD](/packaging/installer/methods/freebsd.md) + +[ Install +from a Git checkout](/packaging/installer/methods/manual.md) + +[ Install on +offline/air-gapped systems](/packaging/installer/methods/offline.md) + +[ +Installation on PFSense](/packaging/installer/methods/pfsense.md) + +[ Install +Netdata on Synology](/packaging/installer/methods/synology.md) + +[ Manual +installation on FreeNAS](/packaging/installer/methods/freenas.md) + +[ Manual +installation on Alpine](/packaging/installer/methods/alpine.md) + +[ +Build manually from source](/packaging/installer/methods/source.md) + +</div> + +## Automatic updates + +By default, Netdata's installation scripts enable automatic updates for both nightly and stable release channels. + +If you would prefer to update your Netdata agent manually, you can disable automatic updates by using the `--no-updates` +option when you install or update Netdata using the [automatic one-line installation +script](#automatic-one-line-installation-script). + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) --no-updates +``` + +With automatic updates disabled, you can choose exactly when and how you [update +Netdata](/packaging/installer/UPDATE.md). + +### Network usage of Netdata’s automatic updater + +The auto-update functionality set up by the installation scripts requires working internet access to function +correctly. In particular, it currently requires access to GitHub (to check if a newer version of the updater script +is available or not, as well as potentially fetching build-time dependencies that are bundled as part of the install), +and Google Cloud Storage (to check for newer versions of Netdata and download the sources if there is a newer version). + +Note that the auto-update functionality will check for updates to itself independently of updates to Netdata, +and will try to use the latest version of the updater script whenever possible. This is intended to reduce the +amount of effort required by users to get updates working again in the event of a bug in the updater code. + +## Nightly vs. stable releases + +The Netdata team maintains two releases of the Netdata agent: **nightly** and **stable**. By default, Netdata's +installation scripts will give you **automatic, nightly** updates, as that is our recommended configuration. + +**Nightly**: We create nightly builds every 24 hours. They contain fully-tested code that fixes bugs or security flaws, +or introduces new features to Netdata. Every nightly release is a candidate for then becoming a stable release—when +we're ready, we simply change the release tags on GitHub. That means nightly releases are stable and proven to function +correctly in the vast majority of Netdata use cases. That's why nightly is the _best choice for most Netdata users_. + +**Stable**: We create stable releases whenever we believe the code has reached a major milestone. Most often, stable +releases correlate with the introduction of new, significant features. Stable releases might be a better choice for +those who run Netdata in _mission-critical production systems_, as updates will come more infrequently, and only after +the community helps fix any bugs that might have been introduced in previous releases. + +**Pros of using nightly releases:** + +- Get the latest features and bug fixes as soon as they're available +- Receive security-related fixes immediately +- Use stable, fully-tested code that's always improving +- Leverage the same Netdata experience our community is using + +**Pros of using stable releases:** + +- Protect yourself from the rare instance when major bugs slip through our testing and negatively affect a Netdata + installation +- Retain more control over the Netdata version you use + +## Troubleshooting and known issues + +We are tracking a few issues related to installation and packaging. + +### Older distributions (Ubuntu 14.04, Debian 8, CentOS 6) and OpenSSL + +If you're running an older Linux distribution or one that has reached EOL, such as Ubuntu 14.04 LTS, Debian 8, or CentOS +6, your Agent may not be able to securely connect to Netdata Cloud due to an outdated version of OpenSSL. These old +versions of OpenSSL cannot perform [hostname validation](https://wiki.openssl.org/index.php/Hostname_validation), which +helps securely encrypt SSL connections. + +We recommend you reinstall Netdata with a [static build](/packaging/installer/methods/kickstart-64.md), which uses an +up-to-date version of OpenSSL with hostname validation enabled. + +If you choose to continue using the outdated version of OpenSSL, your node will still connect to Netdata Cloud, albeit +with hostname verification disabled. Without verification, your Netdata Cloud connection could be vulnerable to +man-in-the-middle attacks. + +### CentOS 6 and CentOS 8 + +To install the Agent on certain CentOS and RHEL systems, you must enable non-default repositories, such as EPEL or +PowerTools, to gather hard dependencies. See the [CentOS 6](/packaging/installer/methods/manual.md#centos-rehel-6-x) and +[CentOS 8](/packaging/installer/methods/manual.md#centos-rehel-8-x) sections for more information. + +### Access to file is not permitted + +If you see an error similar to `Access to file is not permitted: /usr/share/netdata/web//index.html` when you try to +visit the Agent dashboard at `http://NODE:19999`, you need to update Netdata's permissions to match those of your +system. + +Run `ls -la /usr/share/netdata/web/index.html` to find the file's permissions. You may need to change this path based on +the error you're seeing in your browser. In the below example, the file is owned by the user `netdata` and the group +`netdata`. + +```bash +ls -la /usr/share/netdata/web/index.html +-rw-r--r--. 1 netdata netdata 89377 May 5 06:30 /usr/share/netdata/web/index.html +``` + +Open your `netdata.conf` file and find the `[web]` section, plus the `web files owner`/`web files group` settings. Edit +the lines to match the output from `ls -la` above and uncomment them if necessary. + +```conf +[web] + web files owner = netdata + web files group = netdata +``` + +Save the file, [restart the Netdata Agent](/docs/getting-started.md#start-stop-and-restart-netdata), and try accessing +the dashboard again. + +### Multiple versions of OpenSSL + +We've received reports from the community about issues with running the `kickstart.sh` script on systems that have both +a distribution-installed version of OpenSSL and a manually-installed local version. The Agent's installer cannot handle +both. + +We recommend you install Netdata with the [static binary](/packaging/installer/methods/kickstart-64.md) to avoid the +issue altogether. Or, you can manually remove one version of OpenSSL to remove the conflict. + +### Clang compiler on Linux + +Our current build process has some issues when using certain configurations of the `clang` C compiler on Linux. See [the +section on `nonrepresentable section on output` +errors](/packaging/installer/methods/manual.md#nonrepresentable-section-on-output-errors) for a workaround. + +[](<>) diff --git a/packaging/installer/REINSTALL.md b/packaging/installer/REINSTALL.md new file mode 100644 index 0000000..50a15d7 --- /dev/null +++ b/packaging/installer/REINSTALL.md @@ -0,0 +1,68 @@ +<!-- +title: "Reinstall the Netdata Agent" +description: "Troubleshooting installation issues or force an update of the Netdata Agent by reinstalling it using the same method you used during installation." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/REINSTALL.md +--> + +# Reinstall the Netdata Agent + +In certain situations, such as needing to enable a feature or troubleshoot an issue, you may need to reinstall the +Netdata Agent on your node. + +Before you try reinstalling Netdata, figure out which [installation method you +used](/packaging/installer/UPDATE.md#determine-which-installation-method-you-used) if you do not already know. This will +determine the reinstallation method. + +## One-line installer script (`kickstart.sh`) + +Run the one-line installer script with the `--reinstall` parameter to reinstall the Netdata Agent. This will preserve +any [user configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +If you used any [optional +parameters](/packaging/installer/methods/kickstart.md#optional-parameters-to-alter-your-installation) during initial +installation, you need to pass them to the script again during reinstallation. If you cannot remember which options you +used, read the contents of the `.environment` file and look for a `REINSTALL_OPTIONS` line. This line contains a list of +optional parameters. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) --reinstall +``` + +## `.deb` or `.rpm` packages + +If you installed Netdata with [`.deb` or `.rpm` packages](/packaging/installer/methods/packages.md), use your +distribution's package manager to reinstall Netdata. Any custom settings present in your Netdata configuration directory +(typically at `/etc/netdata`) persists during this process. + +```bash +apt-get install --reinstall netdata # Ubuntu/Debian +dnf reinstall netdata # Fedora/RHEL +yum reinstall netdata # CentOS +zypper in -f netdata # openSUSE +``` + +## Pre-built static binary for 64-bit systems (`kickstart-static64.sh`) + +Run the one-line installer script with the `--reinstall` parameter to reinstall the Netdata Agent. This will preserve +any [user configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +If you used any [optional +parameters](/packaging/installer/methods/kickstart-64.md#optional-parameters-to-alter-your-installation) during +initial installation, you need to pass them to the script again during reinstallation. If you cannot remember which +options you used, read the contents of the `.environment` file and look for a `REINSTALL_OPTIONS` line. This line +contains a list of optional parameters. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --reinstall +``` + +## Troubleshooting + +If you still experience problems with your Netdata Agent installation after following one of these processes, the next +best route is to [uninstall](/packaging/installer/UNINSTALL.md) and then try a fresh installation using the [one-line +installer](/packaging/installer/methods/kickstart.md). + +You can also post to our [community forums](https://community.netdata.cloud/c/support/13) or create a new [bug +report](https://github.com/netdata/netdata/issues/new?labels=bug%2C+needs+triage&template=bug_report.md). + +[](<>) diff --git a/packaging/installer/UNINSTALL.md b/packaging/installer/UNINSTALL.md new file mode 100644 index 0000000..c18a1bf --- /dev/null +++ b/packaging/installer/UNINSTALL.md @@ -0,0 +1,54 @@ +<!-- +title: "Uninstall Netdata" +description: "If you are no longer interested in using the Netdata Agent, use the self-contained uninstaller to remove all traces of binaries and configuration files." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/UNINSTALL.md +--> + +# Uninstall Netdata + +> ⚠️ If you're having trouble updating Netdata, moving from one installation method to another, or generally having +> issues with your Netdata Agent installation, consider our [**reinstall Netdata** +> doc](/packaging/installer/REINSTALL.md) instead of removing the Netdata Agent entirely. + +Our self-contained uninstaller is able to remove Netdata installations created with shell installer. It doesn't need any +other Netdata repository files to be run. All it needs is an `.environment` file, which is created during installation +(with shell installer) and put in `${NETDATA_USER_CONFIG_DIR}/.environment` (by default `/etc/netdata/.environment`). +That file contains some parameters which are passed to our installer and which are needed during uninstallation process. +Mainly two parameters are needed: + +```sh +NETDATA_PREFIX +NETDATA_ADDED_TO_GROUPS +``` + +A workflow for uninstallation looks like this: + +1. Find your `.environment` file, which is usually `/etc/netdata/.environment` in a default installation. +2. If you cannot find that file and would like to uninstall Netdata, then create a new file with the following content: + +```sh +NETDATA_PREFIX="<installation prefix>" # put what you used as a parameter to shell installed `--install` flag. Otherwise it should be empty +NETDATA_ADDED_TO_GROUPS="<additional groups>" # Additional groups for a user running the Netdata process +``` + +3. Run `netdata-uninstaller.sh` as follows + +```sh +${NETDATA_PREFIX}/usr/libexec/netdata/netdata-uninstaller.sh --yes --env <environment_file> +``` + +Note: Existing installations may still need to download the file if it's not present. To execute uninstall in that case, +run the following commands: + +```sh +wget https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/netdata-uninstaller.sh +chmod +x ./netdata-uninstaller.sh +./netdata-uninstaller.sh --yes --env <environment_file> +``` + +The default `environment_file` is `/etc/netdata/.environment`. + +> Note: This uninstallation method assumes previous installation with `netdata-installer.sh` or the kickstart script. +> Currently using it when Netdata was installed by a package manager can work or cause unexpected results. + +[](<>) diff --git a/packaging/installer/UPDATE.md b/packaging/installer/UPDATE.md new file mode 100644 index 0000000..ba57c8c --- /dev/null +++ b/packaging/installer/UPDATE.md @@ -0,0 +1,153 @@ +<!-- +title: "Update the Netdata Agent" +description: "If you opted out of automatic updates, you need to update your Netdata Agent to the latest nightly or stable version." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/UPDATE.md +--> + +# Update the Netdata Agent + +By default, the Netdata Agent automatically updates with the latest nightly version. If you opted out of automatic +updates, you need to update your Netdata Agent to the latest nightly or stable version. + +> 💡 Looking to reinstall the Netdata Agent to enable a feature, update an Agent that cannot update automatically, or +> troubleshoot an error during the installation process? See our [reinstallation doc](/packaging/installer/REINSTALL.md) +> for reinstallation steps. + +Before you update the Netdata Agent, check to see if your Netdata Agent is already up-to-date by clicking on the update +icon in the local Agent dashboard's top navigation. This modal informs you whether your Agent needs an update or not. + + + +## Determine which installation method you used + +If you are not sure where your Netdata config directory is, see the [configuration doc](/docs/configure/nodes.md). In +most installations, this is `/etc/netdata`. + +Use `cd` to navigate to the Netdata config directory, then use `ls -a` to look for a file called `.environment`. + +- If the `.environment` file _does not_ exist, reinstall with your [package manager](#deb-or-rpm-packages). +- If the `.environment` file _does_ exist, check its contents with `less .environment`. + - If `IS_NETDATA_STATIC_BINARY` is `"yes"`, update using the [pre-built static + binary](#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh). + - In all other cases, update using the [one-line installer script](#one-line-installer-script-kickstartsh). + +Next, use the appropriate method to update the Netdata Agent: + +- [One-line installer script (`kickstart.sh`)](#one-line-installer-script-kickstartsh) +- [`.deb` or `.rpm` packages](#deb-or-rpm-packages) +- [Pre-built static binary for 64-bit systems (`kickstart-static64.sh`)](#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh) +- [Docker](#docker) +- [macOS](#macos) +- [Manual installation from Git](#manual-installation-from-git) + +## One-line installer script (`kickstart.sh`) + +If you installed Netdata using our one-line automatic installation script, run it again to update Netdata. Any custom +settings present in your Netdata configuration directory (typically at `/etc/netdata`) persists during this process. + +This script will automatically run the update script that was installed as part of the initial install (even if +you disabled automatic updates) and preserve the existing install options you specified. + +If you installed Netdata using an installation prefix, you will need to add an `--install` option specifying +that prefix to this command to make sure it finds Netdata. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) +``` + +> ❗ If the above command fails, you can [reinstall +> Netdata](/packaging/installer/REINSTALL.md#one-line-installer-script-kickstartsh) to get the latest version. This also +> preserves your [configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +## `.deb` or `.rpm` packages + +If you installed Netdata with [`.deb` or `.rpm` packages](/packaging/installer/methods/packages.md), use your +distribution's package manager to update Netdata. Any custom settings present in your Netdata configuration directory +(typically at `/etc/netdata`) persists during this process. + +Your package manager grabs a new package from our hosted repository, updates the Netdata Agent, and restarts it. + +```bash +apt-get install netdata # Ubuntu/Debian +dnf install netdata # Fedora/RHEL +yum install netdata # CentOS +zypper in netdata # openSUSE +``` + +> You may need to escalate privileges using `sudo`. + +## Pre-built static binary for 64-bit systems (`kickstart-static64.sh`) + +If you installed Netdata using the pre-built static binary, run the `kickstart-static64.sh` script again to update +Netdata. Any custom settings present in your Netdata configuration directory (typically at `/etc/netdata`) persists +during this process. + +This script will automatically run the update script that was installed as part of the initial install (even if +you disabled automatic updates) and preserve the existing install options you specified. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) +``` + +> ❗ If the above command fails, you can [reinstall +> Netdata](/packaging/installer/REINSTALL.md#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh) to get the +> latest version. This also preserves your [configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +## Docker + +Docker-based installations do not update automatically. To update an Netdata Agent running in a Docker container, you +must pull the [latest image from Docker Hub](https://hub.docker.com/r/netdata/netdata), stop and remove the container, +and re-create it using the latest image. + +First, pull the latest version of the image. + +```bash +docker pull netdata/netdata:latest +``` + +Next, to stop and remove any containers using the `netdata/netdata` image. Replace `netdata` if you changed it from the +default. + +```bash +docker stop netdata +docker rm netdata +``` + +You can now re-create your Netdata container using the `docker` command or a `docker-compose.yml` file. See our [Docker +installation instructions](/packaging/docker/README.md#create-a-new-netdata-agent-container) for details. + +## macOS + +If you installed Netdata on your macOS system using Homebrew, you can explicitly request an update: + +```bash +brew upgrade netdata +``` + +Homebrew downloads the latest Netdata via the +[formulae](https://github.com/Homebrew/homebrew-core/blob/master/Formula/netdata.rb), ensures all dependencies are met, +and updates Netdata via reinstallation. + +## Manual installation from Git + +If you installed [Netdata manually from Git](/packaging/installer/methods/manual.md), you can run that installer again +to update your agent. First, run our automatic requirements installer, which works on many Linux distributions, to +ensure your system has the dependencies necessary for new features. + +```bash +bash <(curl -sSL https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh) +``` + +Navigate to the directory where you first cloned the Netdata repository, pull the latest source code, and run +`netdata-install.sh` again. This process compiles Netdata with the latest source code and updates it via reinstallation. + +```bash +cd /path/to/netdata/git +git pull origin master +sudo ./netdata-installer.sh +``` + +> ⚠️ If you installed Netdata with any optional parameters, such as `--no-updates` to disable automatic updates, and +> want to retain those settings, you need to set them again during this process. + +[](<>) diff --git a/packaging/installer/functions.sh b/packaging/installer/functions.sh new file mode 100644 index 0000000..03ceb2a --- /dev/null +++ b/packaging/installer/functions.sh @@ -0,0 +1,1054 @@ +#!/bin/bash + +# SPDX-License-Identifier: GPL-3.0-or-later + +# make sure we have a UID +[ -z "${UID}" ] && UID="$(id -u)" + +# ----------------------------------------------------------------------------- + +setup_terminal() { + TPUT_RESET="" + TPUT_BLACK="" + TPUT_RED="" + TPUT_GREEN="" + TPUT_YELLOW="" + TPUT_BLUE="" + TPUT_PURPLE="" + TPUT_CYAN="" + TPUT_WHITE="" + TPUT_BGBLACK="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BGYELLOW="" + TPUT_BGBLUE="" + TPUT_BGPURPLE="" + TPUT_BGCYAN="" + TPUT_BGWHITE="" + TPUT_BOLD="" + TPUT_DIM="" + TPUT_UNDERLINED="" + TPUT_BLINK="" + TPUT_INVERTED="" + TPUT_STANDOUT="" + TPUT_BELL="" + TPUT_CLEAR="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput 1> /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + # shellcheck disable=SC2034 + TPUT_BLACK="$(tput setaf 0)" + TPUT_RED="$(tput setaf 1)" + TPUT_GREEN="$(tput setaf 2)" + # shellcheck disable=SC2034 + TPUT_YELLOW="$(tput setaf 3)" + # shellcheck disable=SC2034 + TPUT_BLUE="$(tput setaf 4)" + # shellcheck disable=SC2034 + TPUT_PURPLE="$(tput setaf 5)" + TPUT_CYAN="$(tput setaf 6)" + TPUT_WHITE="$(tput setaf 7)" + # shellcheck disable=SC2034 + TPUT_BGBLACK="$(tput setab 0)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + # shellcheck disable=SC2034 + TPUT_BGYELLOW="$(tput setab 3)" + # shellcheck disable=SC2034 + TPUT_BGBLUE="$(tput setab 4)" + # shellcheck disable=SC2034 + TPUT_BGPURPLE="$(tput setab 5)" + # shellcheck disable=SC2034 + TPUT_BGCYAN="$(tput setab 6)" + # shellcheck disable=SC2034 + TPUT_BGWHITE="$(tput setab 7)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + # shellcheck disable=SC2034 + TPUT_UNDERLINED="$(tput smul)" + # shellcheck disable=SC2034 + TPUT_BLINK="$(tput blink)" + # shellcheck disable=SC2034 + TPUT_INVERTED="$(tput rev)" + # shellcheck disable=SC2034 + TPUT_STANDOUT="$(tput smso)" + # shellcheck disable=SC2034 + TPUT_BELL="$(tput bel)" + # shellcheck disable=SC2034 + TPUT_CLEAR="$(tput clear)" + fi + fi + + return 0 +} +setup_terminal || echo > /dev/null + +progress() { + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " +} + +get() { + url="${1}" + if command -v curl > /dev/null 2>&1; then + curl -q -o - -sSL --connect-timeout 10 --retry 3 "${url}" + elif command -v wget > /dev/null 2>&1; then + wget -T 15 -O - "${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi +} + +download_file() { + url="${1}" + dest="${2}" + name="${3}" + opt="${4}" + + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" + else + echo >&2 + echo >&2 "Downloading ${name} from '${url}' failed because of missing mandatory packages." + if [ -n "$opt" ]; then + echo >&2 "Either add packages or disable it by issuing '--disable-${opt}' in the installer" + fi + echo >&2 + + run_failed "I need curl or wget to proceed, but neither is available on this system." + fi +} + +# ----------------------------------------------------------------------------- +# external component handling + +fetch_and_verify() { + local component=${1} + local url=${2} + local base_name=${3} + local tmp=${4} + local override=${5} + + if [ -z "${override}" ]; then + download_file "${url}" "${tmp}/${base_name}" "${component}" + else + progress "Using provided ${component} archive ${override}" + run cp "${override}" "${tmp}/${base_name}" + fi + + if [ ! -f "${tmp}/${base_name}" ] || [ ! -s "${tmp}/${base_name}" ]; then + run_failed "Unable to find usable archive for ${component}" + return 1 + fi + + grep "${base_name}\$" "${INSTALLER_DIR}/packaging/${component}.checksums" > "${tmp}/sha256sums.txt" 2> /dev/null + + # Checksum validation + if ! (cd "${tmp}" && safe_sha256sum -c "sha256sums.txt"); then + run_failed "${component} files checksum validation failed." + return 1 + fi +} + +# ----------------------------------------------------------------------------- + +netdata_banner() { + local l1=" ^" \ + l2=" |.-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-" \ + l3=" | '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' " \ + l4=" +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->" \ + sp=" " \ + netdata="netdata" start end msg="${*}" chartcolor="${TPUT_DIM}" + + [ ${#msg} -lt ${#netdata} ] && msg="${msg}${sp:0:$((${#netdata} - ${#msg}))}" + [ ${#msg} -gt $((${#l2} - 20)) ] && msg="${msg:0:$((${#l2} - 23))}..." + + start="$((${#l2} / 2 - 4))" + [ $((start + ${#msg} + 4)) -gt ${#l2} ] && start=$((${#l2} - ${#msg} - 4)) + end=$((start + ${#msg} + 4)) + + echo >&2 + echo >&2 "${chartcolor}${l1}${TPUT_RESET}" + echo >&2 "${chartcolor}${l2:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_GREEN}${netdata}${TPUT_RESET}${chartcolor}${sp:0:$((end - start - 2 - ${#netdata}))}${l2:end:$((${#l2} - end))}${TPUT_RESET}" + echo >&2 "${chartcolor}${l3:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_CYAN}${msg}${TPUT_RESET}${chartcolor}${sp:0:2}${l3:end:$((${#l2} - end))}${TPUT_RESET}" + echo >&2 "${chartcolor}${l4}${TPUT_RESET}" + echo >&2 +} + +# ----------------------------------------------------------------------------- +# portable service command + +service_cmd="$(command -v service 2> /dev/null)" +rcservice_cmd="$(command -v rc-service 2> /dev/null)" +systemctl_cmd="$(command -v systemctl 2> /dev/null)" +service() { + + local cmd="${1}" action="${2}" + + if [ -n "${systemctl_cmd}" ]; then + run "${systemctl_cmd}" "${action}" "${cmd}" + return $? + elif [ -n "${service_cmd}" ]; then + run "${service_cmd}" "${cmd}" "${action}" + return $? + elif [ -n "${rcservice_cmd}" ]; then + run "${rcservice_cmd}" "${cmd}" "${action}" + return $? + fi + return 1 +} + +# ----------------------------------------------------------------------------- +# portable pidof + +safe_pidof() { + local pidof_cmd + pidof_cmd="$(command -v pidof 2> /dev/null)" + if [ -n "${pidof_cmd}" ]; then + ${pidof_cmd} "${@}" + return $? + else + ps -acxo pid,comm | + sed "s/^ *//g" | + grep netdata | + cut -d ' ' -f 1 + return $? + fi +} + +# ----------------------------------------------------------------------------- +find_processors() { + # Most UNIX systems have `nproc` as part of their userland (including macOS, Linux and BSD) + if command -v nproc > /dev/null; then + nproc && return + fi + + local cpus + if [ -f "/proc/cpuinfo" ]; then + # linux + cpus=$(grep -c ^processor /proc/cpuinfo) + else + # freebsd + cpus=$(sysctl hw.ncpu 2> /dev/null | grep ^hw.ncpu | cut -d ' ' -f 2) + fi + if [ -z "${cpus}" ] || [ $((cpus)) -lt 1 ]; then + echo 1 + else + echo "${cpus}" + fi +} + +# ----------------------------------------------------------------------------- +fatal() { + printf >&2 "%s ABORTED %s %s \n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" + exit 1 +} + +run_ok() { + printf >&2 "%s OK %s %s \n\n" "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" +} + +run_failed() { + printf >&2 "%s FAILED %s %s \n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" +} + +ESCAPED_PRINT_METHOD= +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi +escaped_print() { + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 +} + +run_logfile="/dev/null" +run() { + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "%s" "${info}" + escaped_print "${@}" + printf "%s" " ... " + } >> "${run_logfile}" + + printf >&2 "%s" "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "%s\n" "${TPUT_RESET}" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code %s\n" "${ret}" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} +} + +iscontainer() { + # man systemd-detect-virt + local cmd + cmd=$(command -v systemd-detect-virt 2> /dev/null) + if [ -n "${cmd}" ] && [ -x "${cmd}" ]; then + "${cmd}" --container > /dev/null 2>&1 && return 0 + fi + + # /proc/1/sched exposes the host's pid of our init ! + # http://stackoverflow.com/a/37016302 + local pid + pid=$(head -n 1 /proc/1/sched 2> /dev/null | { + # shellcheck disable=SC2034 + IFS='(),#:' read -r name pid th threads + echo "$pid" + }) + if [ -n "${pid}" ]; then + pid=$((pid + 0)) + [ ${pid} -gt 1 ] && return 0 + fi + + # lxc sets environment variable 'container' + # shellcheck disable=SC2154 + [ -n "${container}" ] && return 0 + + # docker creates /.dockerenv + # http://stackoverflow.com/a/25518345 + [ -f "/.dockerenv" ] && return 0 + + # ubuntu and debian supply /bin/running-in-container + # https://www.apt-browse.org/browse/ubuntu/trusty/main/i386/upstart/1.12.1-0ubuntu4/file/bin/running-in-container + if [ -x "/bin/running-in-container" ]; then + "/bin/running-in-container" > /dev/null 2>&1 && return 0 + fi + + return 1 +} + +get_os_key() { + if [ -f /etc/os-release ]; then + # shellcheck disable=SC1091 + source /etc/os-release || return 1 + echo "${ID}-${VERSION_ID}" + + elif [ -f /etc/redhat-release ]; then + echo "$(< /etc/redhat-release)" + else + echo "unknown" + fi +} + +issystemd() { + local pids p myns ns systemctl + + # if the directory /lib/systemd/system OR /usr/lib/systemd/system (SLES 12.x) does not exit, it is not systemd + if [ ! -d /lib/systemd/system ] && [ ! -d /usr/lib/systemd/system ]; then + return 1 + fi + + # if there is no systemctl command, it is not systemd + systemctl=$(command -v systemctl 2> /dev/null) + if [ -z "${systemctl}" ] || [ ! -x "${systemctl}" ]; then + return 1 + fi + + # if pid 1 is systemd, it is systemd + [ "$(basename "$(readlink /proc/1/exe)" 2> /dev/null)" = "systemd" ] && return 0 + + # if systemd is not running, it is not systemd + pids=$(safe_pidof systemd 2> /dev/null) + [ -z "${pids}" ] && return 1 + + # check if the running systemd processes are not in our namespace + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + for p in ${pids}; do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + # if pid of systemd is in our namespace, it is systemd + [ -n "${myns}" ] && [ "${myns}" = "${ns}" ] && return 0 + done + + # else, it is not systemd + return 1 +} + +get_systemd_service_dir() { + local SYSTEMD_DIRECTORY="" + local key + key="$(get_os_key)" + + if [ -w "/lib/systemd/system" ]; then + SYSTEMD_DIRECTORY="/lib/systemd/system" + elif [ -w "/usr/lib/systemd/system" ]; then + SYSTEMD_DIRECTORY="/usr/lib/systemd/system" + elif [ -w "/etc/systemd/system" ]; then + SYSTEMD_DIRECTORY="/etc/systemd/system" + fi + + if [[ ${key} =~ ^devuan* ]] || [ "${key}" = "debian-7" ] || [ "${key}" = "ubuntu-12.04" ] || [ "${key}" = "ubuntu-14.04" ]; then + SYSTEMD_DIRECTORY="/etc/systemd/system" + fi + + echo "${SYSTEMD_DIRECTORY}" +} + +install_non_systemd_init() { + [ "${UID}" != 0 ] && return 1 + + local key + key="$(get_os_key)" + + if [ -d /etc/init.d ] && [ ! -f /etc/init.d/netdata ]; then + if [[ ${key} =~ ^(gentoo|alpine).* ]]; then + echo >&2 "Installing OpenRC init file..." + run cp system/netdata-openrc /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run rc-update add netdata default && + return 0 + + elif [[ ${key} =~ ^devuan* ]] || [ "${key}" = "debian-7" ] || [ "${key}" = "ubuntu-12.04" ] || [ "${key}" = "ubuntu-14.04" ]; then + echo >&2 "Installing LSB init file..." + run cp system/netdata-lsb /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run update-rc.d netdata defaults && + run update-rc.d netdata enable && + return 0 + elif [[ ${key} =~ ^(amzn-201[5678]|ol|CentOS release 6|Red Hat Enterprise Linux Server release 6|Scientific Linux CERN SLC release 6|CloudLinux Server release 6).* ]]; then + echo >&2 "Installing init.d file..." + run cp system/netdata-init-d /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run chkconfig netdata on && + return 0 + else + echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." + return 1 + fi + elif [ -f /etc/init.d/netdata ]; then + echo >&2 "file '/etc/init.d/netdata' already exists." + return 0 + else + echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." + fi + + return 1 +} + +# This is used by netdata-installer.sh +# shellcheck disable=SC2034 +NETDATA_STOP_CMD="netdatacli shutdown-agent" + +NETDATA_START_CMD="netdata" +NETDATA_INSTALLER_START_CMD="" + +install_netdata_service() { + local uname + uname="$(uname 2> /dev/null)" + + if [ "${UID}" -eq 0 ]; then + if [ "${uname}" = "Darwin" ]; then + + if [ -f "/Library/LaunchDaemons/com.github.netdata.plist" ]; then + echo >&2 "file '/Library/LaunchDaemons/com.github.netdata.plist' already exists." + return 0 + else + echo >&2 "Installing MacOS X plist file..." + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + run cp system/netdata.plist /Library/LaunchDaemons/com.github.netdata.plist && + run launchctl load /Library/LaunchDaemons/com.github.netdata.plist && + NETDATA_START_CMD="launchctl start com.github.netdata" && + NETDATA_STOP_CMD="launchctl stop com.github.netdata" + return 0 + fi + + elif [ "${uname}" = "FreeBSD" ]; then + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + run cp system/netdata-freebsd /etc/rc.d/netdata && NETDATA_START_CMD="service netdata start" && + NETDATA_STOP_CMD="service netdata stop" && + NETDATA_INSTALLER_START_CMD="service netdata onestart" && + myret=$? + + echo >&2 "Note: To explicitly enable netdata automatic start, set 'netdata_enable' to 'YES' in /etc/rc.conf" + echo >&2 "" + + return ${myret} + + elif issystemd; then + # systemd is running on this system + NETDATA_START_CMD="systemctl start netdata" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="systemctl stop netdata" + NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" + + SYSTEMD_DIRECTORY="$(get_systemd_service_dir)" + + if [ "${SYSTEMD_DIRECTORY}x" != "x" ]; then + ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="run systemctl enable netdata" + IS_NETDATA_ENABLED="$(systemctl is-enabled netdata 2> /dev/null || echo "Netdata not there")" + if [ "${IS_NETDATA_ENABLED}" == "disabled" ]; then + echo >&2 "Netdata was there and disabled, make sure we don't re-enable it ourselves" + ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="true" + fi + + echo >&2 "Installing systemd service..." + run cp system/netdata.service "${SYSTEMD_DIRECTORY}/netdata.service" && + run systemctl daemon-reload && + ${ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED} && + return 0 + else + echo >&2 "no systemd directory; cannot install netdata.service" + fi + else + install_non_systemd_init + local ret=$? + + if [ ${ret} -eq 0 ]; then + if [ -n "${service_cmd}" ]; then + NETDATA_START_CMD="service netdata start" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="service netdata stop" + elif [ -n "${rcservice_cmd}" ]; then + NETDATA_START_CMD="rc-service netdata start" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="rc-service netdata stop" + fi + NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" + fi + + return ${ret} + fi + fi + + return 1 +} + +# ----------------------------------------------------------------------------- +# stop netdata + +pidisnetdata() { + if [ -d /proc/self ]; then + if [ -z "$1" ] || [ ! -f "/proc/$1/stat" ]; then + return 1 + fi + [ "$(cut -d '(' -f 2 "/proc/$1/stat" | cut -d ')' -f 1)" = "netdata" ] && return 0 + return 1 + fi + return 0 +} + +stop_netdata_on_pid() { + local pid="${1}" ret=0 count=0 + + pidisnetdata "${pid}" || return 0 + + printf >&2 "Stopping netdata on pid %s ..." "${pid}" + while [ -n "$pid" ] && [ ${ret} -eq 0 ]; do + if [ ${count} -gt 24 ]; then + echo >&2 "Cannot stop the running netdata on pid ${pid}." + return 1 + fi + + count=$((count + 1)) + + pidisnetdata "${pid}" || ret=1 + if [ ${ret} -eq 1 ]; then + break + fi + + if [ ${count} -lt 12 ]; then + run kill "${pid}" 2> /dev/null + ret=$? + else + run kill -9 "${pid}" 2> /dev/null + ret=$? + fi + + test ${ret} -eq 0 && printf >&2 "." && sleep 5 + + done + + echo >&2 + if [ ${ret} -eq 0 ]; then + echo >&2 "SORRY! CANNOT STOP netdata ON PID ${pid} !" + return 1 + fi + + echo >&2 "netdata on pid ${pid} stopped." + return 0 +} + +netdata_pids() { + local p myns ns + + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + + for p in \ + $(cat /var/run/netdata.pid 2> /dev/null) \ + $(cat /var/run/netdata/netdata.pid 2> /dev/null) \ + $(safe_pidof netdata 2> /dev/null); do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then + pidisnetdata "${p}" && echo "${p}" + fi + done +} + +stop_all_netdata() { + local p uname + + if [ "${UID}" -eq 0 ]; then + uname="$(uname 2> /dev/null)" + + # Any of these may fail, but we need to not bail if they do. + if issystemd; then + if systemctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "Darwin" ]; then + if launchctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "FreeBSD" ]; then + if /etc/rc.d/netdata stop; then + sleep 5 + fi + else + if service netdata stop; then + sleep 5 + fi + fi + fi + + if [ -n "$(netdata_pids)" ] && [ -n "$(builtin type -P netdatacli)" ]; then + netdatacli shutdown-agent + sleep 20 + fi + + for p in $(netdata_pids); do + # shellcheck disable=SC2086 + stop_netdata_on_pid ${p} + done +} + +# ----------------------------------------------------------------------------- +# restart netdata + +restart_netdata() { + local netdata="${1}" + shift + + local started=0 + + progress "Restarting netdata instance" + + if [ -z "${NETDATA_INSTALLER_START_CMD}" ]; then + NETDATA_INSTALLER_START_CMD="${netdata}" + fi + + if [ "${UID}" -eq 0 ]; then + echo >&2 + echo >&2 "Stopping all netdata threads" + run stop_all_netdata + + echo >&2 "Starting netdata using command '${NETDATA_INSTALLER_START_CMD}'" + # shellcheck disable=SC2086 + run ${NETDATA_INSTALLER_START_CMD} && started=1 + + if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then + echo >&2 "Ooops! it seems netdata is not started." + started=0 + fi + + if [ ${started} -eq 0 ]; then + echo >&2 "Attempting another netdata start using command '${NETDATA_INSTALLER_START_CMD}'" + # shellcheck disable=SC2086 + run ${NETDATA_INSTALLER_START_CMD} && started=1 + fi + fi + + if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then + echo >&2 "Hm... it seems netdata is still not started." + started=0 + fi + + if [ ${started} -eq 0 ]; then + # still not started... another forced attempt, just run the binary + echo >&2 "Netdata service still not started, attempting another forced restart by running '${netdata} ${*}'" + run stop_all_netdata + run "${netdata}" "${@}" + return $? + fi + + return 0 +} + +# ----------------------------------------------------------------------------- +# install netdata logrotate + +install_netdata_logrotate() { + if [ "${UID}" -eq 0 ]; then + if [ -d /etc/logrotate.d ]; then + if [ ! -f /etc/logrotate.d/netdata ]; then + run cp system/netdata.logrotate /etc/logrotate.d/netdata + fi + + if [ -f /etc/logrotate.d/netdata ]; then + run chmod 644 /etc/logrotate.d/netdata + fi + + return 0 + fi + fi + + return 1 +} + +# ----------------------------------------------------------------------------- +# create netdata.conf + +create_netdata_conf() { + local path="${1}" url="${2}" + + if [ -s "${path}" ]; then + return 0 + fi + + if [ -n "$url" ]; then + echo >&2 "Downloading default configuration from netdata..." + sleep 5 + + # remove a possibly obsolete configuration file + [ -f "${path}.new" ] && rm "${path}.new" + + # disable a proxy to get data from the local netdata + export http_proxy= + export https_proxy= + + if command -v curl 1> /dev/null 2>&1; then + run curl -sSL --connect-timeout 10 --retry 3 "${url}" > "${path}.new" + elif command -v wget 1> /dev/null 2>&1; then + run wget -T 15 -O - "${url}" > "${path}.new" + fi + + if [ -s "${path}.new" ]; then + run mv "${path}.new" "${path}" + run_ok "New configuration saved for you to edit at ${path}" + else + [ -f "${path}.new" ] && rm "${path}.new" + run_failed "Cannnot download configuration from netdata daemon using url '${url}'" + url='' + fi + fi + + if [ -z "$url" ]; then + echo "# netdata can generate its own config which is available at 'http://<netdata_ip>/netdata.conf'" > "${path}" + echo "# You can download it with command like: 'wget -O ${path} http://localhost:19999/netdata.conf'" >> "${path}" + fi + +} + +portable_add_user() { + local username="${1}" homedir="${2}" + + [ -z "${homedir}" ] && homedir="/tmp" + + # Check if user exists + if cut -d ':' -f 1 < /etc/passwd | grep "^${username}$" 1> /dev/null 2>&1; then + echo >&2 "User '${username}' already exists." + return 0 + fi + + echo >&2 "Adding ${username} user account with home ${homedir} ..." + + local nologin + nologin="$(command -v nologin || echo '/bin/false')" + + # Linux + if command -v useradd 1> /dev/null 2>&1; then + run useradd -r -g "${username}" -c "${username}" -s "${nologin}" --no-create-home -d "${homedir}" "${username}" && return 0 + fi + + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw useradd "${username}" -d "${homedir}" -g "${username}" -s "${nologin}" && return 0 + fi + + # BusyBox + if command -v adduser 1> /dev/null 2>&1; then + run adduser -h "${homedir}" -s "${nologin}" -D -G "${username}" "${username}" && return 0 + fi + + # mac OS + if command -v sysadminctl 1> /dev/null 2>&1; then + run sysadminctl -addUser "${username}" && return 0 + fi + + echo >&2 "Failed to add ${username} user account !" + + return 1 +} + +portable_add_group() { + local groupname="${1}" + + # Check if group exist + if cut -d ':' -f 1 < /etc/group | grep "^${groupname}$" 1> /dev/null 2>&1; then + echo >&2 "Group '${groupname}' already exists." + return 0 + fi + + echo >&2 "Adding ${groupname} user group ..." + + # Linux + if command -v groupadd 1> /dev/null 2>&1; then + run groupadd -r "${groupname}" && return 0 + fi + + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupadd "${groupname}" && return 0 + fi + + # BusyBox + if command -v addgroup 1> /dev/null 2>&1; then + run addgroup "${groupname}" && return 0 + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + dseditgroup -o create "${groupname}" && return 0 + fi + + echo >&2 "Failed to add ${groupname} user group !" + return 1 +} + +portable_add_user_to_group() { + local groupname="${1}" username="${2}" + + # Check if group exist + if ! cut -d ':' -f 1 < /etc/group | grep "^${groupname}$" > /dev/null 2>&1; then + echo >&2 "Group '${groupname}' does not exist." + return 1 + fi + + # Check if user is in group + if [[ ",$(grep "^${groupname}:" < /etc/group | cut -d ':' -f 4)," =~ ,${username}, ]]; then + # username is already there + echo >&2 "User '${username}' is already in group '${groupname}'." + return 0 + else + # username is not in group + echo >&2 "Adding ${username} user to the ${groupname} group ..." + + # Linux + if command -v usermod 1> /dev/null 2>&1; then + run usermod -a -G "${groupname}" "${username}" && return 0 + fi + + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupmod "${groupname}" -m "${username}" && return 0 + fi + + # BusyBox + if command -v addgroup 1> /dev/null 2>&1; then + run addgroup "${username}" "${groupname}" && return 0 + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + dseditgroup -u "${username}" "${groupname}" && return 0 + fi + echo >&2 "Failed to add user ${username} to group ${groupname} !" + return 1 + fi +} + +safe_sha256sum() { + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi +} + +_get_scheduler_type() { + if _get_intervaldir > /dev/null ; then + echo 'interval' + elif issystemd ; then + echo 'systemd' + elif [ -d /etc/cron.d ] ; then + echo 'crontab' + else + echo 'none' + fi +} + +_get_intervaldir() { + if [ -d /etc/cron.daily ]; then + echo /etc/cron.daily + elif [ -d /etc/periodic/daily ]; then + echo /etc/periodic/daily + else + return 1 + fi + + return 0 +} + +install_netdata_updater() { + if [ "${INSTALLER_DIR}" ] && [ -f "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" ]; then + cat "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + fi + + if [ "${NETDATA_SOURCE_DIR}" ] && [ -f "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" ]; then + cat "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + fi + + if issystemd && [ -n "$(get_systemd_service_dir)" ]; then + cat "${NETDATA_SOURCE_DIR}/system/netdata-updater.timer" > "$(get_systemd_service_dir)/netdata-updater.timer" + cat "${NETDATA_SOURCE_DIR}/system/netdata-updater.service" > "$(get_systemd_service_dir)/netdata-updater.service" + fi + + sed -i -e "s|THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT|${NETDATA_USER_CONFIG_DIR}/.environment|" "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + + chmod 0755 "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" + echo >&2 "Update script is located at ${TPUT_GREEN}${TPUT_BOLD}${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh${TPUT_RESET}" + echo >&2 + + return 0 +} + +cleanup_old_netdata_updater() { + if [ -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh ]; then + echo >&2 "Removing updater from deprecated location" + rm -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh + fi + + if issystemd && [ -n "$(get_systemd_service_dir)" ] ; then + systemctl disable netdata-updater.timer + rm -f "$(get_systemd_service_dir)/netdata-updater.timer" + rm -f "$(get_systemd_service_dir)/netdata-updater.service" + fi + + if [ -d /etc/cron.daily ]; then + rm -f /etc/cron.daily/netdata-updater.sh + rm -f /etc/cron.daily/netdata-updater + fi + + if [ -d /etc/periodic/daily ]; then + rm -f /etc/periodic/daily/netdata-updater.sh + rm -f /etc/periodic/daily/netdata-updater + fi + + if [ -d /etc/cron.d ]; then + rm -f /etc/cron.d/netdata-updater + fi + + return 0 +} + +enable_netdata_updater() { + local updater_type + + if [ -n "${1}" ] ; then + updater_type="${1}" + else + updater_type="$(_get_scheduler_type)" + fi + + case "${updater_type}" in + "systemd") + systemctl enable netdata-updater.timer + + echo >&2 "Auto-updating has been enabled using a systemd timer unit." + echo >&2 + echo >&2 "If the update process fails, the failure will be logged to the systemd journal just like a regular service failure." + echo >&2 "Successful updates should produce empty logs." + echo >&2 + ;; + "interval") + ln -sf "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" "$(_get_intervaldir)/netdata-updater" + + echo >&2 "Auto-updating has been enabled through cron, updater script linked to ${TPUT_RED}${TPUT_BOLD}$(_get_intervaldir)/netdata-updater${TPUT_RESET}" + echo >&2 + echo >&2 "If the update process fails and you have email notifications set up correctly for cron on this system, you should receive an email notification of the failure." + echo >&2 "Successful updates will not send an email." + echo >&2 + ;; + "crontab") + cat "${NETDATA_SOURCE_DIR}/system/netdata.crontab" > "/etc/cron.d/netdata-updater" + + echo >&2 "Auto-updating has been enabled through cron, using a crontab at ${TPUT_RED}${TPUT_BOLD}/etc/cron.d/netdata-updater${TPUT_RESET}" + echo >&2 + echo >&2 "If the update process fails and you have email notifications set up correctly for cron on this system, you should receive an email notification of the failure." + echo >&2 "Successful updates will not send an email." + echo >&2 + ;; + *) + echo >&2 "Unable to determine what type of auto-update scheduling to use, not enabling auto-updates." + echo >&2 + return 1 + esac + + return 0 +} + +disable_netdata_updater() { + echo >&2 "You chose *NOT* to enable auto-update, removing any links to the updater from cron (it may have happened if you are reinstalling)" + echo >&2 + + if issystemd && [ -n "$(get_systemd_service_dir)" ] ; then + systemctl disable netdata-updater.timer + fi + + if [ -d /etc/cron.daily ]; then + rm -f /etc/cron.daily/netdata-updater.sh + rm -f /etc/cron.daily/netdata-updater + fi + + if [ -d /etc/periodic/daily ]; then + rm -f /etc/periodic/daily/netdata-updater.sh + rm -f /etc/periodic/daily/netdata-updater + fi + + if [ -d /etc/cron.d ]; then + rm -f /etc/cron.d/netdata-updater + fi + + return 0 +} + +set_netdata_updater_channel() { + sed -i -e "s/^RELEASE_CHANNEL=.*/RELEASE_CHANNEL=\"${RELEASE_CHANNEL}\"/" "${NETDATA_USER_CONFIG_DIR}/.environment" +} diff --git a/packaging/installer/install-required-packages.sh b/packaging/installer/install-required-packages.sh new file mode 100755 index 0000000..23cbe12 --- /dev/null +++ b/packaging/installer/install-required-packages.sh @@ -0,0 +1,2175 @@ +#!/usr/bin/env bash +# shellcheck disable=SC2034 +# We use lots of computed variable names in here, so we need to disable shellcheck 2034 + +export PATH="${PATH}:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin" +export LC_ALL=C + +# Be nice on production environments +renice 19 $$ > /dev/null 2> /dev/null + +ME="${0}" + +if [ "${BASH_VERSINFO[0]}" -lt "4" ]; then + echo >&2 "Sorry! This script needs BASH version 4+, but you have BASH version ${BASH_VERSION}" + exit 1 +fi + +# These options control which packages we are going to install +# They can be pre-set, but also can be controlled with command line options +PACKAGES_NETDATA=${PACKAGES_NETDATA-0} +PACKAGES_NETDATA_NODEJS=${PACKAGES_NETDATA_NODEJS-0} +PACKAGES_NETDATA_PYTHON=${PACKAGES_NETDATA_PYTHON-0} +PACKAGES_NETDATA_PYTHON3=${PACKAGES_NETDATA_PYTHON3-0} +PACKAGES_NETDATA_PYTHON_MYSQL=${PACKAGES_NETDATA_PYTHON_MYSQL-0} +PACKAGES_NETDATA_PYTHON_POSTGRES=${PACKAGES_NETDATA_PYTHON_POSTGRES-0} +PACKAGES_NETDATA_PYTHON_MONGO=${PACKAGES_NETDATA_PYTHON_MONGO-0} +PACKAGES_DEBUG=${PACKAGES_DEBUG-0} +PACKAGES_IPRANGE=${PACKAGES_IPRANGE-0} +PACKAGES_FIREHOL=${PACKAGES_FIREHOL-0} +PACKAGES_FIREQOS=${PACKAGES_FIREQOS-0} +PACKAGES_UPDATE_IPSETS=${PACKAGES_UPDATE_IPSETS-0} +PACKAGES_NETDATA_DEMO_SITE=${PACKAGES_NETDATA_DEMO_SITE-0} +PACKAGES_NETDATA_SENSORS=${PACKAGES_NETDATA_SENSORS-0} +PACKAGES_NETDATA_DATABASE=${PACKAGES_NETDATA_DATABASE-0} +PACKAGES_NETDATA_EBPF=${PACKAGES_NETDATA_EBPF-0} + +# needed commands +lsb_release=$(command -v lsb_release 2> /dev/null) + +# Check which package managers are available +apk=$(command -v apk 2> /dev/null) +apt_get=$(command -v apt-get 2> /dev/null) +brew=$(command -v brew 2> /dev/null) +pkg=$(command -v pkg 2> /dev/null) +dnf=$(command -v dnf 2> /dev/null) +emerge=$(command -v emerge 2> /dev/null) +equo=$(command -v equo 2> /dev/null) +pacman=$(command -v pacman 2> /dev/null) +swupd=$(command -v swupd 2> /dev/null) +yum=$(command -v yum 2> /dev/null) +zypper=$(command -v zypper 2> /dev/null) + +distribution= +release= +version= +codename= +package_installer= +tree= +detection= +NAME= +ID= +ID_LIKE= +VERSION= +VERSION_ID= + +usage() { + cat << EOF +OPTIONS: + +${ME} [--dont-wait] [--non-interactive] \\ + [distribution DD [version VV] [codename CN]] [installer IN] [packages] + +Supported distributions (DD): + + - arch (all Arch Linux derivatives) + - centos (all CentOS derivatives) + - gentoo (all Gentoo Linux derivatives) + - sabayon (all Sabayon Linux derivatives) + - debian, ubuntu (all Debian and Ubuntu derivatives) + - redhat, fedora (all Red Hat and Fedora derivatives) + - suse, opensuse (all SUSE and openSUSE derivatives) + - clearlinux (all Clear Linux derivatives) + - macos (Apple's macOS) + +Supported installers (IN): + + - apt-get all Debian / Ubuntu Linux derivatives + - dnf newer Red Hat / Fedora Linux + - emerge all Gentoo Linux derivatives + - equo all Sabayon Linux derivatives + - pacman all Arch Linux derivatives + - yum all Red Hat / Fedora / CentOS Linux derivatives + - zypper all SUSE Linux derivatives + - apk all Alpine derivatives + - swupd all Clear Linux derivatives + - brew macOS Homebrew + - pkg FreeBSD Ports + +Supported packages (you can append many of them): + + - netdata-all all packages required to install netdata + including mysql client, postgres client, + node.js, python, sensors, etc + + - netdata minimum packages required to install netdata + (no mysql client, no nodejs, includes python) + + - nodejs install nodejs + (required for monitoring named and SNMP) + + - python install python + + - python3 install python3 + + - python-mysql install MySQLdb + (for monitoring mysql, will install python3 version + if python3 is enabled or detected) + + - python-postgres install psycopg2 + (for monitoring postgres, will install python3 version + if python3 is enabled or detected) + + - python-pymongo install python-pymongo (or python3-pymongo for python3) + + - sensors install lm_sensors for monitoring h/w sensors + + - firehol-all packages required for FireHOL, FireQoS, update-ipsets + - firehol packages required for FireHOL + - fireqos packages required for FireQoS + - update-ipsets packages required for update-ipsets + + - demo packages required for running a netdata demo site + (includes nginx and various debugging tools) + + +If you don't supply the --dont-wait option, the program +will ask you before touching your system. + +EOF +} + +release2lsb_release() { + # loads the given /etc/x-release file + # this file is normaly a single line containing something like + # + # X Linux release 1.2.3 (release-name) + # + # It attempts to parse it + # If it succeeds, it returns 0 + # otherwise it returns 1 + + local file="${1}" x DISTRIB_ID="" DISTRIB_RELEASE="" DISTRIB_CODENAME="" + echo >&2 "Loading ${file} ..." + + x="$(grep -v "^$" "${file}" | head -n 1)" + + if [[ "${x}" =~ ^.*[[:space:]]+Linux[[:space:]]+release[[:space:]]+.*[[:space:]]+(.*)[[:space:]]*$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+Linux[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]\+(\(.*\))[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"\nDISTRIB_CODENAME=\"\3\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+Linux[[:space:]]+release[[:space:]]+.*[[:space:]]+$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+Linux[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+release[[:space:]]+.*[[:space:]]+(.*)[[:space:]]*$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]\+(\(.*\))[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"\nDISTRIB_CODENAME=\"\3\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+release[[:space:]]+.*[[:space:]]+$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"|g" | grep "^DISTRIB")" + fi + + distribution="${DISTRIB_ID}" + version="${DISTRIB_RELEASE}" + codename="${DISTRIB_CODENAME}" + + [ -z "${distribution}" ] && echo >&2 "Cannot parse this lsb-release: ${x}" && return 1 + detection="${file}" + return 0 +} + +get_os_release() { + # Loads the /etc/os-release or /usr/lib/os-release file(s) + # Only the required fields are loaded + # + # If it manages to load a valid os-release, it returns 0 + # otherwise it returns 1 + # + # It searches the ID_LIKE field for a compatible distribution + + os_release_file= + if [ -s "/etc/os-release" ]; then + os_release_file="/etc/os-release" + elif [ -s "/usr/lib/os-release" ]; then + os_release_file="/usr/lib/os-release" + else + echo >&2 "Cannot find an os-release file ..." + return 1 + fi + + local x + echo >&2 "Loading ${os_release_file} ..." + + eval "$(grep -E "^(NAME|ID|ID_LIKE|VERSION|VERSION_ID)=" "${os_release_file}")" + for x in "${ID}" ${ID_LIKE}; do + case "${x,,}" in + alpine | arch | centos | clear-linux-os | debian | fedora | gentoo | manjaro | opensuse-leap | rhel | sabayon | sles | suse | ubuntu) + distribution="${x}" + version="${VERSION_ID}" + codename="${VERSION}" + detection="${os_release_file}" + break + ;; + *) + echo >&2 "Unknown distribution ID: ${x}" + ;; + esac + done + [ -z "${distribution}" ] && echo >&2 "Cannot find valid distribution in: ${ID} ${ID_LIKE}" && return 1 + + [ -z "${distribution}" ] && return 1 + return 0 +} + +get_lsb_release() { + # Loads the /etc/lsb-release file + # If it fails, it attempts to run the command: lsb_release -a + # and parse its output + # + # If it manages to find the lsb-release, it returns 0 + # otherwise it returns 1 + + if [ -f "/etc/lsb-release" ]; then + echo >&2 "Loading /etc/lsb-release ..." + local DISTRIB_ID="" DISTRIB_RELEASE="" DISTRIB_CODENAME="" + eval "$(grep -E "^(DISTRIB_ID|DISTRIB_RELEASE|DISTRIB_CODENAME)=" /etc/lsb-release)" + distribution="${DISTRIB_ID}" + version="${DISTRIB_RELEASE}" + codename="${DISTRIB_CODENAME}" + detection="/etc/lsb-release" + fi + + if [ -z "${distribution}" ] && [ -n "${lsb_release}" ]; then + echo >&2 "Cannot find distribution with /etc/lsb-release" + echo >&2 "Running command: lsb_release ..." + eval "declare -A release=( $(lsb_release -a 2> /dev/null | sed -e "s|^\(.*\):[[:space:]]*\(.*\)$|[\1]=\"\2\"|g") )" + distribution="${release["Distributor ID"]}" + version="${release[Release]}" + codename="${release[Codename]}" + detection="lsb_release" + fi + + [ -z "${distribution}" ] && echo >&2 "Cannot find valid distribution with lsb-release" && return 1 + return 0 +} + +find_etc_any_release() { + # Check for any of the known /etc/x-release files + # If it finds one, it loads it and returns 0 + # otherwise it returns 1 + + if [ -f "/etc/arch-release" ]; then + release2lsb_release "/etc/arch-release" && return 0 + fi + + if [ -f "/etc/centos-release" ]; then + release2lsb_release "/etc/centos-release" && return 0 + fi + + if [ -f "/etc/redhat-release" ]; then + release2lsb_release "/etc/redhat-release" && return 0 + fi + + if [ -f "/etc/SuSe-release" ]; then + release2lsb_release "/etc/SuSe-release" && return 0 + fi + + return 1 +} + +autodetect_distribution() { + # autodetection of distribution/OS + case "$(uname -s)" in + "Linux") + get_os_release || get_lsb_release || find_etc_any_release + ;; + "FreeBSD") + distribution="freebsd" + version="$(uname -r)" + detection="uname" + ;; + "Darwin") + distribution="macos" + version="$(uname -r)" + detection="uname" + + if [ ${EUID} -eq 0 ]; then + echo >&2 "This script does not support running as EUID 0 on macOS. Please run it as a regular user." + exit 1 + fi + ;; + *) + return 1 + ;; + esac +} + +user_picks_distribution() { + # let the user pick a distribution + + echo >&2 + echo >&2 "I NEED YOUR HELP" + echo >&2 "It seems I cannot detect your system automatically." + + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + echo >&2 " > Bailing out..." + exit 1 + fi + + if [ -z "${equo}" ] && [ -z "${emerge}" ] && [ -z "${apt_get}" ] && [ -z "${yum}" ] && [ -z "${dnf}" ] && [ -z "${pacman}" ] && [ -z "${apk}" ] && [ -z "${swupd}" ]; then + echo >&2 "And it seems I cannot find a known package manager in this system." + echo >&2 "Please open a github issue to help us support your system too." + exit 1 + fi + + local opts= + echo >&2 "I found though that the following installers are available:" + echo >&2 + [ -n "${apt_get}" ] && echo >&2 " - Debian/Ubuntu based (installer is: apt-get)" && opts="apt-get ${opts}" + [ -n "${yum}" ] && echo >&2 " - Redhat/Fedora/Centos based (installer is: yum)" && opts="yum ${opts}" + [ -n "${dnf}" ] && echo >&2 " - Redhat/Fedora/Centos based (installer is: dnf)" && opts="dnf ${opts}" + [ -n "${zypper}" ] && echo >&2 " - SuSe based (installer is: zypper)" && opts="zypper ${opts}" + [ -n "${pacman}" ] && echo >&2 " - Arch Linux based (installer is: pacman)" && opts="pacman ${opts}" + [ -n "${emerge}" ] && echo >&2 " - Gentoo based (installer is: emerge)" && opts="emerge ${opts}" + [ -n "${equo}" ] && echo >&2 " - Sabayon based (installer is: equo)" && opts="equo ${opts}" + [ -n "${apk}" ] && echo >&2 " - Alpine Linux based (installer is: apk)" && opts="apk ${opts}" + [ -n "${swupd}" ] && echo >&2 " - Clear Linux based (installer is: swupd)" && opts="swupd ${opts}" + [ -n "${brew}" ] && echo >&2 " - macOS based (installer is: brew)" && opts="brew ${opts}" + # XXX: This is being removed in another PR. + echo >&2 + + REPLY= + while [ -z "${REPLY}" ]; do + echo "To proceed please write one of these:" + echo "${opts// /, }" + if ! read -r -p ">" REPLY; then + continue + fi + + if [ "${REPLY}" = "yum" ] && [ -z "${distribution}" ]; then + REPLY= + while [ -z "${REPLY}" ]; do + if ! read -r -p "yum in centos, rhel or fedora? > "; then + continue + fi + + case "${REPLY,,}" in + fedora | rhel) + distribution="rhel" + ;; + centos) + distribution="centos" + ;; + *) + echo >&2 "Please enter 'centos', 'fedora' or 'rhel'." + REPLY= + ;; + esac + done + REPLY="yum" + fi + check_package_manager "${REPLY}" || REPLY= + done +} + +detect_package_manager_from_distribution() { + case "${1,,}" in + arch* | manjaro*) + package_installer="install_pacman" + tree="arch" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pacman}" ]; then + echo >&2 "command 'pacman' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + sabayon*) + package_installer="install_equo" + tree="sabayon" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${equo}" ]; then + echo >&2 "command 'equo' is required to install packages on a '${distribution} ${version}' system." + # Maybe offer to fall back on emerge? Both installers exist in Sabayon... + exit 1 + fi + ;; + + alpine*) + package_installer="install_apk" + tree="alpine" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apk}" ]; then + echo >&2 "command 'apk' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + gentoo*) + package_installer="install_emerge" + tree="gentoo" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${emerge}" ]; then + echo >&2 "command 'emerge' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + debian* | ubuntu*) + package_installer="install_apt_get" + tree="debian" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apt_get}" ]; then + echo >&2 "command 'apt-get' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + centos* | clearos*) + echo >&2 "You should have EPEL enabled to install all the prerequisites." + echo >&2 "Check: http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/" + package_installer="install_yum" + tree="centos" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${yum}" ]; then + echo >&2 "command 'yum' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + fedora* | redhat* | red\ hat* | rhel*) + package_installer= + tree="rhel" + [ -n "${yum}" ] && package_installer="install_yum" + [ -n "${dnf}" ] && package_installer="install_dnf" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${package_installer}" ]; then + echo >&2 "command 'yum' or 'dnf' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + suse* | opensuse* | sles*) + package_installer="install_zypper" + tree="suse" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${zypper}" ]; then + echo >&2 "command 'zypper' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + clear-linux* | clearlinux*) + package_installer="install_swupd" + tree="clearlinux" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${swupd}" ]; then + echo >&2 "command 'swupd' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + freebsd) + package_installer="install_pkg" + tree="freebsd" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pkg}" ]; then + echo >&2 "command 'pkg' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + macos) + package_installer="install_brew" + tree="macos" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${brew}" ]; then + echo >&2 "command 'brew' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + *) + # oops! unknown system + user_picks_distribution + ;; + esac +} + +# XXX: This is being removed in another PR. +check_package_manager() { + # This is called only when the user is selecting a package manager + # It is used to verify the user selection is right + + echo >&2 "Checking package manager: ${1}" + + case "${1}" in + apt-get) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apt_get}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_apt_get" + tree="debian" + detection="user-input" + return 0 + ;; + + dnf) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${dnf}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_dnf" + tree="rhel" + detection="user-input" + return 0 + ;; + + apk) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apk}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_apk" + tree="alpine" + detection="user-input" + return 0 + ;; + + equo) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${equo}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_equo" + tree="sabayon" + detection="user-input" + return 0 + ;; + + emerge) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${emerge}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_emerge" + tree="gentoo" + detection="user-input" + return 0 + ;; + + pacman) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pacman}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_pacman" + tree="arch" + detection="user-input" + + return 0 + ;; + + zypper) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${zypper}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_zypper" + tree="suse" + detection="user-input" + return 0 + ;; + + yum) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${yum}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_yum" + if [ "${distribution}" = "centos" ]; then + tree="centos" + else + tree="rhel" + fi + detection="user-input" + return 0 + ;; + + swupd) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${swupd}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_swupd" + tree="clear-linux" + detection="user-input" + return 0 + ;; + + brew) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${brew}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_brew" + tree="macos" + detection="user-input" + + return 0 + ;; + + *) + echo >&2 "Invalid package manager: '${1}'." + return 1 + ;; + esac +} + +require_cmd() { + # check if any of the commands given as argument + # are present on this system + # If any of them is available, it returns 0 + # otherwise 1 + + [ "${IGNORE_INSTALLED}" -eq 1 ] && return 1 + + local wanted found + for wanted in "${@}"; do + if command -v "${wanted}" > /dev/null 2>&1; then + found="$(command -v "$wanted" 2> /dev/null)" + fi + [ -n "${found}" ] && [ -x "${found}" ] && return 0 + done + return 1 +} + +declare -A pkg_find=( + ['gentoo']="sys-apps/findutils" + ['fedora']="findutils" + ['clearlinux']="findutils" + ['macos']="NOTREQUIRED" + ['freebsd']="NOTREQUIRED" + ['default']="WARNING|" +) + +declare -A pkg_distro_sdk=( + ['alpine']="alpine-sdk" + ['default']="NOTREQUIRED" +) + +declare -A pkg_autoconf=( + ['gentoo']="sys-devel/autoconf" + ['clearlinux']="c-basic" + ['default']="autoconf" +) + +# required to compile netdata with --enable-sse +# https://github.com/firehol/netdata/pull/450 +declare -A pkg_autoconf_archive=( + ['gentoo']="sys-devel/autoconf-archive" + ['clearlinux']="c-basic" + ['alpine']="WARNING|" + ['default']="autoconf-archive" + + # exceptions + ['centos-6']="WARNING|" + ['rhel-6']="WARNING|" + ['rhel-7']="WARNING|" +) + +declare -A pkg_autogen=( + ['gentoo']="sys-devel/autogen" + ['clearlinux']="c-basic" + ['alpine']="WARNING|" + ['default']="autogen" + + # exceptions + ['centos-6']="WARNING|" + ['rhel-6']="WARNING|" +) + +declare -A pkg_automake=( + ['gentoo']="sys-devel/automake" + ['clearlinux']="c-basic" + ['default']="automake" +) + +# required to bundle libJudy +declare -A pkg_libtool=( + ['gentoo']="sys-devel/libtool" + ['clearlinux']="c-basic" + ['default']="libtool" +) + +# Required to build libwebsockets and libmosquitto on some systems. +declare -A pkg_cmake=( + ['gentoo']="dev-util/cmake" + ['clearlinux']="c-basic" + ['default']="cmake" +) + +declare -A pkg_json_c_dev=( + ['alpine']="json-c-dev" + ['arch']="json-c" + ['clearlinux']="devpkg-json-c" + ['debian']="libjson-c-dev" + ['gentoo']="dev-libs/json-c" + ['sabayon']="dev-libs/json-c" + ['suse']="libjson-c-devel" + ['freebsd']="json-c" + ['default']="json-c-devel" +) + +declare -A pkg_bridge_utils=( + ['gentoo']="net-misc/bridge-utils" + ['clearlinux']="network-basic" + ['macos']="WARNING|" + ['default']="bridge-utils" +) + +declare -A pkg_chrony=( + ['gentoo']="net-misc/chrony" + ['clearlinux']="time-server-basic" + ['macos']="WARNING|" + ['default']="chrony" +) + +declare -A pkg_curl=( + ['gentoo']="net-misc/curl" + ['sabayon']="net-misc/curl" + ['default']="curl" +) + +declare -A pkg_gzip=( + ['gentoo']="app-arch/gzip" + ['macos']="NOTREQUIRED" + ['default']="gzip" +) + +declare -A pkg_tar=( + ['gentoo']="app-arch/tar" + ['clearlinux']="os-core-update" + ['macos']="NOTREQUIRED" + ['default']="tar" +) + +declare -A pkg_git=( + ['gentoo']="dev-vcs/git" + ['default']="git" +) + +declare -A pkg_gcc=( + ['gentoo']="sys-devel/gcc" + ['clearlinux']="c-basic" + ['macos']="NOTREQUIRED" + ['default']="gcc" +) + +declare -A pkg_gdb=( + ['gentoo']="sys-devel/gdb" + ['macos']="NOTREQUIRED" + ['default']="gdb" +) + +declare -A pkg_iotop=( + ['gentoo']="sys-process/iotop" + ['macos']="WARNING|" + ['default']="iotop" +) + +declare -A pkg_iproute2=( + ['alpine']="iproute2" + ['debian']="iproute2" + ['gentoo']="sys-apps/iproute2" + ['sabayon']="sys-apps/iproute2" + ['clearlinux']="iproute2" + ['macos']="WARNING|" + ['default']="iproute" + + # exceptions + ['ubuntu-12.04']="iproute" +) + +declare -A pkg_ipset=( + ['gentoo']="net-firewall/ipset" + ['clearlinux']="network-basic" + ['macos']="WARNING|" + ['default']="ipset" +) + +declare -A pkg_jq=( + ['gentoo']="app-misc/jq" + ['default']="jq" +) + +declare -A pkg_iptables=( + ['gentoo']="net-firewall/iptables" + ['macos']="WARNING|" + ['default']="iptables" +) + +declare -A pkg_libz_dev=( + ['alpine']="zlib-dev" + ['arch']="zlib" + ['centos']="zlib-devel" + ['debian']="zlib1g-dev" + ['gentoo']="sys-libs/zlib" + ['sabayon']="sys-libs/zlib" + ['rhel']="zlib-devel" + ['suse']="zlib-devel" + ['clearlinux']="devpkg-zlib" + ['macos']="NOTREQUIRED" + ['freebsd']="lzlib" + ['default']="" +) + +declare -A pkg_libuuid_dev=( + ['alpine']="util-linux-dev" + ['arch']="util-linux" + ['centos']="libuuid-devel" + ['clearlinux']="devpkg-util-linux" + ['debian']="uuid-dev" + ['gentoo']="sys-apps/util-linux" + ['sabayon']="sys-apps/util-linux" + ['rhel']="libuuid-devel" + ['suse']="libuuid-devel" + ['macos']="NOTREQUIRED" + ['freebsd']="e2fsprogs-libuuid" + ['default']="" +) + +declare -A pkg_libmnl_dev=( + ['alpine']="libmnl-dev" + ['arch']="libmnl" + ['centos']="libmnl-devel" + ['debian']="libmnl-dev" + ['gentoo']="net-libs/libmnl" + ['sabayon']="net-libs/libmnl" + ['rhel']="libmnl-devel" + ['suse']="libmnl-devel" + ['clearlinux']="devpkg-libmnl" + ['macos']="NOTREQUIRED" + ['default']="" +) + +declare -A pkg_lm_sensors=( + ['alpine']="lm_sensors" + ['arch']="lm_sensors" + ['centos']="lm_sensors" + ['debian']="lm-sensors" + ['gentoo']="sys-apps/lm-sensors" + ['sabayon']="sys-apps/lm_sensors" + ['rhel']="lm_sensors" + ['suse']="sensors" + ['clearlinux']="lm-sensors" + ['macos']="WARNING|" + ['freebsd']="NOTREQUIRED" + ['default']="lm_sensors" +) + +declare -A pkg_logwatch=( + ['gentoo']="sys-apps/logwatch" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="logwatch" +) + +declare -A pkg_lxc=( + ['gentoo']="app-emulation/lxc" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="lxc" +) + +declare -A pkg_mailutils=( + ['gentoo']="net-mail/mailutils" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="mailutils" +) + +declare -A pkg_make=( + ['gentoo']="sys-devel/make" + ['macos']="NOTREQUIRED" + ['freebsd']="gmake" + ['default']="make" +) + +declare -A pkg_netcat=( + ['alpine']="netcat-openbsd" + ['arch']="netcat" + ['centos']="nmap-ncat" + ['debian']="netcat" + ['gentoo']="net-analyzer/netcat" + ['sabayon']="net-analyzer/gnu-netcat" + ['rhel']="nmap-ncat" + ['suse']="netcat-openbsd" + ['clearlinux']="sysadmin-basic" + ['arch']="gnu-netcat" + ['macos']="NOTREQUIRED" + ['default']="netcat" + + # exceptions + ['centos-6']="nc" + ['rhel-6']="nc" +) + +declare -A pkg_nginx=( + ['gentoo']="www-servers/nginx" + ['default']="nginx" +) + +declare -A pkg_nodejs=( + ['gentoo']="net-libs/nodejs" + ['clearlinux']="nodejs-basic" + ['freebsd']="node" + ['default']="nodejs" + + # exceptions + ['rhel-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['rhel-7']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['centos-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['debian-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['debian-7']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" +) + +declare -A pkg_postfix=( + ['gentoo']="mail-mta/postfix" + ['macos']="WARNING|" + ['default']="postfix" +) + +declare -A pkg_pkg_config=( + ['alpine']="pkgconfig" + ['arch']="pkgconfig" + ['centos']="pkgconfig" + ['debian']="pkg-config" + ['gentoo']="virtual/pkgconfig" + ['sabayon']="virtual/pkgconfig" + ['rhel']="pkgconfig" + ['suse']="pkg-config" + ['freebsd']="pkgconf" + ['clearlinux']="c-basic" + ['default']="pkg-config" +) + +declare -A pkg_python=( + ['gentoo']="dev-lang/python" + ['sabayon']="dev-lang/python:2.7" + ['clearlinux']="python-basic" + ['default']="python" + + # Exceptions + ['macos']="WARNING|" + ['centos-8']="python2" +) + +declare -A pkg_python_mysqldb=( + ['alpine']="py-mysqldb" + ['arch']="mysql-python" + ['centos']="MySQL-python" + ['debian']="python-mysqldb" + ['gentoo']="dev-python/mysqlclient" + ['sabayon']="dev-python/mysqlclient" + ['rhel']="MySQL-python" + ['suse']="python-PyMySQL" + ['clearlinux']="WARNING|" + ['default']="python-mysql" + + # exceptions + ['fedora-24']="python2-mysql" +) + +declare -A pkg_python3_mysqldb=( + ['alpine']="WARNING|" + ['arch']="WARNING|" + ['centos']="WARNING|" + ['debian']="python3-mysqldb" + ['gentoo']="dev-python/mysqlclient" + ['sabayon']="dev-python/mysqlclient" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="WARNING|" + + # exceptions + ['debian-6']="WARNING|" + ['debian-7']="WARNING|" + ['debian-8']="WARNING|" + ['ubuntu-12.04']="WARNING|" + ['ubuntu-12.10']="WARNING|" + ['ubuntu-13.04']="WARNING|" + ['ubuntu-13.10']="WARNING|" + ['ubuntu-14.04']="WARNING|" + ['ubuntu-14.10']="WARNING|" + ['ubuntu-15.04']="WARNING|" + ['ubuntu-15.10']="WARNING|" + ['centos-7']="python36-mysql" + ['centos-8']="python38-mysql" + ['rhel-7']="python36-mysql" + ['rhel-8']="python38-mysql" +) + +declare -A pkg_python_psycopg2=( + ['alpine']="py-psycopg2" + ['arch']="python2-psycopg2" + ['centos']="python-psycopg2" + ['debian']="python-psycopg2" + ['gentoo']="dev-python/psycopg" + ['sabayon']="dev-python/psycopg:2" + ['rhel']="python-psycopg2" + ['suse']="python-psycopg2" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="python-psycopg2" +) + +declare -A pkg_python3_psycopg2=( + ['alpine']="py3-psycopg2" + ['arch']="python-psycopg2" + ['centos']="WARNING|" + ['debian']="WARNING|" + ['gentoo']="dev-python/psycopg" + ['sabayon']="dev-python/psycopg:2" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="WARNING|" + + ['centos-7']="python3-psycopg2" + ['centos-8']="python38-psycopg2" + ['rhel-7']="python3-psycopg2" + ['rhel-8']="python38-psycopg2" +) + +declare -A pkg_python_pip=( + ['alpine']="py-pip" + ['gentoo']="dev-python/pip" + ['sabayon']="dev-python/pip" + ['clearlinux']="python-basic" + ['macos']="WARNING|" + ['default']="python-pip" +) + +declare -A pkg_python3_pip=( + ['alpine']="py3-pip" + ['arch']="python-pip" + ['gentoo']="dev-python/pip" + ['sabayon']="dev-python/pip" + ['clearlinux']="python3-basic" + ['macos']="NOTREQUIRED" + ['default']="python3-pip" +) + +declare -A pkg_python_pymongo=( + ['alpine']="WARNING|" + ['arch']="python2-pymongo" + ['centos']="WARNING|" + ['debian']="python-pymongo" + ['gentoo']="dev-python/pymongo" + ['suse']="python-pymongo" + ['clearlinux']="WARNING|" + ['rhel']="WARNING|" + ['macos']="WARNING|" + ['default']="python-pymongo" +) + +declare -A pkg_python3_pymongo=( + ['alpine']="WARNING|" + ['arch']="python-pymongo" + ['centos']="WARNING|" + ['debian']="python3-pymongo" + ['gentoo']="dev-python/pymongo" + ['suse']="python3-pymongo" + ['clearlinux']="WARNING|" + ['rhel']="WARNING|" + ['freebsd']="py37-pymongo" + ['macos']="WARNING|" + ['default']="python3-pymongo" + + ['centos-7']="python36-pymongo" + ['centos-8']="python3-pymongo" + ['rhel-7']="python36-pymongo" + ['rhel-8']="python3-pymongo" +) + +declare -A pkg_python_requests=( + ['alpine']="py-requests" + ['arch']="python2-requests" + ['centos']="python-requests" + ['debian']="python-requests" + ['gentoo']="dev-python/requests" + ['sabayon']="dev-python/requests" + ['rhel']="python-requests" + ['suse']="python-requests" + ['clearlinux']="python-extras" + ['macos']="WARNING|" + ['default']="python-requests" + ['alpine-3.1.4']="WARNING|" + ['alpine-3.2.3']="WARNING|" +) + +declare -A pkg_python3_requests=( + ['alpine']="py3-requests" + ['arch']="python-requests" + ['centos']="WARNING|" + ['debian']="WARNING|" + ['gentoo']="dev-python/requests" + ['sabayon']="dev-python/requests" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="python-extras" + ['macos']="WARNING|" + ['default']="WARNING|" + + ['centos-7']="python36-requests" + ['centos-8']="python3-requests" + ['rhel-7']="python36-requests" + ['rhel-8']="python3-requests" +) + +declare -A pkg_lz4=( + ['alpine']="lz4-dev" + ['debian']="liblz4-dev" + ['ubuntu']="liblz4-dev" + ['suse']="liblz4-devel" + ['gentoo']="app-arch/lz4" + ['clearlinux']="devpkg-lz4" + ['arch']="lz4" + ['macos']="lz4" + ['freebsd']="liblz4" + ['default']="lz4-devel" +) + +declare -A pkg_libuv=( + ['alpine']="libuv-dev" + ['debian']="libuv1-dev" + ['ubuntu']="libuv1-dev" + ['gentoo']="dev-libs/libuv" + ['arch']="libuv" + ['clearlinux']="devpkg-libuv" + ['macos']="libuv" + ['freebsd']="libuv" + ['default']="libuv-devel" +) + +declare -A pkg_openssl=( + ['alpine']="openssl-dev" + ['debian']="libssl-dev" + ['ubuntu']="libssl-dev" + ['suse']="libopenssl-devel" + ['clearlinux']="devpkg-openssl" + ['gentoo']="dev-libs/openssl" + ['arch']="openssl" + ['freebsd']="openssl" + ['macos']="openssl@1.1" + ['default']="openssl-devel" +) + +declare -A pkg_judy=( + ['debian']="libjudy-dev" + ['ubuntu']="libjudy-dev" + ['suse']="judy-devel" + ['gentoo']="dev-libs/judy" + ['arch']="judy" + ['freebsd']="Judy" + ['fedora']="Judy-devel" + ['default']="NOTREQUIRED" +) + +declare -A pkg_python3=( + ['gentoo']="dev-lang/python" + ['sabayon']="dev-lang/python:3.4" + ['clearlinux']="python3-basic" + ['macos']="python" + ['default']="python3" + + # exceptions + ['centos-6']="WARNING|" +) + +declare -A pkg_screen=( + ['gentoo']="app-misc/screen" + ['sabayon']="app-misc/screen" + ['clearlinux']="sysadmin-basic" + ['default']="screen" +) + +declare -A pkg_sudo=( + ['gentoo']="app-admin/sudo" + ['macos']="NOTREQUIRED" + ['default']="sudo" +) + +declare -A pkg_sysstat=( + ['gentoo']="app-admin/sysstat" + ['macos']="WARNING|" + ['default']="sysstat" +) + +declare -A pkg_tcpdump=( + ['gentoo']="net-analyzer/tcpdump" + ['clearlinux']="network-basic" + ['default']="tcpdump" +) + +declare -A pkg_traceroute=( + ['alpine']=" " + ['gentoo']="net-analyzer/traceroute" + ['clearlinux']="network-basic" + ['macos']="NOTREQUIRED" + ['default']="traceroute" +) + +declare -A pkg_valgrind=( + ['gentoo']="dev-util/valgrind" + ['default']="valgrind" +) + +declare -A pkg_ulogd=( + ['centos']="WARNING|" + ['rhel']="WARNING|" + ['clearlinux']="WARNING|" + ['gentoo']="app-admin/ulogd" + ['arch']="ulogd" + ['macos']="WARNING|" + ['default']="ulogd2" +) + +declare -A pkg_unzip=( + ['gentoo']="app-arch/unzip" + ['macos']="NOTREQUIRED" + ['default']="unzip" +) + +declare -A pkg_zip=( + ['gentoo']="app-arch/zip" + ['macos']="NOTREQUIRED" + ['default']="zip" +) + +declare -A pkg_libelf=( + ['alpine']="elfutils-dev" + ['arch']="libelf" + ['gentoo']="virtual/libelf" + ['sabayon']="virtual/libelf" + ['debian']="libelf-dev" + ['ubuntu']="libelf-dev" + ['fedora']="elfutils-libelf-devel" + ['centos']="elfutils-libelf-devel" + ['rhel']="elfutils-libelf-devel" + ['clearlinux']="devpkg-elfutils" + ['suse']="libelf-devel" + ['macos']="NOTREQUIRED" + ['freebsd']="NOTREQUIRED" + ['default']="libelf-devel" + + # exceptions + ['alpine-3.5']="libelf-dev" + ['alpine-3.4']="libelf-dev" + ['alpine-3.3']="libelf-dev" +) + +validate_package_trees() { + if type -t validate_tree_${tree} > /dev/null; then + validate_tree_${tree} + fi +} + +validate_installed_package() { + validate_${package_installer} "${p}" +} + +suitable_package() { + local package="${1//-/_}" p="" v="${version//.*/}" + + echo >&2 "Searching for ${package} ..." + + eval "p=\${pkg_${package}['${distribution,,}-${version,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${distribution,,}-${v,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${distribution,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}-${version}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}-${v}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['default']}" + + if [[ "${p/|*/}" =~ ^(ERROR|WARNING|INFO)$ ]]; then + echo >&2 "${p/|*/}" + echo >&2 "package ${1} is not available in this system." + if [ -z "${p/*|/}" ]; then + echo >&2 "You may try to install without it." + else + echo >&2 "${p/*|/}" + fi + echo >&2 + return 1 + elif [ "${p}" = "NOTREQUIRED" ]; then + return 0 + elif [ -z "${p}" ]; then + echo >&2 "WARNING" + echo >&2 "package ${1} is not availabe in this system." + echo >&2 + return 1 + else + if [ "${IGNORE_INSTALLED}" -eq 0 ]; then + validate_installed_package "${p}" + else + echo "${p}" + fi + return 0 + fi +} + +packages() { + # detect the packages we need to install on this system + + # ------------------------------------------------------------------------- + # basic build environment + + suitable_package distro-sdk + + require_cmd git || suitable_package git + require_cmd find || suitable_package find + + require_cmd gcc || + require_cmd gcc-multilib || suitable_package gcc + + require_cmd make || suitable_package make + require_cmd autoconf || suitable_package autoconf + suitable_package autoconf-archive + require_cmd autogen || suitable_package autogen + require_cmd automake || suitable_package automake + require_cmd libtoolize || suitable_package libtool + require_cmd pkg-config || suitable_package pkg-config + require_cmd cmake || suitable_package cmake + + # ------------------------------------------------------------------------- + # debugging tools for development + + if [ "${PACKAGES_DEBUG}" -ne 0 ]; then + require_cmd traceroute || suitable_package traceroute + require_cmd tcpdump || suitable_package tcpdump + require_cmd screen || suitable_package screen + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + require_cmd gdb || suitable_package gdb + require_cmd valgrind || suitable_package valgrind + fi + fi + + # ------------------------------------------------------------------------- + # common command line tools + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + require_cmd tar || suitable_package tar + require_cmd curl || suitable_package curl + require_cmd gzip || suitable_package gzip + require_cmd nc || suitable_package netcat + fi + + # ------------------------------------------------------------------------- + # firehol/fireqos/update-ipsets command line tools + + if [ "${PACKAGES_FIREQOS}" -ne 0 ]; then + require_cmd ip || suitable_package iproute2 + fi + + if [ "${PACKAGES_FIREHOL}" -ne 0 ]; then + require_cmd iptables || suitable_package iptables + require_cmd ipset || suitable_package ipset + require_cmd ulogd ulogd2 || suitable_package ulogd + require_cmd traceroute || suitable_package traceroute + require_cmd bridge || suitable_package bridge-utils + fi + + if [ "${PACKAGES_UPDATE_IPSETS}" -ne 0 ]; then + require_cmd ipset || suitable_package ipset + require_cmd zip || suitable_package zip + require_cmd funzip || suitable_package unzip + fi + + # ------------------------------------------------------------------------- + # netdata libraries + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + suitable_package libz-dev + suitable_package libuuid-dev + suitable_package libmnl-dev + suitable_package json-c-dev + fi + + # ------------------------------------------------------------------------- + # sensors + + if [ "${PACKAGES_NETDATA_SENSORS}" -ne 0 ]; then + require_cmd sensors || suitable_package lm_sensors + fi + + # ------------------------------------------------------------------------- + # netdata database + if [ "${PACKAGES_NETDATA_DATABASE}" -ne 0 ]; then + suitable_package libuv + suitable_package lz4 + suitable_package openssl + suitable_package judy + fi + + # ------------------------------------------------------------------------- + # ebpf plugin + if [ "${PACKAGES_NETDATA_EBPF}" -ne 0 ]; then + suitable_package libelf + fi + + # ------------------------------------------------------------------------- + # scripting interpreters for netdata plugins + + if [ "${PACKAGES_NETDATA_NODEJS}" -ne 0 ]; then + require_cmd nodejs node js || suitable_package nodejs + fi + + # ------------------------------------------------------------------------- + # python2 + + if [ "${PACKAGES_NETDATA_PYTHON}" -ne 0 ]; then + require_cmd python || suitable_package python + + [ "${PACKAGES_NETDATA_PYTHON_MONGO}" -ne 0 ] && suitable_package python-pymongo + # suitable_package python-requests + # suitable_package python-pip + + [ "${PACKAGES_NETDATA_PYTHON_MYSQL}" -ne 0 ] && suitable_package python-mysqldb + [ "${PACKAGES_NETDATA_PYTHON_POSTGRES}" -ne 0 ] && suitable_package python-psycopg2 + fi + + # ------------------------------------------------------------------------- + # python3 + + if [ "${PACKAGES_NETDATA_PYTHON3}" -ne 0 ]; then + require_cmd python3 || suitable_package python3 + + [ "${PACKAGES_NETDATA_PYTHON_MONGO}" -ne 0 ] && suitable_package python3-pymongo + # suitable_package python3-requests + # suitable_package python3-pip + + [ "${PACKAGES_NETDATA_PYTHON_MYSQL}" -ne 0 ] && suitable_package python3-mysqldb + [ "${PACKAGES_NETDATA_PYTHON_POSTGRES}" -ne 0 ] && suitable_package python3-psycopg2 + fi + + # ------------------------------------------------------------------------- + # applications needed for the netdata demo sites + + if [ "${PACKAGES_NETDATA_DEMO_SITE}" -ne 0 ]; then + require_cmd sudo || suitable_package sudo + require_cmd jq || suitable_package jq + require_cmd nginx || suitable_package nginx + require_cmd postconf || suitable_package postfix + require_cmd lxc-create || suitable_package lxc + require_cmd logwatch || suitable_package logwatch + require_cmd mail || suitable_package mailutils + require_cmd iostat || suitable_package sysstat + require_cmd iotop || suitable_package iotop + fi +} + +DRYRUN=0 +run() { + + printf >&2 "%q " "${@}" + printf >&2 "\n" + + if [ ! "${DRYRUN}" -eq 1 ]; then + "${@}" + return $? + fi + return 0 +} + +sudo= +if [ ${UID} -ne 0 ]; then + sudo="sudo" +fi + +# ----------------------------------------------------------------------------- +# debian / ubuntu + +validate_install_apt_get() { + echo >&2 " > Checking if package '${*}' is installed..." + [ "$(dpkg-query -W --showformat='${Status}\n' "${*}")" = "install ok installed" ] || echo "${*}" +} + +install_apt_get() { + local opts="" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://serverfault.com/questions/227190/how-do-i-ask-apt-get-to-skip-any-interactive-post-install-configuration-steps + export DEBIAN_FRONTEND="noninteractive" + opts="${opts} -yq" + fi + + read -r -a apt_opts <<< "$opts" + + # update apt repository caches + + echo >&2 "NOTE: Running apt-get update and updating your APT caches ..." + if [ "${version}" = 8 ]; then + echo >&2 "WARNING: You seem to be on Debian 8 (jessie) which is old enough we have to disable Check-Valid-Until checks" + if ! cat /etc/apt/sources.list /etc/apt/sources.list.d/* 2> /dev/null | grep -q jessie-backports; then + echo >&2 "We also have to enable the jessie-backports repository" + if prompt "Is this okay?"; then + ${sudo} /bin/sh -c 'echo "deb http://archive.debian.org/debian/ jessie-backports main contrib non-free" >> /etc/apt/sources.list.d/99-archived.list' + fi + fi + run ${sudo} apt-get "${apt_opts[@]}" -o Acquire::Check-Valid-Until=false update + else + run ${sudo} apt-get "${apt_opts[@]}" update + fi + + # install the required packages + run ${sudo} apt-get "${apt_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# centos / rhel + +prompt() { + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode, assuming yes (y)" + echo >&2 " > Would have promptedfor ${1} ..." + return 0 + fi + + while true; do + read -r -p "${1} [y/n] " yn + case $yn in + [Yy]*) return 0 ;; + [Nn]*) return 1 ;; + *) echo >&2 "Please answer with yes (y) or no (n)." ;; + esac + done +} + +validate_tree_freebsd() { + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + echo >&2 " > FreeBSD Version: ${version} ..." + + make="make" + echo >&2 " > Checking for gmake ..." + if ! pkg query %n-%v | grep -q gmake; then + if prompt "gmake is required to build on FreeBSD and is not installed. Shall I install it?"; then + run ${sudo} pkg install ${opts} gmake + fi + fi +} + +validate_tree_centos() { + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + echo >&2 " > CentOS Version: ${version} ..." + + echo >&2 " > Checking for epel ..." + if ! rpm -qa | grep epel > /dev/null; then + if prompt "epel not found, shall I install it?"; then + run ${sudo} yum ${opts} install epel-release + fi + fi + + if [[ "${version}" =~ ^8(\..*)?$ ]]; then + echo >&2 " > Checking for config-manager ..." + if ! run yum ${sudo} config-manager; then + if prompt "config-manager not found, shall I install it?"; then + run ${sudo} yum ${opts} install 'dnf-command(config-manager)' + fi + fi + + echo >&2 " > Checking for PowerTools ..." + if ! run yum ${sudo} repolist | grep PowerTools; then + if prompt "PowerTools not found, shall I install it?"; then + run ${sudo} yum ${opts} config-manager --set-enabled powertools + fi + fi + + echo >&2 " > Checking for Okay ..." + if ! rpm -qa | grep okay > /dev/null; then + if prompt "okay not found, shall I install it?"; then + run ${sudo} yum ${opts} install http://repo.okay.com.mx/centos/8/x86_64/release/okay-release-1-3.el8.noarch.rpm + fi + fi + + echo >&2 " > Installing Judy-devel directly ..." + run ${sudo} yum ${opts} install http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/Judy-devel-1.0.5-18.module_el8.1.0+217+4d875839.x86_64.rpm + + elif [[ "${version}" =~ ^6\..*$ ]]; then + echo >&2 " > Detected CentOS 6.x ..." + echo >&2 " > Checking for Okay ..." + if ! rpm -qa | grep okay > /dev/null; then + if prompt "okay not found, shall I install it?"; then + run ${sudo} yum ${opts} install http://repo.okay.com.mx/centos/6/x86_64/release/okay-release-1-3.el6.noarch.rpm + fi + fi + + fi +} + +validate_install_yum() { + echo >&2 " > Checking if package '${*}' is installed..." + yum list installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_yum() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} yum update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/87822/does-yum-have-an-equivalent-to-apt-aptitudes-debian-frontend-noninteractive + opts="-y" + fi + + read -r -a yum_opts <<< "${opts}" + + # install the required packages + run ${sudo} yum "${yum_opts[@]}" install "${@}" # --enablerepo=epel-testing +} + +# ----------------------------------------------------------------------------- +# fedora + +validate_install_dnf() { + echo >&2 " > Checking if package '${*}' is installed..." + dnf list installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_dnf() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} dnf update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # man dnf + opts="-y" + fi + + # install the required packages + # --setopt=strict=0 allows dnf to proceed + # installing whatever is available + # even if a package is not found + opts="$opts --setopt=strict=0" + read -r -a dnf_opts <<< "$opts" + run ${sudo} dnf "${dnf_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# gentoo + +validate_install_emerge() { + echo "${*}" +} + +install_emerge() { + # download the latest package info + # we don't do this for emerge - it is very slow + # and most users are expected to do this daily + # emerge --sync + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} emerge --sync or ${sudo} eix-sync " + echo >&2 + fi + + local opts="--ask" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="" + fi + + read -r -a emerge_opts <<< "$opts" + + # install the required packages + run ${sudo} emerge "${emerge_opts[@]}" -v --noreplace "${@}" +} + +# ----------------------------------------------------------------------------- +# alpine + +validate_install_apk() { + echo "${*}" +} + +install_apk() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} apk update " + echo >&2 + fi + + local opts="--force-broken-world" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + else + opts="${opts} -i" + fi + + read -r -a apk_opts <<< "$opts" + + # install the required packages + run ${sudo} apk add "${apk_opts[@]}" "${@}" +} + +# ----------------------------------------------------------------------------- +# sabayon + +validate_install_equo() { + echo >&2 " > Checking if package '${*}' is installed..." + equo s --installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_equo() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} equo up " + echo >&2 + fi + + local opts="-av" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-v" + fi + + read -r -a equo_opts <<< "$opts" + + # install the required packages + run ${sudo} equo i "${equo_opts[@]}" "${@}" +} + +# ----------------------------------------------------------------------------- +# arch + +PACMAN_DB_SYNCED=0 +validate_install_pacman() { + + if [ ${PACMAN_DB_SYNCED} -eq 0 ]; then + echo >&2 " > Running pacman -Sy to sync the database" + local x + x=$(pacman -Sy) + [ -z "${x}" ] && echo "${*}" + PACMAN_DB_SYNCED=1 + fi + echo >&2 " > Checking if package '${*}' is installed..." + + # In pacman, you can utilize alternative flags to exactly match package names, + # but is highly likely we require pattern matching too in this so we keep -s and match + # the exceptional cases like so + local x="" + case "${package}" in + "gcc") + # Temporary workaround: In archlinux, default installation includes runtime libs under package "gcc" + # These are not sufficient for netdata install, so we need to make sure that the appropriate libraries are there + # by ensuring devel libs are available + x=$(pacman -Qs "${*}" | grep "base-devel") + ;; + "tar") + x=$(pacman -Qs "${*}" | grep "local/tar") + ;; + "make") + x=$(pacman -Qs "${*}" | grep "local/make ") + ;; + *) + x=$(pacman -Qs "${*}") + ;; + esac + + [ -z "${x}" ] && echo "${*}" +} + +install_pacman() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} pacman -Syu " + echo >&2 + fi + + # install the required packages + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/52277/pacman-option-to-assume-yes-to-every-question/52278 + # Try the noconfirm option, if that fails, go with the legacy way for non-interactive + run ${sudo} pacman --noconfirm --needed -S "${@}" || yes | run ${sudo} pacman --needed -S "${@}" + else + run ${sudo} pacman --needed -S "${@}" + fi +} + +# ----------------------------------------------------------------------------- +# suse / opensuse + +validate_install_zypper() { + rpm -q "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_zypper() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} zypper update " + echo >&2 + fi + + local opts="--ignore-unknown" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/82016/how-to-use-zypper-in-bash-scripts-for-someone-coming-from-apt-get + opts="${opts} --non-interactive" + fi + + read -r -a zypper_opts <<< "$opts" + + # install the required packages + run ${sudo} zypper "${zypper_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# clearlinux + +validate_install_swupd() { + swupd bundle-list | grep -q "${*}" || echo "${*}" +} + +install_swupd() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} swupd update " + echo >&2 + fi + + run ${sudo} swupd bundle-add "${@}" +} + +# ----------------------------------------------------------------------------- +# macOS + +validate_install_pkg() { + pkg query %n-%v | grep -q "${*}" || echo "${*}" +} + +validate_install_brew() { + brew list | grep -q "${*}" || echo "${*}" +} + +install_pkg() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: pkg update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + read -r -a pkg_opts <<< "${opts}" + + run ${sudo} pkg install "${pkg_opts[@]}" "${@}" +} + +install_brew() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: brew upgrade " + echo >&2 + fi + + run brew install "${@}" +} + +# ----------------------------------------------------------------------------- + +install_failed() { + local ret="${1}" + cat << EOF + + + +We are very sorry! + +Installation of required packages failed. + +What to do now: + + 1. Make sure your system is updated. + Most of the times, updating your system will resolve the issue. + + 2. If the error message is about a specific package, try removing + that package from the command and run it again. + Depending on the broken package, you may be able to continue. + + 3. Let us know. We may be able to help. + Open a github issue with the above log, at: + + https://github.com/netdata/netdata/issues + + +EOF + remote_log "FAILED" "${ret}" + exit 1 +} + +remote_log() { + # log success or failure on our system + # to help us solve installation issues + curl > /dev/null 2>&1 -Ss --max-time 3 "https://registry.my-netdata.io/log/installer?status=${1}&error=${2}&distribution=${distribution}&version=${version}&installer=${package_installer}&tree=${tree}&detection=${detection}&netdata=${PACKAGES_NETDATA}&nodejs=${PACKAGES_NETDATA_NODEJS}&python=${PACKAGES_NETDATA_PYTHON}&python3=${PACKAGES_NETDATA_PYTHON3}&mysql=${PACKAGES_NETDATA_PYTHON_MYSQL}&postgres=${PACKAGES_NETDATA_PYTHON_POSTGRES}&pymongo=${PACKAGES_NETDATA_PYTHON_MONGO}&sensors=${PACKAGES_NETDATA_SENSORS}&database=${PACKAGES_NETDATA_DATABASE}&ebpf=${PACKAGES_NETDATA_EBPF}&firehol=${PACKAGES_FIREHOL}&fireqos=${PACKAGES_FIREQOS}&iprange=${PACKAGES_IPRANGE}&update_ipsets=${PACKAGES_UPDATE_IPSETS}&demo=${PACKAGES_NETDATA_DEMO_SITE}" +} + +if [ -z "${1}" ]; then + usage + exit 1 +fi + +pv=$(python --version 2>&1) +if [ "${tree}" = macos ]; then + pv=3 +elif [[ "${pv}" =~ ^Python\ 2.* ]]; then + pv=2 +elif [[ "${pv}" =~ ^Python\ 3.* ]]; then + pv=3 +elif [[ "${tree}" == "centos" ]] && [ "${version}" -lt 8 ]; then + pv=2 +else + pv=3 +fi + +# parse command line arguments +DONT_WAIT=0 +NON_INTERACTIVE=0 +IGNORE_INSTALLED=0 +while [ -n "${1}" ]; do + case "${1}" in + distribution) + distribution="${2}" + shift + ;; + + version) + version="${2}" + shift + ;; + + codename) + codename="${2}" + shift + ;; + + installer) + check_package_manager "${2}" || exit 1 + shift + ;; + + dont-wait | --dont-wait | -n) + DONT_WAIT=1 + ;; + + non-interactive | --non-interactive | -y) + NON_INTERACTIVE=1 + ;; + + ignore-installed | --ignore-installed | -i) + IGNORE_INSTALLED=1 + ;; + + netdata-all) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + PACKAGES_NETDATA_SENSORS=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + netdata) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + python | netdata-python) + PACKAGES_NETDATA_PYTHON=1 + ;; + + python3 | netdata-python3) + PACKAGES_NETDATA_PYTHON3=1 + ;; + + python-mysql | mysql-python | mysqldb | netdata-mysql) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + fi + ;; + + python-postgres | postgres-python | psycopg2 | netdata-postgres) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + fi + ;; + + python-pymongo) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + ;; + + nodejs | netdata-nodejs) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + PACKAGES_NETDATA_DATABASE=1 + ;; + + sensors | netdata-sensors) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_SENSORS=1 + PACKAGES_NETDATA_DATABASE=1 + ;; + + firehol | update-ipsets | firehol-all | fireqos) + PACKAGES_IPRANGE=1 + PACKAGES_FIREHOL=1 + PACKAGES_FIREQOS=1 + PACKAGES_IPRANGE=1 + PACKAGES_UPDATE_IPSETS=1 + ;; + + demo | all) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + PACKAGES_DEBUG=1 + PACKAGES_IPRANGE=1 + PACKAGES_FIREHOL=1 + PACKAGES_FIREQOS=1 + PACKAGES_UPDATE_IPSETS=1 + PACKAGES_NETDATA_DEMO_SITE=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + help | -h | --help) + usage + exit 1 + ;; + + *) + echo >&2 "ERROR: Cannot understand option '${1}'" + echo >&2 + usage + exit 1 + ;; + esac + shift +done + +# Check for missing core commands like grep, warn the user to install it and bail out cleanly +if ! command -v grep > /dev/null 2>&1; then + echo >&2 + echo >&2 "ERROR: 'grep' is required for the install to run correctly and was not found on the system." + echo >&2 "Please install grep and run the installer again." + echo >&2 + exit 1 +fi + +if [ -z "${package_installer}" ] || [ -z "${tree}" ]; then + if [ -z "${distribution}" ]; then + # we dont know the distribution + autodetect_distribution || user_picks_distribution + fi + + # When no package installer is detected, try again from distro info if any + if [ -z "${package_installer}" ]; then + detect_package_manager_from_distribution "${distribution}" + fi + + # Validate package manager trees + validate_package_trees +fi + +[ "${detection}" = "/etc/os-release" ] && cat << EOF + +/etc/os-release information: +NAME : ${NAME} +VERSION : ${VERSION} +ID : ${ID} +ID_LIKE : ${ID_LIKE} +VERSION_ID : ${VERSION_ID} +EOF + +cat << EOF + +We detected these: +Distribution : ${distribution} +Version : ${version} +Codename : ${codename} +Package Manager : ${package_installer} +Packages Tree : ${tree} +Detection Method: ${detection} +Default Python v: ${pv} $([ ${pv} -eq 2 ] && [ "${PACKAGES_NETDATA_PYTHON3}" -eq 1 ] && echo "(will install python3 too)") + +EOF + +mapfile -t PACKAGES_TO_INSTALL < <(packages | sort -u) + +if [ ${#PACKAGES_TO_INSTALL[@]} -gt 0 ]; then + echo >&2 + echo >&2 "The following command will be run:" + echo >&2 + DRYRUN=1 + "${package_installer}" "${PACKAGES_TO_INSTALL[@]}" + DRYRUN=0 + echo >&2 + echo >&2 + + if [ "${DONT_WAIT}" -eq 0 ] && [ "${NON_INTERACTIVE}" -eq 0 ]; then + read -r -p "Press ENTER to run it > " || exit 1 + fi + + "${package_installer}" "${PACKAGES_TO_INSTALL[@]}" || install_failed $? + + echo >&2 + echo >&2 "All Done! - Now proceed to the next step." + echo >&2 + +else + echo >&2 + echo >&2 "All required packages are already installed. Now proceed to the next step." + echo >&2 +fi + +remote_log "OK" + +exit 0 diff --git a/packaging/installer/kickstart-static64.sh b/packaging/installer/kickstart-static64.sh new file mode 100755 index 0000000..3c45d9e --- /dev/null +++ b/packaging/installer/kickstart-static64.sh @@ -0,0 +1,368 @@ +#!/usr/bin/env sh + +# SPDX-License-Identifier: GPL-3.0-or-later +# shellcheck disable=SC1117,SC2039,SC2059,SC2086 +# +# Options to run +# --dont-wait do not wait for input +# --non-interactive do not wait for input +# --dont-start-it do not start netdata after install +# --stable-channel Use the stable release channel, rather than the nightly to fetch sources +# --disable-telemetry Opt-out of anonymous telemetry program (DO_NOT_TRACK=1) +# --local-files Use a manually provided tarball for the installation +# --allow-duplicate-install do not bail if we detect a duplicate install +# --reinstall if an existing install would be updated, reinstall instead +# +# Environment options: +# +# NETDATA_TARBALL_BASEURL set the base url for downloading the dist tarball +# +# ---------------------------------------------------------------------------- +# library functions copied from packaging/installer/functions.sh + +setup_terminal() { + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput > /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 +} +setup_terminal || echo > /dev/null + +# ---------------------------------------------------------------------------- +fatal() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 +} + +run_ok() { + printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} \n\n" +} + +run_failed() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} \n\n" +} + +ESCAPED_PRINT_METHOD= +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi +escaped_print() { + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 +} + +progress() { + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " +} + +run_logfile="/dev/null" +run() { + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "${info}" + escaped_print "${@}" + printf " ... " + } >> "${run_logfile}" + + printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "${TPUT_RESET}\n" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code ${ret}\n" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} +} + +fatal() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 +} + +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" +} + +create_tmp_directory() { + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + + mktemp -d -t netdata-kickstart-XXXXXXXXXX +} + +download() { + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" || fatal "Cannot download ${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi +} + +set_tarball_urls() { + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + progress "Not fetching remote tarballs, local override was given" + return + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.gz.run" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_TARBALL_BASEURL/netdata-latest.gz.run" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_TARBALL_BASEURL/sha256sums.txt" + fi +} + +safe_sha256sum() { + # Within the context of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start using non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi +} + +# ---------------------------------------------------------------------------- +umask 022 + +sudo="" +[ -z "${UID}" ] && UID="$(id -u)" +[ "${UID}" -ne "0" ] && sudo="sudo" + +# ---------------------------------------------------------------------------- +if [ "$(uname -m)" != "x86_64" ]; then + fatal "Static binary versions of netdata are available only for 64bit Intel/AMD CPUs (x86_64), but yours is: $(uname -m)." +fi + +if [ "$(uname -s)" != "Linux" ]; then + fatal "Static binary versions of netdata are available only for Linux, but this system is $(uname -s)" +fi + +# ---------------------------------------------------------------------------- +opts= +NETDATA_INSTALLER_OPTIONS="" +NETDATA_UPDATES="--auto-update" +RELEASE_CHANNEL="nightly" +while [ -n "${1}" ]; do + if [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ] || [ "${1}" = "--accept" ]; then + opts="${opts} --accept" + shift 1 + elif [ "${1}" = "--dont-start-it" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--no-updates" ]; then + NETDATA_UPDATES="" + shift 1 + elif [ "${1}" = "--auto-update" ]; then + true # This is the default behaviour, so ignore it. + shift 1 + elif [ "${1}" = "--stable-channel" ]; then + RELEASE_CHANNEL="stable" + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--disable-telemetry" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--local-files" ]; then + NETDATA_UPDATES="" # Disable autoupdates if using pre-downloaded files. + shift 1 + if [ -z "${1}" ]; then + fatal "Option --local-files requires extra information. The desired tarball full filename is needed" + fi + + NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" + shift 1 + if [ -z "${1}" ]; then + fatal "Option --local-files requires a pair of the tarball source and the checksum file" + fi + + NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" + shift 1 + elif [ "${1}" = "--allow-duplicate-install" ]; then + NETDATA_ALLOW_DUPLICATE_INSTALL=1 + shift 1 + elif [ "${1}" = "--reinstall" ]; then + NETDATA_REINSTALL=1 + shift 1 + else + echo >&2 "Unknown option '${1}' or invalid number of arguments. Please check the README for the available arguments of ${0} and try again" + exit 1 + fi +done + +if [ ! "${DO_NOT_TRACK:-0}" -eq 0 ] || [ -n "$DO_NOT_TRACK" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }--disable-telemtry" +fi + +# Netdata Tarball Base URL (defaults to our Google Storage Bucket) +[ -z "$NETDATA_TARBALL_BASEURL" ] && NETDATA_TARBALL_BASEURL=https://storage.googleapis.com/netdata-nightlies + +# --------------------------------------------------------------------------------------------------------------------- +# look for an existing install and try to update that instead if it exists + +ndpath="$(command -v netdata 2>/dev/null)" +if [ -z "$ndpath" ] && [ -x /opt/netdata/bin/netdata ] ; then + ndpath="/opt/netdata/bin/netdata" +fi + +if [ -n "$ndpath" ] ; then + ndprefix="$(dirname "$(dirname "${ndpath}")")" + + if [ "${ndprefix}" = /usr ] ; then + ndprefix="/" + fi + + progress "Found existing install of Netdata under: ${ndprefix}" + + if [ -r "${ndprefix}/etc/netdata/.environment" ] ; then + ndstatic="$(grep IS_NETDATA_STATIC_BINARY "${ndprefix}/etc/netdata/.environment" | cut -d "=" -f 2 | tr -d \")" + if [ -z "${NETDATA_REINSTALL}" ] && [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ] ; then + if [ -x "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" ] ; then + progress "Attempting to update existing install instead of creating a new one" + if run ${sudo} "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" --not-running-from-cron ; then + progress "Updated existing install at ${ndpath}" + exit 0 + else + fatal "Failed to update existing Netdata install" + exit 1 + fi + else + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] || [ "${ndstatic}" = "no" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi + else + if [ "${ndstatic}" = "yes" ] ; then + progress "User requested reinstall instead of update, proceeding." + else + fatal "Existing install is not a static install, please use kickstart.sh instead." + exit 1 + fi + fi + else + progress "Existing install appears to be handled manually or through the system package manager." + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi +fi + +# ---------------------------------------------------------------------------- +TMPDIR=$(create_tmp_directory) +cd "${TMPDIR}" || exit 1 + +if [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + set_tarball_urls "${RELEASE_CHANNEL}" + progress "Downloading static netdata binary: ${NETDATA_TARBALL_URL}" + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${TMPDIR}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${TMPDIR}/netdata-latest.gz.run" +else + progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${TMPDIR}/netdata-latest.gz.run" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${TMPDIR}/sha256sum.txt" +fi + +if ! grep netdata-latest.gz.run "${TMPDIR}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Static binary checksum validation failed. Stopping Netdata Agent installation and leaving binary in ${TMPDIR}\nUsually this is a result of an older copy of the file being cached somewhere upstream and can be resolved by retrying in an hour." +fi + +# ---------------------------------------------------------------------------- +progress "Installing netdata" +run ${sudo} sh "${TMPDIR}/netdata-latest.gz.run" ${opts} -- ${NETDATA_UPDATES} ${NETDATA_INSTALLER_OPTIONS} + +#shellcheck disable=SC2181 +if [ $? -eq 0 ]; then + run ${sudo} rm "${TMPDIR}/netdata-latest.gz.run" + if [ ! "${TMPDIR}" = "/" ] && [ -d "${TMPDIR}" ]; then + run ${sudo} rm -rf "${TMPDIR}" + fi +else + echo >&2 "NOTE: did not remove: ${TMPDIR}/netdata-latest.gz.run" +fi diff --git a/packaging/installer/kickstart.sh b/packaging/installer/kickstart.sh new file mode 100755 index 0000000..c97587c --- /dev/null +++ b/packaging/installer/kickstart.sh @@ -0,0 +1,491 @@ +#!/usr/bin/env sh +# SPDX-License-Identifier: GPL-3.0-or-later +# +# Run me with: +# +# bash <(curl -Ss https://my-netdata.io/kickstart.sh) +# +# or (to install all netdata dependencies): +# +# bash <(curl -Ss https://my-netdata.io/kickstart.sh) all +# +# Other options: +# --dont-wait do not prompt for user input +# --non-interactive do not prompt for user input +# --no-updates do not install script for daily updates +# --local-files set the full path of the desired tarball to run install with +# --allow-duplicate-install do not bail if we detect a duplicate install +# --reinstall if an existing install would be updated, reinstall instead +# +# Environment options: +# +# TMPDIR specify where to save temporary files +# NETDATA_TARBALL_BASEURL set the base url for downloading the dist tarball +# +# This script will: +# +# 1. install all netdata compilation dependencies +# using the package manager of the system +# +# 2. download netdata nightly package to temporary directory +# +# 3. install netdata + +# shellcheck disable=SC2039,SC2059,SC2086 + +# External files +PACKAGES_SCRIPT="https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh" + +# Netdata Tarball Base URL (defaults to our Google Storage Bucket) +[ -z "$NETDATA_TARBALL_BASEURL" ] && NETDATA_TARBALL_BASEURL=https://storage.googleapis.com/netdata-nightlies + +# --------------------------------------------------------------------------------------------------------------------- +# library functions copied from packaging/installer/functions.sh + +setup_terminal() { + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput > /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 +} +setup_terminal || echo > /dev/null + +# ----------------------------------------------------------------------------- +fatal() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 +} + +run_ok() { + printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} \n\n" +} + +run_failed() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} \n\n" +} + +ESCAPED_PRINT_METHOD= +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi +escaped_print() { + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 +} + +progress() { + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " +} + +run_logfile="/dev/null" +run() { + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "${info}" + escaped_print "${@}" + printf " ... " + } >> "${run_logfile}" + + printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "${TPUT_RESET}" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code ${ret}\n" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} +} + +fatal() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 +} + +warning() { + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} WARNING ${TPUT_RESET} ${*} \n\n" + if [ "${INTERACTIVE}" = "0" ]; then + fatal "Stopping due to non-interactive mode. Fix the issue or retry installation in an interactive mode." + else + read -r -p "Press ENTER to attempt netdata installation > " + progress "OK, let's give it a try..." + fi +} + +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" +} + +create_tmp_directory() { + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + + mktemp -d -t netdata-kickstart-XXXXXXXXXX +} + +download() { + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" || fatal "Cannot download ${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi +} + +set_tarball_urls() { + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + progress "Not fetching remote tarballs, local override was given" + return + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.tar.gz" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_TARBALL_BASEURL/netdata-latest.tar.gz" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_TARBALL_BASEURL/sha256sums.txt" + fi +} + +detect_bash4() { + bash="${1}" + if [ -z "${BASH_VERSION}" ]; then + # we don't run under bash + if [ -n "${bash}" ] && [ -x "${bash}" ]; then + # shellcheck disable=SC2016 + BASH_MAJOR_VERSION=$(${bash} -c 'echo "${BASH_VERSINFO[0]}"') + fi + else + # we run under bash + BASH_MAJOR_VERSION="${BASH_VERSINFO[0]}" + fi + + if [ -z "${BASH_MAJOR_VERSION}" ]; then + echo >&2 "No BASH is available on this system" + return 1 + elif [ $((BASH_MAJOR_VERSION)) -lt 4 ]; then + echo >&2 "No BASH v4+ is available on this system (installed bash is v${BASH_MAJOR_VERSION}" + return 1 + fi + return 0 +} + +dependencies() { + SYSTEM="$(uname -s 2> /dev/null || uname -v)" + OS="$(uname -o 2> /dev/null || uname -rs)" + MACHINE="$(uname -m 2> /dev/null)" + + echo "System : ${SYSTEM}" + echo "Operating System : ${OS}" + echo "Machine : ${MACHINE}" + echo "BASH major version: ${BASH_MAJOR_VERSION}" + + if [ "${OS}" != "GNU/Linux" ] && [ "${SYSTEM}" != "Linux" ]; then + warning "Cannot detect the packages to be installed on a ${SYSTEM} - ${OS} system." + else + bash="$(command -v bash 2> /dev/null)" + if ! detect_bash4 "${bash}"; then + warning "Cannot detect packages to be installed in this system, without BASH v4+." + else + progress "Fetching script to detect required packages..." + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then + if [ -f "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" "${ndtmpdir}/install-required-packages.sh" + else + fatal "Invalid given dependency file, please check your --local-files parameter options and try again" + fi + else + download "${PACKAGES_SCRIPT}" "${ndtmpdir}/install-required-packages.sh" + fi + + if [ ! -s "${ndtmpdir}/install-required-packages.sh" ]; then + warning "Downloaded dependency installation script is empty." + else + progress "Running downloaded script to detect required packages..." + run ${sudo} "${bash}" "${ndtmpdir}/install-required-packages.sh" ${PACKAGES_INSTALLER_OPTIONS} + # shellcheck disable=SC2181 + if [ $? -ne 0 ]; then + warning "It failed to install all the required packages, but installation might still be possible." + fi + fi + + fi + fi +} + +safe_sha256sum() { + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi +} + +# --------------------------------------------------------------------------------------------------------------------- +umask 022 + +sudo="" +[ -z "${UID}" ] && UID="$(id -u)" +[ "${UID}" -ne "0" ] && sudo="sudo" +export PATH="${PATH}:/usr/local/bin:/usr/local/sbin" + + +# --------------------------------------------------------------------------------------------------------------------- + +INTERACTIVE=1 +PACKAGES_INSTALLER_OPTIONS="netdata" +NETDATA_INSTALLER_OPTIONS="" +NETDATA_UPDATES="--auto-update" +RELEASE_CHANNEL="nightly" +while [ -n "${1}" ]; do + if [ "${1}" = "all" ]; then + PACKAGES_INSTALLER_OPTIONS="netdata-all" + shift 1 + elif [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ]; then + INTERACTIVE=0 + shift 1 + elif [ "${1}" = "--no-updates" ]; then + # echo >&2 "netdata will not auto-update" + NETDATA_UPDATES= + shift 1 + elif [ "${1}" = "--stable-channel" ]; then + RELEASE_CHANNEL="stable" + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --stable-channel" + shift 1 + elif [ "${1}" = "--allow-duplicate-install" ]; then + NETDATA_ALLOW_DUPLICATE_INSTALL=1 + shift 1 + elif [ "${1}" = "--reinstall" ]; then + NETDATA_REINSTALL=1 + shift 1 + elif [ "${1}" = "--local-files" ]; then + shift 1 + if [ -z "${1}" ]; then + fatal "Missing netdata: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing checksum file: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing go.d tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing go.d config tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN_CONFIG="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing dependencies management scriptlet: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT="${1}" + shift 1 + else + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS ${1}" + shift 1 + fi +done + +if [ "${INTERACTIVE}" = "0" ]; then + PACKAGES_INSTALLER_OPTIONS="--dont-wait --non-interactive ${PACKAGES_INSTALLER_OPTIONS}" + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --dont-wait" +fi + +# --------------------------------------------------------------------------------------------------------------------- +# look for an existing install and try to update that instead if it exists + +ndpath="$(command -v netdata 2>/dev/null)" +if [ -z "$ndpath" ] && [ -x /opt/netdata/bin/netdata ] ; then + ndpath="/opt/netdata/bin/netdata" +fi + +if [ -n "$ndpath" ] ; then + ndprefix="$(dirname "$(dirname "${ndpath}")")" + + if [ "${ndprefix}" = /usr ] ; then + ndprefix="/" + fi + + progress "Found existing install of Netdata under: ${ndprefix}" + + if [ -r "${ndprefix}/etc/netdata/.environment" ] ; then + ndstatic="$(grep IS_NETDATA_STATIC_BINARY "${ndprefix}/etc/netdata/.environment" | cut -d "=" -f 2 | tr -d \")" + if [ -z "${NETDATA_REINSTALL}" ] && [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ] ; then + if [ -x "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" ] ; then + progress "Attempting to update existing install instead of creating a new one" + if run ${sudo} "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" --not-running-from-cron ; then + progress "Updated existing install at ${ndpath}" + exit 0 + else + fatal "Failed to update existing Netdata install" + exit 1 + fi + else + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] || [ "${ndstatic}" = "yes" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi + else + if [ "${ndstatic}" = "no" ] ; then + progress "User requested reinstall instead of update, proceeding." + else + fatal "Existing install is a static install, please use kickstart-static64.sh instead." + exit 1 + fi + fi + else + progress "Existing install appears to be handled manually or through the system package manager." + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi +fi + +# --------------------------------------------------------------------------------------------------------------------- +# install required system packages + +ndtmpdir=$(create_tmp_directory) +cd "${ndtmpdir}" || exit 1 + +dependencies + +# --------------------------------------------------------------------------------------------------------------------- +# download netdata package + +if [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + set_tarball_urls "${RELEASE_CHANNEL}" + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.tar.gz" +else + progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${ndtmpdir}/sha256sum.txt" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${ndtmpdir}/netdata-latest.tar.gz" +fi + +if ! grep netdata-latest.tar.gz "${ndtmpdir}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Tarball checksum validation failed. Stopping Netdata Agent installation and leaving tarball in ${ndtmpdir}.\nUsually this is a result of an older copy of the file being cached somewhere upstream and can be resolved by retrying in an hour." +fi +run tar -xf netdata-latest.tar.gz +rm -rf netdata-latest.tar.gz > /dev/null 2>&1 +cd netdata-* || fatal "Cannot cd to netdata source tree" + +# --------------------------------------------------------------------------------------------------------------------- +# install netdata from source + +install() { + progress "Installing netdata..." + run ${sudo} ./netdata-installer.sh ${NETDATA_UPDATES} ${NETDATA_INSTALLER_OPTIONS} || fatal "netdata-installer.sh exited with error" + if [ -d "${ndtmpdir}" ] && [ ! "${ndtmpdir}" = "/" ]; then + run ${sudo} rm -rf "${ndtmpdir}" > /dev/null 2>&1 + fi +} + +if [ -x netdata-installer.sh ]; then + install "$@" +else + if [ "$(find . -mindepth 1 -maxdepth 1 -type d | wc -l)" -eq 1 ] && [ -x "$(find . -mindepth 1 -maxdepth 1 -type d)/netdata-installer.sh" ]; then + cd "$(find . -mindepth 1 -maxdepth 1 -type d)" && install "$@" + else + fatal "Cannot install netdata from source (the source directory does not include netdata-installer.sh). Leaving all files in ${ndtmpdir}" + fi +fi diff --git a/packaging/installer/methods/alpine.md b/packaging/installer/methods/alpine.md new file mode 100644 index 0000000..fb44895 --- /dev/null +++ b/packaging/installer/methods/alpine.md @@ -0,0 +1,36 @@ +<!-- +title: "Install Netdata on Alpine 3.x" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/alpine.md +--> + +# Install Netdata on Alpine 3.x + +Execute these commands to install Netdata in Alpine Linux 3.x: + +```sh +# install required packages +apk add alpine-sdk bash curl libuv-dev zlib-dev util-linux-dev libmnl-dev gcc make git autoconf automake pkgconfig python logrotate + +# if you plan to run node.js Netdata plugins +apk add nodejs + +# download Netdata - the directory 'netdata' will be created +git clone https://github.com/netdata/netdata.git --depth=100 +cd netdata + +# build it, install it, start it +./netdata-installer.sh + +# make Netdata start at boot +echo -e "#!/usr/bin/env bash\n/usr/sbin/netdata" >/etc/local.d/netdata.start +chmod 755 /etc/local.d/netdata.start + +# make Netdata stop at shutdown +echo -e "#!/usr/bin/env bash\nkillall netdata" >/etc/local.d/netdata.stop +chmod 755 /etc/local.d/netdata.stop + +# enable the local service to start automatically +rc-update add local +``` + +[](<>) diff --git a/packaging/installer/methods/cloud-providers.md b/packaging/installer/methods/cloud-providers.md new file mode 100644 index 0000000..943a649 --- /dev/null +++ b/packaging/installer/methods/cloud-providers.md @@ -0,0 +1,130 @@ +<!-- +title: "Install Netdata on cloud providers" +description: "The Netdata Agent runs on all popular cloud providers, but often requires additional steps and configuration for full functionality." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/cloud-providers.md +--> + +# Install Netdata on cloud providers + +Netdata is fully compatible with popular cloud providers like Google Cloud Platform (GCP), Amazon Web Services (AWS), +Azure, and others. You can install Netdata on cloud instances to monitor the apps/services running there, or use +multiple instances in a [parent-child streaming](/streaming/README.md) configuration. + +In some cases, using Netdata on these cloud providers requires unique installation or configuration steps. This page +aims to document some of those steps for popular cloud providers. + +> This document is a work-in-progress! If you find new issues specific to a cloud provider, or would like to help +> clarify the correct workaround, please [create an +> issue](https://github.com/netdata/netdata/issues/new?labels=feature+request,+needs+triage&template=feature_request.md) +> with your process and instructions on using the provider's interface to complete the workaround. + +- [Recommended installation methods for cloud providers](#recommended-installation-methods-for-cloud-providers) +- [Post-installation configuration](#post-installation-configuration) + - [Add a firewall rule to access Netdata's dashboard](#add-a-firewall-rule-to-access-netdatas-dashboard) + +## Recommended installation methods for cloud providers + +The best installation method depends on the instance's operating system, distribution, and version. For Linux instances, +we recommend either the [`kickstart.sh` automatic installation script](kickstart.md) or [.deb/.rpm +packages](packages.md). + +To see the full list of approved methods for each operating system/version we support, see our [distribution +matrix](../../DISTRIBUTIONS.md). That table will guide you to the various supported methods for your cloud instance. + +If you have issues with Netdata after installation, look to the sections below to find the issue you're experiencing, +followed by the solution for your provider. + +## Post-installation configuration + +Some cloud providers require you take additional steps to properly configure your instance or its networking to access +all of Netdata's features. + +### Add a firewall rule to access Netdata's dashboard + +If you cannot access Netdata's dashboard on your cloud instance via `http://HOST:19999`, and instead get an error page +from your browser that says, "This site can't be reached" (Chrome) or "Unable to connect" (Firefox), you may need to +configure your cloud provider's firewall. + +Cloud providers often create network-level firewalls that run separately from the instance itself. Both AWS and Google +Cloud Platform calls them Virtual Private Cloud (VPC) networks. These firewalls can apply even if you've disabled +firewalls on the instance itself. Because you can modify these firewalls only via the cloud provider's web interface, +it's easy to overlook them when trying to configure and access Netdata's dashboard. + +You can often confirm a firewall issue by querying the dashboard while connected to the instance via SSH: `curl +http://localhost:19999/api/v1/info`. If you see JSON output, Netdata is running properly. If you try the same `curl` +command from a remote system, and it fails, it's likely that a firewall is blocking your requests. + +Another option is to put Netdata behind web server, which will proxy requests through standard HTTP/HTTPS ports +(80/443), which are likely already open on your instance. We have a number of guides available: + +- [Apache](/docs/Running-behind-apache.md) +- [Nginx](/docs/Running-behind-nginx.md) +- [Caddy](/docs/Running-behind-caddy.md) +- [HAProxy](/docs/Running-behind-haproxy.md) +- [lighttpd](/docs/Running-behind-lighttpd.md) + +The next few sections outline how to add firewall rules to GCP, AWS, and Azure instances. + +#### Google Cloud Platform (GCP) + +To add a firewall rule, go to the [Firewall rules page](https://console.cloud.google.com/networking/firewalls/list) and +click **Create firewall rule**. + +The following configuration has previously worked for Netdata running on GCP instances +([see #7786](https://github.com/netdata/netdata/issues/7786)): + +```conf +Name: <name> +Type: Ingress +Targets: <name-tag> +Filters: 0.0.0.0/0 +Protocols/ports: 19999 +Action: allow +Priority: 1000 +``` + +Read GCP's [firewall documentation](https://cloud.google.com/vpc/docs/using-firewalls) for specific instructions on how +to create a new firewall rule. + +#### Amazon Web Services (AWS) / EC2 + +Sign in to the [AWS console](https://console.aws.amazon.com/) and navigate to the EC2 dashboard. Click on the **Security +Groups** link in the navigation, beneath the **Network & Security** heading. Find the Security Group your instance +belongs to, and either right-click on it or click the **Actions** button above to see a dropdown menu with **Edit +inbound rules**. + +Add a new rule with the following options: + +```conf +Type: Custom TCP +Protocol: TCP +Port Range: 19999 +Source: Anywhere +Description: Netdata +``` + +You can also choose **My IP** as the source if you prefer. + +Click **Save** to apply your new inbound firewall rule. + +#### Azure + +Sign in to the [Azure portal](https://portal.azure.com) and open the virtual machine running Netdata. Click on the +**Networking** link beneath the **Settings** header, then click on the **Add inbound security rule** button. + +Add a new rule with the following options: + +```conf +Source: Any +Source port ranges: 19999 +Destination: Any +Destination port randes: 19999 +Protocol: TCP +Action: Allow +Priority: 310 +Name: Netdata +``` + +Click **Add** to apply your new inbound security rule. + +[](<>) diff --git a/packaging/installer/methods/freebsd.md b/packaging/installer/methods/freebsd.md new file mode 100644 index 0000000..e2af417 --- /dev/null +++ b/packaging/installer/methods/freebsd.md @@ -0,0 +1,108 @@ +<!-- +title: "Install Netdata on FreeBSD" +description: "Install Netdata on FreeBSD to monitor the health and performance of bare metal or VMs with thousands of real-time, per-second metrics." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freebsd.md +--> + +# Install Netdata on FreeBSD + +> 💡 This document is maintained by Netdata's community, and may not be completely up-to-date. Please double-check the +> details of the installation process, such as version numbers for downloadable packages, before proceeding. +> +> You can help improve this document by [submitting a +> PR](https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freebsd.md) with your recommended +> improvements or changes. Thank you! + +## Install latest version + +This is how to install the latest Netdata version on FreeBSD: + +Install required packages (**need root permission**): + +```sh +pkg install bash e2fsprogs-libuuid git curl autoconf automake pkgconf pidof Judy liblz4 libuv json-c cmake gmake +``` + +Download Netdata: + +```sh +fetch https://github.com/netdata/netdata/releases/download/v1.26.0/netdata-v1.26.0.tar.gz +``` + +> ⚠️ Verify the latest version by either navigating to [Netdata's latest +> release](https://github.com/netdata/netdata/releases/latest) or using `curl`: +> +> ```bash +> basename $(curl -Ls -o /dev/null -w %{url_effective} https://github.com/netdata/netdata/releases/latest) +> ``` + +Unzip the downloaded file: + +```sh +gunzip netdata*.tar.gz && tar xf netdata*.tar && rm -rf netdata*.tar +``` + +Install Netdata in `/opt/netdata`. If you want to enable automatic updates, add `--auto-update` or `-u` to install `netdata-updater` in `cron` (**need root permission**): + +```sh +cd netdata-v* && ./netdata-installer.sh --install /opt && cp /opt/netdata/usr/sbin/netdata-claim.sh /usr/sbin/ +``` + +You also need to enable the `netdata` service in `/etc/rc.conf`: + +```sh +sysrc netdata_enable="YES" +``` + +Finally, and very importantly, update Netdata using the script provided by the Netdata team (**need root permission**): + +```sh +cd /opt/netdata/usr/libexec/netdata/ && ./netdata-updater.sh +``` + +You can now access the Netdata dashboard by navigating to `http://NODE:19999`, replacing `NODE` with the IP address or hostname of your system. + + + +From Netdata v1.12 and above, anonymous usage information is collected by default and sent to Google Analytics. To read +more about the information collected and how to opt-out, check the [anonymous statistics +page](/docs/anonymous-statistics.md). + +## Updating the Agent on FreeBSD +If you have not passed the `--auto-update` or `-u` parameter for the installer to enable automatic updating, repeat the last step to update Netdata whenever a new version becomes available. +The `netdata-updater.sh` script will update your Agent. + +## Optional parameters to alter your installation +| parameters | Description | +|:-----:|-----------| +|`--install <path>`| Install netdata in `<path>.` Ex: `--install /opt` will put netdata in `/opt/netdata`| +| `--dont-start-it` | Do not (re)start netdata after installation| +| `--dont-wait` | Run installation in non-interactive mode| +| `--auto-update` or `-u` | Install netdata-updater in cron to update netdata automatically once per day| +| `--stable-channel` | Use packages from GitHub release pages instead of GCS (nightly updates). This results in less frequent updates| +| `--nightly-channel` | Use most recent nightly updates instead of GitHub releases. This results in more frequent updates| +| `--disable-go` | Disable installation of go.d.plugin| +| `--disable-ebpf` | Disable eBPF Kernel plugin (Default: enabled)| +| `--disable-cloud` | Disable all Netdata Cloud functionality| +| `--require-cloud` | Fail the install if it can't build Netdata Cloud support| +| `--enable-plugin-freeipmi` | Enable the FreeIPMI plugin. Default: enable it when libipmimonitoring is available| +| `--disable-plugin-freeipmi` | Enable the FreeIPMI plugin| +| `--disable-https` | Explicitly disable TLS support| +| `--disable-dbengine` | Explicitly disable DB engine support| +| `--enable-plugin-nfacct` | Enable nfacct plugin. Default: enable it when libmnl and libnetfilter_acct are available| +| `--disable-plugin-nfacct` | Disable nfacct plugin. Default: enable it when libmnl and libnetfilter_acct are available| +| `--enable-plugin-xenstat` | Enable the xenstat plugin. Default: enable it when libxenstat and libyajl are available| +| `--disable-plugin-xenstat` | Disable the xenstat plugin| +| `--enable-backend-kinesis` | Enable AWS Kinesis backend. Default: enable it when libaws_cpp_sdk_kinesis and libraries (it depends on are available)| +| `--disable-backend-kinesis` | Disable AWS Kinesis backend. Default: enable it when libaws_cpp_sdk_kinesis and libraries (it depends on are available)| +| `--enable-backend-prometheus-remote-write` | Enable Prometheus remote write backend. Default: enable it when libprotobuf and libsnappy are available| +| `--disable-backend-prometheus-remote-write` | Disable Prometheus remote write backend. Default: enable it when libprotobuf and libsnappy are available| +| `--enable-backend-mongodb` | Enable MongoDB backend. Default: enable it when libmongoc is available| +| `--disable-backend-mongodb` | Disable MongoDB backend| +| `--enable-lto` | Enable Link-Time-Optimization. Default: enabled| +| `--disable-lto` | Disable Link-Time-Optimization. Default: enabled| +| `--disable-x86-sse` | Disable SSE instructions. By default SSE optimizations are enabled| +| `--zlib-is-really-here` or `--libs-are-really-here` | If you get errors about missing zlib or libuuid but you know it is available, you might have a broken pkg-config. Use this option to proceed without checking pkg-config| +|`--disable-telemetry` | Use this flag to opt-out from our anonymous telemetry program. (DO_NOT_TRACK=1)| + +[](<>) diff --git a/packaging/installer/methods/freenas.md b/packaging/installer/methods/freenas.md new file mode 100644 index 0000000..a0dafdf --- /dev/null +++ b/packaging/installer/methods/freenas.md @@ -0,0 +1,24 @@ +<!-- +title: "Install Netdata on FreeNAS" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freenas.md +--> + +# Install Netdata on FreeNAS + +On FreeNAS-Corral-RELEASE (>=10.0.3 and <11.3), Netdata is pre-installed. + +To use Netdata, the service will need to be enabled and started from the FreeNAS [CLI](https://github.com/freenas/cli). + +To enable the Netdata service: + +```sh +service netdata config set enable=true +``` + +To start the Netdata service: + +```sh +service netdata start +``` + +[](<>) diff --git a/packaging/installer/methods/kickstart-64.md b/packaging/installer/methods/kickstart-64.md new file mode 100644 index 0000000..120cc9e --- /dev/null +++ b/packaging/installer/methods/kickstart-64.md @@ -0,0 +1,96 @@ +<!-- +title: "Install Netdata with kickstart-static64.sh" +description: "The kickstart-static64.sh script installs a pre-compiled static binary, including all dependencies required to connect to Netdata Cloud, with one command." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kickstart-64.md +--> + +# Install Netdata with kickstart-static64.sh + +  + +This page covers detailed instructions on using and configuring the installation script named `kickstart-static64.sh`. + +This method uses a pre-compiled static binary to install Netdata on any Intel/AMD 64bit Linux system and on any Linux +distribution, even those with a broken or unsupported package manager. + +To install Netdata from a static binary package, including all dependencies required to connect to Netdata Cloud, and +get _automatic nightly updates_, run the following as your normal user: + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) +``` + +> This script installs Netdata at `/opt/netdata`. + +> See our [installation guide](/packaging/installer/README.md) for details about [automatic +> updates](/packaging/installer/README.md#automatic-updates) or [nightly vs. stable +> releases](/packaging/installer/README.md#nightly-vs-stable-releases). + +## What does `kickstart-static64.sh` do? + +The `kickstart.sh` script does the following after being downloaded and run: + +- Checks to see if there is an existing installation, and if there is updates that in preference to reinstalling. +- Downloads the latest Netdata binary from the [binary-packages](https://github.com/netdata/binary-packages) + repository. You can also run any of these `.run` files with [makeself](https://github.com/megastep/makeself). +- Installs Netdata by running `./netdata-installer.sh` from the source tree, including any options you might have + added. +- Installs `netdata-updater.sh` to `cron.daily` to enable automatic updates, unless you added the `--no-updates` + option. +- Prints a message about whether the installation succeeded for failed for QA purposes. + +If your shell fails to handle the above one-liner, you can download and run the `kickstart-static64.sh` script manually. + +```sh +# download the script with curl +curl https://my-netdata.io/kickstart-static64.sh >/tmp/kickstart-static64.sh + +# or, download the script with wget +wget -O /tmp/kickstart-static64.sh https://my-netdata.io/kickstart-static64.sh + +# run the downloaded script (any sh is fine, no need for bash) +sh /tmp/kickstart-static64.sh +``` + +## Optional parameters to alter your installation + +The `kickstart-static64.sh` script passes all its parameters to `netdata-installer.sh`, which you can use to customize +your installation. Here are a few important parameters: + +- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. +- `--dont-start-it`: Prevent the installer from starting Netdata automatically. +- `--stable-channel`: Automatically update only on the release of new major versions. +- `--nightly-channel`: Automatically update on every new nightly build. +- `--disable-telemetry`: Opt-out of [anonymous statistics](/docs/anonymous-statistics.md) we use to make + Netdata better. +- `--no-updates`: Prevent automatic updates of any kind. +- `--reinstall`: If an existing installation is detected, reinstall instead of attempting to update it. Note + that this cannot be used to switch between installation types. +- `--local-files`: Used for [offline installations](/packaging/installer/methods/offline.md). Pass four file paths: + the Netdata tarball, the checksum file, the go.d plugin tarball, and the go.d plugin config tarball, to force + kickstart run the process using those files. This option conflicts with the `--stable-channel` option. If you set + this _and_ `--stable-channel`, Netdata will use the local files. + +## Verify script integrity + +To use `md5sum` to verify the integrity of the `kickstart-static64.sh` script you will download using the one-line +command above, run the following: + +```bash +[ "047c86a7c8905955bee39b6980a28e30" = "$(curl -Ss https://my-netdata.io/kickstart-static64.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" +``` + +If the script is valid, this command will return `OK, VALID`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/kickstart.md b/packaging/installer/methods/kickstart.md new file mode 100644 index 0000000..f825f80 --- /dev/null +++ b/packaging/installer/methods/kickstart.md @@ -0,0 +1,79 @@ +<!-- +title: "Install Netdata with kickstart.sh" +description: "The kickstart.sh script installs Netdata from source, including all dependencies required to connect to Netdata Cloud, with a single command." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kickstart.md +--> + +# Install Netdata with kickstart.sh + +  + +This page covers detailed instructions on using and configuring the automatic one-line installation script named +`kickstart.sh`. + +This method is fully automatic on all Linux distributions. To install Netdata from source, including all dependencies +required to connect to Netdata Cloud, and get _automatic nightly updates_, run the following as your normal user: + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) +``` + +> See our [installation guide](../README.md) for details about [automatic updates](../README.md#automatic-updates) or +> [nightly vs. stable releases](../README.md#nightly-vs-stable-releases). + +## What does `kickstart.sh` do? + +The `kickstart.sh` script does the following after being downloaded and run using `bash`: + +- Detects the Linux distribution and **installs the required system packages** for building Netdata. Unless you added + the `--dont-wait` option, it will ask for your permission first. +- Checks for an existing installation, and if found updates that instead of creating a new install. +- Downloads the latest Netdata source tree to `/usr/src/netdata.git`. +- Installs Netdata by running `./netdata-installer.sh` from the source tree, using any [optional + parameters](#optional-parameters-to-alter-your-installation) you have specified. +- Installs `netdata-updater.sh` to `cron.daily`, so your Netdata installation will be updated with new nightly + versions, unless you override that with an [optional parameter](#optional-parameters-to-alter-your-installation). +- Prints a message whether installation succeeded or failed for QA purposes. + +## Optional parameters to alter your installation + +The `kickstart.sh` script passes all its parameters to `netdata-installer.sh`, which you can use to customize your +installation. Here are a few important parameters: + +- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. +- `--dont-start-it`: Prevent the installer from starting Netdata automatically. +- `--stable-channel`: Automatically update only on the release of new major versions. +- `--nightly-channel`: Automatically update on every new nightly build. +- `--disable-telemetry`: Opt-out of [anonymous statistics](/docs/anonymous-statistics.md) we use to make + Netdata better. +- `--no-updates`: Prevent automatic updates of any kind. +- `--reinstall`: If an existing install is detected, reinstall instead of trying to update it. Note that this + cannot be used to change installation types. +- `--local-files`: Used for [offline installations](offline.md). Pass four file paths: the Netdata + tarball, the checksum file, the go.d plugin tarball, and the go.d plugin config tarball, to force kickstart run the + process using those files. This option conflicts with the `--stable-channel` option. If you set this _and_ + `--stable-channel`, Netdata will use the local files. + +## Verify script integrity + +To use `md5sum` to verify the integrity of the `kickstart.sh` script you will download using the one-line command above, +run the following: + +```bash +[ "8df7a45b2abb336c84507b7c107bcba3" = "$(curl -Ss https://my-netdata.io/kickstart.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" +``` + +If the script is valid, this command will return `OK, VALID`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/kubernetes.md b/packaging/installer/methods/kubernetes.md new file mode 100644 index 0000000..3e85928 --- /dev/null +++ b/packaging/installer/methods/kubernetes.md @@ -0,0 +1,209 @@ +<!-- +title: "Install Netdata on a Kubernetes cluster" +description: "Use Netdata's Helm chart to bootstrap a Netdata monitoring and troubleshooting toolkit on your Kubernetes (k8s) cluster." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kubernetes.md +--> + +# Install Netdata on a Kubernetes cluster + +This document details how to install Netdata on an existing Kubernetes (k8s) cluster. By following these directions, you +will use Netdata's [Helm chart](https://github.com/netdata/helmchart) to bootstrap a Netdata deployment on your cluster. +The Helm chart installs one parent pod for storing metrics and managing alarm notifications plus an additional child pod +for every node in the cluster. + +Each child pod will collect metrics from the node it runs on, in addition to [compatible +applications](https://github.com/netdata/helmchart#service-discovery-and-supported-services), plus any endpoints covered +by our [generic Prometheus collector](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus), +via [service discovery](https://github.com/netdata/agent-service-discovery/). Each child pod will also collect +[cgroups](/collectors/cgroups.plugin/README.md), +[Kubelet](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubelet), and +[kube-proxy](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubeproxy) metrics from its node. + +To install Netdata on a Kubernetes cluster, you need: + +- A working cluster running Kubernetes v1.9 or newer. +- The [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) command line tool, within [one minor version + difference](https://kubernetes.io/docs/tasks/tools/install-kubectl/#before-you-begin) of your cluster, on an + administrative system. +- The [Helm package manager](https://helm.sh/) v3.0.0 or newer on the same administrative system. + +The default configuration creates one `parent` pod, installed on one of your cluster's nodes, and a DaemonSet for +additional `child` pods. This DaemonSet ensures that every node in your k8s cluster also runs a `child` pod, including +the node that also runs `parent`. The `child` pods collect metrics and stream the information to the `parent` pod, which +uses two persistent volumes to store metrics and alarms. The `parent` pod also handles alarm notifications and enables +the Netdata dashboard using an ingress controller. + +## Install the Netdata Helm chart + +We recommend you install the Helm chart using our Helm repository. In the `helm install` command, replace `netdata` with +the release name of your choice. + +```bash +helm repo add netdata https://netdata.github.io/helmchart/ +helm install netdata netdata/netdata +``` + +> You can also install the Netdata Helm chart by cloning the +> [repository](https://artifacthub.io/packages/helm/netdata/netdata#install-by-cloning-the-repository) and manually +> running Helm against the included chart. + +### Post-installation + +Run `kubectl get services` and `kubectl get pods` to confirm that your cluster now runs a `netdata` service, one +`parent` pod, and three `child` pods. + +You've now installed Netdata on your Kubernetes cluster. See how to [access the Netdata +dashboard](#access-the-netdata-dashboard) to confirm it's working as expected, or see the next section to [configure the +Helm chart](#configure-the-netdata-helm-chart) to suit your cluster's particular setup. + +## Configure the Netdata Helm chart + +Read up on the various configuration options in the [Helm chart +documentation](https://github.com/netdata/helmchart#configuration) to see if you need to change any of the options based +on your cluster's setup. + +To change a setting, use the `--set` or `--values` arguments with `helm install`, for the initial deployment, or `helm upgrade` to upgrade an existing deployment. + +```bash +helm install --set a.b.c=xyz netdata netdata/netdata +helm upgrade --set a.b.c=xyz netdata netdata/netdata +``` + +For example, to change the size of the persistent metrics volume on the parent node: + +```bash +helm install --set parent.database.volumesize=4Gi netdata netdata/netdata +helm upgrade --set parent.database.volumesize=4Gi netdata netdata/netdata +``` + +### Configure service discovery + +As mentioned in the introduction, Netdata has a [service discovery +plugin](https://github.com/netdata/agent-service-discovery/#service-discovery) to identify compatible pods and collect +metrics from the service they run. The Netdata Helm chart installs this service discovery plugin into your k8s cluster. + +Service discovery scans your cluster for pods exposed on certain ports and with certain image names. By default, it +looks for its supported services on the ports they most commonly listen on, and using default image names. Service +discovery currently supports [popular +applications](https://github.com/netdata/helmchart#service-discovery-and-supported-services), plus any endpoints covered +by our [generic Prometheus collector](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus). + +If you haven't changed listening ports, image names, or other defaults, service discovery should find your pods, create +the proper configurations based on the service that pod runs, and begin monitoring them immediately after deployment. + +However, if you have changed some of these defaults, you need to copy a file from the Netdata Helm chart repository, +make your edits, and pass the changed file to `helm install`/`helm upgrade`. + +First, copy the file to your administrative system. + +```bash +curl https://raw.githubusercontent.com/netdata/helmchart/master/charts/netdata/sdconfig/child.yml -o child.yml +``` + +Edit the new `child.yml` file according to your needs. See the [Helm chart +configuration](https://github.com/netdata/helmchart#configuration) and the file itself for details. + +You can then run `helm install`/`helm upgrade` with the `--set-file` argument to use your configured `child.yml` file +instead of the default, changing the path if you copied it elsewhere. + +```bash +helm install --set-file sd.child.configmap.from.value=./child.yml netdata netdata/netdata +helm upgrade --set-file sd.child.configmap.from.value=./child.yml netdata netdata/netdata +``` + +Your configured service discovery is now pushed to your cluster. + +## Access the Netdata dashboard + +Accessing the Netdata dashboard itself depends on how you set up your k8s cluster and the Netdata Helm chart. If you +installed the Helm chart with the default `service.type=ClusterIP`, you will need to forward a port to the parent pod. + +```bash +kubectl port-forward netdata-parent-0 19999:19999 +``` + +You can now access the dashboard at `http://CLUSTER:19999`, replacing `CLUSTER` with the IP address or hostname of your +k8s cluster. + +If you set up the Netdata Helm chart with `service.type=LoadBalancer`, you can find the external IP for the load +balancer with `kubectl get services`, under the `EXTERNAL-IP` column. + +```bash +kubectl get services +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +cockroachdb ClusterIP None <none> 26257/TCP,8080/TCP 46h +cockroachdb-public ClusterIP 10.245.148.233 <none> 26257/TCP,8080/TCP 46h +kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 47h +netdata LoadBalancer 10.245.160.131 203.0.113.0 19999:32231/TCP 74m +``` + +In the above example, access the dashboard by navigating to `http://203.0.113.0:19999`. + +## Claim a Kubernetes cluster's parent pod + +You can [claim](/claim/README.md) a cluster's parent Netdata pod to see its real-time metrics alongside any other nodes +you monitor using [Netdata Cloud](https://app.netdata.cloud). + +> Netdata Cloud does not currently support claiming child nodes because the Helm chart does not allocate a persistent +> volume for them. + +Ensure persistence is enabled on the parent pod by running the following `helm upgrade` command. + +```bash +helm upgrade \ + --set parent.database.persistence=true \ + --set parent.alarms.persistence=true \ + netdata netdata/netdata +``` + +Next, find your claiming script in Netdata Cloud by clicking on your Space's dropdown, then **Manage your Space**. Click +the **Nodes** tab. Netdata Cloud shows a script similar to the following: + +```bash +sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud +``` + +You will need the values of `TOKEN` and `ROOM1,ROOM2` for the command, which sets `parent.claiming.enabled`, +`parent.claiming.token`, and `parent.claiming.rooms` to complete the parent pod claiming process. + +Run the following `helm upgrade` command after replacing `TOKEN` and `ROOM1,ROOM2` with the values found in the claiming +script from Netdata Cloud. The quotations are required. + +```bash +helm upgrade \ + --set parent.claiming.enabled=true \ + --set parent.claiming.token="TOKEN" \ + --set parent.claiming.rooms="ROOM1,ROOM2" \ + netdata netdata/netdata +``` + +The cluster terminates the old parent pod and creates a new one with the proper claiming configuration. You can see your +parent pod in Netdata Cloud after a few moments. You can now [build new +dashboards](https://learn.netdata.cloud/docs/cloud/visualize/dashboards) using the parent pod's metrics or run [Metric +Correlations](https://learn.netdata.cloud/docs/cloud/insights/metric-correlations) to troubleshoot anomalies. + + + +## Update/reinstall the Netdata Helm chart + +If you update the Helm chart's configuration, run `helm upgrade` to redeploy your Netdata service, replacing `netdata` +with the name of the release, if you changed it upon installation: + +```bash +helm upgrade netdata netdata/netdata +``` + +## What's next? + +Read the [monitoring a Kubernetes cluster guide](/docs/guides/monitor/kubernetes-k8s-netdata.md) for details on the +various metrics and charts created by the Helm chart and some best practices on real-time troubleshooting using Netdata. + +Check out our [infrastructure](/docs/quickstart/infrastructure.md) for details about additional k8s monitoring features, +and learn more about [configuring the Netdata Agent](/docs/configure/nodes.md) to better understand the settings you +might be interested in changing. + +To further configure Netdata for your cluster, see our [Helm chart repository](https://github.com/netdata/helmchart) and +the [service discovery repository](https://github.com/netdata/agent-service-discovery/). + +[](<>) diff --git a/packaging/installer/methods/macos.md b/packaging/installer/methods/macos.md new file mode 100644 index 0000000..05883a7 --- /dev/null +++ b/packaging/installer/methods/macos.md @@ -0,0 +1,90 @@ +<!-- +title: "Install Netdata on macOS" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/macos.md +--> + +# Install Netdata on macOS + +Netdata works on macOS, albeit with some limitations. The number of charts displaying system metrics is limited, but you +can use any of Netdata's [external plugins](../../../collectors/plugins.d/README.md) to monitor any services you might +have installed on your macOS system. You could also use a macOS system as the parent node in a [streaming +configuration](/streaming/README.md). + +We recommend installing Netdata with the community-created and -maintained [**Homebrew +package**](#install-netdata-with-the-homebrew-package). + +- [Install Netdata via the Homebrew package](#install-netdata-with-the-homebrew-package) +- [Install Netdata from source](#install-netdata-from-source) + +## Install Netdata with the Homebrew package + +If you don't have [Homebrew](https://brew.sh/) installed already, begin with their installation script: + +```bash +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" +``` + +Next, you can use Homebrew's package, which installs Netdata all its dependencies in a single step: + +```sh +brew install netdata +``` + +> Homebrew will place your Netdata configuration directory at `/usr/local/etc/netdata/`. Use the `edit-config` script +> and the files in this directory to configure Netdata. For reference, you can find stock configuration files at +> `/usr/local/Cellar/netdata/{NETDATA_VERSION}/lib/netdata/conf.d/`. + +Skip on ahead to the [What's next?](#whats-next) section to find links to helpful post-installation guides. + +## Install Netdata from source + +We don't recommend installing Netdata from source on macOS, as it can be difficult to configure and install dependencies +manually. + +First open your terminal of choice and install the Xcode development packages. + +```bash +xcode-select --install +``` + +Click **Install** on the Software Update popup window that appears. Then, use the same terminal session to use Homebrew +to install some of Netdata's prerequisites. You can omit `cmake` in case you do not want to use +[Netdata Cloud](https://learn.netdata.cloud/docs/cloud/). + +```bash +brew install ossp-uuid autoconf automake pkg-config libuv lz4 json-c openssl@1.1 libtool cmake +``` + +If you want to use the [database engine](/database/engine/README.md) to store your metrics, you need to download +and install the [Judy library](https://sourceforge.net/projects/judy/) before proceeding compiling Netdata. + +Next, download Netdata from our GitHub repository: + +```bash +git clone https://github.com/netdata/netdata.git +``` + +Finally, `cd` into the newly-created directory and then start the installer script: + +```bash +cd netdata/ +sudo ./netdata-installer.sh --install /usr/local +``` + +> Your Netdata configuration directory will be at `/usr/local/netdata/`, and your stock configuration directory will +> be at **`/usr/local/lib/netdata/conf.d/`.** +> +> The installer will also install a startup plist to start Netdata when your macOS system boots. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/manual.md b/packaging/installer/methods/manual.md new file mode 100644 index 0000000..6ece952 --- /dev/null +++ b/packaging/installer/methods/manual.md @@ -0,0 +1,229 @@ +<!-- +title: "Install Netdata on Linux from a Git checkout" +description: "Use the Netdata Agent source code from GitHub, plus helper scripts to set up your system, to install Netdata without packages or binaries." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/manual.md +--> + +# Install Netdata on Linux from a Git checkout + +To install the latest git version of Netdata, please follow these 2 steps: + +1. [Prepare your system](#prepare-your-system) + + Install the required packages on your system. + +2. [Install Netdata](#install-netdata) + + Download and install Netdata. You can also update it the same way. + +## Prepare your system + +Use our automatic requirements installer (_no need to be `root`_), which attempts to find the packages that +should be installed on your system to build and run Netdata. It supports a large variety of major Linux distributions +and other operating systems and is regularly tested. You can find this tool [here](https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh) or run it directly with `bash <(curl -sSL https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh)`. Otherwise read on for how to get requires packages manually: + +- **Alpine** Linux and its derivatives + - You have to install `bash` yourself, before using the installer. + +- **Arch** Linux and its derivatives + - You need arch/aur for package Judy. + +- **Gentoo** Linux and its derivatives + +- **Debian** Linux and its derivatives (including **Ubuntu**, **Mint**) + +- **Red Hat Enterprise Linux** and its derivatives (including **Fedora**, **CentOS**, **Amazon Machine Image**) + - Please note that for RHEL/CentOS you need + [EPEL](http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/). + In addition, RHEL/CentOS version 6 also need + [OKay](https://okay.com.mx/blog-news/rpm-repositories-for-centos-6-and-7.html) for package libuv version 1. + - CentOS 8 / RHEL 8 requires a bit of extra work. See the dedicated section below. + +- **SUSE** Linux and its derivatives (including **openSUSE**) + +- **SLE12** Must have your system registered with SUSE Customer Center or have the DVD. See + [#1162](https://github.com/netdata/netdata/issues/1162) + +Install the packages for having a **basic Netdata installation** (system monitoring and many applications, without `mysql` / `mariadb`, `postgres`, `named`, hardware sensors and `SNMP`): + +```sh +curl -Ss 'https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata +``` + +Install all the required packages for **monitoring everything Netdata can monitor**: + +```sh +curl -Ss 'https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata-all +``` + +If the above do not work for you, please [open a github +issue](https://github.com/netdata/netdata/issues/new?title=packages%20installer%20failed&labels=installation%20help&body=The%20experimental%20packages%20installer%20failed.%0A%0AThis%20is%20what%20it%20says:%0A%0A%60%60%60txt%0A%0Aplease%20paste%20your%20screen%20here%0A%0A%60%60%60) +with a copy of the message you get on screen. We are trying to make it work everywhere (this is also why the script +[reports back](https://github.com/netdata/netdata/issues/2054) success or failure for all its runs). + +--- + +This is how to do it by hand: + +```sh +# Debian / Ubuntu +apt-get install zlib1g-dev uuid-dev libuv1-dev liblz4-dev libjudy-dev libssl-dev libelf-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl python cmake + +# Fedora +dnf install zlib-devel libuuid-devel libuv-devel lz4-devel Judy-devel openssl-devel elfutils-libelf-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python cmake + +# CentOS / Red Hat Enterprise Linux +yum install autoconf automake curl gcc git libmnl-devel libuuid-devel openssl-devel libuv-devel lz4-devel Judy-devel elfutils-libelf-devel make nc pkgconfig python zlib-devel cmake + +# openSUSE +zypper install zlib-devel libuuid-devel libuv-devel liblz4-devel judy-devel libopenssl-devel libelf-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python cmake +``` + +Once Netdata is compiled, to run it the following packages are required (already installed using the above commands): + +| package | description| +|:-----:|-----------| +| `libuuid` | part of `util-linux` for GUIDs management| +| `zlib` | gzip compression for the internal Netdata web server| +| `libuv` | Multi-platform support library with a focus on asynchronous I/O, version 1 or greater| + +*Netdata will fail to start without the above.* + +Netdata plugins and various aspects of Netdata can be enabled or benefit when these are installed (they are optional): + +| package |description| +|:-----:|-----------| +| `bash`|for shell plugins and **alarm notifications**| +| `curl`|for shell plugins and **alarm notifications**| +| `iproute` or `iproute2`|for monitoring **Linux traffic QoS**<br/>use `iproute2` if `iproute` reports as not available or obsolete| +| `python`|for most of the external plugins| +| `python-yaml`|used for monitoring **beanstalkd**| +| `python-beanstalkc`|used for monitoring **beanstalkd**| +| `python-dnspython`|used for monitoring DNS query time| +| `python-ipaddress`|used for monitoring **DHCPd**<br/>this package is required only if the system has python v2. python v3 has this functionality embedded| +| `python-mysqldb`<br/>or<br/>`python-pymysql`|used for monitoring **mysql** or **mariadb** databases<br/>`python-mysqldb` is a lot faster and thus preferred| +| `python-psycopg2`|used for monitoring **postgresql** databases| +| `python-pymongo`|used for monitoring **mongodb** databases| +| `nodejs`|used for `node.js` plugins for monitoring **named** and **SNMP** devices| +| `lm-sensors`|for monitoring **hardware sensors**| +| `libelf`|for monitoring kernel-level metrics using eBPF| +| `libmnl`|for collecting netfilter metrics| +| `netcat`|for shell plugins to collect metrics from remote systems| + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +Netdata DB engine can be enabled when these are installed (they are optional): + +| package | description| +|:-----:|-----------| +| `liblz4` | Extremely fast compression algorithm, version r129 or greater| +| `Judy` | General purpose dynamic array| +| `openssl`| Cryptography and SSL/TLS toolkit| + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +Netdata Cloud support may require the following packages to be installed: + +| package | description +|:--------:| ----------------------- +| `cmake` | Needed at build time if you aren't using your distribution's version of libwebsockets or are building on a platform other than Linux +| `openssl` | Needed to secure communications with the Netdata Cloud + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +### CentOS / RHEL 6.x + +On CentOS / RHEL 6.x, many of the dependencies for Netdata are only +available with versions older than what we need, so special setup is +required if manually installing packages. + +CentOS 6.x: + +- Enable the EPEL repo +- Enable the additional repo from [okay.network](https://okay.network/blog-news/rpm-repositories-for-centos-6-and-7.html) + +And install the minimum required dependencies. + +### CentOS / RHEL 8.x + +For CentOS / RHEL 8.x a lot of development packages have moved out into their +own separate repositories. Some other dependencies are either missing completely +or have to be sourced by 3rd-parties. + +CentOS 8.x: + +- Enable the PowerTools repo +- Enable the EPEL repo +- Enable the Extra repo from [OKAY](https://okay.network/blog-news/rpm-repositories-for-centos-6-and-7.html) + +And install the minimum required dependencies: + +```sh +# Enable config-manager +yum install -y 'dnf-command(config-manager)' + +# Enable PowerTools +yum config-manager --set-enabled powertools + +# Enable EPEL +yum install -y epel-release + +# Install Repo for libuv-devl (NEW) +yum install -y http://repo.okay.com.mx/centos/8/x86_64/release/okay-release-1-3.el8.noarch.rpm + +# Install Devel Packages +yum install autoconf automake curl gcc git cmake libuuid-devel openssl-devel libuv-devel lz4-devel make nc pkgconfig python3 zlib-devel + +# Install Judy-Devel directly +yum install -y http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/Judy-devel-1.0.5-18.module_el8.1.0+217+4d875839.x86_64.rpm +``` + +--- + +### Install Netdata + +Do this to install and run Netdata: + +```sh +# download it - the directory 'netdata' will be created +git clone https://github.com/netdata/netdata.git --depth=100 +cd netdata + +# run script with root privileges to build, install, start Netdata +./netdata-installer.sh +``` + +- If you don't want to run it straight-away, add `--dont-start-it` option. + +- You can also append `--stable-channel` to fetch and install only the official releases from GitHub, instead of the nightly builds. + +- If you don't want to install it on the default directories, you can run the installer like this: `./netdata-installer.sh --install /opt`. This one will install Netdata in `/opt/netdata`. + +- If your server does not have access to the internet and you have manually put the installation directory on your server, you will need to pass the option `--disable-go` to the installer. The option will prevent the installer from attempting to download and install `go.d.plugin`. + +Once the installer completes, the file `/etc/netdata/netdata.conf` will be created (if you changed the installation directory, the configuration will appear in that directory too). + +You can edit this file to set options. One common option to tweak is `history`, which controls the size of the memory database Netdata will use. By default is `3600` seconds (an hour of data at the charts) which makes Netdata use about 10-15MB of RAM (depending on the number of charts detected on your system). Check **\[[Memory Requirements]]**. + +To apply the changes you made, you have to restart Netdata. + +### 'nonrepresentable section on output' errors + +Our current build process unfortunately has some issues when using certain configurations of the `clang` C compiler on Linux. + +If the installation fails with errors like `/bin/ld: externaldeps/libwebsockets/libwebsockets.a(context.c.o): relocation R_X86_64_32 against '.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIC`, and you are trying to build with `clang` on Linux, you will need to build Netdata using GCC to get a fully functional install. + +In most cases, you can do this by running `CC=gcc ./netdata-installer.sh`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/offline.md b/packaging/installer/methods/offline.md new file mode 100644 index 0000000..c978dd6 --- /dev/null +++ b/packaging/installer/methods/offline.md @@ -0,0 +1,90 @@ +<!-- +title: "Install Netdata on offline systems" +description: "Install the Netdata Agent on offline/air gapped systems to benefit from real-time, per-second monitoring without connecting to the internet." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/offline.md +--> + +# Install Netdata on offline systems + +The Netdata Agent installs on offline or air gapped systems with a few additional steps. + +By default, the `kickstart.sh` and `kickstart-static64.sh` download Netdata assets, like the precompiled binary and a +few dependencies, using the system's internet connection, but the Agent installer can also use equivalent files already +present on the local filesystem. + +First, download the required files. If you're using `kickstart.sh`, you need the Netdata tarball, the checksums, the +go.d plugin binary, and the go.d plugin configuration. If you're using `kickstart-static64.sh`, you need only the +Netdata tarball and checksums. + +Download the files you need to a system of yours that's connected to the internet. Use the commands below, or visit the +[latest Netdata release page](https://github.com/netdata/netdata/releases/latest) and [latest go.d plugin release +page](https://github.com/netdata/go.d.plugin/releases) to download the required files manually. + +**If you're using `kickstart.sh`**, use the following commands: + +```bash +cd /tmp + +curl -s https://my-netdata.io/kickstart.sh > kickstart.sh + +# Netdata tarball +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*tar.gz" | cut -d '"' -f 4 | wget -qi - + +# Netdata checksums +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - + +# Netdata dependency handling script +wget -q - https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh + +# go.d plugin +# For binaries for OS types and architectures not listed on [go.d releases](https://github.com/netdata/go.d.plugin/releases/latest), kindly open a github issue and we will do our best to serve your request +export OS=$(uname -s | tr '[:upper:]' '[:lower:]') ARCH=$(uname -m | sed -e 's/i386/386/g' -e 's/i686/386/g' -e 's/x86_64/amd64/g' -e 's/aarch64/arm64/g' -e 's/armv64/arm64/g' -e 's/armv6l/arm/g' -e 's/armv7l/arm/g' -e 's/armv5tel/arm/g') && curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*${OS}-${ARCH}.tar.gz" | cut -d '"' -f 4 | wget -qi - + +# go.d configuration +curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*config.tar.gz" | cut -d '"' -f 4 | wget -qi - +``` + +**If you're using `kickstart-static64.sh`**, use the following commands: + +```bash +cd /tmp + +curl -s https://my-netdata.io/kickstart-static64.sh > kickstart-static64.sh + +# Netdata static64 tarball +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*gz.run" | cut -d '"' -f 4 | wget -qi - + +# Netdata checksums +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - +``` + +Move downloaded files to the `/tmp` directory on the offline system in whichever way your defined policy allows (if +any). + +Now you can run either the `kickstart.sh` or `kickstart-static64.sh` scripts using the `--local-files` option. This +option requires you to specify the location and names of the files you just downloaded. + +> When using `--local-files`, the `kickstart.sh` or `kickstart-static64.sh` scripts won't download any Netdata assets +> from the internet. But, you may still need a connection to install dependencies using your system's package manager. +> The scripts will warn you if your system doesn't have all the dependencies. + +```bash +# kickstart.sh +bash kickstart.sh --local-files /tmp/netdata-(version-number-here).tar.gz /tmp/sha256sums.txt /tmp/go.d.plugin-(version-number-here).(OS)-(architecture).tar.gz /tmp/config.tar.gz /tmp/install-required-packages.sh + +# kickstart-static64.sh +bash kickstart-static64.sh --local-files /tmp/netdata-(version-number-here).gz.run /tmp/sha256sums.txt +``` + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[]() diff --git a/packaging/installer/methods/packages.md b/packaging/installer/methods/packages.md new file mode 100644 index 0000000..cf1e335 --- /dev/null +++ b/packaging/installer/methods/packages.md @@ -0,0 +1,48 @@ +<!-- +title: "Install Netdata with .deb/.rpm packages" +description: "Install the Netdata Agent with Linux packages that support Ubuntu, Debian, Fedora, RHEL, CentOS, openSUSE, and more." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/packages.md +--> + +# Install Netdata with .deb/.rpm packages + + + +Netdata provides our own flavour of binary packages for the most common operating systems that use with `.deb` and +`.rpm` packaging formats. + +We provide two separate repositories, one for our stable releases and one for our nightly releases. Visit the repository +pages and follow the quick set-up instructions to get started. + +1. Stable releases: Our stable production releases are hosted in the + [netdata/netdata](https://packagecloud.io/netdata/netdata) repository on packagecloud +2. Nightly releases: Our latest releases are hosted in the + [netdata/netdata-edge](https://packagecloud.io/netdata/netdata-edge) repository on packagecloud + +## Using caching proxies with packagecloud repositories + +packagecloud only provides HTTPS access to repositories they host, which means in turn that Netdata's package +repositories are only accessible via HTTPS. This is known to cause issues with some setups that use a caching proxy for +package downloads. + +If you are using such a setup, there are a couple of ways to work around this: + +- Configure your proxy to automatically pass through HTTPS connections without caching them. This is the simplest + solution, but means that downloads of Netdata packages will not be cached. +- Mirror the repository locally on your proxy system, and use that mirror when installing on other systems. This + requires more setup and more disk space on the caching host, but it lets you cache the packages locally. +- Some specific caching proxies may have alternative configuration options to deal with these issues. Find + such options in their documentation. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[]() diff --git a/packaging/installer/methods/pfsense.md b/packaging/installer/methods/pfsense.md new file mode 100644 index 0000000..ee1a453 --- /dev/null +++ b/packaging/installer/methods/pfsense.md @@ -0,0 +1,84 @@ +<!-- +title: "Install Netdata on pfSense" +description: "Install Netdata on pfSense to monitor the health and performance of firewalls with thousands of real-time, per-second metrics." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/pfsense.md +--> + +# Install Netdata on pfSense + +> 💡 This document is maintained by Netdata's community, and may not be completely up-to-date. Please double-check the +> details of the installation process, such as version numbers for downloadable packages, before proceeding. +> +> You can help improve this document by [submitting a +> PR](https://github.com/netdata/netdata/edit/master/packaging/installer/methods/pfsense.md) with your recommended +> improvements or changes. Thank you! + +## Install prerequisites/dependencies + +To install Netdata on pfSense, first run the following command (within a shell or under the **Diagnostics/Command** +prompt within the pfSense web interface). + +```bash +pkg install -y pkgconf bash e2fsprogs-libuuid libuv nano +``` + +Then run the following commands to download various dependencies from the FreeBSD repository. + +```sh +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/Judy-1.0.5_2.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/json-c-0.15_1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-certifi-2020.6.20.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-asn1crypto-1.3.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-pycparser-2.20.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-cffi-1.14.3.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-six-1.15.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-cryptography-2.6.1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-idna-2.10.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-openssl-19.0.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-pysocks-1.7.1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-urllib3-1.25.11,1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-yaml-5.3.1.txz +``` + +> ⚠️ If any of the above commands return a `Not Found` error, you need to manually search for the latest package in the +> [FreeBSD repository](https://www.freebsd.org/ports/). Search for the package's name, such as `py37-cffi`, find the +> latest version number, and update the command accordingly. + +> ⚠️ On pfSense 2.4.5, Python version 3.7 may be installed by the system, in which case you should should not install +> Python from the FreeBSD repository as instructed above. + +> ⚠️ If you are using the `apcupsd` collector, you need to make sure that apcupsd is up before starting Netdata. +> Otherwise a infinitely running `cat` process triggered by the default activated apcupsd charts plugin will eat up CPU +> and RAM (`/tmp/.netdata-charts.d-*/run-*`). This also applies to `OPNsense`. + +## Install Netdata + +You can now install Netdata from the FreeBSD repository. + +```bash +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/netdata-1.28.0.txz +``` + +> ⚠️ If the above command returns a `Not Found` error, you need to manually search for the latest version of Netdata in +> the [FreeBSD repository](https://www.freebsd.org/ports/). Search for `netdata`, find the latest version number, and +> update the command accordingly. + +You must edit `/usr/local/etc/netdata/netdata.conf` and change `bind to = 127.0.0.1` to `bind to = 0.0.0.0`. + +To start Netdata manually, run `service netdata onestart`. + +Visit the Netdata dashboard to confirm it's working: `http://<pfsenseIP>:19999` + +To start Netdata automatically every boot, add `service netdata onestart` as a Shellcmd entry within the pfSense web +interface under **Services/Shellcmd**. You'll need to install the Shellcmd package beforehand under **System/Package +Manager/Available Packages**. The Shellcmd Type should be set to `Shellcmd`. + Alternatively more information can be found in +<https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages>, for achieving the same via the command line and +scripts. + +If you experience an issue with `/usr/bin/install` being absent in pfSense 2.3 or earlier, update pfSense or use a +workaround from <https://redmine.pfsense.org/issues/6643> + +**Note:** In pfSense, the Netdata configuration files are located under `/usr/local/etc/netdata`. + +[](<>) diff --git a/packaging/installer/methods/source.md b/packaging/installer/methods/source.md new file mode 100644 index 0000000..e0827fc --- /dev/null +++ b/packaging/installer/methods/source.md @@ -0,0 +1,291 @@ +<!-- +title: "Manually build Netdata from source" +description: "Package maintainers and power users may be interested in manually building Netdata from source without using any of our installation scripts." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/source.md +--> + +# Manually build Netdata from source + +These instructions are for advanced users and distribution package +maintainers. Unless this describes you, you almost certainly want +to follow [our guide for manually installing Netdata from a git +checkout](/packaging/installer/methods/manual.md) instead. + +## Required dependencies + +At a bare minimum, Netdata requires the following libraries and tools +to build and run successfully: + +- libuuid +- libuv version 1.0 or newer +- zlib +- GNU autoconf +- GNU automake +- GCC or Xcode (Clang is known to have issues in certain configurations, see [Using Clang](#using-clang)) +- A version of `make` compatible with GNU automake +- Git (we use git in the build system to generate version info, don't need a full install, just a working `git show` command) + +Additionally, the following build time features require additional dependencies: + +- TLS support for the web GUI: + - OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer. +- dbengine metric storage: + - liblz4 r129 or newer + - OpenSSL 1.0 or newer (LibreSSL _amy_ work, but is largely untested). + - [libJudy](http://judy.sourceforge.net/) +- Netdata Cloud support: + - A working internet connection + - A recent version of CMake + - OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer. + - JSON-C (may be provided by the user as shown below, or by the system) + +## Preparing the source tree + +Certain features in Netdata require custom versions of specific libraries, +which the the build system will link statically into Netdata. These +libraries and their header files must be copied into specific locations +in the source tree to be used. + +### Netdata cloud + +Netdata Cloud functionality requires custom builds of libmosquitto and +libwebsockets. + +#### libmosquitto + +Netdata maintains a custom fork of libmosquitto at +https://github.com/netdata/mosquitto with patches to allow for proper +integration with libwebsockets, which is needed for correct operation of +Netdata Cloud functionality. To prepare this library for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/mosquitto.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/mosquitto/releases and + downloading and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. If building on a platform other than Linux, prepare the mosquitto + sources by running `cmake -D WITH_STATIC_LIBRARIES:boolean=YES .` in + the mosquitto source directory. +4. Build mosquitto by running `make -C lib` in the mosquitto source directory. +5. In the Netdata source directory, create a directory called `externaldeps/mosquitto`. +6. Copy `lib/mosquitto.h` from the mosquitto source directory to + `externaldeps/mosquitto/mosquitto.h` in the Netdata source tree. +7. Copy `lib/libmosquitto.a` from the mosquitto source directory to + `externaldeps/mosquitto/libmosquitto.a` in the Netdata source tree. If + building on a platform other than Linux, the file that needs to be + copied will instead be named `lib/libmosquitto_static.a`, but it + still needs to be copied to `externaldeps/mosquitto/libmosquitto.a`. + +#### libwebsockets + +Netdata uses the standard upstream version of libwebsockets located at +https://github.com/warmcat/libwebsockets, but requires a build with SOCKS5 +support, which is not enabled by most pre-built versions. Currently, +we do not support using a system copy of libwebsockets. To prepare this +library for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/libwebsockets.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/warmcat/libwebsockets/releases and + downloading and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Prepare the libwebsockets sources by running `cmake -D + LWS_WITH_SOCKS5:bool=ON .` in the libwebsockets source directory. +4. Build libwebsockets by running `make` in the libwebsockets source + directory. +5. In the Netdata source directory, create a directory called + `externaldeps/libwebsockets`. +6. Copy `lib/libwebsockets.a` from the libwebsockets source directory to + `externaldeps/libwebsockets/libwebsockets.a` in the Netdata source tree. +7. Copy the entire contents of `lib/include` from the libwebsockets source + directory to `externaldeps/libwebsockets/include` in the Netdata source tree. + +#### JSON-C + +Netdata requires the use of JSON-C for JSON parsing when using Netdata +Cloud. Netdata is able to use a system-provided copy of JSON-C, but +some systems may not provide it. If your system does not provide JSON-C, +you can do the following to prepare a copy for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/jsonc.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/json-c/json-c and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Prepare the JSON-C sources by running `cmake -DBUILD_SHARED_LIBS=OFF .` + in the JSON-C source directory. +4. Build JSON-C by running `make` in the JSON-C source directory. +5. In the Netdata source directory, create a directory called + `externaldeps/jsonc`. +6. Copy `libjson-c.a` fro the JSON-C source directory to + `externaldeps/jsonc/libjson-c.a` in the Netdata source tree. +7. Copy all of the header files (`*.h`) from the JSON-C source directory + to `externaldeps/jsonc/json-c` in the Netdata source tree. + +## Building Netdata + +Once the source tree has been prepared, Netdata is ready to be configured +and built. Netdata currently uses GNU autotools as it's primary build +system. To build Netdata this way: + +1. Run `autoreconf -ivf` in the Netdata source tree. +2. Run `./configure` in the Netdata source tree. +3. Run `make` in the Netdata source tree. + +### Configure options + +Netdata provides a number of build time configure options. This section +lists some of the ones you are most likely to need: + +- `--prefix`: Specify the prefix under which Netdata will be installed. +- `--with-webdir`: Specify a path relative to the prefix in which to + install the web UI files. +- `--disable-cloud`: Disables all Netdata Cloud functionality for + this build. + +### Using Clang + +Netdata is primarily developed using GCC, but in most cases we also +build just fine using Clang. Under some build configurations of Clang +itself, you may see build failures with the linker reporting errors +about `nonrepresentable section on output`. We currently do not have a +conclusive fix for this issue (the obvious fix leads to other issues which +we haven't been able to fix yet), and unfortunately the only workaround +is to use a different build of Clang or to use GCC. + +### Linking errors relating to OpenSSL + +Netdata's build system currently does not reliably support building +on systems which have multiple ABI incompatible versions of OpenSSL +installed. In such situations, you may encounter linking errors due to +Netdata trying to build against headers for one version but link to a +different version. + +## Additional components + +A full featured install of Netdata requires some additional components +which must be built and installed separately from the main Netdata +agent. All of these should be handled _after_ installing Netdata itself. + +### React dashboard + +The above build steps include a deprecated web UI for Netdata that lacks +support for Netdata Cloud. To get a fully featured dashboard, you must +install our new React dashboard. + +#### Installing the pre-built React dashboard + +We provide pre-built archives of the React dashboard for each release +(these are also used during our normal install process). To use one +of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/dashboard.version` in your Netdata sources. +2. Go to https://github.com/netdata/dashboard/releases and download the + `dashboard.tar.gz` file for the required release. +3. Unpack the downloaded archive to a temporary directory. +4. Copy the contents of the `build` directory from the extracted + archive to `/usr/share/netdata/web` or the equivalent location for + your build of Netdata. This _will_ overwrite some files in the target + location. + +#### Building the React dashboard locally + +Alternatively, you may wish to build the React dashboard locally. Doing +so requires a recent version of Node.JS with a working install of +NPM. Once you have the required tools, do the following: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/dashboard.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/dashboard and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Run `npm install` in the dashboard source tree. +4. Run `npm run build` in the dashboard source tree. +5. Copy the contents of the `build` directory just like step 4 of + installing the pre-built React dashboard. + +### Go collectors + +A number of the collectors for Netdata are written in Go instead of C, +and are developed in a separate repository from the mian Netdata code. +An installation without these collectors is still usable, but will be +unable to collect metrics for a number of network services the system +may be providing. You can either install a pre-built copy of these +collectors, or build them locally. + +#### Installing the pre-built Go collectors + +We provide pre-built binaries of the Go collectors for all the platforms +we officially support. To use one of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/go.d.version` in your Netdata sources. +2. Go to https://github.com/netdata/go.d.plugin/releases, select the + required release, and download the `go.d.plugin-*.tar.gz` file + for your system type and CPu architecture and the `config.tar.gz` + configuration file archive. +3. Extract the `go.d.plugin-*.tar.gz` archive into a temporary + location, and then copy the single file in the archive to + `/usr/libexec/netdata/plugins.d` or the equivalent location for your + build of Netdata and rename it to `go.d.plugin`. +4. Extract the `config.tar.gz` archive to a temporarylocation and then + copy the contents of the archive to `/etc/netdata` or the equivalent + location for your build of Netdata. + +#### Building the Go collectors locally + +Alternatively, you may wish to build the Go collectors locally +yourself. Doing so requires a working installation of Golang 1.13 or +newer. Once you have the required tools, do the following: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/go.d.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/go.d.plugin and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Run `make` in the go.d.plugin source tree. +4. Copy `bin/godplugin` to `/usr/libexec/netdata/plugins.d` or th + equivalent location for your build of Netdata and rename it to + `go.d.plugin`. +5. Copy the contents of the `config` directory to `/etc/netdata` or the + equivalent location for your build of Netdata. + +### eBPF collector + +On Linux systems, Netdata has support for using the kernel's eBPF +interface to monitor performance-related VFS, network, and process events, +allowing for insights into process lifetimes and file access +patterns. Using this functionality requires additional code managed in +a separate repository from the core Netdata agent. You can either install +a pre-built copy of the required code, or build it locally. + +#### Installing the pre-built eBPF code + +We provide pre-built copies of the eBPF code for 64-bit x86 systems +using glibc or musl. To use one of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/ebpf.version` in your Netdata sources. +2. Go to https://github.com/netdata/kernel-collector/releases, select the + required release, and download the `netdata-kernel-collector-*.tar.xz` + file for the libc variant your system uses (either rmusl or glibc). +3. Extract the contents of the archive to a temporary location, and then + copy all of the `.o` and `.so.*` files and the contents of the `library/` + directory to `/usr/libexec/netdata/plugins.d` or the equivalent location + for your build of Netdata. + +#### Building the eBPF code locally + +Alternatively, you may wish to build the eBPF code locally yourself. For +instructions, please consult [the README file for our kernel-collector +repository](https://github.com/netdata/kernel-collector/blob/master/README.md), +which outlines both the required dependencies, as well as multiple +options for building the code. + +[](<>) diff --git a/packaging/installer/methods/synology.md b/packaging/installer/methods/synology.md new file mode 100644 index 0000000..4a0ae35 --- /dev/null +++ b/packaging/installer/methods/synology.md @@ -0,0 +1,52 @@ +<!-- +title: "Install Netdata on Synology" +description: "The Netdata Agent can be installed on AMD64-compatible NAS systems using the 64-bit pre-compiled static binary." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/synology.md +--> + +# Install Netdata on Synology + +The documentation previously recommended installing the Debian Chroot package from the Synology community package +sources and then running Netdata from within the chroot. This does not work, as the chroot environment does not have +access to `/proc`, and therefore exposes very few metrics to Netdata. Additionally, [this +issue](https://github.com/SynoCommunity/spksrc/issues/2758), still open as of 2018/06/24, indicates that the Debian +Chroot package is not suitable for DSM versions greater than version 5 and may corrupt system libraries and render the +NAS unable to boot. + +The good news is that the [64-bit static installer](kickstart-64.md) works fine if your NAS is one that uses the amd64 architecture. It +will install the content into `/opt/netdata`, making future removal safe and simple. + +## Run as netdata user + +When Netdata is first installed, it will run as _root_. This may or may not be acceptable for you, and since other +installations run it as the `netdata` user, you might wish to do the same. This requires some extra work: + +1. Creat a group `netdata` via the Synology group interface. Give it no access to anything. +2. Create a user `netdata` via the Synology user interface. Give it no access to anything and a random password. Assign + the user to the `netdata` group. Netdata will chuid to this user when running. +3. Change ownership of the following directories, as defined in [Netdata + Security](/docs/netdata-security.md#security-design): + +```sh +chown -R root:netdata /opt/netdata/usr/share/netdata +chown -R netdata:netdata /opt/netdata/var/lib/netdata /opt/netdata/var/cache/netdata +chown -R netdata:root /opt/netdata/var/log/netdata +``` + +## Create startup script + +Additionally, as of 2018/06/24, the Netdata installer doesn't recognize DSM as an operating system, so no init script is +installed. You'll have to do this manually: + +1. Add [this file](https://gist.github.com/oskapt/055d474d7bfef32c49469c1b53e8225f) as `/etc/rc.netdata`. Make it + executable with `chmod 0755 /etc/rc.netdata`. +2. Add or edit `/etc/rc.local` and add a line calling `/etc/rc.netdata` to have it start on boot: + +```conf +# Netdata startup +[ -x /etc/rc.netdata ] && /etc/rc.netdata start +``` + +3. Make sure `/etc/rc.netdata` is executable: `chmod 0755 /etc/rc.netdata`. + +[](<>) diff --git a/packaging/installer/netdata-uninstaller.sh b/packaging/installer/netdata-uninstaller.sh new file mode 100755 index 0000000..f62934f --- /dev/null +++ b/packaging/installer/netdata-uninstaller.sh @@ -0,0 +1,499 @@ +#!/usr/bin/env bash +# +# This is the netdata uninstaller script +# +# Variables needed by script and taken from '.environment' file: +# - NETDATA_PREFIX +# - NETDATA_ADDED_TO_GROUPS +# +# Copyright: SPDX-License-Identifier: GPL-3.0-or-later +# +# Author: Paweł Krupa <paulfantom@gmail.com> +# Author: Pavlos Emm. Katsoulakis <paul@netdata.cloud> + +usage="$(basename "$0") [-h] [-f ] -- program to calculate the answer to life, the universe and everything + +where: + -e, --env path to environment file (defauls to '/etc/netdata/.environment' + -f, --force force uninstallation and do not ask any questions + -h show this help text + -y, --yes flag needs to be set to proceed with uninstallation" + +FILE_REMOVAL_STATUS=0 +ENVIRONMENT_FILE="/etc/netdata/.environment" +INTERACTIVITY="-i" +YES=0 +while :; do + case "$1" in + -h | --help) + echo "$usage" >&2 + exit 1 + ;; + -f | --force) + INTERACTIVITY="-f" + shift + ;; + -y | --yes) + YES=1 + shift + ;; + -e | --env) + ENVIRONMENT_FILE="$2" + shift 2 + ;; + -*) + echo "$usage" >&2 + exit 1 + ;; + *) break ;; + esac +done + +if [ "$YES" != "1" ]; then + echo >&2 "This script will REMOVE netdata from your system." + echo >&2 "Run it again with --yes to do it." + exit 1 +fi + +if [[ $EUID -ne 0 ]]; then + echo >&2 "This script SHOULD be run as root or otherwise it won't delete all installed components." + key="n" + read -r -s -n 1 -p "Do you want to continue as non-root user [y/n] ? " key + if [ "$key" != "y" ] && [ "$key" != "Y" ]; then + exit 1 + fi +fi + +# ----------------------------------------------------------------------------- +# portable service command + +service_cmd="$(command -v service 2> /dev/null)" +rcservice_cmd="$(command -v rc-service 2> /dev/null)" +systemctl_cmd="$(command -v systemctl 2> /dev/null)" +service() { + + local cmd="${1}" action="${2}" + + if [ -n "${systemctl_cmd}" ]; then + run "${systemctl_cmd}" "${action}" "${cmd}" + return $? + elif [ -n "${service_cmd}" ]; then + run "${service_cmd}" "${cmd}" "${action}" + return $? + elif [ -n "${rcservice_cmd}" ]; then + run "${rcservice_cmd}" "${cmd}" "${action}" + return $? + fi + return 1 +} + +# ----------------------------------------------------------------------------- + +setup_terminal() { + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput 1> /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 +} +setup_terminal || echo > /dev/null + +run_ok() { + printf >&2 "%s OK %s\n\n" "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" +} + +run_failed() { + printf >&2 "%s FAILED %s\n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" +} + +ESCAPED_PRINT_METHOD= +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi +escaped_print() { + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 +} + +run_logfile="/dev/null" +run() { + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "%s" "${info}" + escaped_print "${@}" + printf "%s" " ... " + } >> "${run_logfile}" + + printf "%s" "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" >&2 + escaped_print >&2 "${@}" + printf "%s\n" "${TPUT_RESET}" >&2 + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code %s\n" "${ret}" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} +} + +portable_del_group() { + local groupname="${1}" + + # Check if group exist + echo >&2 "Removing ${groupname} user group ..." + + # Linux + if command -v groupdel 1> /dev/null 2>&1; then + if grep -q "${groupname}" /etc/group; then + run groupdel "${groupname}" && return 0 + else + echo >&2 "Group ${groupname} already removed in a previous step." + return 0 + fi + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + if dseditgroup -o read netdata 1> /dev/null 2>&1; then + run dseditgroup -o delete "${groupname}" && return 0 + else + echo >&2 "Could not find group ${groupname}, nothing to do" + return 0 + fi + fi + + echo >&2 "Group ${groupname} was not automatically removed, you might have to remove it manually" + return 1 +} + +issystemd() { + local pids p myns ns systemctl + + # if the directory /lib/systemd/system OR /usr/lib/systemd/system (SLES 12.x) does not exit, it is not systemd + if [ ! -d /lib/systemd/system ] && [ ! -d /usr/lib/systemd/system ]; then + return 1 + fi + + # if there is no systemctl command, it is not systemd + systemctl=$(command -v systemctl 2> /dev/null) + if [ -z "${systemctl}" ] || [ ! -x "${systemctl}" ]; then + return 1 + fi + + # if pid 1 is systemd, it is systemd + [ "$(basename "$(readlink /proc/1/exe)" 2> /dev/null)" = "systemd" ] && return 0 + + # if systemd is not running, it is not systemd + pids=$(safe_pidof systemd 2> /dev/null) + [ -z "${pids}" ] && return 1 + + # check if the running systemd processes are not in our namespace + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + for p in ${pids}; do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + # if pid of systemd is in our namespace, it is systemd + [ -n "${myns}" ] && [ "${myns}" = "${ns}" ] && return 0 + done + + # else, it is not systemd + return 1 +} + +portable_del_user() { + local username="${1}" + echo >&2 "Deleting ${username} user account ..." + + # Linux + if command -v userdel 1> /dev/null 2>&1; then + run userdel -f "${username}" && return 0 + fi + + # mac OS + if command -v sysadminctl 1> /dev/null 2>&1; then + run sysadminctl -deleteUser "${username}" && return 0 + fi + + echo >&2 "User ${username} could not be deleted from system, you might have to remove it manually" + return 1 +} + +portable_del_user_from_group() { + local groupname="${1}" username="${2}" + + # username is not in group + echo >&2 "Deleting ${username} user from ${groupname} group ..." + + # Linux + if command -v gpasswd 1> /dev/null 2>&1; then + run gpasswd -d "netdata" "${group}" && return 0 + fi + + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupmod "${groupname}" -d "${username}" && return 0 + fi + + # BusyBox + if command -v delgroup 1> /dev/null 2>&1; then + run delgroup "${username}" "${groupname}" && return 0 + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + run dseditgroup -o delete -u "${username}" "${groupname}" && return 0 + fi + + echo >&2 "Failed to delete user ${username} from group ${groupname} !" + return 1 +} + +quit_msg() { + echo + if [ "$FILE_REMOVAL_STATUS" -eq 0 ]; then + echo >&2 "Something went wrong :(" + else + echo >&2 "Netdata files were successfully removed from your system" + fi +} + +user_input() { + TEXT="$1" + if [ "${INTERACTIVITY}" = "-i" ]; then + read -r -p "$TEXT" >&2 + fi +} + +rm_file() { + FILE="$1" + if [ -f "${FILE}" ]; then + run rm -v ${INTERACTIVITY} "${FILE}" + fi +} + +rm_dir() { + DIR="$1" + if [ -n "$DIR" ] && [ -d "$DIR" ]; then + user_input "Press ENTER to recursively delete directory '$DIR' > " + run rm -v -f -R "${DIR}" + fi +} + +safe_pidof() { + local pidof_cmd + pidof_cmd="$(command -v pidof 2> /dev/null)" + if [ -n "${pidof_cmd}" ]; then + ${pidof_cmd} "${@}" + return $? + else + ps -acxo pid,comm | + sed "s/^ *//g" | + grep netdata | + cut -d ' ' -f 1 + return $? + fi +} + +pidisnetdata() { + if [ -d /proc/self ]; then + if [ -z "$1" ] || [ ! -f "/proc/$1/stat" ]; then + return 1 + fi + [ "$(cut -d '(' -f 2 "/proc/$1/stat" | cut -d ')' -f 1)" = "netdata" ] && return 0 + return 1 + fi + return 0 +} + +stop_netdata_on_pid() { + local pid="${1}" ret=0 count=0 + + pidisnetdata "${pid}" || return 0 + + printf >&2 "Stopping netdata on pid %s ..." "${pid}" + while [ -n "$pid" ] && [ ${ret} -eq 0 ]; do + if [ ${count} -gt 24 ]; then + echo >&2 "Cannot stop the running netdata on pid ${pid}." + return 1 + fi + + count=$((count + 1)) + + pidisnetdata "${pid}" || ret=1 + if [ ${ret} -eq 1 ]; then + break + fi + + if [ ${count} -lt 12 ]; then + run kill "${pid}" 2> /dev/null + ret=$? + else + run kill -9 "${pid}" 2> /dev/null + ret=$? + fi + + test ${ret} -eq 0 && printf >&2 "." && sleep 5 + + done + + echo >&2 + if [ ${ret} -eq 0 ]; then + echo >&2 "SORRY! CANNOT STOP netdata ON PID ${pid} !" + return 1 + fi + + echo >&2 "netdata on pid ${pid} stopped." + return 0 +} + +netdata_pids() { + local p myns ns + + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + + for p in \ + $(cat /var/run/netdata.pid 2> /dev/null) \ + $(cat /var/run/netdata/netdata.pid 2> /dev/null) \ + $(safe_pidof netdata 2> /dev/null); do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then + pidisnetdata "${p}" && echo "${p}" + fi + done +} + +stop_all_netdata() { + local p + + if [ "${UID}" -eq 0 ]; then + uname="$(uname 2> /dev/null)" + + # Any of these may fail, but we need to not bail if they do. + if issystemd; then + if systemctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "Darwin" ]; then + if launchctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "FreeBSD" ]; then + if /etc/rc.d/netdata stop; then + sleep 5 + fi + else + if service netdata stop; then + sleep 5 + fi + fi + fi + + if [ -n "$(netdata_pids)" ] && [ -n "$(builtin type -P netdatacli)" ]; then + netdatacli shutdown-agent + sleep 20 + fi + + for p in $(netdata_pids); do + # shellcheck disable=SC2086 + stop_netdata_on_pid ${p} + done +} + +trap quit_msg EXIT + +#shellcheck source=/dev/null +source "${ENVIRONMENT_FILE}" || exit 1 + +#### STOP NETDATA +echo >&2 "Stopping a possibly running netdata..." +stop_all_netdata + +#### REMOVE NETDATA FILES +rm_file /etc/logrotate.d/netdata +rm_file /etc/systemd/system/netdata.service +rm_file /lib/systemd/system/netdata.service +rm_file /usr/lib/systemd/system/netdata.service +rm_file /etc/systemd/system/netdata-updater.service +rm_file /lib/systemd/system/netdata-updater.service +rm_file /usr/lib/systemd/system/netdata-updater.service +rm_file /etc/systemd/system/netdata-updater.timer +rm_file /lib/systemd/system/netdata-updater.timer +rm_file /usr/lib/systemd/system/netdata-updater.timer +rm_file /etc/init.d/netdata +rm_file /etc/periodic/daily/netdata-updater +rm_file /etc/cron.daily/netdata-updater +rm_file /etc/cron.d/netdata-updater + + +if [ -n "${NETDATA_PREFIX}" ] && [ -d "${NETDATA_PREFIX}" ]; then + rm_dir "${NETDATA_PREFIX}" +else + rm_file "/usr/sbin/netdata" + rm_file "/usr/sbin/netdatacli" + rm_file "/tmp/netdata-ipc" + rm_file "/usr/sbin/netdata-claim.sh" + rm_dir "/usr/share/netdata" + rm_dir "/usr/libexec/netdata" + rm_dir "/var/lib/netdata" + rm_dir "/var/cache/netdata" + rm_dir "/var/log/netdata" + rm_dir "/etc/netdata" +fi + +FILE_REMOVAL_STATUS=1 + +#### REMOVE NETDATA USER FROM ADDED GROUPS +if [ -n "$NETDATA_ADDED_TO_GROUPS" ]; then + user_input "Press ENTER to delete 'netdata' from following groups: '$NETDATA_ADDED_TO_GROUPS' > " + for group in $NETDATA_ADDED_TO_GROUPS; do + portable_del_user_from_group "${group}" "netdata" + done +fi + +#### REMOVE USER +user_input "Press ENTER to delete 'netdata' system user > " +portable_del_user "netdata" || : + +### REMOVE GROUP +user_input "Press ENTER to delete 'netdata' system group > " +portable_del_group "netdata" || : diff --git a/packaging/installer/netdata-updater.sh b/packaging/installer/netdata-updater.sh new file mode 100755 index 0000000..10835cd --- /dev/null +++ b/packaging/installer/netdata-updater.sh @@ -0,0 +1,428 @@ +#!/usr/bin/env bash + +# Netdata updater utility +# +# Variables needed by script: +# - PATH +# - CFLAGS +# - LDFLAGS +# - IS_NETDATA_STATIC_BINARY +# - NETDATA_CONFIGURE_OPTIONS +# - REINSTALL_OPTIONS +# - NETDATA_TARBALL_URL +# - NETDATA_TARBALL_CHECKSUM_URL +# - NETDATA_TARBALL_CHECKSUM +# - NETDATA_PREFIX / NETDATA_LIB_DIR (After 1.16.1 we will only depend on lib dir) +# +# Optional environment options: +# +# - TMPDIR (set to a usable temporary directory) +# - NETDATA_NIGHTLIES_BASEURL (set the base url for downloading the dist tarball) +# +# Copyright: 2018-2020 Netdata Inc. +# SPDX-License-Identifier: GPL-3.0-or-later +# +# Author: Paweł Krupa <paulfantom@gmail.com> +# Author: Pavlos Emm. Katsoulakis <paul@netdata.cloud> +# Author: Austin S. Hemmelgarn <austin@netdata.cloud> + +set -e + +script_dir="$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd -P)" + +if [ -x "${script_dir}/netdata-updater" ]; then + script_source="${script_dir}/netdata-updater" +else + script_source="${script_dir}/netdata-updater.sh" +fi + +info() { + echo >&3 "$(date) : INFO: " "${@}" +} + +error() { + echo >&3 "$(date) : ERROR: " "${@}" +} + +: "${ENVIRONMENT_FILE:=THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT}" + +if [ "${ENVIRONMENT_FILE}" == "THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT" ]; then + if [ -r "${script_dir}/../../../etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="${script_dir}/../../../etc/netdata/.environment" + elif [ -r "/etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="/etc/netdata/.environment" + elif [ -r "/opt/netdata/etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="/opt/netdata/etc/netdata/.environment" + else + envpath="$(find / -type d \( -path /sys -o -path /proc -o -path /dev \) -prune -false -o -path '*netdata/.environment' -type f 2> /dev/null | head -n 1)" + if [ -r "${envpath}" ]; then + ENVIRONMENT_FILE="${envpath}" + else + error "Cannot find environment file, unable to update." + exit 1 + fi + fi +fi + +safe_sha256sum() { + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi +} + +# this is what we will do if it fails (head-less only) +fatal() { + error "FAILED TO UPDATE NETDATA : ${1}" + exit 1 +} + +cleanup() { + if [ -n "${logfile}" ]; then + cat >&2 "${logfile}" + rm "${logfile}" + fi + + if [ -n "$ndtmpdir" ] && [ -d "$ndtmpdir" ]; then + rm -rf "$ndtmpdir" + fi +} + +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" +} + +create_tmp_directory() { + if [ -n "${NETDATA_TMPDIR_PATH}" ]; then + TMPDIR="${NETDATA_TMPDIR_PATH}" + else + if [ -z "${NETDATA_TMPDIR}" ] || _cannot_use_tmpdir "${NETDATA_TMPDIR}" ; then + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + else + TMPDIR="${NETDATA_TMPDIR}" + fi + + mktemp -d -t netdata-updater-XXXXXXXXXX + fi +} + +_safe_download() { + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + curl -sSL --connect-timeout 10 --retry 3 "${url}" > "${dest}" + return $? + elif command -v wget > /dev/null 2>&1; then + wget -T 15 -O - "${url}" > "${dest}" + return $? + else + return 255 + fi +} + +download() { + url="${1}" + dest="${2}" + + _safe_download "${url}" "${dest}" + ret=$? + + if [ ${ret} -eq 0 ]; then + return 0 + elif [ ${ret} -eq 255 ]; then + fatal "I need curl or wget to proceed, but neither is available on this system." + else + fatal "Cannot download ${url}" + fi +} + +newer_commit_date() { + echo >&3 "Checking if a newer version of the updater script is available." + + if command -v jq > /dev/null 2>&1; then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | jq '.[0].commit.committer.date' | tr -d '"')" + elif command -v python > /dev/null 2>&1;then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | python -c 'from __future__ import print_function;import sys,json;print(json.load(sys.stdin)[0]["commit"]["committer"]["date"])')" + elif command -v python3 > /dev/null 2>&1;then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | python3 -c 'from __future__ import print_function;import sys,json;print(json.load(sys.stdin)[0]["commit"]["committer"]["date"])')" + fi + + if [ -z "${commit_date}" ] ; then + commit_date="9999-12-31T23:59:59Z" + fi + + if [ -e "${script_source}" ]; then + script_date="$(date -r "${script_source}" +%s)" + else + script_date="$(date +%s)" + fi + + [ "$(date -d "${commit_date}" +%s)" -ge "${script_date}" ] +} + +self_update() { + if [ -z "${NETDATA_NO_UPDATER_SELF_UPDATE}" ] && newer_commit_date; then + echo >&3 "Downloading newest version of updater script." + + ndtmpdir=$(create_tmp_directory) + cd "$ndtmpdir" || exit 1 + + if _safe_download "https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/netdata-updater.sh" ./netdata-updater.sh; then + chmod +x ./netdata-updater.sh || exit 1 + export ENVIRONMENT_FILE="${ENVIRONMENT_FILE}" + exec ./netdata-updater.sh --not-running-from-cron --no-self-update --tmpdir-path "$(pwd)" + else + echo >&3 "Failed to download newest version of updater script, continuing with current version." + fi + fi +} + +parse_version() { + r="${1}" + if echo "${r}" | grep -q '^v.*'; then + # shellcheck disable=SC2001 + # XXX: Need a regex group subsitutation here. + r="$(echo "${r}" | sed -e 's/^v\(.*\)/\1/')" + fi + + read -r -a p <<< "$(echo "${r}" | tr '-' ' ')" + + v="${p[0]}" + b="${p[1]}" + _="${p[2]}" # ignore the SHA + + if [[ ! "${b}" =~ ^[0-9]+$ ]]; then + b="0" + fi + + read -r -a pp <<< "$(echo "${v}" | tr '.' ' ')" + printf "%03d%03d%03d%05d" "${pp[0]}" "${pp[1]}" "${pp[2]}" "${b}" +} + +get_latest_version() { + local latest + if [ "${RELEASE_CHANNEL}" == "stable" ]; then + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + else + latest="$(download "$NETDATA_NIGHTLIES_BASEURL/latest-version.txt" /dev/stdout)" + fi + parse_version "$latest" +} + +set_tarball_urls() { + local extension="tar.gz" + + if [ "$2" == "yes" ]; then + extension="gz.run" + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.${extension}" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_NIGHTLIES_BASEURL/netdata-latest.${extension}" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_NIGHTLIES_BASEURL/sha256sums.txt" + fi +} + +update() { + [ -z "${logfile}" ] && info "Running on a terminal - (this script also supports running headless from crontab)" + + RUN_INSTALLER=0 + ndtmpdir=$(create_tmp_directory) + cd "$ndtmpdir" || exit 1 + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" >&3 2>&3 + + current_version="$(command -v netdata > /dev/null && parse_version "$(netdata -v | cut -f 2 -d ' ')")" + latest_version="$(get_latest_version)" + + # If we can't get the current version for some reason assume `0` + current_version="${current_version:-0}" + + # If we can't get the latest version for some reason assume `0` + latest_version="${latest_version:-0}" + + info "Current Version: ${current_version}" + info "Latest Version: ${latest_version}" + + if [ "${latest_version}" -gt 0 ] && [ "${current_version}" -gt 0 ] && [ "${current_version}" -ge "${latest_version}" ]; then + info "Newest version (current=${current_version} >= latest=${latest_version}) is already installed" + elif [ -n "${NETDATA_TARBALL_CHECKSUM}" ] && grep "${NETDATA_TARBALL_CHECKSUM}" sha256sum.txt >&3 2>&3; then + info "Newest version is already installed" + else + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.tar.gz" + if ! grep netdata-latest.tar.gz sha256sum.txt | safe_sha256sum -c - >&3 2>&3; then + fatal "Tarball checksum validation failed. Stopping netdata upgrade and leaving tarball in ${ndtmpdir}\nUsually this is a result of an older copy of the tarball or checksum file being cached somewhere upstream and can be resolved by retrying in an hour." + fi + NEW_CHECKSUM="$(safe_sha256sum netdata-latest.tar.gz 2> /dev/null | cut -d' ' -f1)" + tar -xf netdata-latest.tar.gz >&3 2>&3 + rm netdata-latest.tar.gz >&3 2>&3 + cd netdata-* || exit 1 + RUN_INSTALLER=1 + cd "${NETDATA_LOCAL_TARBAL_OVERRIDE}" || exit 1 + fi + + # We got the sources, run the update now + if [ ${RUN_INSTALLER} -eq 1 ]; then + # signal netdata to start saving its database + # this is handy if your database is big + possible_pids=$(pidof netdata) + do_not_start= + if [ -n "${possible_pids}" ]; then + read -r -a pids_to_kill <<< "${possible_pids}" + kill -USR1 "${pids_to_kill[@]}" + else + # netdata is currently not running, so do not start it after updating + do_not_start="--dont-start-it" + fi + + if [ -n "${NETDATA_SELECTED_DASHBOARD}" ]; then + env="NETDATA_SELECTED_DASHBOARD=${NETDATA_SELECTED_DASHBOARD}" + fi + + if [ ! -x ./netdata-installer.sh ]; then + if [ "$(find . -mindepth 1 -maxdepth 1 -type d | wc -l)" -eq 1 ] && [ -x "$(find . -mindepth 1 -maxdepth 1 -type d)/netdata-installer.sh" ]; then + cd "$(find . -mindepth 1 -maxdepth 1 -type d)" || exit 1 + fi + fi + + info "Re-installing netdata..." + eval "${env} ./netdata-installer.sh ${REINSTALL_OPTIONS} --dont-wait ${do_not_start}" >&3 2>&3 || fatal "FAILED TO COMPILE/INSTALL NETDATA" + + # We no longer store checksum info here. but leave this so that we clean up all environment files upon next update. + sed -i '/NETDATA_TARBALL/d' "${ENVIRONMENT_FILE}" + + info "Updating tarball checksum info" + echo "${NEW_CHECKSUM}" > "${NETDATA_LIB_DIR}/netdata.tarball.checksum" + fi + + rm -rf "${ndtmpdir}" >&3 2>&3 + [ -n "${logfile}" ] && rm "${logfile}" && logfile= + + return 0 +} + +logfile= +ndtmpdir= + +trap cleanup EXIT + +while [ -n "${1}" ]; do + if [ "${1}" = "--not-running-from-cron" ]; then + NETDATA_NOT_RUNNING_FROM_CRON=1 + shift 1 + elif [ "${1}" = "--no-updater-self-update" ]; then + NETDATA_NO_UPDATER_SELF_UPDATE=1 + shift 1 + elif [ "${1}" = "--tmpdir-path" ]; then + NETDATA_TMPDIR_PATH="${2}" + shift 2 + else + break + fi +done + +# Random sleep to aileviate stampede effect of Agents upgrading +# and disconnecting/reconnecting at the same time (or near to). +# But only we're not a controlling terminal (tty) +# Randomly sleep between 1s and 60m +if [ ! -t 1 ] && [ -z "${NETDATA_NOT_RUNNING_FROM_CRON}" ]; then + sleep $(((RANDOM % 3600) + 1)) +fi + +# shellcheck source=/dev/null +source "${ENVIRONMENT_FILE}" || exit 1 + +# We dont expect to find lib dir variable on older installations, so load this path if none found +export NETDATA_LIB_DIR="${NETDATA_LIB_DIR:-${NETDATA_PREFIX}/var/lib/netdata}" + +# Source the tarbal checksum, if not already available from environment (for existing installations with the old logic) +[[ -z "${NETDATA_TARBALL_CHECKSUM}" ]] && [[ -f ${NETDATA_LIB_DIR}/netdata.tarball.checksum ]] && NETDATA_TARBALL_CHECKSUM="$(cat "${NETDATA_LIB_DIR}/netdata.tarball.checksum")" + +# Grab the nightlies baseurl (defaulting to our Google Storage bucket) +export NETDATA_NIGHTLIES_BASEURL="${NETDATA_NIGHTLIES_BASEURL:-https://storage.googleapis.com/netdata-nightlies}" + +if [ "${INSTALL_UID}" != "$(id -u)" ]; then + fatal "You are running this script as user with uid $(id -u). We recommend to run this script as root (user with uid 0)" +fi + +if [ -t 2 ]; then + # we are running on a terminal + # open fd 3 and send it to stderr + exec 3>&2 +else + # we are headless + # create a temporary file for the log + logfile="$(mktemp "${logfile}"/netdata-updater.log.XXXXXX)" + # open fd 3 and send it to logfile + exec 3> "${logfile}" +fi + +self_update + +set_tarball_urls "${RELEASE_CHANNEL}" "${IS_NETDATA_STATIC_BINARY}" + +if [ "${IS_NETDATA_STATIC_BINARY}" == "yes" ]; then + ndtmpdir="$(create_tmp_directory)" + PREVDIR="$(pwd)" + + echo >&2 "Entering ${ndtmpdir}" + cd "${ndtmpdir}" || exit 1 + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.gz.run" + if ! grep netdata-latest.gz.run "${ndtmpdir}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Static binary checksum validation failed. Stopping netdata installation and leaving binary in ${ndtmpdir}\nUsually this is a result of an older copy of the file being cached somehere and can be resolved by simply retrying in an hour." + fi + + # Do not pass any options other than the accept, for now + # shellcheck disable=SC2086 + if sh "${ndtmpdir}/netdata-latest.gz.run" --accept -- ${REINSTALL_OPTIONS}; then + rm -r "${ndtmpdir}" + else + echo >&2 "NOTE: did not remove: ${ndtmpdir}" + fi + echo >&2 "Switching back to ${PREVDIR}" + cd "${PREVDIR}" || exit 1 +else + # the installer updates this script - so we run and exit in a single line + update && exit 0 +fi |