diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2021-02-07 11:45:55 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2021-02-07 11:45:55 +0000 |
| commit | a8220ab2d293bb7f4b014b79d16b2fb05090fa93 (patch) | |
| tree | 77f0a30f016c0925cf7ee9292e644bba183c2774 /packaging/installer | |
| parent | Adding upstream version 1.19.0. (diff) | |
| download | netdata-a8220ab2d293bb7f4b014b79d16b2fb05090fa93.tar.xz netdata-a8220ab2d293bb7f4b014b79d16b2fb05090fa93.zip | |
Adding upstream version 1.29.0.upstream/1.29.0
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to '')
25 files changed, 6384 insertions, 2018 deletions
diff --git a/packaging/installer/.keep b/packaging/installer/.keep deleted file mode 100644 index e69de29b..00000000 --- a/packaging/installer/.keep +++ /dev/null diff --git a/packaging/installer/README.md b/packaging/installer/README.md index 4840d29c..317ac638 100644 --- a/packaging/installer/README.md +++ b/packaging/installer/README.md @@ -1,606 +1,253 @@ -# Installation +<!-- +title: "Installation guide" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/README.md +--> -Netdata is a **monitoring agent**. It is designed to be installed and run on all your systems: **physical** and **virtual** servers, **containers**, even **IoT**. +# Installation guide -The best way to install Netdata is directly from source. Our **automatic installer** will install any required system packages and compile Netdata directly on your systems. +Netdata is a monitoring agent designed to run on all your systems: physical and virtual servers, containers, even +IoT/edge devices. Netdata runs on Linux, FreeBSD, macOS, Kubernetes, Docker, and all their derivatives. -!!! warning - You can find Netdata packages distributed by third parties. In many cases, these packages are either too old or broken. So, the suggested ways to install Netdata are the ones in this page. +The best way to install Netdata is with our [**automatic one-line installation +script**](#automatic-one-line-installation-script), which works with all Linux distributions, or our [**.deb/rpm +packages**](/packaging/installer/methods/packages.md), which seamlessly install with your distribution's package +manager. -1. [Automatic one line installation](#one-line-installation), easy installation from source, **this is the default** -2. [Install pre-built static binary on any 64bit Linux](#linux-64bit-pre-built-static-binary) -3. [Run Netdata in a docker container](#run-netdata-in-a-docker-container) -4. [Manual installation, step by step](#install-netdata-on-linux-manually) -5. [Install on FreeBSD](#freebsd) -6. [Install on pfSense](#pfsense) -7. [Enable on FreeNAS Corral](#freenas) -8. [Install on macOS (OS X)](#macos) -9. [Install on a Kubernetes cluster](https://github.com/netdata/helmchart#netdata-helm-chart-for-kubernetes-deployments) -10. [Install using binary packages](#binary-packages) +If you want to install Netdata with Docker, on a Kubernetes cluster, or a different operating system, see [Have a +different operating system, or want to try another +method?](#have-a-different-operating-system-or-want-to-try-another-method) -See also the list of Netdata [package maintainers](../maintainers) for ASUSTOR NAS, OpenWRT, ReadyNAS, etc. +Some third parties, such as the packaging teams at various Linux distributions, distribute old, broken, or altered +packages. We recommend you install Netdata using one of the methods listed below to guarantee you get the latest +checksum-verified packages. -Note: From Netdata v1.12 and above, anonymous usage information is collected by default and sent to Google Analytics. To read more about the information collected and how to opt-out, check the [anonymous statistics page](../../docs/anonymous-statistics.md). +Starting with v1.12, Netdata collects anonymous usage information by default and sends it to Google Analytics. Read +about the information collected, and learn how to-opt, on our [anonymous statistics](/docs/anonymous-statistics.md) +page. ---- +The usage statistics are _vital_ for us, as we use them to discover bugs and prioritize new features. We thank you for +_actively_ contributing to Netdata's future. -## One-line installation +## Automatic one-line installation script   -This method is **fully automatic on all Linux distributions**. FreeBSD and MacOS systems need some preparations before installing Netdata for the first time. Check the [FreeBSD](#freebsd) and the [MacOS](#macos) sections for more information. +This method is fully automatic on all Linux distributions, including Ubuntu, Debian, Fedora, CentOS, and others. -To install Netdata from source, and keep it up to date with our **nightly releases** automatically, run the following: +To install Netdata from source, including all dependencies required to connect to Netdata Cloud, and get _automatic +nightly updates_, run the following as your normal user: ```bash bash <(curl -Ss https://my-netdata.io/kickstart.sh) ``` -!!! note - Do not use `sudo` for the one-line installer—it will escalate privileges itself if needed. +To see more information about this installation script, including how to disable automatic updates, get nightly vs. +stable releases, or disable anonymous statistics, see the [`kickstart.sh` method +page](/packaging/installer/methods/kickstart.md). +Scroll down for details about [automatic updates](#automatic-updates) or [nightly vs. stable +releases](#nightly-vs-stable-releases). -To learn more about the pros and cons of using *nightly* vs. *stable* releases, see our [notice about the two options](#nightly-vs-stable-releases). +### Post-installation -<details markdown="1"><summary>Click here for more information and advanced use of the one-line installation script.</summary> +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. -Verify the integrity of the script with this: +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). -```bash -[ "0ae8dd3c4c9b976c4342c9fc09d9afae" = "$(curl -Ss https://my-netdata.io/kickstart.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" -``` - -_It should print `OK, VALID` if the script is the one we ship._ - -The `kickstart.sh` script: - -- detects the Linux distro and **installs the required system packages** for building Netdata (will ask for confirmation) -- downloads the latest Netdata source tree to `/usr/src/netdata.git`. -- installs Netdata by running `./netdata-installer.sh` from the source tree. -- installs `netdata-updater.sh` to `cron.daily`, so your Netdata installation will be updated daily (you will get a message from cron only if the update fails). -- For QA purposes, this installation method lets us know if it succeed or failed. - -The `kickstart.sh` script passes all its parameters to `netdata-installer.sh`, so you can add more parameters to customize your installation. Here are a few important parameters: - -- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. -- `--dont-start-it`: Prevent the installer from starting Netdata automatically. -- `--stable-channel`: Automatically update only on the release of new major versions. -- `--no-updates`: Prevent automatic updates of any kind. -- `--local-files`: Used for offline installations. Pass four file paths: the Netdata tarball, the checksum file, the go.d plugin tarball, and the go.d plugin config tarball, to force kickstart run the process using those files. - -Example using all the above parameters: - -```bash -bash <(curl -Ss https://my-netdata.io/kickstart.sh) --dont-wait --dont-start-it --no-updates --stable-channel --local-files /tmp/my-selfdownloaded-tarball.tar.gz /tmp/checksums.txt /tmp/manually.downloaded.go.d.binary.tar.gz /tmp/manually.downloaded.go.d.config.tar.gz -``` -Note: `--stable-channel` and `--local-files` overlap, if you use the tarball override the stable channel option is not effective -</details> - -Once Netdata is installed, see [Getting Started](../../docs/getting-started.md). - ---- - -## Linux 64bit pre-built static binary - -  - -You can install a pre-compiled static binary of Netdata on any Intel/AMD 64bit Linux system (even those that don't have a package manager, like CoreOS, CirrOS, busybox systems, etc). You can also use these packages on systems with broken or unsupported package managers. - -To install Netdata from a binary package on any Linux distro and any kernel version on **Intel/AMD 64bit** systems, and keep it up to date with our **nightly releases** automatically, run the following: - -```bash -bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) -``` - -!!! note - Do not use `sudo` for this installer—it will escalate privileges itself if needed. - -``` -To learn more about the pros and cons of using *nightly* vs. *stable* releases, see our [notice about the two options](README.md#nightly-vs-stable-releases). - -If your system does not have `bash` installed, open the `More information and advanced uses of the kickstart-static64.sh script` dropdown for instructions to run the installer without `bash`. - -This script installs Netdata at `/opt/netdata`. -``` - -<details markdown="1"><summary>Click here for more information and advanced use of this command.</summary> - -Verify the integrity of the script with this: - -```bash -[ "8ad43ff960bf6f2487233682909f7a87" = "$(curl -Ss https://my-netdata.io/kickstart-static64.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" -``` - -*It should print `OK, VALID` if the script is the one we ship.* - -The `kickstart-static64.sh` script passes all its parameters to `netdata-installer.sh`, so you can add more parameters to customize your installation. Here are a few important parameters: - -- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. -- `--dont-start-it`: Prevent the installer from starting Netdata automatically. -- `--stable-channel`: Automatically update only on the release of new major versions. -- `--no-updates`: Prevent automatic updates of any kind. -- `--local-files`: Used for offline installations. Pass two file paths, one for the tarball and one fir the checksum file, to force kickstart run the process using those files. - -Example using all the above parameters: - -```sh -bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --dont-wait --dont-start-it --no-updates --stable-channel --local-files /tmp/my-selfdownloaded-tarball.tar.gz /tmp/checksums.txt -``` - -If your shell fails to handle the above one liner, do this: - -```sh -# download the script with curl -curl https://my-netdata.io/kickstart-static64.sh >/tmp/kickstart-static64.sh - -# or, download the script with wget -wget -O /tmp/kickstart-static64.sh https://my-netdata.io/kickstart-static64.sh - -# run the downloaded script (any sh is fine, no need for bash) -sh /tmp/kickstart-static64.sh -``` - -- The static binary files are kept in repo [binary-packages](https://github.com/netdata/binary-packages). You can download any of the `.run` files, and run it. These files are self-extracting shell scripts built with [makeself](https://github.com/megastep/makeself). -- The target system does **not** need to have bash installed. -- The same files can be used for updates too. -- For QA purposes, this installation method lets us know if it succeed or failed. +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. -</details> +## Have a different operating system, or want to try another method? -Once Netdata is installed, see [Getting Started](../../docs/getting-started.md). +Netdata works on many different operating systems, each with a few possible installation methods. To see the full list +of approved methods for each operating system/version we support, see our [distribution +matrix](/packaging/DISTRIBUTIONS.md). ---- +Below, you can find a few additional installation methods, followed by separate instructions for a variety of unique +operating systems. -## Run Netdata in a Docker container +### Alternative methods -You can [Install Netdata with Docker](../docker/#install-netdata-with-docker). +<div class="installer-grid" markdown="1"> ---- +[ Install +with .deb or .rpm packages](/packaging/installer/methods/packages.md) -## Install Netdata on Linux manually +[ Install +with a pre-built static binary for 64-bit systems](/packaging/installer/methods/kickstart-64.md) -To install the latest git version of Netdata, please follow these 2 steps: +[ Install +Netdata on Docker](/packaging/docker/README.md) -1. [Prepare your system](#prepare-your-system) - - Install the required packages on your system. - -2. [Install Netdata](#install-netdata) - - Download and install Netdata. You can also update it the same way. - ---- - -### Prepare your system - -Try our experimental automatic requirements installer (no need to be root). This will try to find the packages that should be installed on your system to build and run Netdata. It supports most major Linux distributions released after 2010: - -- **Alpine** Linux and its derivatives - - You have to install `bash` yourself, before using the installer. - -- **Arch** Linux and its derivatives - - You need arch/aur for package Judy. - -- **Gentoo** Linux and its derivatives - -- **Debian** Linux and its derivatives (including **Ubuntu**, **Mint**) - -- **Redhat Enterprise Linux** and its derivatives (including **Fedora**, **CentOS**, **Amazon Machine Image**) - - Please note that for RHEL/CentOS you need - [EPEL](http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/). - In addition, RHEL/CentOS version 6 also need - [OKay](https://okay.com.mx/blog-news/rpm-repositories-for-centos-6-and-7.html) for package libuv version 1. - -- **SuSe** Linux and its derivatives (including **openSuSe**) - -- **SLE12** Must have your system registered with Suse Customer Center or have the DVD. See [#1162](https://github.com/netdata/netdata/issues/1162) - -Install the packages for having a **basic Netdata installation** (system monitoring and many applications, without `mysql` / `mariadb`, `postgres`, `named`, hardware sensors and `SNMP`): - -```sh -curl -Ss 'https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata -``` - -Install all the required packages for **monitoring everything Netdata can monitor**: - -```sh -curl -Ss 'https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata-all -``` - -If the above do not work for you, please [open a github issue](https://github.com/netdata/netdata/issues/new?title=packages%20installer%20failed&labels=installation%20help&body=The%20experimental%20packages%20installer%20failed.%0A%0AThis%20is%20what%20it%20says:%0A%0A%60%60%60txt%0A%0Aplease%20paste%20your%20screen%20here%0A%0A%60%60%60) with a copy of the message you get on screen. We are trying to make it work everywhere (this is also why the script [reports back](https://github.com/netdata/netdata/issues/2054) success or failure for all its runs). - ---- - -This is how to do it by hand: - -```sh -# Debian / Ubuntu -apt-get install zlib1g-dev uuid-dev libuv1-dev liblz4-dev libjudy-dev libssl-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl python - -# Fedora -dnf install zlib-devel libuuid-devel libuv-devel lz4-devel Judy-devel openssl-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python - -# CentOS / Red Hat Enterprise Linux -yum install autoconf automake curl gcc git libmnl-devel libuuid-devel openssl-devel libuv-devel lz4-devel Judy-devel make nc pkgconfig python zlib-devel - -# openSUSE -zypper install zlib-devel libuuid-devel libuv-devel liblz4-devel judy-devel libopenssl-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python -``` - -Once Netdata is compiled, to run it the following packages are required (already installed using the above commands): - -| package | description| -|:-----:|-----------| -| `libuuid` | part of `util-linux` for GUIDs management| -| `zlib` | gzip compression for the internal Netdata web server| - -*Netdata will fail to start without the above.* - -Netdata plugins and various aspects of Netdata can be enabled or benefit when these are installed (they are optional): - -| package |description| -|:-----:|-----------| -| `bash`|for shell plugins and **alarm notifications**| -| `curl`|for shell plugins and **alarm notifications**| -| `iproute` or `iproute2`|for monitoring **Linux traffic QoS**<br/>use `iproute2` if `iproute` reports as not available or obsolete| -| `python`|for most of the external plugins| -| `python-yaml`|used for monitoring **beanstalkd**| -| `python-beanstalkc`|used for monitoring **beanstalkd**| -| `python-dnspython`|used for monitoring DNS query time| -| `python-ipaddress`|used for monitoring **DHCPd**<br/>this package is required only if the system has python v2. python v3 has this functionality embedded| -| `python-mysqldb`<br/>or<br/>`python-pymysql`|used for monitoring **mysql** or **mariadb** databases<br/>`python-mysqldb` is a lot faster and thus preferred| -| `python-psycopg2`|used for monitoring **postgresql** databases| -| `python-pymongo`|used for monitoring **mongodb** databases| -| `nodejs`|used for `node.js` plugins for monitoring **named** and **SNMP** devices| -| `lm-sensors`|for monitoring **hardware sensors**| -| `libmnl`|for collecting netfilter metrics| -| `netcat`|for shell plugins to collect metrics from remote systems| - -*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* - -Netdata DB engine can be enabled when these are installed (they are optional): - -| package | description| -|:-----:|-----------| -| `libuv` | Multi-platform support library with a focus on asynchronous I/O, version 1 or greater| -| `liblz4` | Extremely fast compression algorithm, version r129 or greater| -| `Judy` | General purpose dynamic array| -| `openssl`| Cryptography and SSL/TLS toolkit| - -*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* - ---- - -### Install Netdata - -Do this to install and run Netdata: - -```sh -# download it - the directory 'netdata' will be created -git clone https://github.com/netdata/netdata.git --depth=100 -cd netdata - -# run script with root privileges to build, install, start Netdata -./netdata-installer.sh -``` +[ Install +Netdata on a Kubernetes cluster](/packaging/installer/methods/kubernetes.md) -- If you don't want to run it straight-away, add `--dont-start-it` option. +[ +Install Netdata on cloud providers (GCP/AWS/Azure)](/packaging/installer/methods/cloud-providers.md) -- You can also append `--stable-channel` to fetch and install only the official releases from GitHub, instead of the nightly builds. +[ Install +Netdata on macOS](/packaging/installer/methods/macos.md) -- If you don't want to install it on the default directories, you can run the installer like this: `./netdata-installer.sh --install /opt`. This one will install Netdata in `/opt/netdata`. +[ Install +Netdata on FreeBSD](/packaging/installer/methods/freebsd.md) -- If your server does not have access to the internet and you have manually put the installation directory on your server, you will need to pass the option `--disable-go` to the installer. The option will prevent the installer from attempting to download and install `go.d.plugin`. +[ Install +from a Git checkout](/packaging/installer/methods/manual.md) -Once the installer completes, the file `/etc/netdata/netdata.conf` will be created (if you changed the installation directory, the configuration will appear in that directory too). +[ Install on +offline/air-gapped systems](/packaging/installer/methods/offline.md) -You can edit this file to set options. One common option to tweak is `history`, which controls the size of the memory database Netdata will use. By default is `3600` seconds (an hour of data at the charts) which makes Netdata use about 10-15MB of RAM (depending on the number of charts detected on your system). Check **\[[Memory Requirements]]**. +[ +Installation on PFSense](/packaging/installer/methods/pfsense.md) -To apply the changes you made, you have to restart Netdata. +[ Install +Netdata on Synology](/packaging/installer/methods/synology.md) ---- +[ Manual +installation on FreeNAS](/packaging/installer/methods/freenas.md) -### Binary Packages +[ Manual +installation on Alpine](/packaging/installer/methods/alpine.md) - +[ +Build manually from source](/packaging/installer/methods/source.md) -We provide our own flavour of binary packages for the most common operating systems that comply with .RPM and .DEB packaging formats. - -We have currently released packages following the .RPM format with version [1.16.0](https://github.com/netdata/netdata/releases/tag/v1.16.0). -We have planned to release packages following the .DEB format with version [1.17.0](https://github.com/netdata/netdata/releases/tag/v1.17.0). -Early adopters may experiment with our .DEB formatted packages using our nightly releases. Our current packaging infrastructure provider is [Package Cloud](https://packagecloud.io). - -Netdata is committed to support installation of our solution to all operating systems. This is a constant battle for Netdata, as we strive to automate and make things easier for our users. For the operating system support matrix, please visit our [distributions](../../packaging/DISTRIBUTIONS.md) support page. - -We provide two separate repositories, one for our stable releases and one for our nightly releases. - -1. Stable releases: Our stable production releases are hosted in [netdata/netdata](https://packagecloud.io/netdata/netdata) repository of package cloud -2. Nightly releases: Our latest releases are hosted in [netdata/netdata-edge](https://packagecloud.io/netdata/netdata-edge) repository of package cloud - -Visit the repository pages and follow the quick set-up instructions to get started. - ---- - -## Other Systems - -##### FreeBSD - -You can install Netdata from ports or packages collection. - -This is how to install the latest Netdata version from sources on FreeBSD: - -```sh -# install required packages -pkg install bash e2fsprogs-libuuid git curl autoconf automake pkgconf pidof Judy liblz4 libuv json-c - -# download Netdata -git clone https://github.com/netdata/netdata.git --depth=100 - -# install Netdata in /opt/netdata -cd netdata -./netdata-installer.sh --install /opt -``` - -##### pfSense - -To install Netdata on pfSense, run the following commands (within a shell or under the **Diagnostics/Command** prompt within the pfSense web interface). - -Note that the first four packages are downloaded from the pfSense repository for maintaining compatibility with pfSense, Netdata, Judy and Python are downloaded from the FreeBSD repository. - -```sh -pkg install pkgconf -pkg install bash -pkg install e2fsprogs-libuuid -pkg install libuv -pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/Judy-1.0.5_2.txz -pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/python36-3.6.9.txz -ln -s /usr/local/lib/libjson-c.so /usr/local/lib/libjson-c.so.4 -pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/netdata-1.17.1.txz -``` -**Note:** If you receive a ` Not Found` error during the last two commands above, you will either need to manually look in the [repo folder](http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/) for the latest available package and use its URL instead, or you can try manually changing the netdata version in the URL to the latest version. - -You must edit `/usr/local/etc/netdata/netdata.conf` and change `bind to = 127.0.0.1` to `bind to = 0.0.0.0`. - -To start Netdata manually, run `service netdata onestart` - -Visit the Netdata dashboard to confirm it's working: `http://<pfsenseIP>:19999` - -To start Netdata automatically every boot, add `service netdata onestart` as a Shellcmd entry within the pfSense web interface under **Services/Shellcmd**. You'll need to install the Shellcmd package beforehand under **System/Package Manager/Available Packages**. The Shellcmd Type should be set to `Shellcmd`. - -Alternatively more information can be found in <https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages>, for achieving the same via the command line and scripts. - -If you experience an issue with `/usr/bin/install` being absent in pfSense 2.3 or earlier, update pfSense or use a workaround from <https://redmine.pfsense.org/issues/6643> - -**Note:** In pfSense, the Netdata configuration files are located under `/usr/local/etc/netdata` - -##### FreeNAS - -On FreeNAS-Corral-RELEASE (>=10.0.3 and <11.3), Netdata is pre-installed. - -To use Netdata, the service will need to be enabled and started from the FreeNAS **[CLI](https://github.com/freenas/cli)**. - -To enable the Netdata service: - -```sh -service netdata config set enable=true -``` - -To start the Netdata service: - -```sh -service netdata start -``` - -##### macOS - -Netdata on macOS still has limited charts, but external plugins do work. - -You can either install Netdata with [Homebrew](https://brew.sh/) - -```sh -brew install netdata -``` - -or from source: - -```sh -# install Xcode Command Line Tools -xcode-select --install -``` - -click `Install` in the software update popup window, then - -```sh -# install HomeBrew package manager -/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" - -# install required packages -brew install ossp-uuid autoconf automake pkg-config - -# download Netdata -git clone https://github.com/netdata/netdata.git --depth=100 - -# install Netdata in /usr/local/netdata -cd netdata -sudo ./netdata-installer.sh --install /usr/local -``` - -The installer will also install a startup plist to start Netdata when your Mac boots. - -##### Alpine 3.x - -Execute these commands to install Netdata in Alpine Linux 3.x: - -```sh -# install required packages -apk add alpine-sdk bash curl zlib-dev util-linux-dev libmnl-dev gcc make git autoconf automake pkgconfig python logrotate - -# if you plan to run node.js Netdata plugins -apk add nodejs - -# download Netdata - the directory 'netdata' will be created -git clone https://github.com/netdata/netdata.git --depth=100 -cd netdata - - -# build it, install it, start it -./netdata-installer.sh +</div> +## Automatic updates -# make Netdata start at boot -echo -e "#!/usr/bin/env bash\n/usr/sbin/netdata" >/etc/local.d/netdata.start -chmod 755 /etc/local.d/netdata.start +By default, Netdata's installation scripts enable automatic updates for both nightly and stable release channels. -# make Netdata stop at shutdown -echo -e "#!/usr/bin/env bash\nkillall netdata" >/etc/local.d/netdata.stop -chmod 755 /etc/local.d/netdata.stop +If you would prefer to update your Netdata agent manually, you can disable automatic updates by using the `--no-updates` +option when you install or update Netdata using the [automatic one-line installation +script](#automatic-one-line-installation-script). -# enable the local service to start automatically -rc-update add local +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) --no-updates ``` -##### Synology - -The documentation previously recommended installing the Debian Chroot package from the Synology community package sources and then running Netdata from within the chroot. This does not work, as the chroot environment does not have access to `/proc`, and therefore exposes very few metrics to Netdata. Additionally, [this issue](https://github.com/SynoCommunity/spksrc/issues/2758), still open as of 2018/06/24, indicates that the Debian Chroot package is not suitable for DSM versions greater than version 5 and may corrupt system libraries and render the NAS unable to boot. - -The good news is that the 64-bit static installer works fine if your NAS is one that uses the amd64 architecture. It will install the content into `/opt/netdata`, making future removal safe and simple. - -When Netdata is first installed, it will run as *root*. This may or may not be acceptable for you, and since other installations run it as the *netdata* user, you might wish to do the same. This requires some extra work: +With automatic updates disabled, you can choose exactly when and how you [update +Netdata](/packaging/installer/UPDATE.md). -1. Creat a group `netdata` via the Synology group interface. Give it no access to anything. -2. Create a user `netdata` via the Synology user interface. Give it no access to anything and a random password. Assign the user to the `netdata` group. Netdata will chuid to this user when running. -3. Change ownership of the following directories, as defined in [Netdata Security](../../docs/netdata-security.md#security-design): +### Network usage of Netdata’s automatic updater -```sh -chown -R root:netdata /opt/netdata/usr/share/netdata -chown -R netdata:netdata /opt/netdata/var/lib/netdata /opt/netdata/var/cache/netdata -chown -R netdata:root /opt/netdata/var/log/netdata -``` - -Additionally, as of 2018/06/24, the Netdata installer doesn't recognize DSM as an operating system, so no init script is installed. You'll have to do this manually: - -1. Add [this file](https://gist.github.com/oskapt/055d474d7bfef32c49469c1b53e8225f) as `/etc/rc.netdata`. Make it executable with `chmod 0755 /etc/rc.netdata`. -2. Edit `/etc/rc.local` and add a line calling `/etc/rc.netdata` to have it start on boot: +The auto-update functionality set up by the installation scripts requires working internet access to function +correctly. In particular, it currently requires access to GitHub (to check if a newer version of the updater script +is available or not, as well as potentially fetching build-time dependencies that are bundled as part of the install), +and Google Cloud Storage (to check for newer versions of Netdata and download the sources if there is a newer version). -``` -# Netdata startup -[ -x /etc/rc.netdata ] && /etc/rc.netdata start -``` +Note that the auto-update functionality will check for updates to itself independently of updates to Netdata, +and will try to use the latest version of the updater script whenever possible. This is intended to reduce the +amount of effort required by users to get updates working again in the event of a bug in the updater code. ## Nightly vs. stable releases -The Netdata team maintains two releases of the Netdata agent: **nightly** and **stable**. By default, Netdata's installation scripts will give you **automatic, nightly** updates, as that is our recommended configuration. +The Netdata team maintains two releases of the Netdata agent: **nightly** and **stable**. By default, Netdata's +installation scripts will give you **automatic, nightly** updates, as that is our recommended configuration. -**Nightly**: We create nightly builds every 24 hours. They contain fully-tested code that fixes bugs or security flaws, or introduces new features to Netdata. Every nightly release is a candidate for then becoming a stable release—when we're ready, we simply change the release tags on GitHub. That means nightly releases are stable and proven to function correctly in the vast majority of Netdata use cases. That's why nightly is the *best choice for most Netdata users*. +**Nightly**: We create nightly builds every 24 hours. They contain fully-tested code that fixes bugs or security flaws, +or introduces new features to Netdata. Every nightly release is a candidate for then becoming a stable release—when +we're ready, we simply change the release tags on GitHub. That means nightly releases are stable and proven to function +correctly in the vast majority of Netdata use cases. That's why nightly is the _best choice for most Netdata users_. -**Stable**: We create stable releases whenever we believe the code has reached a major milestone. Most often, stable releases correlate with the introduction of new, significant features. Stable releases might be a better choice for those who run Netdata in *mission-critical production systems*, as updates will come more infrequently, and only after the community helps fix any bugs that might have been introduced in previous releases. +**Stable**: We create stable releases whenever we believe the code has reached a major milestone. Most often, stable +releases correlate with the introduction of new, significant features. Stable releases might be a better choice for +those who run Netdata in _mission-critical production systems_, as updates will come more infrequently, and only after +the community helps fix any bugs that might have been introduced in previous releases. **Pros of using nightly releases:** -- Get the latest features and bugfixes as soon as they're available +- Get the latest features and bug fixes as soon as they're available - Receive security-related fixes immediately - Use stable, fully-tested code that's always improving - Leverage the same Netdata experience our community is using **Pros of using stable releases:** -- Protect yourself from the rare instance when major bugs slip through our testing and negatively affect a Netdata installation +- Protect yourself from the rare instance when major bugs slip through our testing and negatively affect a Netdata + installation - Retain more control over the Netdata version you use -## Offline installations +## Troubleshooting and known issues -You can install Netdata on systems without internet access, but you need to take -a few extra steps to make it work. +We are tracking a few issues related to installation and packaging. -By default, the `kickstart.sh` and `kickstart-static64.sh` download Netdata -assets, like the precompiled binary and a few dependencies, using the system's -internet connection, but you can also supply these files from the local filesystem. +### Older distributions (Ubuntu 14.04, Debian 8, CentOS 6) and OpenSSL -First, download the required files. If you're using `kickstart.sh`, you need the -Netdata tarball, the checksums, the go.d plugin binary, and the go.d plugin -configuration. If you're using `kickstart-static64.sh`, you need only the -Netdata tarball and checksums. +If you're running an older Linux distribution or one that has reached EOL, such as Ubuntu 14.04 LTS, Debian 8, or CentOS +6, your Agent may not be able to securely connect to Netdata Cloud due to an outdated version of OpenSSL. These old +versions of OpenSSL cannot perform [hostname validation](https://wiki.openssl.org/index.php/Hostname_validation), which +helps securely encrypt SSL connections. -Download the files you need to a system of yours that's connected to the -internet. You can use the commands below, or visit the [latest Netdata release -page](https://github.com/netdata/netdata/releases/latest) and [latest go.d -plugin release page](https://github.com/netdata/go.d.plugin/releases) to -download the required files manually. +We recommend you reinstall Netdata with a [static build](/packaging/installer/methods/kickstart-64.md), which uses an +up-to-date version of OpenSSL with hostname validation enabled. -#### kickstart.sh -```bash -cd /tmp +If you choose to continue using the outdated version of OpenSSL, your node will still connect to Netdata Cloud, albeit +with hostname verification disabled. Without verification, your Netdata Cloud connection could be vulnerable to +man-in-the-middle attacks. -curl -s https://my-netdata.io/kickstart.sh > kickstart.sh +### CentOS 6 and CentOS 8 -# Netdata tarball -curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*tar.gz" | cut -d '"' -f 4 | wget -qi - +To install the Agent on certain CentOS and RHEL systems, you must enable non-default repositories, such as EPEL or +PowerTools, to gather hard dependencies. See the [CentOS 6](/packaging/installer/methods/manual.md#centos-rehel-6-x) and +[CentOS 8](/packaging/installer/methods/manual.md#centos-rehel-8-x) sections for more information. -# Netdata checksums -curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - +### Access to file is not permitted -# Netdata dependency handling script -curl -s https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh | wget -qi - +If you see an error similar to `Access to file is not permitted: /usr/share/netdata/web//index.html` when you try to +visit the Agent dashboard at `http://NODE:19999`, you need to update Netdata's permissions to match those of your +system. -# go.d plugin -# For binaries for OS types and architectures not listed on [go.d releases](https://github.com/netdata/go.d.plugin/releases/latest), kindly open a github issue and we will do our best to serve your request -export OS=$(uname -s | tr '[:upper:]' '[:lower:]') ARCH=$(uname -m | sed -e 's/i386/386/g' -e 's/i686/386/g' -e 's/x86_64/amd64/g' -e 's/aarch64/arm64/g' -e 's/armv64/arm64/g' -e 's/armv6l/arm/g' -e 's/armv7l/arm/g' -e 's/armv5tel/arm/g') && curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*${OS}-${ARCH}.tar.gz" | cut -d '"' -f 4 | wget -qi - +Run `ls -la /usr/share/netdata/web/index.html` to find the file's permissions. You may need to change this path based on +the error you're seeing in your browser. In the below example, the file is owned by the user `netdata` and the group +`netdata`. -# go.d configuration -curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*config.tar.gz" | cut -d '"' -f 4 | wget -qi - -``` - -#### kickstart-static64.sh ```bash -cd /tmp - -curl -s https://my-netdata.io/kickstart-static64.sh > kickstart-static64.sh - -# Netdata static64 tarball -curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*gz.run" | cut -d '"' -f 4 | wget -qi - - -# Netdata checksums -curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - +ls -la /usr/share/netdata/web/index.html +-rw-r--r--. 1 netdata netdata 89377 May 5 06:30 /usr/share/netdata/web/index.html ``` -Move downloaded files to the `/tmp` directory on the offline system in whichever way -your defined policy allows (if any). +Open your `netdata.conf` file and find the `[web]` section, plus the `web files owner`/`web files group` settings. Edit +the lines to match the output from `ls -la` above and uncomment them if necessary. -Now you can run either the `kickstart.sh` or `kickstart-static64.sh` scripts -using the `--local-files` option. This option requires you to specify -the location and names of the files you just downloaded. - -!!! note When using `--local-files`, the `kickstart.sh` or - `kickstart-static64.sh` scripts won't download any Netdata assets from the - internet. But, you may still need a connection to install dependencies using - your system's package manager. The scripts will warn you if your system - doesn't have all the dependencies. - -```bash -# kickstart.sh -bash kickstart.sh --local-files /tmp/netdata-version-number-here.tar.gz /tmp/sha256sums.txt /tmp/go.d-binary-filename.tar.gz /tmp/config.tar.gz /tmp/install-required-packages.sh - -# kickstart-static64.sh -bash kickstart-static64.sh --local-files /tmp/netdata-version-number-here.gz.run /tmp/sha256sums.txt +```conf +[web] + web files owner = netdata + web files group = netdata ``` -Now that you're finished with your offline installation, you can move on to our -[getting started guide](../../docs/getting-started.md)! - -## Automatic updates +Save the file, [restart the Netdata Agent](/docs/getting-started.md#start-stop-and-restart-netdata), and try accessing +the dashboard again. -By default, Netdata's installation scripts enable automatic updates for both nightly and stable release channels. +### Multiple versions of OpenSSL -If you would prefer to manually update your Netdata agent, you can disable automatic updates by using the `--no-updates` option when you install or update Netdata using the [one-line installation script](#one-line-installation). +We've received reports from the community about issues with running the `kickstart.sh` script on systems that have both +a distribution-installed version of OpenSSL and a manually-installed local version. The Agent's installer cannot handle +both. -```bash -# kickstart.sh -bash <(curl -Ss https://my-netdata.io/kickstart.sh) --no-updates +We recommend you install Netdata with the [static binary](/packaging/installer/methods/kickstart-64.md) to avoid the +issue altogether. Or, you can manually remove one version of OpenSSL to remove the conflict. -# kickstart-static64.sh -bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --no-updates -``` +### Clang compiler on Linux -With automatic updates disabled, you can choose exactly when and how you [update Netdata](UPDATE.md). +Our current build process has some issues when using certain configurations of the `clang` C compiler on Linux. See [the +section on `nonrepresentable section on output` +errors](/packaging/installer/methods/manual.md#nonrepresentable-section-on-output-errors) for a workaround. -[](<>) +[](<>) diff --git a/packaging/installer/REINSTALL.md b/packaging/installer/REINSTALL.md new file mode 100644 index 00000000..50a15d79 --- /dev/null +++ b/packaging/installer/REINSTALL.md @@ -0,0 +1,68 @@ +<!-- +title: "Reinstall the Netdata Agent" +description: "Troubleshooting installation issues or force an update of the Netdata Agent by reinstalling it using the same method you used during installation." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/REINSTALL.md +--> + +# Reinstall the Netdata Agent + +In certain situations, such as needing to enable a feature or troubleshoot an issue, you may need to reinstall the +Netdata Agent on your node. + +Before you try reinstalling Netdata, figure out which [installation method you +used](/packaging/installer/UPDATE.md#determine-which-installation-method-you-used) if you do not already know. This will +determine the reinstallation method. + +## One-line installer script (`kickstart.sh`) + +Run the one-line installer script with the `--reinstall` parameter to reinstall the Netdata Agent. This will preserve +any [user configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +If you used any [optional +parameters](/packaging/installer/methods/kickstart.md#optional-parameters-to-alter-your-installation) during initial +installation, you need to pass them to the script again during reinstallation. If you cannot remember which options you +used, read the contents of the `.environment` file and look for a `REINSTALL_OPTIONS` line. This line contains a list of +optional parameters. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) --reinstall +``` + +## `.deb` or `.rpm` packages + +If you installed Netdata with [`.deb` or `.rpm` packages](/packaging/installer/methods/packages.md), use your +distribution's package manager to reinstall Netdata. Any custom settings present in your Netdata configuration directory +(typically at `/etc/netdata`) persists during this process. + +```bash +apt-get install --reinstall netdata # Ubuntu/Debian +dnf reinstall netdata # Fedora/RHEL +yum reinstall netdata # CentOS +zypper in -f netdata # openSUSE +``` + +## Pre-built static binary for 64-bit systems (`kickstart-static64.sh`) + +Run the one-line installer script with the `--reinstall` parameter to reinstall the Netdata Agent. This will preserve +any [user configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +If you used any [optional +parameters](/packaging/installer/methods/kickstart-64.md#optional-parameters-to-alter-your-installation) during +initial installation, you need to pass them to the script again during reinstallation. If you cannot remember which +options you used, read the contents of the `.environment` file and look for a `REINSTALL_OPTIONS` line. This line +contains a list of optional parameters. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --reinstall +``` + +## Troubleshooting + +If you still experience problems with your Netdata Agent installation after following one of these processes, the next +best route is to [uninstall](/packaging/installer/UNINSTALL.md) and then try a fresh installation using the [one-line +installer](/packaging/installer/methods/kickstart.md). + +You can also post to our [community forums](https://community.netdata.cloud/c/support/13) or create a new [bug +report](https://github.com/netdata/netdata/issues/new?labels=bug%2C+needs+triage&template=bug_report.md). + +[](<>) diff --git a/packaging/installer/UNINSTALL.md b/packaging/installer/UNINSTALL.md index d2bd1ebd..c18a1bfc 100644 --- a/packaging/installer/UNINSTALL.md +++ b/packaging/installer/UNINSTALL.md @@ -1,11 +1,19 @@ -# Uninstalling Netdata - -Our self-contained uninstaller is able to remove Netdata installations created with shell installer. -It doesn't need any other Netdata repository files to be run. -All it needs is an `.environment` file, which is created during installation (with shell installer) -and put in `${NETDATA_USER_CONFIG_DIR}/.environment` (by default `/etc/netdata/.environment`). -That file contains some parameters which are passed to our installer -and which are needed during uninstallation process. +<!-- +title: "Uninstall Netdata" +description: "If you are no longer interested in using the Netdata Agent, use the self-contained uninstaller to remove all traces of binaries and configuration files." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/UNINSTALL.md +--> + +# Uninstall Netdata + +> ⚠️ If you're having trouble updating Netdata, moving from one installation method to another, or generally having +> issues with your Netdata Agent installation, consider our [**reinstall Netdata** +> doc](/packaging/installer/REINSTALL.md) instead of removing the Netdata Agent entirely. + +Our self-contained uninstaller is able to remove Netdata installations created with shell installer. It doesn't need any +other Netdata repository files to be run. All it needs is an `.environment` file, which is created during installation +(with shell installer) and put in `${NETDATA_USER_CONFIG_DIR}/.environment` (by default `/etc/netdata/.environment`). +That file contains some parameters which are passed to our installer and which are needed during uninstallation process. Mainly two parameters are needed: ```sh @@ -29,8 +37,8 @@ NETDATA_ADDED_TO_GROUPS="<additional groups>" # Additional groups for a user ru ${NETDATA_PREFIX}/usr/libexec/netdata/netdata-uninstaller.sh --yes --env <environment_file> ``` -Note: Existing installations may still need to download the file if it's not present. -To execute uninstall in that case, run the following commands: +Note: Existing installations may still need to download the file if it's not present. To execute uninstall in that case, +run the following commands: ```sh wget https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/netdata-uninstaller.sh @@ -40,7 +48,7 @@ chmod +x ./netdata-uninstaller.sh The default `environment_file` is `/etc/netdata/.environment`. -Note: This uninstallation method assumes previous installation with `netdata-installer.sh` or the kickstart script. -Currently using it when Netdata was installed by a package manager can work or cause unexpected results. +> Note: This uninstallation method assumes previous installation with `netdata-installer.sh` or the kickstart script. +> Currently using it when Netdata was installed by a package manager can work or cause unexpected results. [](<>) diff --git a/packaging/installer/UPDATE.md b/packaging/installer/UPDATE.md index ab5b558f..ba57c8cc 100644 --- a/packaging/installer/UPDATE.md +++ b/packaging/installer/UPDATE.md @@ -1,57 +1,153 @@ -# Updating Netdata after its installation +<!-- +title: "Update the Netdata Agent" +description: "If you opted out of automatic updates, you need to update your Netdata Agent to the latest nightly or stable version." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/UPDATE.md +--> - +# Update the Netdata Agent -We suggest to keep your Netdata updated. We are actively developing it and you should always update to the latest version. +By default, the Netdata Agent automatically updates with the latest nightly version. If you opted out of automatic +updates, you need to update your Netdata Agent to the latest nightly or stable version. -The update procedure depends on how you installed it: +> 💡 Looking to reinstall the Netdata Agent to enable a feature, update an Agent that cannot update automatically, or +> troubleshoot an error during the installation process? See our [reinstallation doc](/packaging/installer/REINSTALL.md) +> for reinstallation steps. -## You downloaded it from github using git +Before you update the Netdata Agent, check to see if your Netdata Agent is already up-to-date by clicking on the update +icon in the local Agent dashboard's top navigation. This modal informs you whether your Agent needs an update or not. -### Manual update to get the latest git commit + -Netdata versions older than `v1.12.0-rc2-52` had a `netdata-updater.sh` script in the root directory of the source code, which has now been deprecated. The manual process that works for all versions to get the latest commit in git is to use the `netdata-installer.sh`. The installer preserves your custom configuration and updates the information of the installation in the `.environment` file under the user configuration directory. +## Determine which installation method you used -```sh -# go to the git downloaded directory -cd /path/to/git/downloaded/netdata +If you are not sure where your Netdata config directory is, see the [configuration doc](/docs/configure/nodes.md). In +most installations, this is `/etc/netdata`. -# update your local copy -git pull +Use `cd` to navigate to the Netdata config directory, then use `ls -a` to look for a file called `.environment`. -# run the Netdata installer -sudo ./netdata-installer.sh +- If the `.environment` file _does not_ exist, reinstall with your [package manager](#deb-or-rpm-packages). +- If the `.environment` file _does_ exist, check its contents with `less .environment`. + - If `IS_NETDATA_STATIC_BINARY` is `"yes"`, update using the [pre-built static + binary](#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh). + - In all other cases, update using the [one-line installer script](#one-line-installer-script-kickstartsh). + +Next, use the appropriate method to update the Netdata Agent: + +- [One-line installer script (`kickstart.sh`)](#one-line-installer-script-kickstartsh) +- [`.deb` or `.rpm` packages](#deb-or-rpm-packages) +- [Pre-built static binary for 64-bit systems (`kickstart-static64.sh`)](#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh) +- [Docker](#docker) +- [macOS](#macos) +- [Manual installation from Git](#manual-installation-from-git) + +## One-line installer script (`kickstart.sh`) + +If you installed Netdata using our one-line automatic installation script, run it again to update Netdata. Any custom +settings present in your Netdata configuration directory (typically at `/etc/netdata`) persists during this process. + +This script will automatically run the update script that was installed as part of the initial install (even if +you disabled automatic updates) and preserve the existing install options you specified. + +If you installed Netdata using an installation prefix, you will need to add an `--install` option specifying +that prefix to this command to make sure it finds Netdata. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) +``` + +> ❗ If the above command fails, you can [reinstall +> Netdata](/packaging/installer/REINSTALL.md#one-line-installer-script-kickstartsh) to get the latest version. This also +> preserves your [configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +## `.deb` or `.rpm` packages + +If you installed Netdata with [`.deb` or `.rpm` packages](/packaging/installer/methods/packages.md), use your +distribution's package manager to update Netdata. Any custom settings present in your Netdata configuration directory +(typically at `/etc/netdata`) persists during this process. + +Your package manager grabs a new package from our hosted repository, updates the Netdata Agent, and restarts it. + +```bash +apt-get install netdata # Ubuntu/Debian +dnf install netdata # Fedora/RHEL +yum install netdata # CentOS +zypper in netdata # openSUSE +``` + +> You may need to escalate privileges using `sudo`. + +## Pre-built static binary for 64-bit systems (`kickstart-static64.sh`) + +If you installed Netdata using the pre-built static binary, run the `kickstart-static64.sh` script again to update +Netdata. Any custom settings present in your Netdata configuration directory (typically at `/etc/netdata`) persists +during this process. + +This script will automatically run the update script that was installed as part of the initial install (even if +you disabled automatic updates) and preserve the existing install options you specified. + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) ``` -_Netdata will be restarted with the new version._ +> ❗ If the above command fails, you can [reinstall +> Netdata](/packaging/installer/REINSTALL.md#pre-built-static-binary-for-64-bit-systems-kickstart-static64sh) to get the +> latest version. This also preserves your [configuration](/docs/configure/nodes.md) in `netdata.conf` or other files. + +## Docker -Keep in mind, Netdata may now have new features, or certain old features may now behave differently. So pay some attention to it after updating. +Docker-based installations do not update automatically. To update an Netdata Agent running in a Docker container, you +must pull the [latest image from Docker Hub](https://hub.docker.com/r/netdata/netdata), stop and remove the container, +and re-create it using the latest image. -### Manual update to get the latest nightly build +First, pull the latest version of the image. -The `kickstart.sh` and `kickstart-static64.sh` one-liners will do a one-time update to the latest nightly build, if executed as follows: +```bash +docker pull netdata/netdata:latest +``` -```sh -# kickstart.sh -bash <(curl -Ss https://my-netdata.io/kickstart.sh) --no-updates +Next, to stop and remove any containers using the `netdata/netdata` image. Replace `netdata` if you changed it from the +default. -# kickstart-static64.sh -bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) --no-updates +```bash +docker stop netdata +docker rm netdata ``` -### Auto-update +You can now re-create your Netdata container using the `docker` command or a `docker-compose.yml` file. See our [Docker +installation instructions](/packaging/docker/README.md#create-a-new-netdata-agent-container) for details. -_Please, consider the risks of running an auto-update. Something can always go wrong. Keep an eye on your installation, and run a manual update if something ever fails._ +## macOS -Calling the `netdata-installer.sh` with the `--auto-update` or `-u` option will create the `netdata-updater` script under -either `/etc/cron.daily/`, or `/etc/periodic/daily/`. Whenever the `netdata-updater` is executed, it checks if a newer nightly build is available and then handles the download, installation and Netdata restart. +If you installed Netdata on your macOS system using Homebrew, you can explicitly request an update: -Note that after Jan 2019, the `kickstart.sh` one-liner `bash <(curl -Ss https://my-netdata.io/kickstart.sh)` calls the `netdata-installer.sh` with the auto-update option. So if you just run the one-liner without options once, your Netdata will be kept auto-updated. +```bash +brew upgrade netdata +``` -## You downloaded a binary package +Homebrew downloads the latest Netdata via the +[formulae](https://github.com/Homebrew/homebrew-core/blob/master/Formula/netdata.rb), ensures all dependencies are met, +and updates Netdata via reinstallation. -If you installed it from a binary package, the best way is to **obtain a newer copy** from the source you got it in the first place. +## Manual installation from Git + +If you installed [Netdata manually from Git](/packaging/installer/methods/manual.md), you can run that installer again +to update your agent. First, run our automatic requirements installer, which works on many Linux distributions, to +ensure your system has the dependencies necessary for new features. + +```bash +bash <(curl -sSL https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh) +``` + +Navigate to the directory where you first cloned the Netdata repository, pull the latest source code, and run +`netdata-install.sh` again. This process compiles Netdata with the latest source code and updates it via reinstallation. + +```bash +cd /path/to/netdata/git +git pull origin master +sudo ./netdata-installer.sh +``` -If a newer version of Netdata is not available from the source you got it, we suggest to uninstall the version you have and follow the [installation](README.md) instructions for installing a fresh version of Netdata. +> ⚠️ If you installed Netdata with any optional parameters, such as `--no-updates` to disable automatic updates, and +> want to retain those settings, you need to set them again during this process. [](<>) diff --git a/packaging/installer/functions.sh b/packaging/installer/functions.sh index d2b7b676..03ceb2a5 100644 --- a/packaging/installer/functions.sh +++ b/packaging/installer/functions.sh @@ -1,6 +1,6 @@ -# no shebang necessary - this is a library to be sourced +#!/bin/bash + # SPDX-License-Identifier: GPL-3.0-or-later -# shellcheck disable=SC1091,SC1117,SC2002,SC2004,SC2034,SC2046,SC2059,SC2086,SC2129,SC2148,SC2154,SC2155,SC2162,SC2166,SC2181,SC2193 # make sure we have a UID [ -z "${UID}" ] && UID="$(id -u)" @@ -8,821 +8,1047 @@ # ----------------------------------------------------------------------------- setup_terminal() { - TPUT_RESET="" - TPUT_BLACK="" - TPUT_RED="" - TPUT_GREEN="" - TPUT_YELLOW="" - TPUT_BLUE="" - TPUT_PURPLE="" - TPUT_CYAN="" - TPUT_WHITE="" - TPUT_BGBLACK="" - TPUT_BGRED="" - TPUT_BGGREEN="" - TPUT_BGYELLOW="" - TPUT_BGBLUE="" - TPUT_BGPURPLE="" - TPUT_BGCYAN="" - TPUT_BGWHITE="" - TPUT_BOLD="" - TPUT_DIM="" - TPUT_UNDERLINED="" - TPUT_BLINK="" - TPUT_INVERTED="" - TPUT_STANDOUT="" - TPUT_BELL="" - TPUT_CLEAR="" - - # Is stderr on the terminal? If not, then fail - test -t 2 || return 1 - - if command -v tput 1>/dev/null 2>&1; then - if [ $(($(tput colors 2>/dev/null))) -ge 8 ]; then - # Enable colors - TPUT_RESET="$(tput sgr 0)" - TPUT_BLACK="$(tput setaf 0)" - TPUT_RED="$(tput setaf 1)" - TPUT_GREEN="$(tput setaf 2)" - TPUT_YELLOW="$(tput setaf 3)" - TPUT_BLUE="$(tput setaf 4)" - TPUT_PURPLE="$(tput setaf 5)" - TPUT_CYAN="$(tput setaf 6)" - TPUT_WHITE="$(tput setaf 7)" - TPUT_BGBLACK="$(tput setab 0)" - TPUT_BGRED="$(tput setab 1)" - TPUT_BGGREEN="$(tput setab 2)" - TPUT_BGYELLOW="$(tput setab 3)" - TPUT_BGBLUE="$(tput setab 4)" - TPUT_BGPURPLE="$(tput setab 5)" - TPUT_BGCYAN="$(tput setab 6)" - TPUT_BGWHITE="$(tput setab 7)" - TPUT_BOLD="$(tput bold)" - TPUT_DIM="$(tput dim)" - TPUT_UNDERLINED="$(tput smul)" - TPUT_BLINK="$(tput blink)" - TPUT_INVERTED="$(tput rev)" - TPUT_STANDOUT="$(tput smso)" - TPUT_BELL="$(tput bel)" - TPUT_CLEAR="$(tput clear)" - fi - fi - - return 0 -} -setup_terminal || echo >/dev/null + TPUT_RESET="" + TPUT_BLACK="" + TPUT_RED="" + TPUT_GREEN="" + TPUT_YELLOW="" + TPUT_BLUE="" + TPUT_PURPLE="" + TPUT_CYAN="" + TPUT_WHITE="" + TPUT_BGBLACK="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BGYELLOW="" + TPUT_BGBLUE="" + TPUT_BGPURPLE="" + TPUT_BGCYAN="" + TPUT_BGWHITE="" + TPUT_BOLD="" + TPUT_DIM="" + TPUT_UNDERLINED="" + TPUT_BLINK="" + TPUT_INVERTED="" + TPUT_STANDOUT="" + TPUT_BELL="" + TPUT_CLEAR="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput 1> /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + # shellcheck disable=SC2034 + TPUT_BLACK="$(tput setaf 0)" + TPUT_RED="$(tput setaf 1)" + TPUT_GREEN="$(tput setaf 2)" + # shellcheck disable=SC2034 + TPUT_YELLOW="$(tput setaf 3)" + # shellcheck disable=SC2034 + TPUT_BLUE="$(tput setaf 4)" + # shellcheck disable=SC2034 + TPUT_PURPLE="$(tput setaf 5)" + TPUT_CYAN="$(tput setaf 6)" + TPUT_WHITE="$(tput setaf 7)" + # shellcheck disable=SC2034 + TPUT_BGBLACK="$(tput setab 0)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + # shellcheck disable=SC2034 + TPUT_BGYELLOW="$(tput setab 3)" + # shellcheck disable=SC2034 + TPUT_BGBLUE="$(tput setab 4)" + # shellcheck disable=SC2034 + TPUT_BGPURPLE="$(tput setab 5)" + # shellcheck disable=SC2034 + TPUT_BGCYAN="$(tput setab 6)" + # shellcheck disable=SC2034 + TPUT_BGWHITE="$(tput setab 7)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + # shellcheck disable=SC2034 + TPUT_UNDERLINED="$(tput smul)" + # shellcheck disable=SC2034 + TPUT_BLINK="$(tput blink)" + # shellcheck disable=SC2034 + TPUT_INVERTED="$(tput rev)" + # shellcheck disable=SC2034 + TPUT_STANDOUT="$(tput smso)" + # shellcheck disable=SC2034 + TPUT_BELL="$(tput bel)" + # shellcheck disable=SC2034 + TPUT_CLEAR="$(tput clear)" + fi + fi + + return 0 +} +setup_terminal || echo > /dev/null progress() { - echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " } -# ----------------------------------------------------------------------------- +get() { + url="${1}" + if command -v curl > /dev/null 2>&1; then + curl -q -o - -sSL --connect-timeout 10 --retry 3 "${url}" + elif command -v wget > /dev/null 2>&1; then + wget -T 15 -O - "${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi +} -netdata_banner() { - local l1=" ^" \ - l2=" |.-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-" \ - l3=" | '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' " \ - l4=" +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->" \ - sp=" " \ - netdata="netdata" start end msg="${*}" chartcolor="${TPUT_DIM}" +download_file() { + url="${1}" + dest="${2}" + name="${3}" + opt="${4}" + + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" + else + echo >&2 + echo >&2 "Downloading ${name} from '${url}' failed because of missing mandatory packages." + if [ -n "$opt" ]; then + echo >&2 "Either add packages or disable it by issuing '--disable-${opt}' in the installer" + fi + echo >&2 - [ ${#msg} -lt ${#netdata} ] && msg="${msg}${sp:0:$((${#netdata} - ${#msg}))}" - [ ${#msg} -gt $((${#l2} - 20)) ] && msg="${msg:0:$((${#l2} - 23))}..." + run_failed "I need curl or wget to proceed, but neither is available on this system." + fi +} - start="$((${#l2} / 2 - 4))" - [ $((start + ${#msg} + 4)) -gt ${#l2} ] && start=$((${#l2} - ${#msg} - 4)) - end=$((start + ${#msg} + 4)) +# ----------------------------------------------------------------------------- +# external component handling + +fetch_and_verify() { + local component=${1} + local url=${2} + local base_name=${3} + local tmp=${4} + local override=${5} + + if [ -z "${override}" ]; then + download_file "${url}" "${tmp}/${base_name}" "${component}" + else + progress "Using provided ${component} archive ${override}" + run cp "${override}" "${tmp}/${base_name}" + fi + + if [ ! -f "${tmp}/${base_name}" ] || [ ! -s "${tmp}/${base_name}" ]; then + run_failed "Unable to find usable archive for ${component}" + return 1 + fi + + grep "${base_name}\$" "${INSTALLER_DIR}/packaging/${component}.checksums" > "${tmp}/sha256sums.txt" 2> /dev/null + + # Checksum validation + if ! (cd "${tmp}" && safe_sha256sum -c "sha256sums.txt"); then + run_failed "${component} files checksum validation failed." + return 1 + fi +} - echo >&2 - echo >&2 "${chartcolor}${l1}${TPUT_RESET}" - echo >&2 "${chartcolor}${l2:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_GREEN}${netdata}${TPUT_RESET}${chartcolor}${sp:0:$((end - start - 2 - ${#netdata}))}${l2:end:$((${#l2} - end))}${TPUT_RESET}" - echo >&2 "${chartcolor}${l3:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_CYAN}${msg}${TPUT_RESET}${chartcolor}${sp:0:2}${l3:end:$((${#l2} - end))}${TPUT_RESET}" - echo >&2 "${chartcolor}${l4}${TPUT_RESET}" - echo >&2 +# ----------------------------------------------------------------------------- + +netdata_banner() { + local l1=" ^" \ + l2=" |.-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-. .-" \ + l3=" | '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' '-' " \ + l4=" +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->" \ + sp=" " \ + netdata="netdata" start end msg="${*}" chartcolor="${TPUT_DIM}" + + [ ${#msg} -lt ${#netdata} ] && msg="${msg}${sp:0:$((${#netdata} - ${#msg}))}" + [ ${#msg} -gt $((${#l2} - 20)) ] && msg="${msg:0:$((${#l2} - 23))}..." + + start="$((${#l2} / 2 - 4))" + [ $((start + ${#msg} + 4)) -gt ${#l2} ] && start=$((${#l2} - ${#msg} - 4)) + end=$((start + ${#msg} + 4)) + + echo >&2 + echo >&2 "${chartcolor}${l1}${TPUT_RESET}" + echo >&2 "${chartcolor}${l2:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_GREEN}${netdata}${TPUT_RESET}${chartcolor}${sp:0:$((end - start - 2 - ${#netdata}))}${l2:end:$((${#l2} - end))}${TPUT_RESET}" + echo >&2 "${chartcolor}${l3:0:start}${sp:0:2}${TPUT_RESET}${TPUT_BOLD}${TPUT_CYAN}${msg}${TPUT_RESET}${chartcolor}${sp:0:2}${l3:end:$((${#l2} - end))}${TPUT_RESET}" + echo >&2 "${chartcolor}${l4}${TPUT_RESET}" + echo >&2 } # ----------------------------------------------------------------------------- # portable service command -service_cmd="$(command -v service 2>/dev/null)" -rcservice_cmd="$(command -v rc-service 2>/dev/null)" -systemctl_cmd="$(command -v systemctl 2>/dev/null)" +service_cmd="$(command -v service 2> /dev/null)" +rcservice_cmd="$(command -v rc-service 2> /dev/null)" +systemctl_cmd="$(command -v systemctl 2> /dev/null)" service() { - local cmd="${1}" action="${2}" - - if [ -n "${systemctl_cmd}" ]; then - run "${systemctl_cmd}" "${action}" "${cmd}" - return $? - elif [ -n "${service_cmd}" ]; then - run "${service_cmd}" "${cmd}" "${action}" - return $? - elif [ -n "${rcservice_cmd}" ]; then - run "${rcservice_cmd}" "${cmd}" "${action}" - return $? - fi - return 1 + local cmd="${1}" action="${2}" + + if [ -n "${systemctl_cmd}" ]; then + run "${systemctl_cmd}" "${action}" "${cmd}" + return $? + elif [ -n "${service_cmd}" ]; then + run "${service_cmd}" "${cmd}" "${action}" + return $? + elif [ -n "${rcservice_cmd}" ]; then + run "${rcservice_cmd}" "${cmd}" "${action}" + return $? + fi + return 1 } # ----------------------------------------------------------------------------- # portable pidof safe_pidof() { - local pidof_cmd="$(command -v pidof 2>/dev/null)" - if [ -n "${pidof_cmd}" ]; then - ${pidof_cmd} "${@}" - return $? - else - ps -acxo pid,comm | - sed "s/^ *//g" | - grep netdata | - cut -d ' ' -f 1 - return $? - fi + local pidof_cmd + pidof_cmd="$(command -v pidof 2> /dev/null)" + if [ -n "${pidof_cmd}" ]; then + ${pidof_cmd} "${@}" + return $? + else + ps -acxo pid,comm | + sed "s/^ *//g" | + grep netdata | + cut -d ' ' -f 1 + return $? + fi } # ----------------------------------------------------------------------------- find_processors() { - local cpus - if [ -f "/proc/cpuinfo" ]; then - # linux - cpus=$(grep -c ^processor /proc/cpuinfo) - else - # freebsd - cpus=$(sysctl hw.ncpu 2>/dev/null | grep ^hw.ncpu | cut -d ' ' -f 2) - fi - if [ -z "${cpus}" ] || [ $((cpus)) -lt 1 ]; then - echo 1 - else - echo "${cpus}" - fi + # Most UNIX systems have `nproc` as part of their userland (including macOS, Linux and BSD) + if command -v nproc > /dev/null; then + nproc && return + fi + + local cpus + if [ -f "/proc/cpuinfo" ]; then + # linux + cpus=$(grep -c ^processor /proc/cpuinfo) + else + # freebsd + cpus=$(sysctl hw.ncpu 2> /dev/null | grep ^hw.ncpu | cut -d ' ' -f 2) + fi + if [ -z "${cpus}" ] || [ $((cpus)) -lt 1 ]; then + echo 1 + else + echo "${cpus}" + fi } # ----------------------------------------------------------------------------- fatal() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" - exit 1 + printf >&2 "%s ABORTED %s %s \n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" + exit 1 } run_ok() { - printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} ${*} \n\n" + printf >&2 "%s OK %s %s \n\n" "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" } run_failed() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} ${*} \n\n" + printf >&2 "%s FAILED %s %s \n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" "${*}" } ESCAPED_PRINT_METHOD= -printf "%q " test >/dev/null 2>&1 -[ $? -eq 0 ] && ESCAPED_PRINT_METHOD="printfq" +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi escaped_print() { - if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then - printf "%q " "${@}" - else - printf "%s" "${*}" - fi - return 0 + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 } run_logfile="/dev/null" run() { - local user="${USER--}" dir="${PWD}" info info_console + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "%s" "${info}" + escaped_print "${@}" + printf "%s" " ... " + } >> "${run_logfile}" + + printf >&2 "%s" "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "%s\n" "${TPUT_RESET}" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code %s\n" "${ret}" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} +} - if [ "${UID}" = "0" ]; then - info="[root ${dir}]# " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " - else - info="[${user} ${dir}]$ " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " - fi +iscontainer() { + # man systemd-detect-virt + local cmd + cmd=$(command -v systemd-detect-virt 2> /dev/null) + if [ -n "${cmd}" ] && [ -x "${cmd}" ]; then + "${cmd}" --container > /dev/null 2>&1 && return 0 + fi + + # /proc/1/sched exposes the host's pid of our init ! + # http://stackoverflow.com/a/37016302 + local pid + pid=$(head -n 1 /proc/1/sched 2> /dev/null | { + # shellcheck disable=SC2034 + IFS='(),#:' read -r name pid th threads + echo "$pid" + }) + if [ -n "${pid}" ]; then + pid=$((pid + 0)) + [ ${pid} -gt 1 ] && return 0 + fi + + # lxc sets environment variable 'container' + # shellcheck disable=SC2154 + [ -n "${container}" ] && return 0 + + # docker creates /.dockerenv + # http://stackoverflow.com/a/25518345 + [ -f "/.dockerenv" ] && return 0 + + # ubuntu and debian supply /bin/running-in-container + # https://www.apt-browse.org/browse/ubuntu/trusty/main/i386/upstart/1.12.1-0ubuntu4/file/bin/running-in-container + if [ -x "/bin/running-in-container" ]; then + "/bin/running-in-container" > /dev/null 2>&1 && return 0 + fi + + return 1 +} - printf >>"${run_logfile}" "${info}" - escaped_print >>"${run_logfile}" "${@}" - printf >>"${run_logfile}" " ... " +get_os_key() { + if [ -f /etc/os-release ]; then + # shellcheck disable=SC1091 + source /etc/os-release || return 1 + echo "${ID}-${VERSION_ID}" + + elif [ -f /etc/redhat-release ]; then + echo "$(< /etc/redhat-release)" + else + echo "unknown" + fi +} - printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" - escaped_print >&2 "${@}" - printf >&2 "${TPUT_RESET}\n" +issystemd() { + local pids p myns ns systemctl - "${@}" + # if the directory /lib/systemd/system OR /usr/lib/systemd/system (SLES 12.x) does not exit, it is not systemd + if [ ! -d /lib/systemd/system ] && [ ! -d /usr/lib/systemd/system ]; then + return 1 + fi - local ret=$? - if [ ${ret} -ne 0 ]; then - run_failed - printf >>"${run_logfile}" "FAILED with exit code ${ret}\n" - else - run_ok - printf >>"${run_logfile}" "OK\n" - fi + # if there is no systemctl command, it is not systemd + systemctl=$(command -v systemctl 2> /dev/null) + if [ -z "${systemctl}" ] || [ ! -x "${systemctl}" ]; then + return 1 + fi - return ${ret} -} + # if pid 1 is systemd, it is systemd + [ "$(basename "$(readlink /proc/1/exe)" 2> /dev/null)" = "systemd" ] && return 0 -iscontainer() { - # man systemd-detect-virt - local cmd=$(command -v systemd-detect-virt 2>/dev/null) - if [ -n "${cmd}" ] && [ -x "${cmd}" ]; then - "${cmd}" --container >/dev/null 2>&1 && return 0 - fi - - # /proc/1/sched exposes the host's pid of our init ! - # http://stackoverflow.com/a/37016302 - local pid=$(cat /proc/1/sched 2>/dev/null | head -n 1 | { - IFS='(),#:' read name pid th threads - echo "$pid" - }) - if [ -n "${pid}" ]; then - pid=$(( pid + 0 )) - [ ${pid} -gt 1 ] && return 0 - fi - - # lxc sets environment variable 'container' - [ -n "${container}" ] && return 0 - - # docker creates /.dockerenv - # http://stackoverflow.com/a/25518345 - [ -f "/.dockerenv" ] && return 0 - - # ubuntu and debian supply /bin/running-in-container - # https://www.apt-browse.org/browse/ubuntu/trusty/main/i386/upstart/1.12.1-0ubuntu4/file/bin/running-in-container - if [ -x "/bin/running-in-container" ]; then - "/bin/running-in-container" >/dev/null 2>&1 && return 0 - fi - - return 1 -} + # if systemd is not running, it is not systemd + pids=$(safe_pidof systemd 2> /dev/null) + [ -z "${pids}" ] && return 1 -issystemd() { - local pids p myns ns systemctl + # check if the running systemd processes are not in our namespace + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + for p in ${pids}; do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" - # if the directory /lib/systemd/system OR /usr/lib/systemd/system (SLES 12.x) does not exit, it is not systemd - [ ! -d /lib/systemd/system -a ! -d /usr/lib/systemd/system ] && return 1 + # if pid of systemd is in our namespace, it is systemd + [ -n "${myns}" ] && [ "${myns}" = "${ns}" ] && return 0 + done - # if there is no systemctl command, it is not systemd - # shellcheck disable=SC2230 - systemctl=$(command -v systemctl 2>/dev/null) - [ -z "${systemctl}" -o ! -x "${systemctl}" ] && return 1 - - # if pid 1 is systemd, it is systemd - [ "$(basename $(readlink /proc/1/exe) 2>/dev/null)" = "systemd" ] && return 0 + # else, it is not systemd + return 1 +} - # if systemd is not running, it is not systemd - pids=$(safe_pidof systemd 2>/dev/null) - [ -z "${pids}" ] && return 1 +get_systemd_service_dir() { + local SYSTEMD_DIRECTORY="" + local key + key="$(get_os_key)" - # check if the running systemd processes are not in our namespace - myns="$(readlink /proc/self/ns/pid 2>/dev/null)" - for p in ${pids}; do - ns="$(readlink "/proc/${p}/ns/pid" 2>/dev/null)" + if [ -w "/lib/systemd/system" ]; then + SYSTEMD_DIRECTORY="/lib/systemd/system" + elif [ -w "/usr/lib/systemd/system" ]; then + SYSTEMD_DIRECTORY="/usr/lib/systemd/system" + elif [ -w "/etc/systemd/system" ]; then + SYSTEMD_DIRECTORY="/etc/systemd/system" + fi - # if pid of systemd is in our namespace, it is systemd - [ -n "${myns}" ] && [ "${myns}" = "${ns}" ] && return 0 - done + if [[ ${key} =~ ^devuan* ]] || [ "${key}" = "debian-7" ] || [ "${key}" = "ubuntu-12.04" ] || [ "${key}" = "ubuntu-14.04" ]; then + SYSTEMD_DIRECTORY="/etc/systemd/system" + fi - # else, it is not systemd - return 1 + echo "${SYSTEMD_DIRECTORY}" } install_non_systemd_init() { - [ "${UID}" != 0 ] && return 1 - - local key="unknown" - if [ -f /etc/os-release ]; then - source /etc/os-release || return 1 - key="${ID}-${VERSION_ID}" - - elif [ -f /etc/redhat-release ]; then - key=$(</etc/redhat-release) - fi - - if [ -d /etc/init.d ] && [ ! -f /etc/init.d/netdata ]; then - if [[ ${key} =~ ^(gentoo|alpine).* ]]; then - echo >&2 "Installing OpenRC init file..." - run cp system/netdata-openrc /etc/init.d/netdata && - run chmod 755 /etc/init.d/netdata && - run rc-update add netdata default && - return 0 - - elif [ "${key}" =~ ^devuan* ] || [ "${key}" = "debian-7" ] || [ "${key}" = "ubuntu-12.04" ] || [ "${key}" = "ubuntu-14.04" ]; then - echo >&2 "Installing LSB init file..." - run cp system/netdata-lsb /etc/init.d/netdata && - run chmod 755 /etc/init.d/netdata && - run update-rc.d netdata defaults && - run update-rc.d netdata enable && - return 0 - elif [[ ${key} =~ ^(amzn-201[5678]|ol|CentOS release 6|Red Hat Enterprise Linux Server release 6|Scientific Linux CERN SLC release 6|CloudLinux Server release 6).* ]]; then - echo >&2 "Installing init.d file..." - run cp system/netdata-init-d /etc/init.d/netdata && - run chmod 755 /etc/init.d/netdata && - run chkconfig netdata on && - return 0 - else - echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." - return 1 - fi - elif [ -f /etc/init.d/netdata ]; then - echo >&2 "file '/etc/init.d/netdata' already exists." - return 0 - else - echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." - fi - - return 1 + [ "${UID}" != 0 ] && return 1 + + local key + key="$(get_os_key)" + + if [ -d /etc/init.d ] && [ ! -f /etc/init.d/netdata ]; then + if [[ ${key} =~ ^(gentoo|alpine).* ]]; then + echo >&2 "Installing OpenRC init file..." + run cp system/netdata-openrc /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run rc-update add netdata default && + return 0 + + elif [[ ${key} =~ ^devuan* ]] || [ "${key}" = "debian-7" ] || [ "${key}" = "ubuntu-12.04" ] || [ "${key}" = "ubuntu-14.04" ]; then + echo >&2 "Installing LSB init file..." + run cp system/netdata-lsb /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run update-rc.d netdata defaults && + run update-rc.d netdata enable && + return 0 + elif [[ ${key} =~ ^(amzn-201[5678]|ol|CentOS release 6|Red Hat Enterprise Linux Server release 6|Scientific Linux CERN SLC release 6|CloudLinux Server release 6).* ]]; then + echo >&2 "Installing init.d file..." + run cp system/netdata-init-d /etc/init.d/netdata && + run chmod 755 /etc/init.d/netdata && + run chkconfig netdata on && + return 0 + else + echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." + return 1 + fi + elif [ -f /etc/init.d/netdata ]; then + echo >&2 "file '/etc/init.d/netdata' already exists." + return 0 + else + echo >&2 "I don't know what init file to install on system '${key}'. Open a github issue to help us fix it." + fi + + return 1 } +# This is used by netdata-installer.sh +# shellcheck disable=SC2034 +NETDATA_STOP_CMD="netdatacli shutdown-agent" + NETDATA_START_CMD="netdata" -NETDATA_STOP_CMD="killall netdata" -NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" -NETDATA_INSTALLER_STOP_CMD="${NETDATA_STOP_CMD}" +NETDATA_INSTALLER_START_CMD="" install_netdata_service() { - local uname="$(uname 2>/dev/null)" - - if [ "${UID}" -eq 0 ]; then - if [ "${uname}" = "Darwin" ]; then - - if [ -f "/Library/LaunchDaemons/com.github.netdata.plist" ]; then - echo >&2 "file '/Library/LaunchDaemons/com.github.netdata.plist' already exists." - return 0 - else - echo >&2 "Installing MacOS X plist file..." - run cp system/netdata.plist /Library/LaunchDaemons/com.github.netdata.plist && - run launchctl load /Library/LaunchDaemons/com.github.netdata.plist && - return 0 - fi - - elif [ "${uname}" = "FreeBSD" ]; then - - run cp system/netdata-freebsd /etc/rc.d/netdata && NETDATA_START_CMD="service netdata start" && - NETDATA_STOP_CMD="service netdata stop" && - NETDATA_INSTALLER_START_CMD="service netdata onestart" && - NETDATA_INSTALLER_STOP_CMD="${NETDATA_STOP_CMD}" - myret=$? - - echo >&2 "Note: To explicitly enable netdata automatic start, set 'netdata_enable' to 'YES' in /etc/rc.conf" - echo >&2 "" - - return ${myret} - - elif issystemd; then - # systemd is running on this system - NETDATA_START_CMD="systemctl start netdata" - NETDATA_STOP_CMD="systemctl stop netdata" - NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" - NETDATA_INSTALLER_STOP_CMD="${NETDATA_STOP_CMD}" - - SYSTEMD_DIRECTORY="" - - if [ -d "/lib/systemd/system" ]; then - SYSTEMD_DIRECTORY="/lib/systemd/system" - elif [ -d "/usr/lib/systemd/system" ]; then - SYSTEMD_DIRECTORY="/usr/lib/systemd/system" - fi - - if [ "${SYSTEMD_DIRECTORY}x" != "x" ]; then - ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="run systemctl enable netdata" - IS_NETDATA_ENABLED="$(systemctl is-enabled netdata 2> /dev/null || echo "Netdata not there")" - if [ "${IS_NETDATA_ENABLED}" == "disabled" ]; then - echo >&2 "Netdata was there and disabled, make sure we don't re-enable it ourselves" - ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="true" - fi - - echo >&2 "Installing systemd service..." - run cp system/netdata.service "${SYSTEMD_DIRECTORY}/netdata.service" && - run systemctl daemon-reload && - ${ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED} && - return 0 - else - echo >&2 "no systemd directory; cannot install netdata.service" - fi - else - install_non_systemd_init - local ret=$? - - if [ ${ret} -eq 0 ]; then - if [ -n "${service_cmd}" ]; then - NETDATA_START_CMD="service netdata start" - NETDATA_STOP_CMD="service netdata stop" - elif [ -n "${rcservice_cmd}" ]; then - NETDATA_START_CMD="rc-service netdata start" - NETDATA_STOP_CMD="rc-service netdata stop" - fi - NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" - NETDATA_INSTALLER_STOP_CMD="${NETDATA_STOP_CMD}" - fi - - return ${ret} - fi - fi - - return 1 + local uname + uname="$(uname 2> /dev/null)" + + if [ "${UID}" -eq 0 ]; then + if [ "${uname}" = "Darwin" ]; then + + if [ -f "/Library/LaunchDaemons/com.github.netdata.plist" ]; then + echo >&2 "file '/Library/LaunchDaemons/com.github.netdata.plist' already exists." + return 0 + else + echo >&2 "Installing MacOS X plist file..." + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + run cp system/netdata.plist /Library/LaunchDaemons/com.github.netdata.plist && + run launchctl load /Library/LaunchDaemons/com.github.netdata.plist && + NETDATA_START_CMD="launchctl start com.github.netdata" && + NETDATA_STOP_CMD="launchctl stop com.github.netdata" + return 0 + fi + + elif [ "${uname}" = "FreeBSD" ]; then + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + run cp system/netdata-freebsd /etc/rc.d/netdata && NETDATA_START_CMD="service netdata start" && + NETDATA_STOP_CMD="service netdata stop" && + NETDATA_INSTALLER_START_CMD="service netdata onestart" && + myret=$? + + echo >&2 "Note: To explicitly enable netdata automatic start, set 'netdata_enable' to 'YES' in /etc/rc.conf" + echo >&2 "" + + return ${myret} + + elif issystemd; then + # systemd is running on this system + NETDATA_START_CMD="systemctl start netdata" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="systemctl stop netdata" + NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" + + SYSTEMD_DIRECTORY="$(get_systemd_service_dir)" + + if [ "${SYSTEMD_DIRECTORY}x" != "x" ]; then + ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="run systemctl enable netdata" + IS_NETDATA_ENABLED="$(systemctl is-enabled netdata 2> /dev/null || echo "Netdata not there")" + if [ "${IS_NETDATA_ENABLED}" == "disabled" ]; then + echo >&2 "Netdata was there and disabled, make sure we don't re-enable it ourselves" + ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED="true" + fi + + echo >&2 "Installing systemd service..." + run cp system/netdata.service "${SYSTEMD_DIRECTORY}/netdata.service" && + run systemctl daemon-reload && + ${ENABLE_NETDATA_IF_PREVIOUSLY_ENABLED} && + return 0 + else + echo >&2 "no systemd directory; cannot install netdata.service" + fi + else + install_non_systemd_init + local ret=$? + + if [ ${ret} -eq 0 ]; then + if [ -n "${service_cmd}" ]; then + NETDATA_START_CMD="service netdata start" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="service netdata stop" + elif [ -n "${rcservice_cmd}" ]; then + NETDATA_START_CMD="rc-service netdata start" + # This is used by netdata-installer.sh + # shellcheck disable=SC2034 + NETDATA_STOP_CMD="rc-service netdata stop" + fi + NETDATA_INSTALLER_START_CMD="${NETDATA_START_CMD}" + fi + + return ${ret} + fi + fi + + return 1 } # ----------------------------------------------------------------------------- # stop netdata pidisnetdata() { - if [ -d /proc/self ]; then - [ -z "$1" -o ! -f "/proc/$1/stat" ] && return 1 - [ "$(cat "/proc/$1/stat" | cut -d '(' -f 2 | cut -d ')' -f 1)" = "netdata" ] && return 0 - return 1 - fi - return 0 + if [ -d /proc/self ]; then + if [ -z "$1" ] || [ ! -f "/proc/$1/stat" ]; then + return 1 + fi + [ "$(cut -d '(' -f 2 "/proc/$1/stat" | cut -d ')' -f 1)" = "netdata" ] && return 0 + return 1 + fi + return 0 } stop_netdata_on_pid() { - local pid="${1}" ret=0 count=0 + local pid="${1}" ret=0 count=0 - pidisnetdata "${pid}" || return 0 + pidisnetdata "${pid}" || return 0 - printf >&2 "Stopping netdata on pid %s ..." "${pid}" - while [ -n "$pid" ] && [ ${ret} -eq 0 ]; do - if [ ${count} -gt 45 ]; then - echo >&2 "Cannot stop the running netdata on pid ${pid}." - return 1 - fi + printf >&2 "Stopping netdata on pid %s ..." "${pid}" + while [ -n "$pid" ] && [ ${ret} -eq 0 ]; do + if [ ${count} -gt 24 ]; then + echo >&2 "Cannot stop the running netdata on pid ${pid}." + return 1 + fi + + count=$((count + 1)) - count=$((count + 1)) + pidisnetdata "${pid}" || ret=1 + if [ ${ret} -eq 1 ]; then + break + fi - run kill "${pid}" 2>/dev/null - ret=$? + if [ ${count} -lt 12 ]; then + run kill "${pid}" 2> /dev/null + ret=$? + else + run kill -9 "${pid}" 2> /dev/null + ret=$? + fi - test ${ret} -eq 0 && printf >&2 "." && sleep 2 + test ${ret} -eq 0 && printf >&2 "." && sleep 5 - done + done - echo >&2 - if [ ${ret} -eq 0 ]; then - echo >&2 "SORRY! CANNOT STOP netdata ON PID ${pid} !" - return 1 - fi + echo >&2 + if [ ${ret} -eq 0 ]; then + echo >&2 "SORRY! CANNOT STOP netdata ON PID ${pid} !" + return 1 + fi - echo >&2 "netdata on pid ${pid} stopped." - return 0 + echo >&2 "netdata on pid ${pid} stopped." + return 0 } netdata_pids() { - local p myns ns + local p myns ns - myns="$(readlink /proc/self/ns/pid 2>/dev/null)" + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" - for p in \ - $(cat /var/run/netdata.pid 2>/dev/null) \ - $(cat /var/run/netdata/netdata.pid 2>/dev/null) \ - $(safe_pidof netdata 2>/dev/null); do - ns="$(readlink "/proc/${p}/ns/pid" 2>/dev/null)" + for p in \ + $(cat /var/run/netdata.pid 2> /dev/null) \ + $(cat /var/run/netdata/netdata.pid 2> /dev/null) \ + $(safe_pidof netdata 2> /dev/null); do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" - if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then - pidisnetdata "${p}" && echo "${p}" - fi - done + if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then + pidisnetdata "${p}" && echo "${p}" + fi + done } stop_all_netdata() { - local p - for p in $(netdata_pids); do - # shellcheck disable=SC2086 - stop_netdata_on_pid ${p} - done + local p uname + + if [ "${UID}" -eq 0 ]; then + uname="$(uname 2> /dev/null)" + + # Any of these may fail, but we need to not bail if they do. + if issystemd; then + if systemctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "Darwin" ]; then + if launchctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "FreeBSD" ]; then + if /etc/rc.d/netdata stop; then + sleep 5 + fi + else + if service netdata stop; then + sleep 5 + fi + fi + fi + + if [ -n "$(netdata_pids)" ] && [ -n "$(builtin type -P netdatacli)" ]; then + netdatacli shutdown-agent + sleep 20 + fi + + for p in $(netdata_pids); do + # shellcheck disable=SC2086 + stop_netdata_on_pid ${p} + done } # ----------------------------------------------------------------------------- # restart netdata restart_netdata() { - local netdata="${1}" - shift - - local started=0 + local netdata="${1}" + shift - progress "Restarting netdata instance" + local started=0 - if [ "${UID}" -eq 0 ]; then - echo >&2 - echo >&2 "Stopping all netdata threads" - run stop_all_netdata + progress "Restarting netdata instance" - echo >&2 "Starting netdata using command '${NETDATA_INSTALLER_START_CMD}'" - run ${NETDATA_INSTALLER_START_CMD} && started=1 + if [ -z "${NETDATA_INSTALLER_START_CMD}" ]; then + NETDATA_INSTALLER_START_CMD="${netdata}" + fi - if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then - echo >&2 "Ooops! it seems netdata is not started." - started=0 - fi + if [ "${UID}" -eq 0 ]; then + echo >&2 + echo >&2 "Stopping all netdata threads" + run stop_all_netdata - if [ ${started} -eq 0 ]; then - echo >&2 "Attempting another netdata start using command '${NETDATA_INSTALLER_START_CMD}'" - run ${NETDATA_INSTALLER_START_CMD} && started=1 - fi - fi + echo >&2 "Starting netdata using command '${NETDATA_INSTALLER_START_CMD}'" + # shellcheck disable=SC2086 + run ${NETDATA_INSTALLER_START_CMD} && started=1 - if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then - echo >&2 "Hm... it seems netdata is still not started." - started=0 - fi - - if [ ${started} -eq 0 ]; then - # still not started... another forced attempt, just run the binary - echo >&2 "Netdata service still not started, attempting another forced restart by running '${netdata} ${@}'" - run stop_all_netdata - run "${netdata}" "${@}" - return $? - fi + if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then + echo >&2 "Ooops! it seems netdata is not started." + started=0 + fi - return 0 + if [ ${started} -eq 0 ]; then + echo >&2 "Attempting another netdata start using command '${NETDATA_INSTALLER_START_CMD}'" + # shellcheck disable=SC2086 + run ${NETDATA_INSTALLER_START_CMD} && started=1 + fi + fi + + if [ ${started} -eq 1 ] && [ -z "$(netdata_pids)" ]; then + echo >&2 "Hm... it seems netdata is still not started." + started=0 + fi + + if [ ${started} -eq 0 ]; then + # still not started... another forced attempt, just run the binary + echo >&2 "Netdata service still not started, attempting another forced restart by running '${netdata} ${*}'" + run stop_all_netdata + run "${netdata}" "${@}" + return $? + fi + + return 0 } # ----------------------------------------------------------------------------- # install netdata logrotate install_netdata_logrotate() { - if [ "${UID}" -eq 0 ]; then - if [ -d /etc/logrotate.d ]; then - if [ ! -f /etc/logrotate.d/netdata ]; then - run cp system/netdata.logrotate /etc/logrotate.d/netdata - fi + if [ "${UID}" -eq 0 ]; then + if [ -d /etc/logrotate.d ]; then + if [ ! -f /etc/logrotate.d/netdata ]; then + run cp system/netdata.logrotate /etc/logrotate.d/netdata + fi - if [ -f /etc/logrotate.d/netdata ]; then - run chmod 644 /etc/logrotate.d/netdata - fi + if [ -f /etc/logrotate.d/netdata ]; then + run chmod 644 /etc/logrotate.d/netdata + fi - return 0 - fi - fi + return 0 + fi + fi - return 1 + return 1 } # ----------------------------------------------------------------------------- # create netdata.conf create_netdata_conf() { - local path="${1}" url="${2}" + local path="${1}" url="${2}" - if [ -s "${path}" ]; then - return 0 - fi + if [ -s "${path}" ]; then + return 0 + fi - if [ -n "$url" ]; then - echo >&2 "Downloading default configuration from netdata..." - sleep 5 + if [ -n "$url" ]; then + echo >&2 "Downloading default configuration from netdata..." + sleep 5 - # remove a possibly obsolete configuration file - [ -f "${path}.new" ] && rm "${path}.new" + # remove a possibly obsolete configuration file + [ -f "${path}.new" ] && rm "${path}.new" - # disable a proxy to get data from the local netdata - export http_proxy= - export https_proxy= + # disable a proxy to get data from the local netdata + export http_proxy= + export https_proxy= - if command -v curl 1>/dev/null 2>&1; then - run curl -sSL --connect-timeout 10 --retry 3 "${url}" >"${path}.new" - elif command -v wget 1>/dev/null 2>&1; then - run wget -T 15 -O - "${url}" >"${path}.new" - fi + if command -v curl 1> /dev/null 2>&1; then + run curl -sSL --connect-timeout 10 --retry 3 "${url}" > "${path}.new" + elif command -v wget 1> /dev/null 2>&1; then + run wget -T 15 -O - "${url}" > "${path}.new" + fi - if [ -s "${path}.new" ]; then - run mv "${path}.new" "${path}" - run_ok "New configuration saved for you to edit at ${path}" - else - [ -f "${path}.new" ] && rm "${path}.new" - run_failed "Cannnot download configuration from netdata daemon using url '${url}'" - url='' - fi - fi + if [ -s "${path}.new" ]; then + run mv "${path}.new" "${path}" + run_ok "New configuration saved for you to edit at ${path}" + else + [ -f "${path}.new" ] && rm "${path}.new" + run_failed "Cannnot download configuration from netdata daemon using url '${url}'" + url='' + fi + fi - if [ -z "$url" ]; then - echo "# netdata can generate its own config which is available at 'http://<netdata_ip>/netdata.conf'" >"${path}" - echo "# You can download it with command like: 'wget -O ${path} http://localhost:19999/netdata.conf'" >>"${path}" - fi + if [ -z "$url" ]; then + echo "# netdata can generate its own config which is available at 'http://<netdata_ip>/netdata.conf'" > "${path}" + echo "# You can download it with command like: 'wget -O ${path} http://localhost:19999/netdata.conf'" >> "${path}" + fi } portable_add_user() { - local username="${1}" homedir="${2}" + local username="${1}" homedir="${2}" - [ -z "${homedir}" ] && homedir="/tmp" + [ -z "${homedir}" ] && homedir="/tmp" - # Check if user exists - if cut -d ':' -f 1 </etc/passwd | grep "^${username}$" 1>/dev/null 2>&1; then - echo >&2 "User '${username}' already exists." - return 0 - fi + # Check if user exists + if cut -d ':' -f 1 < /etc/passwd | grep "^${username}$" 1> /dev/null 2>&1; then + echo >&2 "User '${username}' already exists." + return 0 + fi - echo >&2 "Adding ${username} user account with home ${homedir} ..." + echo >&2 "Adding ${username} user account with home ${homedir} ..." - # shellcheck disable=SC2230 - local nologin="$(command -v nologin >/dev/null 2>&1 || echo '/bin/false')" + local nologin + nologin="$(command -v nologin || echo '/bin/false')" - # Linux - if command -v useradd 1>/dev/null 2>&1; then - run useradd -r -g "${username}" -c "${username}" -s "${nologin}" --no-create-home -d "${homedir}" "${username}" && return 0 - fi + # Linux + if command -v useradd 1> /dev/null 2>&1; then + run useradd -r -g "${username}" -c "${username}" -s "${nologin}" --no-create-home -d "${homedir}" "${username}" && return 0 + fi - # FreeBSD - if command -v pw 1>/dev/null 2>&1; then - run pw useradd "${username}" -d "${homedir}" -g "${username}" -s "${nologin}" && return 0 - fi + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw useradd "${username}" -d "${homedir}" -g "${username}" -s "${nologin}" && return 0 + fi - # BusyBox - if command -v adduser 1>/dev/null 2>&1; then - run adduser -h "${homedir}" -s "${nologin}" -D -G "${username}" "${username}" && return 0 - fi + # BusyBox + if command -v adduser 1> /dev/null 2>&1; then + run adduser -h "${homedir}" -s "${nologin}" -D -G "${username}" "${username}" && return 0 + fi - # mac OS - if command -v sysadminctl 1> /dev/null 2>&1; then - run sysadminctl -addUser ${username} && return 0 - fi + # mac OS + if command -v sysadminctl 1> /dev/null 2>&1; then + run sysadminctl -addUser "${username}" && return 0 + fi - echo >&2 "Failed to add ${username} user account !" + echo >&2 "Failed to add ${username} user account !" - return 1 + return 1 } portable_add_group() { - local groupname="${1}" - - # Check if group exist - if cut -d ':' -f 1 </etc/group | grep "^${groupname}$" 1>/dev/null 2>&1; then - echo >&2 "Group '${groupname}' already exists." - return 0 - fi - - echo >&2 "Adding ${groupname} user group ..." - - # Linux - if command -v groupadd 1>/dev/null 2>&1; then - run groupadd -r "${groupname}" && return 0 - fi - - # FreeBSD - if command -v pw 1>/dev/null 2>&1; then - run pw groupadd "${groupname}" && return 0 - fi + local groupname="${1}" + + # Check if group exist + if cut -d ':' -f 1 < /etc/group | grep "^${groupname}$" 1> /dev/null 2>&1; then + echo >&2 "Group '${groupname}' already exists." + return 0 + fi + + echo >&2 "Adding ${groupname} user group ..." + + # Linux + if command -v groupadd 1> /dev/null 2>&1; then + run groupadd -r "${groupname}" && return 0 + fi + + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupadd "${groupname}" && return 0 + fi + + # BusyBox + if command -v addgroup 1> /dev/null 2>&1; then + run addgroup "${groupname}" && return 0 + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + dseditgroup -o create "${groupname}" && return 0 + fi + + echo >&2 "Failed to add ${groupname} user group !" + return 1 +} - # BusyBox - if command -v addgroup 1>/dev/null 2>&1; then - run addgroup "${groupname}" && return 0 - fi +portable_add_user_to_group() { + local groupname="${1}" username="${2}" + + # Check if group exist + if ! cut -d ':' -f 1 < /etc/group | grep "^${groupname}$" > /dev/null 2>&1; then + echo >&2 "Group '${groupname}' does not exist." + return 1 + fi + + # Check if user is in group + if [[ ",$(grep "^${groupname}:" < /etc/group | cut -d ':' -f 4)," =~ ,${username}, ]]; then + # username is already there + echo >&2 "User '${username}' is already in group '${groupname}'." + return 0 + else + # username is not in group + echo >&2 "Adding ${username} user to the ${groupname} group ..." + + # Linux + if command -v usermod 1> /dev/null 2>&1; then + run usermod -a -G "${groupname}" "${username}" && return 0 + fi - # mac OS - if command -v dseditgroup 1> /dev/null 2>&1; then - dseditgroup -o create "${groupname}" && return 0 - fi + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupmod "${groupname}" -m "${username}" && return 0 + fi - echo >&2 "Failed to add ${groupname} user group !" - return 1 -} + # BusyBox + if command -v addgroup 1> /dev/null 2>&1; then + run addgroup "${username}" "${groupname}" && return 0 + fi -portable_add_user_to_group() { - local groupname="${1}" username="${2}" - - # Check if group exist - if ! cut -d ':' -f 1 </etc/group | grep "^${groupname}$" >/dev/null 2>&1; then - echo >&2 "Group '${groupname}' does not exist." - return 1 - fi - - # Check if user is in group - if [[ ",$(grep "^${groupname}:" </etc/group | cut -d ':' -f 4)," =~ ,${username}, ]]; then - # username is already there - echo >&2 "User '${username}' is already in group '${groupname}'." - return 0 - else - # username is not in group - echo >&2 "Adding ${username} user to the ${groupname} group ..." - - # Linux - if command -v usermod 1>/dev/null 2>&1; then - run usermod -a -G "${groupname}" "${username}" && return 0 - fi - - # FreeBSD - if command -v pw 1>/dev/null 2>&1; then - run pw groupmod "${groupname}" -m "${username}" && return 0 - fi - - # BusyBox - if command -v addgroup 1>/dev/null 2>&1; then - run addgroup "${username}" "${groupname}" && return 0 - fi - - # mac OS - if command -v dseditgroup 1> /dev/null 2>&1; then - dseditgroup -u "${username}" "${groupname}" && return 0 - fi - echo >&2 "Failed to add user ${username} to group ${groupname} !" - return 1 - fi + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + dseditgroup -u "${username}" "${groupname}" && return 0 + fi + echo >&2 "Failed to add user ${username} to group ${groupname} !" + return 1 + fi } - safe_sha256sum() { - # Within the contexct of the installer, we only use -c option that is common between the two commands - # We will have to reconsider if we start non-common options - if command -v sha256sum >/dev/null 2>&1; then - sha256sum $@ - elif command -v shasum >/dev/null 2>&1; then - shasum -a 256 $@ - else - fatal "I could not find a suitable checksum binary to use" - fi + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi } -_get_crondir() { - if [ -d /etc/cron.daily ]; then - echo /etc/cron.daily - elif [ -d /etc/periodic/daily ]; then - echo /etc/periodic/daily - else - echo >&2 "Cannot figure out the cron directory to handle netdata-updater.sh activation/deactivation" - return 1 - fi - - return 0 +_get_scheduler_type() { + if _get_intervaldir > /dev/null ; then + echo 'interval' + elif issystemd ; then + echo 'systemd' + elif [ -d /etc/cron.d ] ; then + echo 'crontab' + else + echo 'none' + fi } -_check_crondir_permissions() { - if [ "${UID}" -ne "0" ]; then - # We cant touch cron if we are not running as root - echo >&2 "You need to run the installer as root for auto-updating via cron" - return 1 - fi +_get_intervaldir() { + if [ -d /etc/cron.daily ]; then + echo /etc/cron.daily + elif [ -d /etc/periodic/daily ]; then + echo /etc/periodic/daily + else + return 1 + fi - return 0 + return 0 } install_netdata_updater() { - if [ "${INSTALLER_DIR}" ] && [ -f "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" ]; then - cat "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 - fi + if [ "${INSTALLER_DIR}" ] && [ -f "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" ]; then + cat "${INSTALLER_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + fi - if [ "${NETDATA_SOURCE_DIR}" ] && [ -f "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" ]; then - cat "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 - fi + if [ "${NETDATA_SOURCE_DIR}" ] && [ -f "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" ]; then + cat "${NETDATA_SOURCE_DIR}/packaging/installer/netdata-updater.sh" > "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + fi - sed -e "s|THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT|${NETDATA_USER_CONFIG_DIR}/.environment|" -i "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 + if issystemd && [ -n "$(get_systemd_service_dir)" ]; then + cat "${NETDATA_SOURCE_DIR}/system/netdata-updater.timer" > "$(get_systemd_service_dir)/netdata-updater.timer" + cat "${NETDATA_SOURCE_DIR}/system/netdata-updater.service" > "$(get_systemd_service_dir)/netdata-updater.service" + fi - chmod 0755 ${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh - echo >&2 "Update script is located at ${TPUT_GREEN}${TPUT_BOLD}${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh${TPUT_RESET}" - echo >&2 + sed -i -e "s|THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT|${NETDATA_USER_CONFIG_DIR}/.environment|" "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" || return 1 - return 0 + chmod 0755 "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" + echo >&2 "Update script is located at ${TPUT_GREEN}${TPUT_BOLD}${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh${TPUT_RESET}" + echo >&2 + + return 0 } cleanup_old_netdata_updater() { - if [ -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh ]; then - echo >&2 "Removing updater from deprecated location" - rm -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh - fi - - crondir="$(_get_crondir)" || return 1 - _check_crondir_permissions "${crondir}" || return 1 - - if [ -f "${crondir}/netdata-updater.sh" ]; then - echo >&2 "Removing incorrect netdata-updater filename in cron" - rm -f "${crondir}/netdata-updater.sh" - fi - - return 0 + if [ -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh ]; then + echo >&2 "Removing updater from deprecated location" + rm -f "${NETDATA_PREFIX}"/usr/libexec/netdata-updater.sh + fi + + if issystemd && [ -n "$(get_systemd_service_dir)" ] ; then + systemctl disable netdata-updater.timer + rm -f "$(get_systemd_service_dir)/netdata-updater.timer" + rm -f "$(get_systemd_service_dir)/netdata-updater.service" + fi + + if [ -d /etc/cron.daily ]; then + rm -f /etc/cron.daily/netdata-updater.sh + rm -f /etc/cron.daily/netdata-updater + fi + + if [ -d /etc/periodic/daily ]; then + rm -f /etc/periodic/daily/netdata-updater.sh + rm -f /etc/periodic/daily/netdata-updater + fi + + if [ -d /etc/cron.d ]; then + rm -f /etc/cron.d/netdata-updater + fi + + return 0 } enable_netdata_updater() { - crondir="$(_get_crondir)" || return 1 - _check_crondir_permissions "${crondir}" || return 1 - - echo >&2 "Adding to cron" - - rm -f "${crondir}/netdata-updater" - ln -sf "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" "${crondir}/netdata-updater" - - echo >&2 "Auto-updating has been enabled. Updater script linked to: ${TPUT_RED}${TPUT_BOLD}${crondir}/netdata-update${TPUT_RESET}" - echo >&2 - echo >&2 "${TPUT_DIM}${TPUT_BOLD}netdata-updater.sh${TPUT_RESET}${TPUT_DIM} works from cron. It will trigger an email from cron" - echo >&2 "only if it fails (it should not print anything when it can update netdata).${TPUT_RESET}" - echo >&2 - - return 0 + local updater_type + + if [ -n "${1}" ] ; then + updater_type="${1}" + else + updater_type="$(_get_scheduler_type)" + fi + + case "${updater_type}" in + "systemd") + systemctl enable netdata-updater.timer + + echo >&2 "Auto-updating has been enabled using a systemd timer unit." + echo >&2 + echo >&2 "If the update process fails, the failure will be logged to the systemd journal just like a regular service failure." + echo >&2 "Successful updates should produce empty logs." + echo >&2 + ;; + "interval") + ln -sf "${NETDATA_PREFIX}/usr/libexec/netdata/netdata-updater.sh" "$(_get_intervaldir)/netdata-updater" + + echo >&2 "Auto-updating has been enabled through cron, updater script linked to ${TPUT_RED}${TPUT_BOLD}$(_get_intervaldir)/netdata-updater${TPUT_RESET}" + echo >&2 + echo >&2 "If the update process fails and you have email notifications set up correctly for cron on this system, you should receive an email notification of the failure." + echo >&2 "Successful updates will not send an email." + echo >&2 + ;; + "crontab") + cat "${NETDATA_SOURCE_DIR}/system/netdata.crontab" > "/etc/cron.d/netdata-updater" + + echo >&2 "Auto-updating has been enabled through cron, using a crontab at ${TPUT_RED}${TPUT_BOLD}/etc/cron.d/netdata-updater${TPUT_RESET}" + echo >&2 + echo >&2 "If the update process fails and you have email notifications set up correctly for cron on this system, you should receive an email notification of the failure." + echo >&2 "Successful updates will not send an email." + echo >&2 + ;; + *) + echo >&2 "Unable to determine what type of auto-update scheduling to use, not enabling auto-updates." + echo >&2 + return 1 + esac + + return 0 } disable_netdata_updater() { - crondir="$(_get_crondir)" || return 1 - _check_crondir_permissions "${crondir}" || return 1 + echo >&2 "You chose *NOT* to enable auto-update, removing any links to the updater from cron (it may have happened if you are reinstalling)" + echo >&2 + + if issystemd && [ -n "$(get_systemd_service_dir)" ] ; then + systemctl disable netdata-updater.timer + fi + + if [ -d /etc/cron.daily ]; then + rm -f /etc/cron.daily/netdata-updater.sh + rm -f /etc/cron.daily/netdata-updater + fi - echo >&2 "You chose *NOT* to enable auto-update, removing any links to the updater from cron (it may have happened if you are reinstalling)" - echo >&2 + if [ -d /etc/periodic/daily ]; then + rm -f /etc/periodic/daily/netdata-updater.sh + rm -f /etc/periodic/daily/netdata-updater + fi - if [ -f "${crondir}/netdata-updater" ]; then - echo >&2 "Removing cron reference: ${crondir}/netdata-updater" - echo >&2 - rm -f "${crondir}/netdata-updater" - else - echo >&2 "Did not find any cron entries to remove" - echo >&2 - fi + if [ -d /etc/cron.d ]; then + rm -f /etc/cron.d/netdata-updater + fi - return 0 + return 0 } set_netdata_updater_channel() { - sed -e "s/^RELEASE_CHANNEL=.*/RELEASE_CHANNEL=\"${RELEASE_CHANNEL}\"/" -i "${NETDATA_USER_CONFIG_DIR}/.environment" + sed -i -e "s/^RELEASE_CHANNEL=.*/RELEASE_CHANNEL=\"${RELEASE_CHANNEL}\"/" "${NETDATA_USER_CONFIG_DIR}/.environment" } diff --git a/packaging/installer/install-required-packages.sh b/packaging/installer/install-required-packages.sh new file mode 100755 index 00000000..23cbe12d --- /dev/null +++ b/packaging/installer/install-required-packages.sh @@ -0,0 +1,2175 @@ +#!/usr/bin/env bash +# shellcheck disable=SC2034 +# We use lots of computed variable names in here, so we need to disable shellcheck 2034 + +export PATH="${PATH}:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin" +export LC_ALL=C + +# Be nice on production environments +renice 19 $$ > /dev/null 2> /dev/null + +ME="${0}" + +if [ "${BASH_VERSINFO[0]}" -lt "4" ]; then + echo >&2 "Sorry! This script needs BASH version 4+, but you have BASH version ${BASH_VERSION}" + exit 1 +fi + +# These options control which packages we are going to install +# They can be pre-set, but also can be controlled with command line options +PACKAGES_NETDATA=${PACKAGES_NETDATA-0} +PACKAGES_NETDATA_NODEJS=${PACKAGES_NETDATA_NODEJS-0} +PACKAGES_NETDATA_PYTHON=${PACKAGES_NETDATA_PYTHON-0} +PACKAGES_NETDATA_PYTHON3=${PACKAGES_NETDATA_PYTHON3-0} +PACKAGES_NETDATA_PYTHON_MYSQL=${PACKAGES_NETDATA_PYTHON_MYSQL-0} +PACKAGES_NETDATA_PYTHON_POSTGRES=${PACKAGES_NETDATA_PYTHON_POSTGRES-0} +PACKAGES_NETDATA_PYTHON_MONGO=${PACKAGES_NETDATA_PYTHON_MONGO-0} +PACKAGES_DEBUG=${PACKAGES_DEBUG-0} +PACKAGES_IPRANGE=${PACKAGES_IPRANGE-0} +PACKAGES_FIREHOL=${PACKAGES_FIREHOL-0} +PACKAGES_FIREQOS=${PACKAGES_FIREQOS-0} +PACKAGES_UPDATE_IPSETS=${PACKAGES_UPDATE_IPSETS-0} +PACKAGES_NETDATA_DEMO_SITE=${PACKAGES_NETDATA_DEMO_SITE-0} +PACKAGES_NETDATA_SENSORS=${PACKAGES_NETDATA_SENSORS-0} +PACKAGES_NETDATA_DATABASE=${PACKAGES_NETDATA_DATABASE-0} +PACKAGES_NETDATA_EBPF=${PACKAGES_NETDATA_EBPF-0} + +# needed commands +lsb_release=$(command -v lsb_release 2> /dev/null) + +# Check which package managers are available +apk=$(command -v apk 2> /dev/null) +apt_get=$(command -v apt-get 2> /dev/null) +brew=$(command -v brew 2> /dev/null) +pkg=$(command -v pkg 2> /dev/null) +dnf=$(command -v dnf 2> /dev/null) +emerge=$(command -v emerge 2> /dev/null) +equo=$(command -v equo 2> /dev/null) +pacman=$(command -v pacman 2> /dev/null) +swupd=$(command -v swupd 2> /dev/null) +yum=$(command -v yum 2> /dev/null) +zypper=$(command -v zypper 2> /dev/null) + +distribution= +release= +version= +codename= +package_installer= +tree= +detection= +NAME= +ID= +ID_LIKE= +VERSION= +VERSION_ID= + +usage() { + cat << EOF +OPTIONS: + +${ME} [--dont-wait] [--non-interactive] \\ + [distribution DD [version VV] [codename CN]] [installer IN] [packages] + +Supported distributions (DD): + + - arch (all Arch Linux derivatives) + - centos (all CentOS derivatives) + - gentoo (all Gentoo Linux derivatives) + - sabayon (all Sabayon Linux derivatives) + - debian, ubuntu (all Debian and Ubuntu derivatives) + - redhat, fedora (all Red Hat and Fedora derivatives) + - suse, opensuse (all SUSE and openSUSE derivatives) + - clearlinux (all Clear Linux derivatives) + - macos (Apple's macOS) + +Supported installers (IN): + + - apt-get all Debian / Ubuntu Linux derivatives + - dnf newer Red Hat / Fedora Linux + - emerge all Gentoo Linux derivatives + - equo all Sabayon Linux derivatives + - pacman all Arch Linux derivatives + - yum all Red Hat / Fedora / CentOS Linux derivatives + - zypper all SUSE Linux derivatives + - apk all Alpine derivatives + - swupd all Clear Linux derivatives + - brew macOS Homebrew + - pkg FreeBSD Ports + +Supported packages (you can append many of them): + + - netdata-all all packages required to install netdata + including mysql client, postgres client, + node.js, python, sensors, etc + + - netdata minimum packages required to install netdata + (no mysql client, no nodejs, includes python) + + - nodejs install nodejs + (required for monitoring named and SNMP) + + - python install python + + - python3 install python3 + + - python-mysql install MySQLdb + (for monitoring mysql, will install python3 version + if python3 is enabled or detected) + + - python-postgres install psycopg2 + (for monitoring postgres, will install python3 version + if python3 is enabled or detected) + + - python-pymongo install python-pymongo (or python3-pymongo for python3) + + - sensors install lm_sensors for monitoring h/w sensors + + - firehol-all packages required for FireHOL, FireQoS, update-ipsets + - firehol packages required for FireHOL + - fireqos packages required for FireQoS + - update-ipsets packages required for update-ipsets + + - demo packages required for running a netdata demo site + (includes nginx and various debugging tools) + + +If you don't supply the --dont-wait option, the program +will ask you before touching your system. + +EOF +} + +release2lsb_release() { + # loads the given /etc/x-release file + # this file is normaly a single line containing something like + # + # X Linux release 1.2.3 (release-name) + # + # It attempts to parse it + # If it succeeds, it returns 0 + # otherwise it returns 1 + + local file="${1}" x DISTRIB_ID="" DISTRIB_RELEASE="" DISTRIB_CODENAME="" + echo >&2 "Loading ${file} ..." + + x="$(grep -v "^$" "${file}" | head -n 1)" + + if [[ "${x}" =~ ^.*[[:space:]]+Linux[[:space:]]+release[[:space:]]+.*[[:space:]]+(.*)[[:space:]]*$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+Linux[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]\+(\(.*\))[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"\nDISTRIB_CODENAME=\"\3\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+Linux[[:space:]]+release[[:space:]]+.*[[:space:]]+$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+Linux[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+release[[:space:]]+.*[[:space:]]+(.*)[[:space:]]*$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]\+(\(.*\))[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"\nDISTRIB_CODENAME=\"\3\"|g" | grep "^DISTRIB")" + elif [[ "${x}" =~ ^.*[[:space:]]+release[[:space:]]+.*[[:space:]]+$ ]]; then + eval "$(echo "${x}" | sed "s|^\(.*\)[[:space:]]\+release[[:space:]]\+\(.*\)[[:space:]]*$|DISTRIB_ID=\"\1\"\nDISTRIB_RELEASE=\"\2\"|g" | grep "^DISTRIB")" + fi + + distribution="${DISTRIB_ID}" + version="${DISTRIB_RELEASE}" + codename="${DISTRIB_CODENAME}" + + [ -z "${distribution}" ] && echo >&2 "Cannot parse this lsb-release: ${x}" && return 1 + detection="${file}" + return 0 +} + +get_os_release() { + # Loads the /etc/os-release or /usr/lib/os-release file(s) + # Only the required fields are loaded + # + # If it manages to load a valid os-release, it returns 0 + # otherwise it returns 1 + # + # It searches the ID_LIKE field for a compatible distribution + + os_release_file= + if [ -s "/etc/os-release" ]; then + os_release_file="/etc/os-release" + elif [ -s "/usr/lib/os-release" ]; then + os_release_file="/usr/lib/os-release" + else + echo >&2 "Cannot find an os-release file ..." + return 1 + fi + + local x + echo >&2 "Loading ${os_release_file} ..." + + eval "$(grep -E "^(NAME|ID|ID_LIKE|VERSION|VERSION_ID)=" "${os_release_file}")" + for x in "${ID}" ${ID_LIKE}; do + case "${x,,}" in + alpine | arch | centos | clear-linux-os | debian | fedora | gentoo | manjaro | opensuse-leap | rhel | sabayon | sles | suse | ubuntu) + distribution="${x}" + version="${VERSION_ID}" + codename="${VERSION}" + detection="${os_release_file}" + break + ;; + *) + echo >&2 "Unknown distribution ID: ${x}" + ;; + esac + done + [ -z "${distribution}" ] && echo >&2 "Cannot find valid distribution in: ${ID} ${ID_LIKE}" && return 1 + + [ -z "${distribution}" ] && return 1 + return 0 +} + +get_lsb_release() { + # Loads the /etc/lsb-release file + # If it fails, it attempts to run the command: lsb_release -a + # and parse its output + # + # If it manages to find the lsb-release, it returns 0 + # otherwise it returns 1 + + if [ -f "/etc/lsb-release" ]; then + echo >&2 "Loading /etc/lsb-release ..." + local DISTRIB_ID="" DISTRIB_RELEASE="" DISTRIB_CODENAME="" + eval "$(grep -E "^(DISTRIB_ID|DISTRIB_RELEASE|DISTRIB_CODENAME)=" /etc/lsb-release)" + distribution="${DISTRIB_ID}" + version="${DISTRIB_RELEASE}" + codename="${DISTRIB_CODENAME}" + detection="/etc/lsb-release" + fi + + if [ -z "${distribution}" ] && [ -n "${lsb_release}" ]; then + echo >&2 "Cannot find distribution with /etc/lsb-release" + echo >&2 "Running command: lsb_release ..." + eval "declare -A release=( $(lsb_release -a 2> /dev/null | sed -e "s|^\(.*\):[[:space:]]*\(.*\)$|[\1]=\"\2\"|g") )" + distribution="${release["Distributor ID"]}" + version="${release[Release]}" + codename="${release[Codename]}" + detection="lsb_release" + fi + + [ -z "${distribution}" ] && echo >&2 "Cannot find valid distribution with lsb-release" && return 1 + return 0 +} + +find_etc_any_release() { + # Check for any of the known /etc/x-release files + # If it finds one, it loads it and returns 0 + # otherwise it returns 1 + + if [ -f "/etc/arch-release" ]; then + release2lsb_release "/etc/arch-release" && return 0 + fi + + if [ -f "/etc/centos-release" ]; then + release2lsb_release "/etc/centos-release" && return 0 + fi + + if [ -f "/etc/redhat-release" ]; then + release2lsb_release "/etc/redhat-release" && return 0 + fi + + if [ -f "/etc/SuSe-release" ]; then + release2lsb_release "/etc/SuSe-release" && return 0 + fi + + return 1 +} + +autodetect_distribution() { + # autodetection of distribution/OS + case "$(uname -s)" in + "Linux") + get_os_release || get_lsb_release || find_etc_any_release + ;; + "FreeBSD") + distribution="freebsd" + version="$(uname -r)" + detection="uname" + ;; + "Darwin") + distribution="macos" + version="$(uname -r)" + detection="uname" + + if [ ${EUID} -eq 0 ]; then + echo >&2 "This script does not support running as EUID 0 on macOS. Please run it as a regular user." + exit 1 + fi + ;; + *) + return 1 + ;; + esac +} + +user_picks_distribution() { + # let the user pick a distribution + + echo >&2 + echo >&2 "I NEED YOUR HELP" + echo >&2 "It seems I cannot detect your system automatically." + + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + echo >&2 " > Bailing out..." + exit 1 + fi + + if [ -z "${equo}" ] && [ -z "${emerge}" ] && [ -z "${apt_get}" ] && [ -z "${yum}" ] && [ -z "${dnf}" ] && [ -z "${pacman}" ] && [ -z "${apk}" ] && [ -z "${swupd}" ]; then + echo >&2 "And it seems I cannot find a known package manager in this system." + echo >&2 "Please open a github issue to help us support your system too." + exit 1 + fi + + local opts= + echo >&2 "I found though that the following installers are available:" + echo >&2 + [ -n "${apt_get}" ] && echo >&2 " - Debian/Ubuntu based (installer is: apt-get)" && opts="apt-get ${opts}" + [ -n "${yum}" ] && echo >&2 " - Redhat/Fedora/Centos based (installer is: yum)" && opts="yum ${opts}" + [ -n "${dnf}" ] && echo >&2 " - Redhat/Fedora/Centos based (installer is: dnf)" && opts="dnf ${opts}" + [ -n "${zypper}" ] && echo >&2 " - SuSe based (installer is: zypper)" && opts="zypper ${opts}" + [ -n "${pacman}" ] && echo >&2 " - Arch Linux based (installer is: pacman)" && opts="pacman ${opts}" + [ -n "${emerge}" ] && echo >&2 " - Gentoo based (installer is: emerge)" && opts="emerge ${opts}" + [ -n "${equo}" ] && echo >&2 " - Sabayon based (installer is: equo)" && opts="equo ${opts}" + [ -n "${apk}" ] && echo >&2 " - Alpine Linux based (installer is: apk)" && opts="apk ${opts}" + [ -n "${swupd}" ] && echo >&2 " - Clear Linux based (installer is: swupd)" && opts="swupd ${opts}" + [ -n "${brew}" ] && echo >&2 " - macOS based (installer is: brew)" && opts="brew ${opts}" + # XXX: This is being removed in another PR. + echo >&2 + + REPLY= + while [ -z "${REPLY}" ]; do + echo "To proceed please write one of these:" + echo "${opts// /, }" + if ! read -r -p ">" REPLY; then + continue + fi + + if [ "${REPLY}" = "yum" ] && [ -z "${distribution}" ]; then + REPLY= + while [ -z "${REPLY}" ]; do + if ! read -r -p "yum in centos, rhel or fedora? > "; then + continue + fi + + case "${REPLY,,}" in + fedora | rhel) + distribution="rhel" + ;; + centos) + distribution="centos" + ;; + *) + echo >&2 "Please enter 'centos', 'fedora' or 'rhel'." + REPLY= + ;; + esac + done + REPLY="yum" + fi + check_package_manager "${REPLY}" || REPLY= + done +} + +detect_package_manager_from_distribution() { + case "${1,,}" in + arch* | manjaro*) + package_installer="install_pacman" + tree="arch" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pacman}" ]; then + echo >&2 "command 'pacman' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + sabayon*) + package_installer="install_equo" + tree="sabayon" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${equo}" ]; then + echo >&2 "command 'equo' is required to install packages on a '${distribution} ${version}' system." + # Maybe offer to fall back on emerge? Both installers exist in Sabayon... + exit 1 + fi + ;; + + alpine*) + package_installer="install_apk" + tree="alpine" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apk}" ]; then + echo >&2 "command 'apk' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + gentoo*) + package_installer="install_emerge" + tree="gentoo" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${emerge}" ]; then + echo >&2 "command 'emerge' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + debian* | ubuntu*) + package_installer="install_apt_get" + tree="debian" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apt_get}" ]; then + echo >&2 "command 'apt-get' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + centos* | clearos*) + echo >&2 "You should have EPEL enabled to install all the prerequisites." + echo >&2 "Check: http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/" + package_installer="install_yum" + tree="centos" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${yum}" ]; then + echo >&2 "command 'yum' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + fedora* | redhat* | red\ hat* | rhel*) + package_installer= + tree="rhel" + [ -n "${yum}" ] && package_installer="install_yum" + [ -n "${dnf}" ] && package_installer="install_dnf" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${package_installer}" ]; then + echo >&2 "command 'yum' or 'dnf' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + suse* | opensuse* | sles*) + package_installer="install_zypper" + tree="suse" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${zypper}" ]; then + echo >&2 "command 'zypper' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + clear-linux* | clearlinux*) + package_installer="install_swupd" + tree="clearlinux" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${swupd}" ]; then + echo >&2 "command 'swupd' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + freebsd) + package_installer="install_pkg" + tree="freebsd" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pkg}" ]; then + echo >&2 "command 'pkg' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + macos) + package_installer="install_brew" + tree="macos" + if [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${brew}" ]; then + echo >&2 "command 'brew' is required to install packages on a '${distribution} ${version}' system." + exit 1 + fi + ;; + + *) + # oops! unknown system + user_picks_distribution + ;; + esac +} + +# XXX: This is being removed in another PR. +check_package_manager() { + # This is called only when the user is selecting a package manager + # It is used to verify the user selection is right + + echo >&2 "Checking package manager: ${1}" + + case "${1}" in + apt-get) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apt_get}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_apt_get" + tree="debian" + detection="user-input" + return 0 + ;; + + dnf) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${dnf}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_dnf" + tree="rhel" + detection="user-input" + return 0 + ;; + + apk) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${apk}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_apk" + tree="alpine" + detection="user-input" + return 0 + ;; + + equo) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${equo}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_equo" + tree="sabayon" + detection="user-input" + return 0 + ;; + + emerge) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${emerge}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_emerge" + tree="gentoo" + detection="user-input" + return 0 + ;; + + pacman) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${pacman}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_pacman" + tree="arch" + detection="user-input" + + return 0 + ;; + + zypper) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${zypper}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_zypper" + tree="suse" + detection="user-input" + return 0 + ;; + + yum) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${yum}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_yum" + if [ "${distribution}" = "centos" ]; then + tree="centos" + else + tree="rhel" + fi + detection="user-input" + return 0 + ;; + + swupd) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${swupd}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_swupd" + tree="clear-linux" + detection="user-input" + return 0 + ;; + + brew) + [ "${IGNORE_INSTALLED}" -eq 0 ] && [ -z "${brew}" ] && echo >&2 "${1} is not available." && return 1 + package_installer="install_brew" + tree="macos" + detection="user-input" + + return 0 + ;; + + *) + echo >&2 "Invalid package manager: '${1}'." + return 1 + ;; + esac +} + +require_cmd() { + # check if any of the commands given as argument + # are present on this system + # If any of them is available, it returns 0 + # otherwise 1 + + [ "${IGNORE_INSTALLED}" -eq 1 ] && return 1 + + local wanted found + for wanted in "${@}"; do + if command -v "${wanted}" > /dev/null 2>&1; then + found="$(command -v "$wanted" 2> /dev/null)" + fi + [ -n "${found}" ] && [ -x "${found}" ] && return 0 + done + return 1 +} + +declare -A pkg_find=( + ['gentoo']="sys-apps/findutils" + ['fedora']="findutils" + ['clearlinux']="findutils" + ['macos']="NOTREQUIRED" + ['freebsd']="NOTREQUIRED" + ['default']="WARNING|" +) + +declare -A pkg_distro_sdk=( + ['alpine']="alpine-sdk" + ['default']="NOTREQUIRED" +) + +declare -A pkg_autoconf=( + ['gentoo']="sys-devel/autoconf" + ['clearlinux']="c-basic" + ['default']="autoconf" +) + +# required to compile netdata with --enable-sse +# https://github.com/firehol/netdata/pull/450 +declare -A pkg_autoconf_archive=( + ['gentoo']="sys-devel/autoconf-archive" + ['clearlinux']="c-basic" + ['alpine']="WARNING|" + ['default']="autoconf-archive" + + # exceptions + ['centos-6']="WARNING|" + ['rhel-6']="WARNING|" + ['rhel-7']="WARNING|" +) + +declare -A pkg_autogen=( + ['gentoo']="sys-devel/autogen" + ['clearlinux']="c-basic" + ['alpine']="WARNING|" + ['default']="autogen" + + # exceptions + ['centos-6']="WARNING|" + ['rhel-6']="WARNING|" +) + +declare -A pkg_automake=( + ['gentoo']="sys-devel/automake" + ['clearlinux']="c-basic" + ['default']="automake" +) + +# required to bundle libJudy +declare -A pkg_libtool=( + ['gentoo']="sys-devel/libtool" + ['clearlinux']="c-basic" + ['default']="libtool" +) + +# Required to build libwebsockets and libmosquitto on some systems. +declare -A pkg_cmake=( + ['gentoo']="dev-util/cmake" + ['clearlinux']="c-basic" + ['default']="cmake" +) + +declare -A pkg_json_c_dev=( + ['alpine']="json-c-dev" + ['arch']="json-c" + ['clearlinux']="devpkg-json-c" + ['debian']="libjson-c-dev" + ['gentoo']="dev-libs/json-c" + ['sabayon']="dev-libs/json-c" + ['suse']="libjson-c-devel" + ['freebsd']="json-c" + ['default']="json-c-devel" +) + +declare -A pkg_bridge_utils=( + ['gentoo']="net-misc/bridge-utils" + ['clearlinux']="network-basic" + ['macos']="WARNING|" + ['default']="bridge-utils" +) + +declare -A pkg_chrony=( + ['gentoo']="net-misc/chrony" + ['clearlinux']="time-server-basic" + ['macos']="WARNING|" + ['default']="chrony" +) + +declare -A pkg_curl=( + ['gentoo']="net-misc/curl" + ['sabayon']="net-misc/curl" + ['default']="curl" +) + +declare -A pkg_gzip=( + ['gentoo']="app-arch/gzip" + ['macos']="NOTREQUIRED" + ['default']="gzip" +) + +declare -A pkg_tar=( + ['gentoo']="app-arch/tar" + ['clearlinux']="os-core-update" + ['macos']="NOTREQUIRED" + ['default']="tar" +) + +declare -A pkg_git=( + ['gentoo']="dev-vcs/git" + ['default']="git" +) + +declare -A pkg_gcc=( + ['gentoo']="sys-devel/gcc" + ['clearlinux']="c-basic" + ['macos']="NOTREQUIRED" + ['default']="gcc" +) + +declare -A pkg_gdb=( + ['gentoo']="sys-devel/gdb" + ['macos']="NOTREQUIRED" + ['default']="gdb" +) + +declare -A pkg_iotop=( + ['gentoo']="sys-process/iotop" + ['macos']="WARNING|" + ['default']="iotop" +) + +declare -A pkg_iproute2=( + ['alpine']="iproute2" + ['debian']="iproute2" + ['gentoo']="sys-apps/iproute2" + ['sabayon']="sys-apps/iproute2" + ['clearlinux']="iproute2" + ['macos']="WARNING|" + ['default']="iproute" + + # exceptions + ['ubuntu-12.04']="iproute" +) + +declare -A pkg_ipset=( + ['gentoo']="net-firewall/ipset" + ['clearlinux']="network-basic" + ['macos']="WARNING|" + ['default']="ipset" +) + +declare -A pkg_jq=( + ['gentoo']="app-misc/jq" + ['default']="jq" +) + +declare -A pkg_iptables=( + ['gentoo']="net-firewall/iptables" + ['macos']="WARNING|" + ['default']="iptables" +) + +declare -A pkg_libz_dev=( + ['alpine']="zlib-dev" + ['arch']="zlib" + ['centos']="zlib-devel" + ['debian']="zlib1g-dev" + ['gentoo']="sys-libs/zlib" + ['sabayon']="sys-libs/zlib" + ['rhel']="zlib-devel" + ['suse']="zlib-devel" + ['clearlinux']="devpkg-zlib" + ['macos']="NOTREQUIRED" + ['freebsd']="lzlib" + ['default']="" +) + +declare -A pkg_libuuid_dev=( + ['alpine']="util-linux-dev" + ['arch']="util-linux" + ['centos']="libuuid-devel" + ['clearlinux']="devpkg-util-linux" + ['debian']="uuid-dev" + ['gentoo']="sys-apps/util-linux" + ['sabayon']="sys-apps/util-linux" + ['rhel']="libuuid-devel" + ['suse']="libuuid-devel" + ['macos']="NOTREQUIRED" + ['freebsd']="e2fsprogs-libuuid" + ['default']="" +) + +declare -A pkg_libmnl_dev=( + ['alpine']="libmnl-dev" + ['arch']="libmnl" + ['centos']="libmnl-devel" + ['debian']="libmnl-dev" + ['gentoo']="net-libs/libmnl" + ['sabayon']="net-libs/libmnl" + ['rhel']="libmnl-devel" + ['suse']="libmnl-devel" + ['clearlinux']="devpkg-libmnl" + ['macos']="NOTREQUIRED" + ['default']="" +) + +declare -A pkg_lm_sensors=( + ['alpine']="lm_sensors" + ['arch']="lm_sensors" + ['centos']="lm_sensors" + ['debian']="lm-sensors" + ['gentoo']="sys-apps/lm-sensors" + ['sabayon']="sys-apps/lm_sensors" + ['rhel']="lm_sensors" + ['suse']="sensors" + ['clearlinux']="lm-sensors" + ['macos']="WARNING|" + ['freebsd']="NOTREQUIRED" + ['default']="lm_sensors" +) + +declare -A pkg_logwatch=( + ['gentoo']="sys-apps/logwatch" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="logwatch" +) + +declare -A pkg_lxc=( + ['gentoo']="app-emulation/lxc" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="lxc" +) + +declare -A pkg_mailutils=( + ['gentoo']="net-mail/mailutils" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="mailutils" +) + +declare -A pkg_make=( + ['gentoo']="sys-devel/make" + ['macos']="NOTREQUIRED" + ['freebsd']="gmake" + ['default']="make" +) + +declare -A pkg_netcat=( + ['alpine']="netcat-openbsd" + ['arch']="netcat" + ['centos']="nmap-ncat" + ['debian']="netcat" + ['gentoo']="net-analyzer/netcat" + ['sabayon']="net-analyzer/gnu-netcat" + ['rhel']="nmap-ncat" + ['suse']="netcat-openbsd" + ['clearlinux']="sysadmin-basic" + ['arch']="gnu-netcat" + ['macos']="NOTREQUIRED" + ['default']="netcat" + + # exceptions + ['centos-6']="nc" + ['rhel-6']="nc" +) + +declare -A pkg_nginx=( + ['gentoo']="www-servers/nginx" + ['default']="nginx" +) + +declare -A pkg_nodejs=( + ['gentoo']="net-libs/nodejs" + ['clearlinux']="nodejs-basic" + ['freebsd']="node" + ['default']="nodejs" + + # exceptions + ['rhel-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['rhel-7']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['centos-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['debian-6']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" + ['debian-7']="WARNING|To install nodejs check: https://nodejs.org/en/download/package-manager/" +) + +declare -A pkg_postfix=( + ['gentoo']="mail-mta/postfix" + ['macos']="WARNING|" + ['default']="postfix" +) + +declare -A pkg_pkg_config=( + ['alpine']="pkgconfig" + ['arch']="pkgconfig" + ['centos']="pkgconfig" + ['debian']="pkg-config" + ['gentoo']="virtual/pkgconfig" + ['sabayon']="virtual/pkgconfig" + ['rhel']="pkgconfig" + ['suse']="pkg-config" + ['freebsd']="pkgconf" + ['clearlinux']="c-basic" + ['default']="pkg-config" +) + +declare -A pkg_python=( + ['gentoo']="dev-lang/python" + ['sabayon']="dev-lang/python:2.7" + ['clearlinux']="python-basic" + ['default']="python" + + # Exceptions + ['macos']="WARNING|" + ['centos-8']="python2" +) + +declare -A pkg_python_mysqldb=( + ['alpine']="py-mysqldb" + ['arch']="mysql-python" + ['centos']="MySQL-python" + ['debian']="python-mysqldb" + ['gentoo']="dev-python/mysqlclient" + ['sabayon']="dev-python/mysqlclient" + ['rhel']="MySQL-python" + ['suse']="python-PyMySQL" + ['clearlinux']="WARNING|" + ['default']="python-mysql" + + # exceptions + ['fedora-24']="python2-mysql" +) + +declare -A pkg_python3_mysqldb=( + ['alpine']="WARNING|" + ['arch']="WARNING|" + ['centos']="WARNING|" + ['debian']="python3-mysqldb" + ['gentoo']="dev-python/mysqlclient" + ['sabayon']="dev-python/mysqlclient" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="WARNING|" + + # exceptions + ['debian-6']="WARNING|" + ['debian-7']="WARNING|" + ['debian-8']="WARNING|" + ['ubuntu-12.04']="WARNING|" + ['ubuntu-12.10']="WARNING|" + ['ubuntu-13.04']="WARNING|" + ['ubuntu-13.10']="WARNING|" + ['ubuntu-14.04']="WARNING|" + ['ubuntu-14.10']="WARNING|" + ['ubuntu-15.04']="WARNING|" + ['ubuntu-15.10']="WARNING|" + ['centos-7']="python36-mysql" + ['centos-8']="python38-mysql" + ['rhel-7']="python36-mysql" + ['rhel-8']="python38-mysql" +) + +declare -A pkg_python_psycopg2=( + ['alpine']="py-psycopg2" + ['arch']="python2-psycopg2" + ['centos']="python-psycopg2" + ['debian']="python-psycopg2" + ['gentoo']="dev-python/psycopg" + ['sabayon']="dev-python/psycopg:2" + ['rhel']="python-psycopg2" + ['suse']="python-psycopg2" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="python-psycopg2" +) + +declare -A pkg_python3_psycopg2=( + ['alpine']="py3-psycopg2" + ['arch']="python-psycopg2" + ['centos']="WARNING|" + ['debian']="WARNING|" + ['gentoo']="dev-python/psycopg" + ['sabayon']="dev-python/psycopg:2" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="WARNING|" + ['macos']="WARNING|" + ['default']="WARNING|" + + ['centos-7']="python3-psycopg2" + ['centos-8']="python38-psycopg2" + ['rhel-7']="python3-psycopg2" + ['rhel-8']="python38-psycopg2" +) + +declare -A pkg_python_pip=( + ['alpine']="py-pip" + ['gentoo']="dev-python/pip" + ['sabayon']="dev-python/pip" + ['clearlinux']="python-basic" + ['macos']="WARNING|" + ['default']="python-pip" +) + +declare -A pkg_python3_pip=( + ['alpine']="py3-pip" + ['arch']="python-pip" + ['gentoo']="dev-python/pip" + ['sabayon']="dev-python/pip" + ['clearlinux']="python3-basic" + ['macos']="NOTREQUIRED" + ['default']="python3-pip" +) + +declare -A pkg_python_pymongo=( + ['alpine']="WARNING|" + ['arch']="python2-pymongo" + ['centos']="WARNING|" + ['debian']="python-pymongo" + ['gentoo']="dev-python/pymongo" + ['suse']="python-pymongo" + ['clearlinux']="WARNING|" + ['rhel']="WARNING|" + ['macos']="WARNING|" + ['default']="python-pymongo" +) + +declare -A pkg_python3_pymongo=( + ['alpine']="WARNING|" + ['arch']="python-pymongo" + ['centos']="WARNING|" + ['debian']="python3-pymongo" + ['gentoo']="dev-python/pymongo" + ['suse']="python3-pymongo" + ['clearlinux']="WARNING|" + ['rhel']="WARNING|" + ['freebsd']="py37-pymongo" + ['macos']="WARNING|" + ['default']="python3-pymongo" + + ['centos-7']="python36-pymongo" + ['centos-8']="python3-pymongo" + ['rhel-7']="python36-pymongo" + ['rhel-8']="python3-pymongo" +) + +declare -A pkg_python_requests=( + ['alpine']="py-requests" + ['arch']="python2-requests" + ['centos']="python-requests" + ['debian']="python-requests" + ['gentoo']="dev-python/requests" + ['sabayon']="dev-python/requests" + ['rhel']="python-requests" + ['suse']="python-requests" + ['clearlinux']="python-extras" + ['macos']="WARNING|" + ['default']="python-requests" + ['alpine-3.1.4']="WARNING|" + ['alpine-3.2.3']="WARNING|" +) + +declare -A pkg_python3_requests=( + ['alpine']="py3-requests" + ['arch']="python-requests" + ['centos']="WARNING|" + ['debian']="WARNING|" + ['gentoo']="dev-python/requests" + ['sabayon']="dev-python/requests" + ['rhel']="WARNING|" + ['suse']="WARNING|" + ['clearlinux']="python-extras" + ['macos']="WARNING|" + ['default']="WARNING|" + + ['centos-7']="python36-requests" + ['centos-8']="python3-requests" + ['rhel-7']="python36-requests" + ['rhel-8']="python3-requests" +) + +declare -A pkg_lz4=( + ['alpine']="lz4-dev" + ['debian']="liblz4-dev" + ['ubuntu']="liblz4-dev" + ['suse']="liblz4-devel" + ['gentoo']="app-arch/lz4" + ['clearlinux']="devpkg-lz4" + ['arch']="lz4" + ['macos']="lz4" + ['freebsd']="liblz4" + ['default']="lz4-devel" +) + +declare -A pkg_libuv=( + ['alpine']="libuv-dev" + ['debian']="libuv1-dev" + ['ubuntu']="libuv1-dev" + ['gentoo']="dev-libs/libuv" + ['arch']="libuv" + ['clearlinux']="devpkg-libuv" + ['macos']="libuv" + ['freebsd']="libuv" + ['default']="libuv-devel" +) + +declare -A pkg_openssl=( + ['alpine']="openssl-dev" + ['debian']="libssl-dev" + ['ubuntu']="libssl-dev" + ['suse']="libopenssl-devel" + ['clearlinux']="devpkg-openssl" + ['gentoo']="dev-libs/openssl" + ['arch']="openssl" + ['freebsd']="openssl" + ['macos']="openssl@1.1" + ['default']="openssl-devel" +) + +declare -A pkg_judy=( + ['debian']="libjudy-dev" + ['ubuntu']="libjudy-dev" + ['suse']="judy-devel" + ['gentoo']="dev-libs/judy" + ['arch']="judy" + ['freebsd']="Judy" + ['fedora']="Judy-devel" + ['default']="NOTREQUIRED" +) + +declare -A pkg_python3=( + ['gentoo']="dev-lang/python" + ['sabayon']="dev-lang/python:3.4" + ['clearlinux']="python3-basic" + ['macos']="python" + ['default']="python3" + + # exceptions + ['centos-6']="WARNING|" +) + +declare -A pkg_screen=( + ['gentoo']="app-misc/screen" + ['sabayon']="app-misc/screen" + ['clearlinux']="sysadmin-basic" + ['default']="screen" +) + +declare -A pkg_sudo=( + ['gentoo']="app-admin/sudo" + ['macos']="NOTREQUIRED" + ['default']="sudo" +) + +declare -A pkg_sysstat=( + ['gentoo']="app-admin/sysstat" + ['macos']="WARNING|" + ['default']="sysstat" +) + +declare -A pkg_tcpdump=( + ['gentoo']="net-analyzer/tcpdump" + ['clearlinux']="network-basic" + ['default']="tcpdump" +) + +declare -A pkg_traceroute=( + ['alpine']=" " + ['gentoo']="net-analyzer/traceroute" + ['clearlinux']="network-basic" + ['macos']="NOTREQUIRED" + ['default']="traceroute" +) + +declare -A pkg_valgrind=( + ['gentoo']="dev-util/valgrind" + ['default']="valgrind" +) + +declare -A pkg_ulogd=( + ['centos']="WARNING|" + ['rhel']="WARNING|" + ['clearlinux']="WARNING|" + ['gentoo']="app-admin/ulogd" + ['arch']="ulogd" + ['macos']="WARNING|" + ['default']="ulogd2" +) + +declare -A pkg_unzip=( + ['gentoo']="app-arch/unzip" + ['macos']="NOTREQUIRED" + ['default']="unzip" +) + +declare -A pkg_zip=( + ['gentoo']="app-arch/zip" + ['macos']="NOTREQUIRED" + ['default']="zip" +) + +declare -A pkg_libelf=( + ['alpine']="elfutils-dev" + ['arch']="libelf" + ['gentoo']="virtual/libelf" + ['sabayon']="virtual/libelf" + ['debian']="libelf-dev" + ['ubuntu']="libelf-dev" + ['fedora']="elfutils-libelf-devel" + ['centos']="elfutils-libelf-devel" + ['rhel']="elfutils-libelf-devel" + ['clearlinux']="devpkg-elfutils" + ['suse']="libelf-devel" + ['macos']="NOTREQUIRED" + ['freebsd']="NOTREQUIRED" + ['default']="libelf-devel" + + # exceptions + ['alpine-3.5']="libelf-dev" + ['alpine-3.4']="libelf-dev" + ['alpine-3.3']="libelf-dev" +) + +validate_package_trees() { + if type -t validate_tree_${tree} > /dev/null; then + validate_tree_${tree} + fi +} + +validate_installed_package() { + validate_${package_installer} "${p}" +} + +suitable_package() { + local package="${1//-/_}" p="" v="${version//.*/}" + + echo >&2 "Searching for ${package} ..." + + eval "p=\${pkg_${package}['${distribution,,}-${version,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${distribution,,}-${v,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${distribution,,}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}-${version}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}-${v}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['${tree}']}" + [ -z "${p}" ] && eval "p=\${pkg_${package}['default']}" + + if [[ "${p/|*/}" =~ ^(ERROR|WARNING|INFO)$ ]]; then + echo >&2 "${p/|*/}" + echo >&2 "package ${1} is not available in this system." + if [ -z "${p/*|/}" ]; then + echo >&2 "You may try to install without it." + else + echo >&2 "${p/*|/}" + fi + echo >&2 + return 1 + elif [ "${p}" = "NOTREQUIRED" ]; then + return 0 + elif [ -z "${p}" ]; then + echo >&2 "WARNING" + echo >&2 "package ${1} is not availabe in this system." + echo >&2 + return 1 + else + if [ "${IGNORE_INSTALLED}" -eq 0 ]; then + validate_installed_package "${p}" + else + echo "${p}" + fi + return 0 + fi +} + +packages() { + # detect the packages we need to install on this system + + # ------------------------------------------------------------------------- + # basic build environment + + suitable_package distro-sdk + + require_cmd git || suitable_package git + require_cmd find || suitable_package find + + require_cmd gcc || + require_cmd gcc-multilib || suitable_package gcc + + require_cmd make || suitable_package make + require_cmd autoconf || suitable_package autoconf + suitable_package autoconf-archive + require_cmd autogen || suitable_package autogen + require_cmd automake || suitable_package automake + require_cmd libtoolize || suitable_package libtool + require_cmd pkg-config || suitable_package pkg-config + require_cmd cmake || suitable_package cmake + + # ------------------------------------------------------------------------- + # debugging tools for development + + if [ "${PACKAGES_DEBUG}" -ne 0 ]; then + require_cmd traceroute || suitable_package traceroute + require_cmd tcpdump || suitable_package tcpdump + require_cmd screen || suitable_package screen + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + require_cmd gdb || suitable_package gdb + require_cmd valgrind || suitable_package valgrind + fi + fi + + # ------------------------------------------------------------------------- + # common command line tools + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + require_cmd tar || suitable_package tar + require_cmd curl || suitable_package curl + require_cmd gzip || suitable_package gzip + require_cmd nc || suitable_package netcat + fi + + # ------------------------------------------------------------------------- + # firehol/fireqos/update-ipsets command line tools + + if [ "${PACKAGES_FIREQOS}" -ne 0 ]; then + require_cmd ip || suitable_package iproute2 + fi + + if [ "${PACKAGES_FIREHOL}" -ne 0 ]; then + require_cmd iptables || suitable_package iptables + require_cmd ipset || suitable_package ipset + require_cmd ulogd ulogd2 || suitable_package ulogd + require_cmd traceroute || suitable_package traceroute + require_cmd bridge || suitable_package bridge-utils + fi + + if [ "${PACKAGES_UPDATE_IPSETS}" -ne 0 ]; then + require_cmd ipset || suitable_package ipset + require_cmd zip || suitable_package zip + require_cmd funzip || suitable_package unzip + fi + + # ------------------------------------------------------------------------- + # netdata libraries + + if [ "${PACKAGES_NETDATA}" -ne 0 ]; then + suitable_package libz-dev + suitable_package libuuid-dev + suitable_package libmnl-dev + suitable_package json-c-dev + fi + + # ------------------------------------------------------------------------- + # sensors + + if [ "${PACKAGES_NETDATA_SENSORS}" -ne 0 ]; then + require_cmd sensors || suitable_package lm_sensors + fi + + # ------------------------------------------------------------------------- + # netdata database + if [ "${PACKAGES_NETDATA_DATABASE}" -ne 0 ]; then + suitable_package libuv + suitable_package lz4 + suitable_package openssl + suitable_package judy + fi + + # ------------------------------------------------------------------------- + # ebpf plugin + if [ "${PACKAGES_NETDATA_EBPF}" -ne 0 ]; then + suitable_package libelf + fi + + # ------------------------------------------------------------------------- + # scripting interpreters for netdata plugins + + if [ "${PACKAGES_NETDATA_NODEJS}" -ne 0 ]; then + require_cmd nodejs node js || suitable_package nodejs + fi + + # ------------------------------------------------------------------------- + # python2 + + if [ "${PACKAGES_NETDATA_PYTHON}" -ne 0 ]; then + require_cmd python || suitable_package python + + [ "${PACKAGES_NETDATA_PYTHON_MONGO}" -ne 0 ] && suitable_package python-pymongo + # suitable_package python-requests + # suitable_package python-pip + + [ "${PACKAGES_NETDATA_PYTHON_MYSQL}" -ne 0 ] && suitable_package python-mysqldb + [ "${PACKAGES_NETDATA_PYTHON_POSTGRES}" -ne 0 ] && suitable_package python-psycopg2 + fi + + # ------------------------------------------------------------------------- + # python3 + + if [ "${PACKAGES_NETDATA_PYTHON3}" -ne 0 ]; then + require_cmd python3 || suitable_package python3 + + [ "${PACKAGES_NETDATA_PYTHON_MONGO}" -ne 0 ] && suitable_package python3-pymongo + # suitable_package python3-requests + # suitable_package python3-pip + + [ "${PACKAGES_NETDATA_PYTHON_MYSQL}" -ne 0 ] && suitable_package python3-mysqldb + [ "${PACKAGES_NETDATA_PYTHON_POSTGRES}" -ne 0 ] && suitable_package python3-psycopg2 + fi + + # ------------------------------------------------------------------------- + # applications needed for the netdata demo sites + + if [ "${PACKAGES_NETDATA_DEMO_SITE}" -ne 0 ]; then + require_cmd sudo || suitable_package sudo + require_cmd jq || suitable_package jq + require_cmd nginx || suitable_package nginx + require_cmd postconf || suitable_package postfix + require_cmd lxc-create || suitable_package lxc + require_cmd logwatch || suitable_package logwatch + require_cmd mail || suitable_package mailutils + require_cmd iostat || suitable_package sysstat + require_cmd iotop || suitable_package iotop + fi +} + +DRYRUN=0 +run() { + + printf >&2 "%q " "${@}" + printf >&2 "\n" + + if [ ! "${DRYRUN}" -eq 1 ]; then + "${@}" + return $? + fi + return 0 +} + +sudo= +if [ ${UID} -ne 0 ]; then + sudo="sudo" +fi + +# ----------------------------------------------------------------------------- +# debian / ubuntu + +validate_install_apt_get() { + echo >&2 " > Checking if package '${*}' is installed..." + [ "$(dpkg-query -W --showformat='${Status}\n' "${*}")" = "install ok installed" ] || echo "${*}" +} + +install_apt_get() { + local opts="" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://serverfault.com/questions/227190/how-do-i-ask-apt-get-to-skip-any-interactive-post-install-configuration-steps + export DEBIAN_FRONTEND="noninteractive" + opts="${opts} -yq" + fi + + read -r -a apt_opts <<< "$opts" + + # update apt repository caches + + echo >&2 "NOTE: Running apt-get update and updating your APT caches ..." + if [ "${version}" = 8 ]; then + echo >&2 "WARNING: You seem to be on Debian 8 (jessie) which is old enough we have to disable Check-Valid-Until checks" + if ! cat /etc/apt/sources.list /etc/apt/sources.list.d/* 2> /dev/null | grep -q jessie-backports; then + echo >&2 "We also have to enable the jessie-backports repository" + if prompt "Is this okay?"; then + ${sudo} /bin/sh -c 'echo "deb http://archive.debian.org/debian/ jessie-backports main contrib non-free" >> /etc/apt/sources.list.d/99-archived.list' + fi + fi + run ${sudo} apt-get "${apt_opts[@]}" -o Acquire::Check-Valid-Until=false update + else + run ${sudo} apt-get "${apt_opts[@]}" update + fi + + # install the required packages + run ${sudo} apt-get "${apt_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# centos / rhel + +prompt() { + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode, assuming yes (y)" + echo >&2 " > Would have promptedfor ${1} ..." + return 0 + fi + + while true; do + read -r -p "${1} [y/n] " yn + case $yn in + [Yy]*) return 0 ;; + [Nn]*) return 1 ;; + *) echo >&2 "Please answer with yes (y) or no (n)." ;; + esac + done +} + +validate_tree_freebsd() { + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + echo >&2 " > FreeBSD Version: ${version} ..." + + make="make" + echo >&2 " > Checking for gmake ..." + if ! pkg query %n-%v | grep -q gmake; then + if prompt "gmake is required to build on FreeBSD and is not installed. Shall I install it?"; then + run ${sudo} pkg install ${opts} gmake + fi + fi +} + +validate_tree_centos() { + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + echo >&2 " > CentOS Version: ${version} ..." + + echo >&2 " > Checking for epel ..." + if ! rpm -qa | grep epel > /dev/null; then + if prompt "epel not found, shall I install it?"; then + run ${sudo} yum ${opts} install epel-release + fi + fi + + if [[ "${version}" =~ ^8(\..*)?$ ]]; then + echo >&2 " > Checking for config-manager ..." + if ! run yum ${sudo} config-manager; then + if prompt "config-manager not found, shall I install it?"; then + run ${sudo} yum ${opts} install 'dnf-command(config-manager)' + fi + fi + + echo >&2 " > Checking for PowerTools ..." + if ! run yum ${sudo} repolist | grep PowerTools; then + if prompt "PowerTools not found, shall I install it?"; then + run ${sudo} yum ${opts} config-manager --set-enabled powertools + fi + fi + + echo >&2 " > Checking for Okay ..." + if ! rpm -qa | grep okay > /dev/null; then + if prompt "okay not found, shall I install it?"; then + run ${sudo} yum ${opts} install http://repo.okay.com.mx/centos/8/x86_64/release/okay-release-1-3.el8.noarch.rpm + fi + fi + + echo >&2 " > Installing Judy-devel directly ..." + run ${sudo} yum ${opts} install http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/Judy-devel-1.0.5-18.module_el8.1.0+217+4d875839.x86_64.rpm + + elif [[ "${version}" =~ ^6\..*$ ]]; then + echo >&2 " > Detected CentOS 6.x ..." + echo >&2 " > Checking for Okay ..." + if ! rpm -qa | grep okay > /dev/null; then + if prompt "okay not found, shall I install it?"; then + run ${sudo} yum ${opts} install http://repo.okay.com.mx/centos/6/x86_64/release/okay-release-1-3.el6.noarch.rpm + fi + fi + + fi +} + +validate_install_yum() { + echo >&2 " > Checking if package '${*}' is installed..." + yum list installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_yum() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} yum update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/87822/does-yum-have-an-equivalent-to-apt-aptitudes-debian-frontend-noninteractive + opts="-y" + fi + + read -r -a yum_opts <<< "${opts}" + + # install the required packages + run ${sudo} yum "${yum_opts[@]}" install "${@}" # --enablerepo=epel-testing +} + +# ----------------------------------------------------------------------------- +# fedora + +validate_install_dnf() { + echo >&2 " > Checking if package '${*}' is installed..." + dnf list installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_dnf() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} dnf update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # man dnf + opts="-y" + fi + + # install the required packages + # --setopt=strict=0 allows dnf to proceed + # installing whatever is available + # even if a package is not found + opts="$opts --setopt=strict=0" + read -r -a dnf_opts <<< "$opts" + run ${sudo} dnf "${dnf_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# gentoo + +validate_install_emerge() { + echo "${*}" +} + +install_emerge() { + # download the latest package info + # we don't do this for emerge - it is very slow + # and most users are expected to do this daily + # emerge --sync + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} emerge --sync or ${sudo} eix-sync " + echo >&2 + fi + + local opts="--ask" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="" + fi + + read -r -a emerge_opts <<< "$opts" + + # install the required packages + run ${sudo} emerge "${emerge_opts[@]}" -v --noreplace "${@}" +} + +# ----------------------------------------------------------------------------- +# alpine + +validate_install_apk() { + echo "${*}" +} + +install_apk() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} apk update " + echo >&2 + fi + + local opts="--force-broken-world" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + else + opts="${opts} -i" + fi + + read -r -a apk_opts <<< "$opts" + + # install the required packages + run ${sudo} apk add "${apk_opts[@]}" "${@}" +} + +# ----------------------------------------------------------------------------- +# sabayon + +validate_install_equo() { + echo >&2 " > Checking if package '${*}' is installed..." + equo s --installed "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_equo() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} equo up " + echo >&2 + fi + + local opts="-av" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-v" + fi + + read -r -a equo_opts <<< "$opts" + + # install the required packages + run ${sudo} equo i "${equo_opts[@]}" "${@}" +} + +# ----------------------------------------------------------------------------- +# arch + +PACMAN_DB_SYNCED=0 +validate_install_pacman() { + + if [ ${PACMAN_DB_SYNCED} -eq 0 ]; then + echo >&2 " > Running pacman -Sy to sync the database" + local x + x=$(pacman -Sy) + [ -z "${x}" ] && echo "${*}" + PACMAN_DB_SYNCED=1 + fi + echo >&2 " > Checking if package '${*}' is installed..." + + # In pacman, you can utilize alternative flags to exactly match package names, + # but is highly likely we require pattern matching too in this so we keep -s and match + # the exceptional cases like so + local x="" + case "${package}" in + "gcc") + # Temporary workaround: In archlinux, default installation includes runtime libs under package "gcc" + # These are not sufficient for netdata install, so we need to make sure that the appropriate libraries are there + # by ensuring devel libs are available + x=$(pacman -Qs "${*}" | grep "base-devel") + ;; + "tar") + x=$(pacman -Qs "${*}" | grep "local/tar") + ;; + "make") + x=$(pacman -Qs "${*}" | grep "local/make ") + ;; + *) + x=$(pacman -Qs "${*}") + ;; + esac + + [ -z "${x}" ] && echo "${*}" +} + +install_pacman() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} pacman -Syu " + echo >&2 + fi + + # install the required packages + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/52277/pacman-option-to-assume-yes-to-every-question/52278 + # Try the noconfirm option, if that fails, go with the legacy way for non-interactive + run ${sudo} pacman --noconfirm --needed -S "${@}" || yes | run ${sudo} pacman --needed -S "${@}" + else + run ${sudo} pacman --needed -S "${@}" + fi +} + +# ----------------------------------------------------------------------------- +# suse / opensuse + +validate_install_zypper() { + rpm -q "${*}" > /dev/null 2>&1 || echo "${*}" +} + +install_zypper() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} zypper update " + echo >&2 + fi + + local opts="--ignore-unknown" + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + # http://unix.stackexchange.com/questions/82016/how-to-use-zypper-in-bash-scripts-for-someone-coming-from-apt-get + opts="${opts} --non-interactive" + fi + + read -r -a zypper_opts <<< "$opts" + + # install the required packages + run ${sudo} zypper "${zypper_opts[@]}" install "${@}" +} + +# ----------------------------------------------------------------------------- +# clearlinux + +validate_install_swupd() { + swupd bundle-list | grep -q "${*}" || echo "${*}" +} + +install_swupd() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: ${sudo} swupd update " + echo >&2 + fi + + run ${sudo} swupd bundle-add "${@}" +} + +# ----------------------------------------------------------------------------- +# macOS + +validate_install_pkg() { + pkg query %n-%v | grep -q "${*}" || echo "${*}" +} + +validate_install_brew() { + brew list | grep -q "${*}" || echo "${*}" +} + +install_pkg() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: pkg update " + echo >&2 + fi + + local opts= + if [ "${NON_INTERACTIVE}" -eq 1 ]; then + echo >&2 "Running in non-interactive mode" + opts="-y" + fi + + read -r -a pkg_opts <<< "${opts}" + + run ${sudo} pkg install "${pkg_opts[@]}" "${@}" +} + +install_brew() { + # download the latest package info + if [ "${DRYRUN}" -eq 1 ]; then + echo >&2 " >> IMPORTANT << " + echo >&2 " Please make sure your system is up to date" + echo >&2 " by running: brew upgrade " + echo >&2 + fi + + run brew install "${@}" +} + +# ----------------------------------------------------------------------------- + +install_failed() { + local ret="${1}" + cat << EOF + + + +We are very sorry! + +Installation of required packages failed. + +What to do now: + + 1. Make sure your system is updated. + Most of the times, updating your system will resolve the issue. + + 2. If the error message is about a specific package, try removing + that package from the command and run it again. + Depending on the broken package, you may be able to continue. + + 3. Let us know. We may be able to help. + Open a github issue with the above log, at: + + https://github.com/netdata/netdata/issues + + +EOF + remote_log "FAILED" "${ret}" + exit 1 +} + +remote_log() { + # log success or failure on our system + # to help us solve installation issues + curl > /dev/null 2>&1 -Ss --max-time 3 "https://registry.my-netdata.io/log/installer?status=${1}&error=${2}&distribution=${distribution}&version=${version}&installer=${package_installer}&tree=${tree}&detection=${detection}&netdata=${PACKAGES_NETDATA}&nodejs=${PACKAGES_NETDATA_NODEJS}&python=${PACKAGES_NETDATA_PYTHON}&python3=${PACKAGES_NETDATA_PYTHON3}&mysql=${PACKAGES_NETDATA_PYTHON_MYSQL}&postgres=${PACKAGES_NETDATA_PYTHON_POSTGRES}&pymongo=${PACKAGES_NETDATA_PYTHON_MONGO}&sensors=${PACKAGES_NETDATA_SENSORS}&database=${PACKAGES_NETDATA_DATABASE}&ebpf=${PACKAGES_NETDATA_EBPF}&firehol=${PACKAGES_FIREHOL}&fireqos=${PACKAGES_FIREQOS}&iprange=${PACKAGES_IPRANGE}&update_ipsets=${PACKAGES_UPDATE_IPSETS}&demo=${PACKAGES_NETDATA_DEMO_SITE}" +} + +if [ -z "${1}" ]; then + usage + exit 1 +fi + +pv=$(python --version 2>&1) +if [ "${tree}" = macos ]; then + pv=3 +elif [[ "${pv}" =~ ^Python\ 2.* ]]; then + pv=2 +elif [[ "${pv}" =~ ^Python\ 3.* ]]; then + pv=3 +elif [[ "${tree}" == "centos" ]] && [ "${version}" -lt 8 ]; then + pv=2 +else + pv=3 +fi + +# parse command line arguments +DONT_WAIT=0 +NON_INTERACTIVE=0 +IGNORE_INSTALLED=0 +while [ -n "${1}" ]; do + case "${1}" in + distribution) + distribution="${2}" + shift + ;; + + version) + version="${2}" + shift + ;; + + codename) + codename="${2}" + shift + ;; + + installer) + check_package_manager "${2}" || exit 1 + shift + ;; + + dont-wait | --dont-wait | -n) + DONT_WAIT=1 + ;; + + non-interactive | --non-interactive | -y) + NON_INTERACTIVE=1 + ;; + + ignore-installed | --ignore-installed | -i) + IGNORE_INSTALLED=1 + ;; + + netdata-all) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + PACKAGES_NETDATA_SENSORS=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + netdata) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + python | netdata-python) + PACKAGES_NETDATA_PYTHON=1 + ;; + + python3 | netdata-python3) + PACKAGES_NETDATA_PYTHON3=1 + ;; + + python-mysql | mysql-python | mysqldb | netdata-mysql) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + fi + ;; + + python-postgres | postgres-python | psycopg2 | netdata-postgres) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + fi + ;; + + python-pymongo) + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + ;; + + nodejs | netdata-nodejs) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + PACKAGES_NETDATA_DATABASE=1 + ;; + + sensors | netdata-sensors) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_SENSORS=1 + PACKAGES_NETDATA_DATABASE=1 + ;; + + firehol | update-ipsets | firehol-all | fireqos) + PACKAGES_IPRANGE=1 + PACKAGES_FIREHOL=1 + PACKAGES_FIREQOS=1 + PACKAGES_IPRANGE=1 + PACKAGES_UPDATE_IPSETS=1 + ;; + + demo | all) + PACKAGES_NETDATA=1 + PACKAGES_NETDATA_NODEJS=1 + if [ "${pv}" -eq 2 ]; then + PACKAGES_NETDATA_PYTHON=1 + PACKAGES_NETDATA_PYTHON_MYSQL=1 + PACKAGES_NETDATA_PYTHON_POSTGRES=1 + PACKAGES_NETDATA_PYTHON_MONGO=1 + else + PACKAGES_NETDATA_PYTHON3=1 + PACKAGES_NETDATA_PYTHON3_MYSQL=1 + PACKAGES_NETDATA_PYTHON3_POSTGRES=1 + PACKAGES_NETDATA_PYTHON3_MONGO=1 + fi + PACKAGES_DEBUG=1 + PACKAGES_IPRANGE=1 + PACKAGES_FIREHOL=1 + PACKAGES_FIREQOS=1 + PACKAGES_UPDATE_IPSETS=1 + PACKAGES_NETDATA_DEMO_SITE=1 + PACKAGES_NETDATA_DATABASE=1 + PACKAGES_NETDATA_EBPF=1 + ;; + + help | -h | --help) + usage + exit 1 + ;; + + *) + echo >&2 "ERROR: Cannot understand option '${1}'" + echo >&2 + usage + exit 1 + ;; + esac + shift +done + +# Check for missing core commands like grep, warn the user to install it and bail out cleanly +if ! command -v grep > /dev/null 2>&1; then + echo >&2 + echo >&2 "ERROR: 'grep' is required for the install to run correctly and was not found on the system." + echo >&2 "Please install grep and run the installer again." + echo >&2 + exit 1 +fi + +if [ -z "${package_installer}" ] || [ -z "${tree}" ]; then + if [ -z "${distribution}" ]; then + # we dont know the distribution + autodetect_distribution || user_picks_distribution + fi + + # When no package installer is detected, try again from distro info if any + if [ -z "${package_installer}" ]; then + detect_package_manager_from_distribution "${distribution}" + fi + + # Validate package manager trees + validate_package_trees +fi + +[ "${detection}" = "/etc/os-release" ] && cat << EOF + +/etc/os-release information: +NAME : ${NAME} +VERSION : ${VERSION} +ID : ${ID} +ID_LIKE : ${ID_LIKE} +VERSION_ID : ${VERSION_ID} +EOF + +cat << EOF + +We detected these: +Distribution : ${distribution} +Version : ${version} +Codename : ${codename} +Package Manager : ${package_installer} +Packages Tree : ${tree} +Detection Method: ${detection} +Default Python v: ${pv} $([ ${pv} -eq 2 ] && [ "${PACKAGES_NETDATA_PYTHON3}" -eq 1 ] && echo "(will install python3 too)") + +EOF + +mapfile -t PACKAGES_TO_INSTALL < <(packages | sort -u) + +if [ ${#PACKAGES_TO_INSTALL[@]} -gt 0 ]; then + echo >&2 + echo >&2 "The following command will be run:" + echo >&2 + DRYRUN=1 + "${package_installer}" "${PACKAGES_TO_INSTALL[@]}" + DRYRUN=0 + echo >&2 + echo >&2 + + if [ "${DONT_WAIT}" -eq 0 ] && [ "${NON_INTERACTIVE}" -eq 0 ]; then + read -r -p "Press ENTER to run it > " || exit 1 + fi + + "${package_installer}" "${PACKAGES_TO_INSTALL[@]}" || install_failed $? + + echo >&2 + echo >&2 "All Done! - Now proceed to the next step." + echo >&2 + +else + echo >&2 + echo >&2 "All required packages are already installed. Now proceed to the next step." + echo >&2 +fi + +remote_log "OK" + +exit 0 diff --git a/packaging/installer/kickstart-static64.sh b/packaging/installer/kickstart-static64.sh index 72cd00b8..3c45d9ef 100755 --- a/packaging/installer/kickstart-static64.sh +++ b/packaging/installer/kickstart-static64.sh @@ -1,255 +1,368 @@ #!/usr/bin/env sh -# + # SPDX-License-Identifier: GPL-3.0-or-later # shellcheck disable=SC1117,SC2039,SC2059,SC2086 # # Options to run -# --dont-wait do not wait for input -# --non-interactive do not wait for input -# --dont-start-it do not start netdata after install -# --stable-channel Use the stable release channel, rather than the nightly to fetch sources -# --local-files Use a manually provided tarball for the installation +# --dont-wait do not wait for input +# --non-interactive do not wait for input +# --dont-start-it do not start netdata after install +# --stable-channel Use the stable release channel, rather than the nightly to fetch sources +# --disable-telemetry Opt-out of anonymous telemetry program (DO_NOT_TRACK=1) +# --local-files Use a manually provided tarball for the installation +# --allow-duplicate-install do not bail if we detect a duplicate install +# --reinstall if an existing install would be updated, reinstall instead # -# --------------------------------------------------------------------------------------------------------------------- +# Environment options: +# +# NETDATA_TARBALL_BASEURL set the base url for downloading the dist tarball +# +# ---------------------------------------------------------------------------- # library functions copied from packaging/installer/functions.sh setup_terminal() { - TPUT_RESET="" - TPUT_YELLOW="" - TPUT_WHITE="" - TPUT_BGRED="" - TPUT_BGGREEN="" - TPUT_BOLD="" - TPUT_DIM="" - - # Is stderr on the terminal? If not, then fail - test -t 2 || return 1 - - if command -v tput >/dev/null 2>&1; then - if [ $(($(tput colors 2>/dev/null))) -ge 8 ]; then - # Enable colors - TPUT_RESET="$(tput sgr 0)" - TPUT_YELLOW="$(tput setaf 3)" - TPUT_WHITE="$(tput setaf 7)" - TPUT_BGRED="$(tput setab 1)" - TPUT_BGGREEN="$(tput setab 2)" - TPUT_BOLD="$(tput bold)" - TPUT_DIM="$(tput dim)" - fi - fi - - return 0 + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput > /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 } -setup_terminal || echo >/dev/null +setup_terminal || echo > /dev/null -# ----------------------------------------------------------------------------- +# ---------------------------------------------------------------------------- fatal() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" - exit 1 + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 } run_ok() { - printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} ${*} \n\n" + printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} \n\n" } run_failed() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} ${*} \n\n" + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} \n\n" } ESCAPED_PRINT_METHOD= -printf "%q " test >/dev/null 2>&1 -[ $? -eq 0 ] && ESCAPED_PRINT_METHOD="printfq" +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi escaped_print() { - if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then - printf "%q " "${@}" - else - printf "%s" "${*}" - fi - return 0 + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 } progress() { - echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " } run_logfile="/dev/null" run() { - local user="${USER--}" dir="${PWD}" info info_console - - if [ "${UID}" = "0" ]; then - info="[root ${dir}]# " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " - else - info="[${user} ${dir}]$ " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " - fi - - printf >>"${run_logfile}" "${info}" - escaped_print >>"${run_logfile}" "${@}" - printf >>"${run_logfile}" " ... " - - printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" - escaped_print >&2 "${@}" - printf >&2 "${TPUT_RESET}\n" - - "${@}" - - local ret=$? - if [ ${ret} -ne 0 ]; then - run_failed - printf >>"${run_logfile}" "FAILED with exit code ${ret}\n" - else - run_ok - printf >>"${run_logfile}" "OK\n" - fi - - return ${ret} + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "${info}" + escaped_print "${@}" + printf " ... " + } >> "${run_logfile}" + + printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "${TPUT_RESET}\n" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code ${ret}\n" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} } fatal() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" - exit 1 + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 +} + +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" } create_tmp_directory() { - # Check if tmp is mounted as noexec - if grep -Eq '^[^ ]+ /tmp [^ ]+ ([^ ]*,)?noexec[, ]' /proc/mounts; then - pattern="$(pwd)/netdata-kickstart-XXXXXX" - else - pattern="/tmp/netdata-kickstart-XXXXXX" - fi - - mktemp -d $pattern + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + + mktemp -d -t netdata-kickstart-XXXXXXXXXX } download() { - url="${1}" - dest="${2}" - if command -v curl >/dev/null 2>&1; then - run curl -sSL --connect-timeout 10 --retry 3 "${url}" >"${dest}" || fatal "Cannot download ${url}" - elif command -v wget >/dev/null 2>&1; then - run wget -T 15 -O - "${url}" >"${dest}" || fatal "Cannot download ${url}" - else - fatal "I need curl or wget to proceed, but neither is available on this system." - fi + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" || fatal "Cannot download ${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi } set_tarball_urls() { - if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then - progress "Not fetching remote tarballs, local override was given" - return - fi - - if [ "$1" = "stable" ]; then - local latest - # Simple version - # latest="$(curl -sSL https://api.github.com/repos/netdata/netdata/releases/latest | grep tag_name | cut -d'"' -f4)" - latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" - export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.gz.run" - export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" - else - export NETDATA_TARBALL_URL="https://storage.googleapis.com/netdata-nightlies/netdata-latest.gz.run" - export NETDATA_TARBALL_CHECKSUM_URL="https://storage.googleapis.com/netdata-nightlies/sha256sums.txt" - fi + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + progress "Not fetching remote tarballs, local override was given" + return + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.gz.run" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_TARBALL_BASEURL/netdata-latest.gz.run" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_TARBALL_BASEURL/sha256sums.txt" + fi } safe_sha256sum() { - # Within the contexct of the installer, we only use -c option that is common between the two commands - # We will have to reconsider if we start non-common options - if command -v sha256sum >/dev/null 2>&1; then - sha256sum $@ - elif command -v shasum >/dev/null 2>&1; then - shasum -a 256 $@ - else - fatal "I could not find a suitable checksum binary to use" - fi + # Within the context of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start using non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi } -# --------------------------------------------------------------------------------------------------------------------- +# ---------------------------------------------------------------------------- umask 022 sudo="" [ -z "${UID}" ] && UID="$(id -u)" [ "${UID}" -ne "0" ] && sudo="sudo" -# --------------------------------------------------------------------------------------------------------------------- +# ---------------------------------------------------------------------------- if [ "$(uname -m)" != "x86_64" ]; then - fatal "Static binary versions of netdata are available only for 64bit Intel/AMD CPUs (x86_64), but yours is: $(uname -m)." + fatal "Static binary versions of netdata are available only for 64bit Intel/AMD CPUs (x86_64), but yours is: $(uname -m)." fi if [ "$(uname -s)" != "Linux" ]; then - fatal "Static binary versions of netdata are available only for Linux, but this system is $(uname -s)" + fatal "Static binary versions of netdata are available only for Linux, but this system is $(uname -s)" fi -# --------------------------------------------------------------------------------------------------------------------- +# ---------------------------------------------------------------------------- opts= NETDATA_INSTALLER_OPTIONS="" NETDATA_UPDATES="--auto-update" RELEASE_CHANNEL="nightly" while [ -n "${1}" ]; do - if [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ] || [ "${1}" = "--accept" ]; then - opts="${opts} --accept" - shift 1 - elif [ "${1}" = "--dont-start-it" ]; then - NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" - shift 1 - elif [ "${1}" = "--no-updates" ]; then - NETDATA_UPDATES="" - shift 1 - elif [ "${1}" = "--stable-channel" ]; then - RELEASE_CHANNEL="stable" - NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" - shift 1 - elif [ "${1}" = "--local-files" ]; then - shift 1 - if [ -z "${1}" ]; then - fatal "Option --local-files requires extra information. The desired tarball full filename is needed" - fi - - NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" - shift 1 - if [ -z "${1}" ]; then - fatal "Option --local-files requires a pair of the tarball source and the checksum file" - fi - - NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" - shift 1 - else - echo >&2 "Unknown option '${1}' or invalid number of arguments. Please check the README for the available arguments of ${0} and try again" - exit 1 - fi + if [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ] || [ "${1}" = "--accept" ]; then + opts="${opts} --accept" + shift 1 + elif [ "${1}" = "--dont-start-it" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--no-updates" ]; then + NETDATA_UPDATES="" + shift 1 + elif [ "${1}" = "--auto-update" ]; then + true # This is the default behaviour, so ignore it. + shift 1 + elif [ "${1}" = "--stable-channel" ]; then + RELEASE_CHANNEL="stable" + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--disable-telemetry" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }${1}" + shift 1 + elif [ "${1}" = "--local-files" ]; then + NETDATA_UPDATES="" # Disable autoupdates if using pre-downloaded files. + shift 1 + if [ -z "${1}" ]; then + fatal "Option --local-files requires extra information. The desired tarball full filename is needed" + fi + + NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" + shift 1 + if [ -z "${1}" ]; then + fatal "Option --local-files requires a pair of the tarball source and the checksum file" + fi + + NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" + shift 1 + elif [ "${1}" = "--allow-duplicate-install" ]; then + NETDATA_ALLOW_DUPLICATE_INSTALL=1 + shift 1 + elif [ "${1}" = "--reinstall" ]; then + NETDATA_REINSTALL=1 + shift 1 + else + echo >&2 "Unknown option '${1}' or invalid number of arguments. Please check the README for the available arguments of ${0} and try again" + exit 1 + fi done +if [ ! "${DO_NOT_TRACK:-0}" -eq 0 ] || [ -n "$DO_NOT_TRACK" ]; then + NETDATA_INSTALLER_OPTIONS="${NETDATA_INSTALLER_OPTIONS:+${NETDATA_INSTALLER_OPTIONS} }--disable-telemtry" +fi + +# Netdata Tarball Base URL (defaults to our Google Storage Bucket) +[ -z "$NETDATA_TARBALL_BASEURL" ] && NETDATA_TARBALL_BASEURL=https://storage.googleapis.com/netdata-nightlies + # --------------------------------------------------------------------------------------------------------------------- +# look for an existing install and try to update that instead if it exists + +ndpath="$(command -v netdata 2>/dev/null)" +if [ -z "$ndpath" ] && [ -x /opt/netdata/bin/netdata ] ; then + ndpath="/opt/netdata/bin/netdata" +fi + +if [ -n "$ndpath" ] ; then + ndprefix="$(dirname "$(dirname "${ndpath}")")" + + if [ "${ndprefix}" = /usr ] ; then + ndprefix="/" + fi + + progress "Found existing install of Netdata under: ${ndprefix}" + + if [ -r "${ndprefix}/etc/netdata/.environment" ] ; then + ndstatic="$(grep IS_NETDATA_STATIC_BINARY "${ndprefix}/etc/netdata/.environment" | cut -d "=" -f 2 | tr -d \")" + if [ -z "${NETDATA_REINSTALL}" ] && [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ] ; then + if [ -x "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" ] ; then + progress "Attempting to update existing install instead of creating a new one" + if run ${sudo} "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" --not-running-from-cron ; then + progress "Updated existing install at ${ndpath}" + exit 0 + else + fatal "Failed to update existing Netdata install" + exit 1 + fi + else + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] || [ "${ndstatic}" = "no" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi + else + if [ "${ndstatic}" = "yes" ] ; then + progress "User requested reinstall instead of update, proceeding." + else + fatal "Existing install is not a static install, please use kickstart.sh instead." + exit 1 + fi + fi + else + progress "Existing install appears to be handled manually or through the system package manager." + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi +fi + +# ---------------------------------------------------------------------------- TMPDIR=$(create_tmp_directory) -cd "${TMPDIR}" +cd "${TMPDIR}" || exit 1 if [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then - set_tarball_urls "${RELEASE_CHANNEL}" - progress "Downloading static netdata binary: ${NETDATA_TARBALL_URL}" + set_tarball_urls "${RELEASE_CHANNEL}" + progress "Downloading static netdata binary: ${NETDATA_TARBALL_URL}" - download "${NETDATA_TARBALL_CHECKSUM_URL}" "${TMPDIR}/sha256sum.txt" - download "${NETDATA_TARBALL_URL}" "${TMPDIR}/netdata-latest.gz.run" + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${TMPDIR}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${TMPDIR}/netdata-latest.gz.run" else - progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" - run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${TMPDIR}/netdata-latest.gz.run" - run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${TMPDIR}/sha256sum.txt" + progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${TMPDIR}/netdata-latest.gz.run" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${TMPDIR}/sha256sum.txt" fi -if ! grep netdata-latest.gz.run "${TMPDIR}/sha256sum.txt" | safe_sha256sum -c - >/dev/null 2>&1; then - fatal "Static binary checksum validation failed. Stopping netdata installation and leaving binary in ${TMPDIR}" +if ! grep netdata-latest.gz.run "${TMPDIR}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Static binary checksum validation failed. Stopping Netdata Agent installation and leaving binary in ${TMPDIR}\nUsually this is a result of an older copy of the file being cached somewhere upstream and can be resolved by retrying in an hour." fi -# --------------------------------------------------------------------------------------------------------------------- +# ---------------------------------------------------------------------------- progress "Installing netdata" run ${sudo} sh "${TMPDIR}/netdata-latest.gz.run" ${opts} -- ${NETDATA_UPDATES} ${NETDATA_INSTALLER_OPTIONS} #shellcheck disable=SC2181 if [ $? -eq 0 ]; then - run ${sudo} rm "${TMPDIR}/netdata-latest.gz.run" - if [ ! "${TMPDIR}" = "/" ] && [ -d "${TMPDIR}" ]; then - run ${sudo} rm -rf "${TMPDIR}" - fi + run ${sudo} rm "${TMPDIR}/netdata-latest.gz.run" + if [ ! "${TMPDIR}" = "/" ] && [ -d "${TMPDIR}" ]; then + run ${sudo} rm -rf "${TMPDIR}" + fi else - echo >&2 "NOTE: did not remove: ${TMPDIR}/netdata-latest.gz.run" + echo >&2 "NOTE: did not remove: ${TMPDIR}/netdata-latest.gz.run" fi diff --git a/packaging/installer/kickstart.sh b/packaging/installer/kickstart.sh index c66cff3d..c97587c1 100755 --- a/packaging/installer/kickstart.sh +++ b/packaging/installer/kickstart.sh @@ -13,7 +13,14 @@ # --dont-wait do not prompt for user input # --non-interactive do not prompt for user input # --no-updates do not install script for daily updates -# --local-files set the full path of the desired tarball to run install with +# --local-files set the full path of the desired tarball to run install with +# --allow-duplicate-install do not bail if we detect a duplicate install +# --reinstall if an existing install would be updated, reinstall instead +# +# Environment options: +# +# TMPDIR specify where to save temporary files +# NETDATA_TARBALL_BASEURL set the base url for downloading the dist tarball # # This script will: # @@ -27,237 +34,270 @@ # shellcheck disable=SC2039,SC2059,SC2086 # External files -PACKAGES_SCRIPT="https://raw.githubusercontent.com/netdata/netdata-demo-site/master/install-required-packages.sh" +PACKAGES_SCRIPT="https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh" + +# Netdata Tarball Base URL (defaults to our Google Storage Bucket) +[ -z "$NETDATA_TARBALL_BASEURL" ] && NETDATA_TARBALL_BASEURL=https://storage.googleapis.com/netdata-nightlies # --------------------------------------------------------------------------------------------------------------------- # library functions copied from packaging/installer/functions.sh setup_terminal() { - TPUT_RESET="" - TPUT_YELLOW="" - TPUT_WHITE="" - TPUT_BGRED="" - TPUT_BGGREEN="" - TPUT_BOLD="" - TPUT_DIM="" - - # Is stderr on the terminal? If not, then fail - test -t 2 || return 1 - - if command -v tput >/dev/null 2>&1; then - if [ $(($(tput colors 2>/dev/null))) -ge 8 ]; then - # Enable colors - TPUT_RESET="$(tput sgr 0)" - TPUT_YELLOW="$(tput setaf 3)" - TPUT_WHITE="$(tput setaf 7)" - TPUT_BGRED="$(tput setab 1)" - TPUT_BGGREEN="$(tput setab 2)" - TPUT_BOLD="$(tput bold)" - TPUT_DIM="$(tput dim)" - fi - fi - - return 0 + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput > /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 } -setup_terminal || echo >/dev/null +setup_terminal || echo > /dev/null # ----------------------------------------------------------------------------- fatal() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" - exit 1 + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 } run_ok() { - printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} ${*} \n\n" + printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} \n\n" } run_failed() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} ${*} \n\n" + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} \n\n" } ESCAPED_PRINT_METHOD= -printf "%q " test >/dev/null 2>&1 -[ $? -eq 0 ] && ESCAPED_PRINT_METHOD="printfq" +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi escaped_print() { - if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then - printf "%q " "${@}" - else - printf "%s" "${*}" - fi - return 0 + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 } progress() { - echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " + echo >&2 " --- ${TPUT_DIM}${TPUT_BOLD}${*}${TPUT_RESET} --- " } run_logfile="/dev/null" run() { - local user="${USER--}" dir="${PWD}" info info_console - - if [ "${UID}" = "0" ]; then - info="[root ${dir}]# " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " - else - info="[${user} ${dir}]$ " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " - fi - - printf >>"${run_logfile}" "${info}" - escaped_print >>"${run_logfile}" "${@}" - printf >>"${run_logfile}" " ... " - - printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" - escaped_print >&2 "${@}" - printf >&2 "${TPUT_RESET}\n" - - "${@}" - - local ret=$? - if [ ${ret} -ne 0 ]; then - run_failed - printf >>"${run_logfile}" "FAILED with exit code ${ret}\n" - else - run_ok - printf >>"${run_logfile}" "OK\n" - fi - - return ${ret} + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "${info}" + escaped_print "${@}" + printf " ... " + } >> "${run_logfile}" + + printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" + escaped_print >&2 "${@}" + printf >&2 "${TPUT_RESET}" + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code ${ret}\n" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} } fatal() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" - exit 1 + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} ABORTED ${TPUT_RESET} ${*} \n\n" + exit 1 } warning() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} WARNING ${TPUT_RESET} ${*} \n\n" - if [ "${INTERACTIVE}" = "0" ]; then - fatal "Stopping due to non-interactive mode. Fix the issue or retry installation in an interactive mode." - else - read -r -p "Press ENTER to attempt netdata installation > " - progress "OK, let's give it a try..." - fi + printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} WARNING ${TPUT_RESET} ${*} \n\n" + if [ "${INTERACTIVE}" = "0" ]; then + fatal "Stopping due to non-interactive mode. Fix the issue or retry installation in an interactive mode." + else + read -r -p "Press ENTER to attempt netdata installation > " + progress "OK, let's give it a try..." + fi +} + +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" } create_tmp_directory() { - # Check if tmp is mounted as noexec - if grep -Eq '^[^ ]+ /tmp [^ ]+ ([^ ]*,)?noexec[, ]' /proc/mounts > /dev/null 2>&1; then - pattern="$(pwd)/netdata-kickstart-XXXXXX" - else - pattern="/tmp/netdata-kickstart-XXXXXX" - fi - - mktemp -d $pattern + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + + mktemp -d -t netdata-kickstart-XXXXXXXXXX } download() { - url="${1}" - dest="${2}" - if command -v curl >/dev/null 2>&1; then - run curl -sSL --connect-timeout 10 --retry 3 "${url}" >"${dest}" || fatal "Cannot download ${url}" - elif command -v wget >/dev/null 2>&1; then - run wget -T 15 -O - "${url}" >"${dest}" || fatal "Cannot download ${url}" - else - fatal "I need curl or wget to proceed, but neither is available on this system." - fi + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + run curl -q -sSL --connect-timeout 10 --retry 3 --output "${dest}" "${url}" + elif command -v wget > /dev/null 2>&1; then + run wget -T 15 -O "${dest}" "${url}" || fatal "Cannot download ${url}" + else + fatal "I need curl or wget to proceed, but neither is available on this system." + fi } set_tarball_urls() { - if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then - progress "Not fetching remote tarballs, local override was given" - return - fi - - if [ "$1" = "stable" ]; then - local latest - # Simple version - # latest="$(curl -sSL https://api.github.com/repos/netdata/netdata/releases/latest | grep tag_name | cut -d'"' -f4)" - latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" - export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.tar.gz" - export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" - else - export NETDATA_TARBALL_URL="https://storage.googleapis.com/netdata-nightlies/netdata-latest.tar.gz" - export NETDATA_TARBALL_CHECKSUM_URL="https://storage.googleapis.com/netdata-nightlies/sha256sums.txt" - fi + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then + progress "Not fetching remote tarballs, local override was given" + return + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.tar.gz" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_TARBALL_BASEURL/netdata-latest.tar.gz" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_TARBALL_BASEURL/sha256sums.txt" + fi } detect_bash4() { - bash="${1}" - if [ -z "${BASH_VERSION}" ]; then - # we don't run under bash - if [ -n "${bash}" ] && [ -x "${bash}" ]; then - # shellcheck disable=SC2016 - BASH_MAJOR_VERSION=$(${bash} -c 'echo "${BASH_VERSINFO[0]}"') - fi - else - # we run under bash - BASH_MAJOR_VERSION="${BASH_VERSINFO[0]}" - fi - - if [ -z "${BASH_MAJOR_VERSION}" ]; then - echo >&2 "No BASH is available on this system" - return 1 - elif [ $((BASH_MAJOR_VERSION)) -lt 4 ]; then - echo >&2 "No BASH v4+ is available on this system (installed bash is v${BASH_MAJOR_VERSION}" - return 1 - fi - return 0 + bash="${1}" + if [ -z "${BASH_VERSION}" ]; then + # we don't run under bash + if [ -n "${bash}" ] && [ -x "${bash}" ]; then + # shellcheck disable=SC2016 + BASH_MAJOR_VERSION=$(${bash} -c 'echo "${BASH_VERSINFO[0]}"') + fi + else + # we run under bash + BASH_MAJOR_VERSION="${BASH_VERSINFO[0]}" + fi + + if [ -z "${BASH_MAJOR_VERSION}" ]; then + echo >&2 "No BASH is available on this system" + return 1 + elif [ $((BASH_MAJOR_VERSION)) -lt 4 ]; then + echo >&2 "No BASH v4+ is available on this system (installed bash is v${BASH_MAJOR_VERSION}" + return 1 + fi + return 0 } dependencies() { - SYSTEM="$(uname -s 2> /dev/null || uname -v)" - OS="$(uname -o 2> /dev/null || uname -rs)" - MACHINE="$(uname -m 2> /dev/null)" - - echo "System : ${SYSTEM}" - echo "Operating System : ${OS}" - echo "Machine : ${MACHINE}" - echo "BASH major version: ${BASH_MAJOR_VERSION}" - - if [ "${OS}" != "GNU/Linux" ] && [ "${SYSTEM}" != "Linux" ]; then - warning "Cannot detect the packages to be installed on a ${SYSTEM} - ${OS} system." - else - bash="$(command -v bash 2>/dev/null)" - if ! detect_bash4 "${bash}"; then - warning "Cannot detect packages to be installed in this system, without BASH v4+." - else - progress "Fetching script to detect required packages..." - if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then - if [ -f "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then - run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" "${TMPDIR}/install-required-packages.sh" - else - fatal "Invalid given dependency file, please check your --local-files parameter options and try again" - fi - else - download "${PACKAGES_SCRIPT}" "${TMPDIR}/install-required-packages.sh" - fi - - if [ ! -s "${TMPDIR}/install-required-packages.sh" ]; then - warning "Downloaded dependency installation script is empty." - else - progress "Running downloaded script to detect required packages..." - run ${sudo} "${bash}" "${TMPDIR}/install-required-packages.sh" ${PACKAGES_INSTALLER_OPTIONS} - # shellcheck disable=SC2181 - if [ $? -ne 0 ] ; then - warning "It failed to install all the required packages, but installation might still be possible." - fi - fi - - fi - fi + SYSTEM="$(uname -s 2> /dev/null || uname -v)" + OS="$(uname -o 2> /dev/null || uname -rs)" + MACHINE="$(uname -m 2> /dev/null)" + + echo "System : ${SYSTEM}" + echo "Operating System : ${OS}" + echo "Machine : ${MACHINE}" + echo "BASH major version: ${BASH_MAJOR_VERSION}" + + if [ "${OS}" != "GNU/Linux" ] && [ "${SYSTEM}" != "Linux" ]; then + warning "Cannot detect the packages to be installed on a ${SYSTEM} - ${OS} system." + else + bash="$(command -v bash 2> /dev/null)" + if ! detect_bash4 "${bash}"; then + warning "Cannot detect packages to be installed in this system, without BASH v4+." + else + progress "Fetching script to detect required packages..." + if [ -n "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then + if [ -f "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" ]; then + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT}" "${ndtmpdir}/install-required-packages.sh" + else + fatal "Invalid given dependency file, please check your --local-files parameter options and try again" + fi + else + download "${PACKAGES_SCRIPT}" "${ndtmpdir}/install-required-packages.sh" + fi + + if [ ! -s "${ndtmpdir}/install-required-packages.sh" ]; then + warning "Downloaded dependency installation script is empty." + else + progress "Running downloaded script to detect required packages..." + run ${sudo} "${bash}" "${ndtmpdir}/install-required-packages.sh" ${PACKAGES_INSTALLER_OPTIONS} + # shellcheck disable=SC2181 + if [ $? -ne 0 ]; then + warning "It failed to install all the required packages, but installation might still be possible." + fi + fi + + fi + fi } safe_sha256sum() { - # Within the contexct of the installer, we only use -c option that is common between the two commands - # We will have to reconsider if we start non-common options - if command -v sha256sum >/dev/null 2>&1; then - sha256sum $@ - elif command -v shasum >/dev/null 2>&1; then - shasum -a 256 $@ - else - fatal "I could not find a suitable checksum binary to use" - fi + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi } # --------------------------------------------------------------------------------------------------------------------- @@ -268,26 +308,8 @@ sudo="" [ "${UID}" -ne "0" ] && sudo="sudo" export PATH="${PATH}:/usr/local/bin:/usr/local/sbin" -# --------------------------------------------------------------------------------------------------------------------- -# try to update using autoupdater in the first place - -updater="" -[ -x /etc/periodic/daily/netdata-updater ] && updater=/etc/periodic/daily/netdata-updater -[ -x /etc/cron.daily/netdata-updater ] && updater=/etc/cron.daily/netdata-updater -if [ -L "${updater}" ]; then - # remove old updater (symlink) - run "${sudo}" rm -f "${updater}" - updater="" -fi -if [ -n "${updater}" ]; then - # attempt to run the updater, to respect any compilation settings already in place - progress "Re-installing netdata..." - run "${sudo}" "${updater}" -f || fatal "Failed to forcefully update netdata" - exit 0 -fi # --------------------------------------------------------------------------------------------------------------------- -# install required system packages INTERACTIVE=1 PACKAGES_INSTALLER_OPTIONS="netdata" @@ -295,68 +317,134 @@ NETDATA_INSTALLER_OPTIONS="" NETDATA_UPDATES="--auto-update" RELEASE_CHANNEL="nightly" while [ -n "${1}" ]; do - if [ "${1}" = "all" ]; then - PACKAGES_INSTALLER_OPTIONS="netdata-all" - shift 1 - elif [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ]; then - INTERACTIVE=0 - shift 1 - elif [ "${1}" = "--no-updates" ]; then - # echo >&2 "netdata will not auto-update" - NETDATA_UPDATES= - shift 1 - elif [ "${1}" = "--stable-channel" ]; then - RELEASE_CHANNEL="stable" - NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --stable-channel" - shift 1 - elif [ "${1}" = "--local-files" ]; then - shift 1 - if [ -z "${1}" ]; then - fatal "Missing netdata: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" - fi - - export NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" - shift 1 - - if [ -z "${1}" ]; then - fatal "Missing checksum file: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" - fi - - export NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" - shift 1 - - if [ -z "${1}" ]; then - fatal "Missing go.d tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" - fi - - export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN="${1}" - shift 1 - - if [ -z "${1}" ]; then - fatal "Missing go.d config tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" - fi - - export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN_CONFIG="${1}" - shift 1 - - if [ -z "${1}" ]; then - fatal "Missing dependencies management scriptlet: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" - fi - - export NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT="${1}" - shift 1 - else - break - fi + if [ "${1}" = "all" ]; then + PACKAGES_INSTALLER_OPTIONS="netdata-all" + shift 1 + elif [ "${1}" = "--dont-wait" ] || [ "${1}" = "--non-interactive" ]; then + INTERACTIVE=0 + shift 1 + elif [ "${1}" = "--no-updates" ]; then + # echo >&2 "netdata will not auto-update" + NETDATA_UPDATES= + shift 1 + elif [ "${1}" = "--stable-channel" ]; then + RELEASE_CHANNEL="stable" + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --stable-channel" + shift 1 + elif [ "${1}" = "--allow-duplicate-install" ]; then + NETDATA_ALLOW_DUPLICATE_INSTALL=1 + shift 1 + elif [ "${1}" = "--reinstall" ]; then + NETDATA_REINSTALL=1 + shift 1 + elif [ "${1}" = "--local-files" ]; then + shift 1 + if [ -z "${1}" ]; then + fatal "Missing netdata: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing checksum file: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing go.d tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing go.d config tarball: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_GO_PLUGIN_CONFIG="${1}" + shift 1 + + if [ -z "${1}" ]; then + fatal "Missing dependencies management scriptlet: Option --local-files requires extra information. The desired tarball for netdata, the checksum, the go.d plugin tarball , the go.d plugin config tarball and the dependency management script, in this particular order" + fi + + export NETDATA_LOCAL_TARBALL_OVERRIDE_DEPS_SCRIPT="${1}" + shift 1 + else + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS ${1}" + shift 1 + fi done if [ "${INTERACTIVE}" = "0" ]; then - PACKAGES_INSTALLER_OPTIONS="--dont-wait --non-interactive ${PACKAGES_INSTALLER_OPTIONS}" - NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --dont-wait" + PACKAGES_INSTALLER_OPTIONS="--dont-wait --non-interactive ${PACKAGES_INSTALLER_OPTIONS}" + NETDATA_INSTALLER_OPTIONS="$NETDATA_INSTALLER_OPTIONS --dont-wait" fi -TMPDIR=$(create_tmp_directory) -cd "${TMPDIR}" +# --------------------------------------------------------------------------------------------------------------------- +# look for an existing install and try to update that instead if it exists + +ndpath="$(command -v netdata 2>/dev/null)" +if [ -z "$ndpath" ] && [ -x /opt/netdata/bin/netdata ] ; then + ndpath="/opt/netdata/bin/netdata" +fi + +if [ -n "$ndpath" ] ; then + ndprefix="$(dirname "$(dirname "${ndpath}")")" + + if [ "${ndprefix}" = /usr ] ; then + ndprefix="/" + fi + + progress "Found existing install of Netdata under: ${ndprefix}" + + if [ -r "${ndprefix}/etc/netdata/.environment" ] ; then + ndstatic="$(grep IS_NETDATA_STATIC_BINARY "${ndprefix}/etc/netdata/.environment" | cut -d "=" -f 2 | tr -d \")" + if [ -z "${NETDATA_REINSTALL}" ] && [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ] ; then + if [ -x "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" ] ; then + progress "Attempting to update existing install instead of creating a new one" + if run ${sudo} "${ndprefix}/usr/libexec/netdata/netdata-updater.sh" --not-running-from-cron ; then + progress "Updated existing install at ${ndpath}" + exit 0 + else + fatal "Failed to update existing Netdata install" + exit 1 + fi + else + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] || [ "${ndstatic}" = "yes" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi + else + if [ "${ndstatic}" = "no" ] ; then + progress "User requested reinstall instead of update, proceeding." + else + fatal "Existing install is a static install, please use kickstart-static64.sh instead." + exit 1 + fi + fi + else + progress "Existing install appears to be handled manually or through the system package manager." + if [ -z "${NETDATA_ALLOW_DUPLICATE_INSTALL}" ] ; then + fatal "Existing installation detected which cannot be safely updated by this script, refusing to continue." + exit 1 + else + progress "User explicitly requested duplicate install, proceeding." + fi + fi +fi + +# --------------------------------------------------------------------------------------------------------------------- +# install required system packages + +ndtmpdir=$(create_tmp_directory) +cd "${ndtmpdir}" || exit 1 dependencies @@ -364,32 +452,40 @@ dependencies # download netdata package if [ -z "${NETDATA_LOCAL_TARBALL_OVERRIDE}" ]; then - set_tarball_urls "${RELEASE_CHANNEL}" + set_tarball_urls "${RELEASE_CHANNEL}" - download "${NETDATA_TARBALL_CHECKSUM_URL}" "${TMPDIR}/sha256sum.txt" - download "${NETDATA_TARBALL_URL}" "${TMPDIR}/netdata-latest.tar.gz" + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.tar.gz" else - progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" - run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${TMPDIR}/sha256sum.txt" - run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${TMPDIR}/netdata-latest.tar.gz" + progress "Installation sources were given as input, running installation using \"${NETDATA_LOCAL_TARBALL_OVERRIDE}\"" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE_CHECKSUM}" "${ndtmpdir}/sha256sum.txt" + run cp "${NETDATA_LOCAL_TARBALL_OVERRIDE}" "${ndtmpdir}/netdata-latest.tar.gz" fi -if ! grep netdata-latest.tar.gz "${TMPDIR}/sha256sum.txt" | safe_sha256sum -c - >/dev/null 2>&1; then - fatal "Tarball checksum validation failed. Stopping netdata installation and leaving tarball in ${TMPDIR}" +if ! grep netdata-latest.tar.gz "${ndtmpdir}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Tarball checksum validation failed. Stopping Netdata Agent installation and leaving tarball in ${ndtmpdir}.\nUsually this is a result of an older copy of the file being cached somewhere upstream and can be resolved by retrying in an hour." fi run tar -xf netdata-latest.tar.gz -rm -rf netdata-latest.tar.gz >/dev/null 2>&1 +rm -rf netdata-latest.tar.gz > /dev/null 2>&1 cd netdata-* || fatal "Cannot cd to netdata source tree" # --------------------------------------------------------------------------------------------------------------------- # install netdata from source +install() { + progress "Installing netdata..." + run ${sudo} ./netdata-installer.sh ${NETDATA_UPDATES} ${NETDATA_INSTALLER_OPTIONS} || fatal "netdata-installer.sh exited with error" + if [ -d "${ndtmpdir}" ] && [ ! "${ndtmpdir}" = "/" ]; then + run ${sudo} rm -rf "${ndtmpdir}" > /dev/null 2>&1 + fi +} + if [ -x netdata-installer.sh ]; then - progress "Installing netdata..." - run ${sudo} ./netdata-installer.sh ${NETDATA_UPDATES} ${NETDATA_INSTALLER_OPTIONS} "${@}" || fatal "netdata-installer.sh exited with error" - if [ -d "${TMPDIR}" ] && [ ! "${TMPDIR}" = "/" ]; then - run ${sudo} rm -rf "${TMPDIR}" >/dev/null 2>&1 - fi + install "$@" else - fatal "Cannot install netdata from source (the source directory does not include netdata-installer.sh). Leaving all files in ${TMPDIR}" + if [ "$(find . -mindepth 1 -maxdepth 1 -type d | wc -l)" -eq 1 ] && [ -x "$(find . -mindepth 1 -maxdepth 1 -type d)/netdata-installer.sh" ]; then + cd "$(find . -mindepth 1 -maxdepth 1 -type d)" && install "$@" + else + fatal "Cannot install netdata from source (the source directory does not include netdata-installer.sh). Leaving all files in ${ndtmpdir}" + fi fi diff --git a/packaging/installer/methods/alpine.md b/packaging/installer/methods/alpine.md new file mode 100644 index 00000000..fb448959 --- /dev/null +++ b/packaging/installer/methods/alpine.md @@ -0,0 +1,36 @@ +<!-- +title: "Install Netdata on Alpine 3.x" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/alpine.md +--> + +# Install Netdata on Alpine 3.x + +Execute these commands to install Netdata in Alpine Linux 3.x: + +```sh +# install required packages +apk add alpine-sdk bash curl libuv-dev zlib-dev util-linux-dev libmnl-dev gcc make git autoconf automake pkgconfig python logrotate + +# if you plan to run node.js Netdata plugins +apk add nodejs + +# download Netdata - the directory 'netdata' will be created +git clone https://github.com/netdata/netdata.git --depth=100 +cd netdata + +# build it, install it, start it +./netdata-installer.sh + +# make Netdata start at boot +echo -e "#!/usr/bin/env bash\n/usr/sbin/netdata" >/etc/local.d/netdata.start +chmod 755 /etc/local.d/netdata.start + +# make Netdata stop at shutdown +echo -e "#!/usr/bin/env bash\nkillall netdata" >/etc/local.d/netdata.stop +chmod 755 /etc/local.d/netdata.stop + +# enable the local service to start automatically +rc-update add local +``` + +[](<>) diff --git a/packaging/installer/methods/cloud-providers.md b/packaging/installer/methods/cloud-providers.md new file mode 100644 index 00000000..943a649b --- /dev/null +++ b/packaging/installer/methods/cloud-providers.md @@ -0,0 +1,130 @@ +<!-- +title: "Install Netdata on cloud providers" +description: "The Netdata Agent runs on all popular cloud providers, but often requires additional steps and configuration for full functionality." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/cloud-providers.md +--> + +# Install Netdata on cloud providers + +Netdata is fully compatible with popular cloud providers like Google Cloud Platform (GCP), Amazon Web Services (AWS), +Azure, and others. You can install Netdata on cloud instances to monitor the apps/services running there, or use +multiple instances in a [parent-child streaming](/streaming/README.md) configuration. + +In some cases, using Netdata on these cloud providers requires unique installation or configuration steps. This page +aims to document some of those steps for popular cloud providers. + +> This document is a work-in-progress! If you find new issues specific to a cloud provider, or would like to help +> clarify the correct workaround, please [create an +> issue](https://github.com/netdata/netdata/issues/new?labels=feature+request,+needs+triage&template=feature_request.md) +> with your process and instructions on using the provider's interface to complete the workaround. + +- [Recommended installation methods for cloud providers](#recommended-installation-methods-for-cloud-providers) +- [Post-installation configuration](#post-installation-configuration) + - [Add a firewall rule to access Netdata's dashboard](#add-a-firewall-rule-to-access-netdatas-dashboard) + +## Recommended installation methods for cloud providers + +The best installation method depends on the instance's operating system, distribution, and version. For Linux instances, +we recommend either the [`kickstart.sh` automatic installation script](kickstart.md) or [.deb/.rpm +packages](packages.md). + +To see the full list of approved methods for each operating system/version we support, see our [distribution +matrix](../../DISTRIBUTIONS.md). That table will guide you to the various supported methods for your cloud instance. + +If you have issues with Netdata after installation, look to the sections below to find the issue you're experiencing, +followed by the solution for your provider. + +## Post-installation configuration + +Some cloud providers require you take additional steps to properly configure your instance or its networking to access +all of Netdata's features. + +### Add a firewall rule to access Netdata's dashboard + +If you cannot access Netdata's dashboard on your cloud instance via `http://HOST:19999`, and instead get an error page +from your browser that says, "This site can't be reached" (Chrome) or "Unable to connect" (Firefox), you may need to +configure your cloud provider's firewall. + +Cloud providers often create network-level firewalls that run separately from the instance itself. Both AWS and Google +Cloud Platform calls them Virtual Private Cloud (VPC) networks. These firewalls can apply even if you've disabled +firewalls on the instance itself. Because you can modify these firewalls only via the cloud provider's web interface, +it's easy to overlook them when trying to configure and access Netdata's dashboard. + +You can often confirm a firewall issue by querying the dashboard while connected to the instance via SSH: `curl +http://localhost:19999/api/v1/info`. If you see JSON output, Netdata is running properly. If you try the same `curl` +command from a remote system, and it fails, it's likely that a firewall is blocking your requests. + +Another option is to put Netdata behind web server, which will proxy requests through standard HTTP/HTTPS ports +(80/443), which are likely already open on your instance. We have a number of guides available: + +- [Apache](/docs/Running-behind-apache.md) +- [Nginx](/docs/Running-behind-nginx.md) +- [Caddy](/docs/Running-behind-caddy.md) +- [HAProxy](/docs/Running-behind-haproxy.md) +- [lighttpd](/docs/Running-behind-lighttpd.md) + +The next few sections outline how to add firewall rules to GCP, AWS, and Azure instances. + +#### Google Cloud Platform (GCP) + +To add a firewall rule, go to the [Firewall rules page](https://console.cloud.google.com/networking/firewalls/list) and +click **Create firewall rule**. + +The following configuration has previously worked for Netdata running on GCP instances +([see #7786](https://github.com/netdata/netdata/issues/7786)): + +```conf +Name: <name> +Type: Ingress +Targets: <name-tag> +Filters: 0.0.0.0/0 +Protocols/ports: 19999 +Action: allow +Priority: 1000 +``` + +Read GCP's [firewall documentation](https://cloud.google.com/vpc/docs/using-firewalls) for specific instructions on how +to create a new firewall rule. + +#### Amazon Web Services (AWS) / EC2 + +Sign in to the [AWS console](https://console.aws.amazon.com/) and navigate to the EC2 dashboard. Click on the **Security +Groups** link in the navigation, beneath the **Network & Security** heading. Find the Security Group your instance +belongs to, and either right-click on it or click the **Actions** button above to see a dropdown menu with **Edit +inbound rules**. + +Add a new rule with the following options: + +```conf +Type: Custom TCP +Protocol: TCP +Port Range: 19999 +Source: Anywhere +Description: Netdata +``` + +You can also choose **My IP** as the source if you prefer. + +Click **Save** to apply your new inbound firewall rule. + +#### Azure + +Sign in to the [Azure portal](https://portal.azure.com) and open the virtual machine running Netdata. Click on the +**Networking** link beneath the **Settings** header, then click on the **Add inbound security rule** button. + +Add a new rule with the following options: + +```conf +Source: Any +Source port ranges: 19999 +Destination: Any +Destination port randes: 19999 +Protocol: TCP +Action: Allow +Priority: 310 +Name: Netdata +``` + +Click **Add** to apply your new inbound security rule. + +[](<>) diff --git a/packaging/installer/methods/freebsd.md b/packaging/installer/methods/freebsd.md new file mode 100644 index 00000000..e2af4175 --- /dev/null +++ b/packaging/installer/methods/freebsd.md @@ -0,0 +1,108 @@ +<!-- +title: "Install Netdata on FreeBSD" +description: "Install Netdata on FreeBSD to monitor the health and performance of bare metal or VMs with thousands of real-time, per-second metrics." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freebsd.md +--> + +# Install Netdata on FreeBSD + +> 💡 This document is maintained by Netdata's community, and may not be completely up-to-date. Please double-check the +> details of the installation process, such as version numbers for downloadable packages, before proceeding. +> +> You can help improve this document by [submitting a +> PR](https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freebsd.md) with your recommended +> improvements or changes. Thank you! + +## Install latest version + +This is how to install the latest Netdata version on FreeBSD: + +Install required packages (**need root permission**): + +```sh +pkg install bash e2fsprogs-libuuid git curl autoconf automake pkgconf pidof Judy liblz4 libuv json-c cmake gmake +``` + +Download Netdata: + +```sh +fetch https://github.com/netdata/netdata/releases/download/v1.26.0/netdata-v1.26.0.tar.gz +``` + +> ⚠️ Verify the latest version by either navigating to [Netdata's latest +> release](https://github.com/netdata/netdata/releases/latest) or using `curl`: +> +> ```bash +> basename $(curl -Ls -o /dev/null -w %{url_effective} https://github.com/netdata/netdata/releases/latest) +> ``` + +Unzip the downloaded file: + +```sh +gunzip netdata*.tar.gz && tar xf netdata*.tar && rm -rf netdata*.tar +``` + +Install Netdata in `/opt/netdata`. If you want to enable automatic updates, add `--auto-update` or `-u` to install `netdata-updater` in `cron` (**need root permission**): + +```sh +cd netdata-v* && ./netdata-installer.sh --install /opt && cp /opt/netdata/usr/sbin/netdata-claim.sh /usr/sbin/ +``` + +You also need to enable the `netdata` service in `/etc/rc.conf`: + +```sh +sysrc netdata_enable="YES" +``` + +Finally, and very importantly, update Netdata using the script provided by the Netdata team (**need root permission**): + +```sh +cd /opt/netdata/usr/libexec/netdata/ && ./netdata-updater.sh +``` + +You can now access the Netdata dashboard by navigating to `http://NODE:19999`, replacing `NODE` with the IP address or hostname of your system. + + + +From Netdata v1.12 and above, anonymous usage information is collected by default and sent to Google Analytics. To read +more about the information collected and how to opt-out, check the [anonymous statistics +page](/docs/anonymous-statistics.md). + +## Updating the Agent on FreeBSD +If you have not passed the `--auto-update` or `-u` parameter for the installer to enable automatic updating, repeat the last step to update Netdata whenever a new version becomes available. +The `netdata-updater.sh` script will update your Agent. + +## Optional parameters to alter your installation +| parameters | Description | +|:-----:|-----------| +|`--install <path>`| Install netdata in `<path>.` Ex: `--install /opt` will put netdata in `/opt/netdata`| +| `--dont-start-it` | Do not (re)start netdata after installation| +| `--dont-wait` | Run installation in non-interactive mode| +| `--auto-update` or `-u` | Install netdata-updater in cron to update netdata automatically once per day| +| `--stable-channel` | Use packages from GitHub release pages instead of GCS (nightly updates). This results in less frequent updates| +| `--nightly-channel` | Use most recent nightly updates instead of GitHub releases. This results in more frequent updates| +| `--disable-go` | Disable installation of go.d.plugin| +| `--disable-ebpf` | Disable eBPF Kernel plugin (Default: enabled)| +| `--disable-cloud` | Disable all Netdata Cloud functionality| +| `--require-cloud` | Fail the install if it can't build Netdata Cloud support| +| `--enable-plugin-freeipmi` | Enable the FreeIPMI plugin. Default: enable it when libipmimonitoring is available| +| `--disable-plugin-freeipmi` | Enable the FreeIPMI plugin| +| `--disable-https` | Explicitly disable TLS support| +| `--disable-dbengine` | Explicitly disable DB engine support| +| `--enable-plugin-nfacct` | Enable nfacct plugin. Default: enable it when libmnl and libnetfilter_acct are available| +| `--disable-plugin-nfacct` | Disable nfacct plugin. Default: enable it when libmnl and libnetfilter_acct are available| +| `--enable-plugin-xenstat` | Enable the xenstat plugin. Default: enable it when libxenstat and libyajl are available| +| `--disable-plugin-xenstat` | Disable the xenstat plugin| +| `--enable-backend-kinesis` | Enable AWS Kinesis backend. Default: enable it when libaws_cpp_sdk_kinesis and libraries (it depends on are available)| +| `--disable-backend-kinesis` | Disable AWS Kinesis backend. Default: enable it when libaws_cpp_sdk_kinesis and libraries (it depends on are available)| +| `--enable-backend-prometheus-remote-write` | Enable Prometheus remote write backend. Default: enable it when libprotobuf and libsnappy are available| +| `--disable-backend-prometheus-remote-write` | Disable Prometheus remote write backend. Default: enable it when libprotobuf and libsnappy are available| +| `--enable-backend-mongodb` | Enable MongoDB backend. Default: enable it when libmongoc is available| +| `--disable-backend-mongodb` | Disable MongoDB backend| +| `--enable-lto` | Enable Link-Time-Optimization. Default: enabled| +| `--disable-lto` | Disable Link-Time-Optimization. Default: enabled| +| `--disable-x86-sse` | Disable SSE instructions. By default SSE optimizations are enabled| +| `--zlib-is-really-here` or `--libs-are-really-here` | If you get errors about missing zlib or libuuid but you know it is available, you might have a broken pkg-config. Use this option to proceed without checking pkg-config| +|`--disable-telemetry` | Use this flag to opt-out from our anonymous telemetry program. (DO_NOT_TRACK=1)| + +[](<>) diff --git a/packaging/installer/methods/freenas.md b/packaging/installer/methods/freenas.md new file mode 100644 index 00000000..a0dafdff --- /dev/null +++ b/packaging/installer/methods/freenas.md @@ -0,0 +1,24 @@ +<!-- +title: "Install Netdata on FreeNAS" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/freenas.md +--> + +# Install Netdata on FreeNAS + +On FreeNAS-Corral-RELEASE (>=10.0.3 and <11.3), Netdata is pre-installed. + +To use Netdata, the service will need to be enabled and started from the FreeNAS [CLI](https://github.com/freenas/cli). + +To enable the Netdata service: + +```sh +service netdata config set enable=true +``` + +To start the Netdata service: + +```sh +service netdata start +``` + +[](<>) diff --git a/packaging/installer/methods/kickstart-64.md b/packaging/installer/methods/kickstart-64.md new file mode 100644 index 00000000..120cc9e1 --- /dev/null +++ b/packaging/installer/methods/kickstart-64.md @@ -0,0 +1,96 @@ +<!-- +title: "Install Netdata with kickstart-static64.sh" +description: "The kickstart-static64.sh script installs a pre-compiled static binary, including all dependencies required to connect to Netdata Cloud, with one command." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kickstart-64.md +--> + +# Install Netdata with kickstart-static64.sh + +  + +This page covers detailed instructions on using and configuring the installation script named `kickstart-static64.sh`. + +This method uses a pre-compiled static binary to install Netdata on any Intel/AMD 64bit Linux system and on any Linux +distribution, even those with a broken or unsupported package manager. + +To install Netdata from a static binary package, including all dependencies required to connect to Netdata Cloud, and +get _automatic nightly updates_, run the following as your normal user: + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart-static64.sh) +``` + +> This script installs Netdata at `/opt/netdata`. + +> See our [installation guide](/packaging/installer/README.md) for details about [automatic +> updates](/packaging/installer/README.md#automatic-updates) or [nightly vs. stable +> releases](/packaging/installer/README.md#nightly-vs-stable-releases). + +## What does `kickstart-static64.sh` do? + +The `kickstart.sh` script does the following after being downloaded and run: + +- Checks to see if there is an existing installation, and if there is updates that in preference to reinstalling. +- Downloads the latest Netdata binary from the [binary-packages](https://github.com/netdata/binary-packages) + repository. You can also run any of these `.run` files with [makeself](https://github.com/megastep/makeself). +- Installs Netdata by running `./netdata-installer.sh` from the source tree, including any options you might have + added. +- Installs `netdata-updater.sh` to `cron.daily` to enable automatic updates, unless you added the `--no-updates` + option. +- Prints a message about whether the installation succeeded for failed for QA purposes. + +If your shell fails to handle the above one-liner, you can download and run the `kickstart-static64.sh` script manually. + +```sh +# download the script with curl +curl https://my-netdata.io/kickstart-static64.sh >/tmp/kickstart-static64.sh + +# or, download the script with wget +wget -O /tmp/kickstart-static64.sh https://my-netdata.io/kickstart-static64.sh + +# run the downloaded script (any sh is fine, no need for bash) +sh /tmp/kickstart-static64.sh +``` + +## Optional parameters to alter your installation + +The `kickstart-static64.sh` script passes all its parameters to `netdata-installer.sh`, which you can use to customize +your installation. Here are a few important parameters: + +- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. +- `--dont-start-it`: Prevent the installer from starting Netdata automatically. +- `--stable-channel`: Automatically update only on the release of new major versions. +- `--nightly-channel`: Automatically update on every new nightly build. +- `--disable-telemetry`: Opt-out of [anonymous statistics](/docs/anonymous-statistics.md) we use to make + Netdata better. +- `--no-updates`: Prevent automatic updates of any kind. +- `--reinstall`: If an existing installation is detected, reinstall instead of attempting to update it. Note + that this cannot be used to switch between installation types. +- `--local-files`: Used for [offline installations](/packaging/installer/methods/offline.md). Pass four file paths: + the Netdata tarball, the checksum file, the go.d plugin tarball, and the go.d plugin config tarball, to force + kickstart run the process using those files. This option conflicts with the `--stable-channel` option. If you set + this _and_ `--stable-channel`, Netdata will use the local files. + +## Verify script integrity + +To use `md5sum` to verify the integrity of the `kickstart-static64.sh` script you will download using the one-line +command above, run the following: + +```bash +[ "047c86a7c8905955bee39b6980a28e30" = "$(curl -Ss https://my-netdata.io/kickstart-static64.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" +``` + +If the script is valid, this command will return `OK, VALID`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/kickstart.md b/packaging/installer/methods/kickstart.md new file mode 100644 index 00000000..f825f808 --- /dev/null +++ b/packaging/installer/methods/kickstart.md @@ -0,0 +1,79 @@ +<!-- +title: "Install Netdata with kickstart.sh" +description: "The kickstart.sh script installs Netdata from source, including all dependencies required to connect to Netdata Cloud, with a single command." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kickstart.md +--> + +# Install Netdata with kickstart.sh + +  + +This page covers detailed instructions on using and configuring the automatic one-line installation script named +`kickstart.sh`. + +This method is fully automatic on all Linux distributions. To install Netdata from source, including all dependencies +required to connect to Netdata Cloud, and get _automatic nightly updates_, run the following as your normal user: + +```bash +bash <(curl -Ss https://my-netdata.io/kickstart.sh) +``` + +> See our [installation guide](../README.md) for details about [automatic updates](../README.md#automatic-updates) or +> [nightly vs. stable releases](../README.md#nightly-vs-stable-releases). + +## What does `kickstart.sh` do? + +The `kickstart.sh` script does the following after being downloaded and run using `bash`: + +- Detects the Linux distribution and **installs the required system packages** for building Netdata. Unless you added + the `--dont-wait` option, it will ask for your permission first. +- Checks for an existing installation, and if found updates that instead of creating a new install. +- Downloads the latest Netdata source tree to `/usr/src/netdata.git`. +- Installs Netdata by running `./netdata-installer.sh` from the source tree, using any [optional + parameters](#optional-parameters-to-alter-your-installation) you have specified. +- Installs `netdata-updater.sh` to `cron.daily`, so your Netdata installation will be updated with new nightly + versions, unless you override that with an [optional parameter](#optional-parameters-to-alter-your-installation). +- Prints a message whether installation succeeded or failed for QA purposes. + +## Optional parameters to alter your installation + +The `kickstart.sh` script passes all its parameters to `netdata-installer.sh`, which you can use to customize your +installation. Here are a few important parameters: + +- `--dont-wait`: Enable automated installs by not prompting for permission to install any required packages. +- `--dont-start-it`: Prevent the installer from starting Netdata automatically. +- `--stable-channel`: Automatically update only on the release of new major versions. +- `--nightly-channel`: Automatically update on every new nightly build. +- `--disable-telemetry`: Opt-out of [anonymous statistics](/docs/anonymous-statistics.md) we use to make + Netdata better. +- `--no-updates`: Prevent automatic updates of any kind. +- `--reinstall`: If an existing install is detected, reinstall instead of trying to update it. Note that this + cannot be used to change installation types. +- `--local-files`: Used for [offline installations](offline.md). Pass four file paths: the Netdata + tarball, the checksum file, the go.d plugin tarball, and the go.d plugin config tarball, to force kickstart run the + process using those files. This option conflicts with the `--stable-channel` option. If you set this _and_ + `--stable-channel`, Netdata will use the local files. + +## Verify script integrity + +To use `md5sum` to verify the integrity of the `kickstart.sh` script you will download using the one-line command above, +run the following: + +```bash +[ "8df7a45b2abb336c84507b7c107bcba3" = "$(curl -Ss https://my-netdata.io/kickstart.sh | md5sum | cut -d ' ' -f 1)" ] && echo "OK, VALID" || echo "FAILED, INVALID" +``` + +If the script is valid, this command will return `OK, VALID`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/kubernetes.md b/packaging/installer/methods/kubernetes.md new file mode 100644 index 00000000..3e85928e --- /dev/null +++ b/packaging/installer/methods/kubernetes.md @@ -0,0 +1,209 @@ +<!-- +title: "Install Netdata on a Kubernetes cluster" +description: "Use Netdata's Helm chart to bootstrap a Netdata monitoring and troubleshooting toolkit on your Kubernetes (k8s) cluster." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/kubernetes.md +--> + +# Install Netdata on a Kubernetes cluster + +This document details how to install Netdata on an existing Kubernetes (k8s) cluster. By following these directions, you +will use Netdata's [Helm chart](https://github.com/netdata/helmchart) to bootstrap a Netdata deployment on your cluster. +The Helm chart installs one parent pod for storing metrics and managing alarm notifications plus an additional child pod +for every node in the cluster. + +Each child pod will collect metrics from the node it runs on, in addition to [compatible +applications](https://github.com/netdata/helmchart#service-discovery-and-supported-services), plus any endpoints covered +by our [generic Prometheus collector](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus), +via [service discovery](https://github.com/netdata/agent-service-discovery/). Each child pod will also collect +[cgroups](/collectors/cgroups.plugin/README.md), +[Kubelet](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubelet), and +[kube-proxy](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubeproxy) metrics from its node. + +To install Netdata on a Kubernetes cluster, you need: + +- A working cluster running Kubernetes v1.9 or newer. +- The [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) command line tool, within [one minor version + difference](https://kubernetes.io/docs/tasks/tools/install-kubectl/#before-you-begin) of your cluster, on an + administrative system. +- The [Helm package manager](https://helm.sh/) v3.0.0 or newer on the same administrative system. + +The default configuration creates one `parent` pod, installed on one of your cluster's nodes, and a DaemonSet for +additional `child` pods. This DaemonSet ensures that every node in your k8s cluster also runs a `child` pod, including +the node that also runs `parent`. The `child` pods collect metrics and stream the information to the `parent` pod, which +uses two persistent volumes to store metrics and alarms. The `parent` pod also handles alarm notifications and enables +the Netdata dashboard using an ingress controller. + +## Install the Netdata Helm chart + +We recommend you install the Helm chart using our Helm repository. In the `helm install` command, replace `netdata` with +the release name of your choice. + +```bash +helm repo add netdata https://netdata.github.io/helmchart/ +helm install netdata netdata/netdata +``` + +> You can also install the Netdata Helm chart by cloning the +> [repository](https://artifacthub.io/packages/helm/netdata/netdata#install-by-cloning-the-repository) and manually +> running Helm against the included chart. + +### Post-installation + +Run `kubectl get services` and `kubectl get pods` to confirm that your cluster now runs a `netdata` service, one +`parent` pod, and three `child` pods. + +You've now installed Netdata on your Kubernetes cluster. See how to [access the Netdata +dashboard](#access-the-netdata-dashboard) to confirm it's working as expected, or see the next section to [configure the +Helm chart](#configure-the-netdata-helm-chart) to suit your cluster's particular setup. + +## Configure the Netdata Helm chart + +Read up on the various configuration options in the [Helm chart +documentation](https://github.com/netdata/helmchart#configuration) to see if you need to change any of the options based +on your cluster's setup. + +To change a setting, use the `--set` or `--values` arguments with `helm install`, for the initial deployment, or `helm upgrade` to upgrade an existing deployment. + +```bash +helm install --set a.b.c=xyz netdata netdata/netdata +helm upgrade --set a.b.c=xyz netdata netdata/netdata +``` + +For example, to change the size of the persistent metrics volume on the parent node: + +```bash +helm install --set parent.database.volumesize=4Gi netdata netdata/netdata +helm upgrade --set parent.database.volumesize=4Gi netdata netdata/netdata +``` + +### Configure service discovery + +As mentioned in the introduction, Netdata has a [service discovery +plugin](https://github.com/netdata/agent-service-discovery/#service-discovery) to identify compatible pods and collect +metrics from the service they run. The Netdata Helm chart installs this service discovery plugin into your k8s cluster. + +Service discovery scans your cluster for pods exposed on certain ports and with certain image names. By default, it +looks for its supported services on the ports they most commonly listen on, and using default image names. Service +discovery currently supports [popular +applications](https://github.com/netdata/helmchart#service-discovery-and-supported-services), plus any endpoints covered +by our [generic Prometheus collector](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus). + +If you haven't changed listening ports, image names, or other defaults, service discovery should find your pods, create +the proper configurations based on the service that pod runs, and begin monitoring them immediately after deployment. + +However, if you have changed some of these defaults, you need to copy a file from the Netdata Helm chart repository, +make your edits, and pass the changed file to `helm install`/`helm upgrade`. + +First, copy the file to your administrative system. + +```bash +curl https://raw.githubusercontent.com/netdata/helmchart/master/charts/netdata/sdconfig/child.yml -o child.yml +``` + +Edit the new `child.yml` file according to your needs. See the [Helm chart +configuration](https://github.com/netdata/helmchart#configuration) and the file itself for details. + +You can then run `helm install`/`helm upgrade` with the `--set-file` argument to use your configured `child.yml` file +instead of the default, changing the path if you copied it elsewhere. + +```bash +helm install --set-file sd.child.configmap.from.value=./child.yml netdata netdata/netdata +helm upgrade --set-file sd.child.configmap.from.value=./child.yml netdata netdata/netdata +``` + +Your configured service discovery is now pushed to your cluster. + +## Access the Netdata dashboard + +Accessing the Netdata dashboard itself depends on how you set up your k8s cluster and the Netdata Helm chart. If you +installed the Helm chart with the default `service.type=ClusterIP`, you will need to forward a port to the parent pod. + +```bash +kubectl port-forward netdata-parent-0 19999:19999 +``` + +You can now access the dashboard at `http://CLUSTER:19999`, replacing `CLUSTER` with the IP address or hostname of your +k8s cluster. + +If you set up the Netdata Helm chart with `service.type=LoadBalancer`, you can find the external IP for the load +balancer with `kubectl get services`, under the `EXTERNAL-IP` column. + +```bash +kubectl get services +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +cockroachdb ClusterIP None <none> 26257/TCP,8080/TCP 46h +cockroachdb-public ClusterIP 10.245.148.233 <none> 26257/TCP,8080/TCP 46h +kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 47h +netdata LoadBalancer 10.245.160.131 203.0.113.0 19999:32231/TCP 74m +``` + +In the above example, access the dashboard by navigating to `http://203.0.113.0:19999`. + +## Claim a Kubernetes cluster's parent pod + +You can [claim](/claim/README.md) a cluster's parent Netdata pod to see its real-time metrics alongside any other nodes +you monitor using [Netdata Cloud](https://app.netdata.cloud). + +> Netdata Cloud does not currently support claiming child nodes because the Helm chart does not allocate a persistent +> volume for them. + +Ensure persistence is enabled on the parent pod by running the following `helm upgrade` command. + +```bash +helm upgrade \ + --set parent.database.persistence=true \ + --set parent.alarms.persistence=true \ + netdata netdata/netdata +``` + +Next, find your claiming script in Netdata Cloud by clicking on your Space's dropdown, then **Manage your Space**. Click +the **Nodes** tab. Netdata Cloud shows a script similar to the following: + +```bash +sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud +``` + +You will need the values of `TOKEN` and `ROOM1,ROOM2` for the command, which sets `parent.claiming.enabled`, +`parent.claiming.token`, and `parent.claiming.rooms` to complete the parent pod claiming process. + +Run the following `helm upgrade` command after replacing `TOKEN` and `ROOM1,ROOM2` with the values found in the claiming +script from Netdata Cloud. The quotations are required. + +```bash +helm upgrade \ + --set parent.claiming.enabled=true \ + --set parent.claiming.token="TOKEN" \ + --set parent.claiming.rooms="ROOM1,ROOM2" \ + netdata netdata/netdata +``` + +The cluster terminates the old parent pod and creates a new one with the proper claiming configuration. You can see your +parent pod in Netdata Cloud after a few moments. You can now [build new +dashboards](https://learn.netdata.cloud/docs/cloud/visualize/dashboards) using the parent pod's metrics or run [Metric +Correlations](https://learn.netdata.cloud/docs/cloud/insights/metric-correlations) to troubleshoot anomalies. + + + +## Update/reinstall the Netdata Helm chart + +If you update the Helm chart's configuration, run `helm upgrade` to redeploy your Netdata service, replacing `netdata` +with the name of the release, if you changed it upon installation: + +```bash +helm upgrade netdata netdata/netdata +``` + +## What's next? + +Read the [monitoring a Kubernetes cluster guide](/docs/guides/monitor/kubernetes-k8s-netdata.md) for details on the +various metrics and charts created by the Helm chart and some best practices on real-time troubleshooting using Netdata. + +Check out our [infrastructure](/docs/quickstart/infrastructure.md) for details about additional k8s monitoring features, +and learn more about [configuring the Netdata Agent](/docs/configure/nodes.md) to better understand the settings you +might be interested in changing. + +To further configure Netdata for your cluster, see our [Helm chart repository](https://github.com/netdata/helmchart) and +the [service discovery repository](https://github.com/netdata/agent-service-discovery/). + +[](<>) diff --git a/packaging/installer/methods/macos.md b/packaging/installer/methods/macos.md new file mode 100644 index 00000000..05883a7f --- /dev/null +++ b/packaging/installer/methods/macos.md @@ -0,0 +1,90 @@ +<!-- +title: "Install Netdata on macOS" +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/macos.md +--> + +# Install Netdata on macOS + +Netdata works on macOS, albeit with some limitations. The number of charts displaying system metrics is limited, but you +can use any of Netdata's [external plugins](../../../collectors/plugins.d/README.md) to monitor any services you might +have installed on your macOS system. You could also use a macOS system as the parent node in a [streaming +configuration](/streaming/README.md). + +We recommend installing Netdata with the community-created and -maintained [**Homebrew +package**](#install-netdata-with-the-homebrew-package). + +- [Install Netdata via the Homebrew package](#install-netdata-with-the-homebrew-package) +- [Install Netdata from source](#install-netdata-from-source) + +## Install Netdata with the Homebrew package + +If you don't have [Homebrew](https://brew.sh/) installed already, begin with their installation script: + +```bash +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" +``` + +Next, you can use Homebrew's package, which installs Netdata all its dependencies in a single step: + +```sh +brew install netdata +``` + +> Homebrew will place your Netdata configuration directory at `/usr/local/etc/netdata/`. Use the `edit-config` script +> and the files in this directory to configure Netdata. For reference, you can find stock configuration files at +> `/usr/local/Cellar/netdata/{NETDATA_VERSION}/lib/netdata/conf.d/`. + +Skip on ahead to the [What's next?](#whats-next) section to find links to helpful post-installation guides. + +## Install Netdata from source + +We don't recommend installing Netdata from source on macOS, as it can be difficult to configure and install dependencies +manually. + +First open your terminal of choice and install the Xcode development packages. + +```bash +xcode-select --install +``` + +Click **Install** on the Software Update popup window that appears. Then, use the same terminal session to use Homebrew +to install some of Netdata's prerequisites. You can omit `cmake` in case you do not want to use +[Netdata Cloud](https://learn.netdata.cloud/docs/cloud/). + +```bash +brew install ossp-uuid autoconf automake pkg-config libuv lz4 json-c openssl@1.1 libtool cmake +``` + +If you want to use the [database engine](/database/engine/README.md) to store your metrics, you need to download +and install the [Judy library](https://sourceforge.net/projects/judy/) before proceeding compiling Netdata. + +Next, download Netdata from our GitHub repository: + +```bash +git clone https://github.com/netdata/netdata.git +``` + +Finally, `cd` into the newly-created directory and then start the installer script: + +```bash +cd netdata/ +sudo ./netdata-installer.sh --install /usr/local +``` + +> Your Netdata configuration directory will be at `/usr/local/netdata/`, and your stock configuration directory will +> be at **`/usr/local/lib/netdata/conf.d/`.** +> +> The installer will also install a startup plist to start Netdata when your macOS system boots. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/manual.md b/packaging/installer/methods/manual.md new file mode 100644 index 00000000..6ece9524 --- /dev/null +++ b/packaging/installer/methods/manual.md @@ -0,0 +1,229 @@ +<!-- +title: "Install Netdata on Linux from a Git checkout" +description: "Use the Netdata Agent source code from GitHub, plus helper scripts to set up your system, to install Netdata without packages or binaries." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/manual.md +--> + +# Install Netdata on Linux from a Git checkout + +To install the latest git version of Netdata, please follow these 2 steps: + +1. [Prepare your system](#prepare-your-system) + + Install the required packages on your system. + +2. [Install Netdata](#install-netdata) + + Download and install Netdata. You can also update it the same way. + +## Prepare your system + +Use our automatic requirements installer (_no need to be `root`_), which attempts to find the packages that +should be installed on your system to build and run Netdata. It supports a large variety of major Linux distributions +and other operating systems and is regularly tested. You can find this tool [here](https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh) or run it directly with `bash <(curl -sSL https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh)`. Otherwise read on for how to get requires packages manually: + +- **Alpine** Linux and its derivatives + - You have to install `bash` yourself, before using the installer. + +- **Arch** Linux and its derivatives + - You need arch/aur for package Judy. + +- **Gentoo** Linux and its derivatives + +- **Debian** Linux and its derivatives (including **Ubuntu**, **Mint**) + +- **Red Hat Enterprise Linux** and its derivatives (including **Fedora**, **CentOS**, **Amazon Machine Image**) + - Please note that for RHEL/CentOS you need + [EPEL](http://www.tecmint.com/how-to-enable-epel-repository-for-rhel-centos-6-5/). + In addition, RHEL/CentOS version 6 also need + [OKay](https://okay.com.mx/blog-news/rpm-repositories-for-centos-6-and-7.html) for package libuv version 1. + - CentOS 8 / RHEL 8 requires a bit of extra work. See the dedicated section below. + +- **SUSE** Linux and its derivatives (including **openSUSE**) + +- **SLE12** Must have your system registered with SUSE Customer Center or have the DVD. See + [#1162](https://github.com/netdata/netdata/issues/1162) + +Install the packages for having a **basic Netdata installation** (system monitoring and many applications, without `mysql` / `mariadb`, `postgres`, `named`, hardware sensors and `SNMP`): + +```sh +curl -Ss 'https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata +``` + +Install all the required packages for **monitoring everything Netdata can monitor**: + +```sh +curl -Ss 'https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh' >/tmp/install-required-packages.sh && bash /tmp/install-required-packages.sh -i netdata-all +``` + +If the above do not work for you, please [open a github +issue](https://github.com/netdata/netdata/issues/new?title=packages%20installer%20failed&labels=installation%20help&body=The%20experimental%20packages%20installer%20failed.%0A%0AThis%20is%20what%20it%20says:%0A%0A%60%60%60txt%0A%0Aplease%20paste%20your%20screen%20here%0A%0A%60%60%60) +with a copy of the message you get on screen. We are trying to make it work everywhere (this is also why the script +[reports back](https://github.com/netdata/netdata/issues/2054) success or failure for all its runs). + +--- + +This is how to do it by hand: + +```sh +# Debian / Ubuntu +apt-get install zlib1g-dev uuid-dev libuv1-dev liblz4-dev libjudy-dev libssl-dev libelf-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl python cmake + +# Fedora +dnf install zlib-devel libuuid-devel libuv-devel lz4-devel Judy-devel openssl-devel elfutils-libelf-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python cmake + +# CentOS / Red Hat Enterprise Linux +yum install autoconf automake curl gcc git libmnl-devel libuuid-devel openssl-devel libuv-devel lz4-devel Judy-devel elfutils-libelf-devel make nc pkgconfig python zlib-devel cmake + +# openSUSE +zypper install zlib-devel libuuid-devel libuv-devel liblz4-devel judy-devel libopenssl-devel libelf-devel libmnl-devel gcc make git autoconf autoconf-archive autogen automake pkgconfig curl findutils python cmake +``` + +Once Netdata is compiled, to run it the following packages are required (already installed using the above commands): + +| package | description| +|:-----:|-----------| +| `libuuid` | part of `util-linux` for GUIDs management| +| `zlib` | gzip compression for the internal Netdata web server| +| `libuv` | Multi-platform support library with a focus on asynchronous I/O, version 1 or greater| + +*Netdata will fail to start without the above.* + +Netdata plugins and various aspects of Netdata can be enabled or benefit when these are installed (they are optional): + +| package |description| +|:-----:|-----------| +| `bash`|for shell plugins and **alarm notifications**| +| `curl`|for shell plugins and **alarm notifications**| +| `iproute` or `iproute2`|for monitoring **Linux traffic QoS**<br/>use `iproute2` if `iproute` reports as not available or obsolete| +| `python`|for most of the external plugins| +| `python-yaml`|used for monitoring **beanstalkd**| +| `python-beanstalkc`|used for monitoring **beanstalkd**| +| `python-dnspython`|used for monitoring DNS query time| +| `python-ipaddress`|used for monitoring **DHCPd**<br/>this package is required only if the system has python v2. python v3 has this functionality embedded| +| `python-mysqldb`<br/>or<br/>`python-pymysql`|used for monitoring **mysql** or **mariadb** databases<br/>`python-mysqldb` is a lot faster and thus preferred| +| `python-psycopg2`|used for monitoring **postgresql** databases| +| `python-pymongo`|used for monitoring **mongodb** databases| +| `nodejs`|used for `node.js` plugins for monitoring **named** and **SNMP** devices| +| `lm-sensors`|for monitoring **hardware sensors**| +| `libelf`|for monitoring kernel-level metrics using eBPF| +| `libmnl`|for collecting netfilter metrics| +| `netcat`|for shell plugins to collect metrics from remote systems| + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +Netdata DB engine can be enabled when these are installed (they are optional): + +| package | description| +|:-----:|-----------| +| `liblz4` | Extremely fast compression algorithm, version r129 or greater| +| `Judy` | General purpose dynamic array| +| `openssl`| Cryptography and SSL/TLS toolkit| + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +Netdata Cloud support may require the following packages to be installed: + +| package | description +|:--------:| ----------------------- +| `cmake` | Needed at build time if you aren't using your distribution's version of libwebsockets or are building on a platform other than Linux +| `openssl` | Needed to secure communications with the Netdata Cloud + +*Netdata will greatly benefit if you have the above packages installed, but it will still work without them.* + +### CentOS / RHEL 6.x + +On CentOS / RHEL 6.x, many of the dependencies for Netdata are only +available with versions older than what we need, so special setup is +required if manually installing packages. + +CentOS 6.x: + +- Enable the EPEL repo +- Enable the additional repo from [okay.network](https://okay.network/blog-news/rpm-repositories-for-centos-6-and-7.html) + +And install the minimum required dependencies. + +### CentOS / RHEL 8.x + +For CentOS / RHEL 8.x a lot of development packages have moved out into their +own separate repositories. Some other dependencies are either missing completely +or have to be sourced by 3rd-parties. + +CentOS 8.x: + +- Enable the PowerTools repo +- Enable the EPEL repo +- Enable the Extra repo from [OKAY](https://okay.network/blog-news/rpm-repositories-for-centos-6-and-7.html) + +And install the minimum required dependencies: + +```sh +# Enable config-manager +yum install -y 'dnf-command(config-manager)' + +# Enable PowerTools +yum config-manager --set-enabled powertools + +# Enable EPEL +yum install -y epel-release + +# Install Repo for libuv-devl (NEW) +yum install -y http://repo.okay.com.mx/centos/8/x86_64/release/okay-release-1-3.el8.noarch.rpm + +# Install Devel Packages +yum install autoconf automake curl gcc git cmake libuuid-devel openssl-devel libuv-devel lz4-devel make nc pkgconfig python3 zlib-devel + +# Install Judy-Devel directly +yum install -y http://mirror.centos.org/centos/8/PowerTools/x86_64/os/Packages/Judy-devel-1.0.5-18.module_el8.1.0+217+4d875839.x86_64.rpm +``` + +--- + +### Install Netdata + +Do this to install and run Netdata: + +```sh +# download it - the directory 'netdata' will be created +git clone https://github.com/netdata/netdata.git --depth=100 +cd netdata + +# run script with root privileges to build, install, start Netdata +./netdata-installer.sh +``` + +- If you don't want to run it straight-away, add `--dont-start-it` option. + +- You can also append `--stable-channel` to fetch and install only the official releases from GitHub, instead of the nightly builds. + +- If you don't want to install it on the default directories, you can run the installer like this: `./netdata-installer.sh --install /opt`. This one will install Netdata in `/opt/netdata`. + +- If your server does not have access to the internet and you have manually put the installation directory on your server, you will need to pass the option `--disable-go` to the installer. The option will prevent the installer from attempting to download and install `go.d.plugin`. + +Once the installer completes, the file `/etc/netdata/netdata.conf` will be created (if you changed the installation directory, the configuration will appear in that directory too). + +You can edit this file to set options. One common option to tweak is `history`, which controls the size of the memory database Netdata will use. By default is `3600` seconds (an hour of data at the charts) which makes Netdata use about 10-15MB of RAM (depending on the number of charts detected on your system). Check **\[[Memory Requirements]]**. + +To apply the changes you made, you have to restart Netdata. + +### 'nonrepresentable section on output' errors + +Our current build process unfortunately has some issues when using certain configurations of the `clang` C compiler on Linux. + +If the installation fails with errors like `/bin/ld: externaldeps/libwebsockets/libwebsockets.a(context.c.o): relocation R_X86_64_32 against '.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIC`, and you are trying to build with `clang` on Linux, you will need to build Netdata using GCC to get a fully functional install. + +In most cases, you can do this by running `CC=gcc ./netdata-installer.sh`. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[](<>) diff --git a/packaging/installer/methods/offline.md b/packaging/installer/methods/offline.md new file mode 100644 index 00000000..c978dd61 --- /dev/null +++ b/packaging/installer/methods/offline.md @@ -0,0 +1,90 @@ +<!-- +title: "Install Netdata on offline systems" +description: "Install the Netdata Agent on offline/air gapped systems to benefit from real-time, per-second monitoring without connecting to the internet." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/offline.md +--> + +# Install Netdata on offline systems + +The Netdata Agent installs on offline or air gapped systems with a few additional steps. + +By default, the `kickstart.sh` and `kickstart-static64.sh` download Netdata assets, like the precompiled binary and a +few dependencies, using the system's internet connection, but the Agent installer can also use equivalent files already +present on the local filesystem. + +First, download the required files. If you're using `kickstart.sh`, you need the Netdata tarball, the checksums, the +go.d plugin binary, and the go.d plugin configuration. If you're using `kickstart-static64.sh`, you need only the +Netdata tarball and checksums. + +Download the files you need to a system of yours that's connected to the internet. Use the commands below, or visit the +[latest Netdata release page](https://github.com/netdata/netdata/releases/latest) and [latest go.d plugin release +page](https://github.com/netdata/go.d.plugin/releases) to download the required files manually. + +**If you're using `kickstart.sh`**, use the following commands: + +```bash +cd /tmp + +curl -s https://my-netdata.io/kickstart.sh > kickstart.sh + +# Netdata tarball +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*tar.gz" | cut -d '"' -f 4 | wget -qi - + +# Netdata checksums +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - + +# Netdata dependency handling script +wget -q - https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/install-required-packages.sh + +# go.d plugin +# For binaries for OS types and architectures not listed on [go.d releases](https://github.com/netdata/go.d.plugin/releases/latest), kindly open a github issue and we will do our best to serve your request +export OS=$(uname -s | tr '[:upper:]' '[:lower:]') ARCH=$(uname -m | sed -e 's/i386/386/g' -e 's/i686/386/g' -e 's/x86_64/amd64/g' -e 's/aarch64/arm64/g' -e 's/armv64/arm64/g' -e 's/armv6l/arm/g' -e 's/armv7l/arm/g' -e 's/armv5tel/arm/g') && curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*${OS}-${ARCH}.tar.gz" | cut -d '"' -f 4 | wget -qi - + +# go.d configuration +curl -s https://api.github.com/repos/netdata/go.d.plugin/releases/latest | grep "browser_download_url.*config.tar.gz" | cut -d '"' -f 4 | wget -qi - +``` + +**If you're using `kickstart-static64.sh`**, use the following commands: + +```bash +cd /tmp + +curl -s https://my-netdata.io/kickstart-static64.sh > kickstart-static64.sh + +# Netdata static64 tarball +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*gz.run" | cut -d '"' -f 4 | wget -qi - + +# Netdata checksums +curl -s https://api.github.com/repos/netdata/netdata/releases/latest | grep "browser_download_url.*txt" | cut -d '"' -f 4 | wget -qi - +``` + +Move downloaded files to the `/tmp` directory on the offline system in whichever way your defined policy allows (if +any). + +Now you can run either the `kickstart.sh` or `kickstart-static64.sh` scripts using the `--local-files` option. This +option requires you to specify the location and names of the files you just downloaded. + +> When using `--local-files`, the `kickstart.sh` or `kickstart-static64.sh` scripts won't download any Netdata assets +> from the internet. But, you may still need a connection to install dependencies using your system's package manager. +> The scripts will warn you if your system doesn't have all the dependencies. + +```bash +# kickstart.sh +bash kickstart.sh --local-files /tmp/netdata-(version-number-here).tar.gz /tmp/sha256sums.txt /tmp/go.d.plugin-(version-number-here).(OS)-(architecture).tar.gz /tmp/config.tar.gz /tmp/install-required-packages.sh + +# kickstart-static64.sh +bash kickstart-static64.sh --local-files /tmp/netdata-(version-number-here).gz.run /tmp/sha256sums.txt +``` + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[]() diff --git a/packaging/installer/methods/packages.md b/packaging/installer/methods/packages.md new file mode 100644 index 00000000..cf1e3359 --- /dev/null +++ b/packaging/installer/methods/packages.md @@ -0,0 +1,48 @@ +<!-- +title: "Install Netdata with .deb/.rpm packages" +description: "Install the Netdata Agent with Linux packages that support Ubuntu, Debian, Fedora, RHEL, CentOS, openSUSE, and more." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/packages.md +--> + +# Install Netdata with .deb/.rpm packages + + + +Netdata provides our own flavour of binary packages for the most common operating systems that use with `.deb` and +`.rpm` packaging formats. + +We provide two separate repositories, one for our stable releases and one for our nightly releases. Visit the repository +pages and follow the quick set-up instructions to get started. + +1. Stable releases: Our stable production releases are hosted in the + [netdata/netdata](https://packagecloud.io/netdata/netdata) repository on packagecloud +2. Nightly releases: Our latest releases are hosted in the + [netdata/netdata-edge](https://packagecloud.io/netdata/netdata-edge) repository on packagecloud + +## Using caching proxies with packagecloud repositories + +packagecloud only provides HTTPS access to repositories they host, which means in turn that Netdata's package +repositories are only accessible via HTTPS. This is known to cause issues with some setups that use a caching proxy for +package downloads. + +If you are using such a setup, there are a couple of ways to work around this: + +- Configure your proxy to automatically pass through HTTPS connections without caching them. This is the simplest + solution, but means that downloads of Netdata packages will not be cached. +- Mirror the repository locally on your proxy system, and use that mirror when installing on other systems. This + requires more setup and more disk space on the caching host, but it lets you cache the packages locally. +- Some specific caching proxies may have alternative configuration options to deal with these issues. Find + such options in their documentation. + +## What's next? + +When you're finished with installation, check out our [single-node](/docs/quickstart/single-node.md) or +[infrastructure](/docs/quickstart/infrastructure.md) monitoring quickstart guides based on your use case. + +Or, skip straight to [configuring the Netdata Agent](/docs/configure/nodes.md). + +Read through Netdata's [documentation](https://learn.netdata.cloud/docs), which is structured based on actions and +solutions, to enable features like health monitoring, alarm notifications, long-term metrics storage, exporting to +external databases, and more. + +[]() diff --git a/packaging/installer/methods/pfsense.md b/packaging/installer/methods/pfsense.md new file mode 100644 index 00000000..ee1a453d --- /dev/null +++ b/packaging/installer/methods/pfsense.md @@ -0,0 +1,84 @@ +<!-- +title: "Install Netdata on pfSense" +description: "Install Netdata on pfSense to monitor the health and performance of firewalls with thousands of real-time, per-second metrics." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/pfsense.md +--> + +# Install Netdata on pfSense + +> 💡 This document is maintained by Netdata's community, and may not be completely up-to-date. Please double-check the +> details of the installation process, such as version numbers for downloadable packages, before proceeding. +> +> You can help improve this document by [submitting a +> PR](https://github.com/netdata/netdata/edit/master/packaging/installer/methods/pfsense.md) with your recommended +> improvements or changes. Thank you! + +## Install prerequisites/dependencies + +To install Netdata on pfSense, first run the following command (within a shell or under the **Diagnostics/Command** +prompt within the pfSense web interface). + +```bash +pkg install -y pkgconf bash e2fsprogs-libuuid libuv nano +``` + +Then run the following commands to download various dependencies from the FreeBSD repository. + +```sh +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/Judy-1.0.5_2.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/json-c-0.15_1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-certifi-2020.6.20.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-asn1crypto-1.3.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-pycparser-2.20.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-cffi-1.14.3.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-six-1.15.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-cryptography-2.6.1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-idna-2.10.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-openssl-19.0.0.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-pysocks-1.7.1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-urllib3-1.25.11,1.txz +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/py37-yaml-5.3.1.txz +``` + +> ⚠️ If any of the above commands return a `Not Found` error, you need to manually search for the latest package in the +> [FreeBSD repository](https://www.freebsd.org/ports/). Search for the package's name, such as `py37-cffi`, find the +> latest version number, and update the command accordingly. + +> ⚠️ On pfSense 2.4.5, Python version 3.7 may be installed by the system, in which case you should should not install +> Python from the FreeBSD repository as instructed above. + +> ⚠️ If you are using the `apcupsd` collector, you need to make sure that apcupsd is up before starting Netdata. +> Otherwise a infinitely running `cat` process triggered by the default activated apcupsd charts plugin will eat up CPU +> and RAM (`/tmp/.netdata-charts.d-*/run-*`). This also applies to `OPNsense`. + +## Install Netdata + +You can now install Netdata from the FreeBSD repository. + +```bash +pkg add http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/netdata-1.28.0.txz +``` + +> ⚠️ If the above command returns a `Not Found` error, you need to manually search for the latest version of Netdata in +> the [FreeBSD repository](https://www.freebsd.org/ports/). Search for `netdata`, find the latest version number, and +> update the command accordingly. + +You must edit `/usr/local/etc/netdata/netdata.conf` and change `bind to = 127.0.0.1` to `bind to = 0.0.0.0`. + +To start Netdata manually, run `service netdata onestart`. + +Visit the Netdata dashboard to confirm it's working: `http://<pfsenseIP>:19999` + +To start Netdata automatically every boot, add `service netdata onestart` as a Shellcmd entry within the pfSense web +interface under **Services/Shellcmd**. You'll need to install the Shellcmd package beforehand under **System/Package +Manager/Available Packages**. The Shellcmd Type should be set to `Shellcmd`. + Alternatively more information can be found in +<https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages>, for achieving the same via the command line and +scripts. + +If you experience an issue with `/usr/bin/install` being absent in pfSense 2.3 or earlier, update pfSense or use a +workaround from <https://redmine.pfsense.org/issues/6643> + +**Note:** In pfSense, the Netdata configuration files are located under `/usr/local/etc/netdata`. + +[](<>) diff --git a/packaging/installer/methods/source.md b/packaging/installer/methods/source.md new file mode 100644 index 00000000..e0827fc1 --- /dev/null +++ b/packaging/installer/methods/source.md @@ -0,0 +1,291 @@ +<!-- +title: "Manually build Netdata from source" +description: "Package maintainers and power users may be interested in manually building Netdata from source without using any of our installation scripts." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/source.md +--> + +# Manually build Netdata from source + +These instructions are for advanced users and distribution package +maintainers. Unless this describes you, you almost certainly want +to follow [our guide for manually installing Netdata from a git +checkout](/packaging/installer/methods/manual.md) instead. + +## Required dependencies + +At a bare minimum, Netdata requires the following libraries and tools +to build and run successfully: + +- libuuid +- libuv version 1.0 or newer +- zlib +- GNU autoconf +- GNU automake +- GCC or Xcode (Clang is known to have issues in certain configurations, see [Using Clang](#using-clang)) +- A version of `make` compatible with GNU automake +- Git (we use git in the build system to generate version info, don't need a full install, just a working `git show` command) + +Additionally, the following build time features require additional dependencies: + +- TLS support for the web GUI: + - OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer. +- dbengine metric storage: + - liblz4 r129 or newer + - OpenSSL 1.0 or newer (LibreSSL _amy_ work, but is largely untested). + - [libJudy](http://judy.sourceforge.net/) +- Netdata Cloud support: + - A working internet connection + - A recent version of CMake + - OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer. + - JSON-C (may be provided by the user as shown below, or by the system) + +## Preparing the source tree + +Certain features in Netdata require custom versions of specific libraries, +which the the build system will link statically into Netdata. These +libraries and their header files must be copied into specific locations +in the source tree to be used. + +### Netdata cloud + +Netdata Cloud functionality requires custom builds of libmosquitto and +libwebsockets. + +#### libmosquitto + +Netdata maintains a custom fork of libmosquitto at +https://github.com/netdata/mosquitto with patches to allow for proper +integration with libwebsockets, which is needed for correct operation of +Netdata Cloud functionality. To prepare this library for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/mosquitto.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/mosquitto/releases and + downloading and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. If building on a platform other than Linux, prepare the mosquitto + sources by running `cmake -D WITH_STATIC_LIBRARIES:boolean=YES .` in + the mosquitto source directory. +4. Build mosquitto by running `make -C lib` in the mosquitto source directory. +5. In the Netdata source directory, create a directory called `externaldeps/mosquitto`. +6. Copy `lib/mosquitto.h` from the mosquitto source directory to + `externaldeps/mosquitto/mosquitto.h` in the Netdata source tree. +7. Copy `lib/libmosquitto.a` from the mosquitto source directory to + `externaldeps/mosquitto/libmosquitto.a` in the Netdata source tree. If + building on a platform other than Linux, the file that needs to be + copied will instead be named `lib/libmosquitto_static.a`, but it + still needs to be copied to `externaldeps/mosquitto/libmosquitto.a`. + +#### libwebsockets + +Netdata uses the standard upstream version of libwebsockets located at +https://github.com/warmcat/libwebsockets, but requires a build with SOCKS5 +support, which is not enabled by most pre-built versions. Currently, +we do not support using a system copy of libwebsockets. To prepare this +library for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/libwebsockets.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/warmcat/libwebsockets/releases and + downloading and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Prepare the libwebsockets sources by running `cmake -D + LWS_WITH_SOCKS5:bool=ON .` in the libwebsockets source directory. +4. Build libwebsockets by running `make` in the libwebsockets source + directory. +5. In the Netdata source directory, create a directory called + `externaldeps/libwebsockets`. +6. Copy `lib/libwebsockets.a` from the libwebsockets source directory to + `externaldeps/libwebsockets/libwebsockets.a` in the Netdata source tree. +7. Copy the entire contents of `lib/include` from the libwebsockets source + directory to `externaldeps/libwebsockets/include` in the Netdata source tree. + +#### JSON-C + +Netdata requires the use of JSON-C for JSON parsing when using Netdata +Cloud. Netdata is able to use a system-provided copy of JSON-C, but +some systems may not provide it. If your system does not provide JSON-C, +you can do the following to prepare a copy for the build system: + +1. Verify the tag that Netdata expects to be used by checking the contents + of `packaging/jsonc.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/json-c/json-c and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Prepare the JSON-C sources by running `cmake -DBUILD_SHARED_LIBS=OFF .` + in the JSON-C source directory. +4. Build JSON-C by running `make` in the JSON-C source directory. +5. In the Netdata source directory, create a directory called + `externaldeps/jsonc`. +6. Copy `libjson-c.a` fro the JSON-C source directory to + `externaldeps/jsonc/libjson-c.a` in the Netdata source tree. +7. Copy all of the header files (`*.h`) from the JSON-C source directory + to `externaldeps/jsonc/json-c` in the Netdata source tree. + +## Building Netdata + +Once the source tree has been prepared, Netdata is ready to be configured +and built. Netdata currently uses GNU autotools as it's primary build +system. To build Netdata this way: + +1. Run `autoreconf -ivf` in the Netdata source tree. +2. Run `./configure` in the Netdata source tree. +3. Run `make` in the Netdata source tree. + +### Configure options + +Netdata provides a number of build time configure options. This section +lists some of the ones you are most likely to need: + +- `--prefix`: Specify the prefix under which Netdata will be installed. +- `--with-webdir`: Specify a path relative to the prefix in which to + install the web UI files. +- `--disable-cloud`: Disables all Netdata Cloud functionality for + this build. + +### Using Clang + +Netdata is primarily developed using GCC, but in most cases we also +build just fine using Clang. Under some build configurations of Clang +itself, you may see build failures with the linker reporting errors +about `nonrepresentable section on output`. We currently do not have a +conclusive fix for this issue (the obvious fix leads to other issues which +we haven't been able to fix yet), and unfortunately the only workaround +is to use a different build of Clang or to use GCC. + +### Linking errors relating to OpenSSL + +Netdata's build system currently does not reliably support building +on systems which have multiple ABI incompatible versions of OpenSSL +installed. In such situations, you may encounter linking errors due to +Netdata trying to build against headers for one version but link to a +different version. + +## Additional components + +A full featured install of Netdata requires some additional components +which must be built and installed separately from the main Netdata +agent. All of these should be handled _after_ installing Netdata itself. + +### React dashboard + +The above build steps include a deprecated web UI for Netdata that lacks +support for Netdata Cloud. To get a fully featured dashboard, you must +install our new React dashboard. + +#### Installing the pre-built React dashboard + +We provide pre-built archives of the React dashboard for each release +(these are also used during our normal install process). To use one +of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/dashboard.version` in your Netdata sources. +2. Go to https://github.com/netdata/dashboard/releases and download the + `dashboard.tar.gz` file for the required release. +3. Unpack the downloaded archive to a temporary directory. +4. Copy the contents of the `build` directory from the extracted + archive to `/usr/share/netdata/web` or the equivalent location for + your build of Netdata. This _will_ overwrite some files in the target + location. + +#### Building the React dashboard locally + +Alternatively, you may wish to build the React dashboard locally. Doing +so requires a recent version of Node.JS with a working install of +NPM. Once you have the required tools, do the following: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/dashboard.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/dashboard and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Run `npm install` in the dashboard source tree. +4. Run `npm run build` in the dashboard source tree. +5. Copy the contents of the `build` directory just like step 4 of + installing the pre-built React dashboard. + +### Go collectors + +A number of the collectors for Netdata are written in Go instead of C, +and are developed in a separate repository from the mian Netdata code. +An installation without these collectors is still usable, but will be +unable to collect metrics for a number of network services the system +may be providing. You can either install a pre-built copy of these +collectors, or build them locally. + +#### Installing the pre-built Go collectors + +We provide pre-built binaries of the Go collectors for all the platforms +we officially support. To use one of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/go.d.version` in your Netdata sources. +2. Go to https://github.com/netdata/go.d.plugin/releases, select the + required release, and download the `go.d.plugin-*.tar.gz` file + for your system type and CPu architecture and the `config.tar.gz` + configuration file archive. +3. Extract the `go.d.plugin-*.tar.gz` archive into a temporary + location, and then copy the single file in the archive to + `/usr/libexec/netdata/plugins.d` or the equivalent location for your + build of Netdata and rename it to `go.d.plugin`. +4. Extract the `config.tar.gz` archive to a temporarylocation and then + copy the contents of the archive to `/etc/netdata` or the equivalent + location for your build of Netdata. + +#### Building the Go collectors locally + +Alternatively, you may wish to build the Go collectors locally +yourself. Doing so requires a working installation of Golang 1.13 or +newer. Once you have the required tools, do the following: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/go.d.version` in your Netdata sources. +2. Obtain the sources for that version by either: + - Navigating to https://github.com/netdata/go.d.plugin and downloading + and unpacking the source code archive for that release. + - Cloning the repository with `git` and checking out the required tag. +3. Run `make` in the go.d.plugin source tree. +4. Copy `bin/godplugin` to `/usr/libexec/netdata/plugins.d` or th + equivalent location for your build of Netdata and rename it to + `go.d.plugin`. +5. Copy the contents of the `config` directory to `/etc/netdata` or the + equivalent location for your build of Netdata. + +### eBPF collector + +On Linux systems, Netdata has support for using the kernel's eBPF +interface to monitor performance-related VFS, network, and process events, +allowing for insights into process lifetimes and file access +patterns. Using this functionality requires additional code managed in +a separate repository from the core Netdata agent. You can either install +a pre-built copy of the required code, or build it locally. + +#### Installing the pre-built eBPF code + +We provide pre-built copies of the eBPF code for 64-bit x86 systems +using glibc or musl. To use one of these: + +1. Verify the release version that Netdata expects to be used by checking + the contents of `packaging/ebpf.version` in your Netdata sources. +2. Go to https://github.com/netdata/kernel-collector/releases, select the + required release, and download the `netdata-kernel-collector-*.tar.xz` + file for the libc variant your system uses (either rmusl or glibc). +3. Extract the contents of the archive to a temporary location, and then + copy all of the `.o` and `.so.*` files and the contents of the `library/` + directory to `/usr/libexec/netdata/plugins.d` or the equivalent location + for your build of Netdata. + +#### Building the eBPF code locally + +Alternatively, you may wish to build the eBPF code locally yourself. For +instructions, please consult [the README file for our kernel-collector +repository](https://github.com/netdata/kernel-collector/blob/master/README.md), +which outlines both the required dependencies, as well as multiple +options for building the code. + +[](<>) diff --git a/packaging/installer/methods/synology.md b/packaging/installer/methods/synology.md new file mode 100644 index 00000000..4a0ae355 --- /dev/null +++ b/packaging/installer/methods/synology.md @@ -0,0 +1,52 @@ +<!-- +title: "Install Netdata on Synology" +description: "The Netdata Agent can be installed on AMD64-compatible NAS systems using the 64-bit pre-compiled static binary." +custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/synology.md +--> + +# Install Netdata on Synology + +The documentation previously recommended installing the Debian Chroot package from the Synology community package +sources and then running Netdata from within the chroot. This does not work, as the chroot environment does not have +access to `/proc`, and therefore exposes very few metrics to Netdata. Additionally, [this +issue](https://github.com/SynoCommunity/spksrc/issues/2758), still open as of 2018/06/24, indicates that the Debian +Chroot package is not suitable for DSM versions greater than version 5 and may corrupt system libraries and render the +NAS unable to boot. + +The good news is that the [64-bit static installer](kickstart-64.md) works fine if your NAS is one that uses the amd64 architecture. It +will install the content into `/opt/netdata`, making future removal safe and simple. + +## Run as netdata user + +When Netdata is first installed, it will run as _root_. This may or may not be acceptable for you, and since other +installations run it as the `netdata` user, you might wish to do the same. This requires some extra work: + +1. Creat a group `netdata` via the Synology group interface. Give it no access to anything. +2. Create a user `netdata` via the Synology user interface. Give it no access to anything and a random password. Assign + the user to the `netdata` group. Netdata will chuid to this user when running. +3. Change ownership of the following directories, as defined in [Netdata + Security](/docs/netdata-security.md#security-design): + +```sh +chown -R root:netdata /opt/netdata/usr/share/netdata +chown -R netdata:netdata /opt/netdata/var/lib/netdata /opt/netdata/var/cache/netdata +chown -R netdata:root /opt/netdata/var/log/netdata +``` + +## Create startup script + +Additionally, as of 2018/06/24, the Netdata installer doesn't recognize DSM as an operating system, so no init script is +installed. You'll have to do this manually: + +1. Add [this file](https://gist.github.com/oskapt/055d474d7bfef32c49469c1b53e8225f) as `/etc/rc.netdata`. Make it + executable with `chmod 0755 /etc/rc.netdata`. +2. Add or edit `/etc/rc.local` and add a line calling `/etc/rc.netdata` to have it start on boot: + +```conf +# Netdata startup +[ -x /etc/rc.netdata ] && /etc/rc.netdata start +``` + +3. Make sure `/etc/rc.netdata` is executable: `chmod 0755 /etc/rc.netdata`. + +[](<>) diff --git a/packaging/installer/netdata-uninstaller.sh b/packaging/installer/netdata-uninstaller.sh index 5c8caeaf..f62934f7 100755 --- a/packaging/installer/netdata-uninstaller.sh +++ b/packaging/installer/netdata-uninstaller.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -#shellcheck disable=SC2181 # # This is the netdata uninstaller script # @@ -25,256 +24,419 @@ ENVIRONMENT_FILE="/etc/netdata/.environment" INTERACTIVITY="-i" YES=0 while :; do - case "$1" in - -h | --help) - echo "$usage" >&2 - exit 1 - ;; - -f | --force) - INTERACTIVITY="-f" - shift - ;; - -y | --yes) - YES=1 - shift - ;; - -e | --env) - ENVIRONMENT_FILE="$2" - shift 2 - ;; - -*) - echo "$usage" >&2 - exit 1 - ;; - *) break ;; - esac + case "$1" in + -h | --help) + echo "$usage" >&2 + exit 1 + ;; + -f | --force) + INTERACTIVITY="-f" + shift + ;; + -y | --yes) + YES=1 + shift + ;; + -e | --env) + ENVIRONMENT_FILE="$2" + shift 2 + ;; + -*) + echo "$usage" >&2 + exit 1 + ;; + *) break ;; + esac done if [ "$YES" != "1" ]; then - echo >&2 "This script will REMOVE netdata from your system." - echo >&2 "Run it again with --yes to do it." - exit 1 + echo >&2 "This script will REMOVE netdata from your system." + echo >&2 "Run it again with --yes to do it." + exit 1 fi if [[ $EUID -ne 0 ]]; then - echo >&2 "This script SHOULD be run as root or otherwise it won't delete all installed components." - key="n" - read -r -s -n 1 -p "Do you want to continue as non-root user [y/n] ? " key - if [ "$key" != "y" ] && [ "$key" != "Y" ]; then - exit 1 - fi + echo >&2 "This script SHOULD be run as root or otherwise it won't delete all installed components." + key="n" + read -r -s -n 1 -p "Do you want to continue as non-root user [y/n] ? " key + if [ "$key" != "y" ] && [ "$key" != "Y" ]; then + exit 1 + fi fi # ----------------------------------------------------------------------------- +# portable service command + +service_cmd="$(command -v service 2> /dev/null)" +rcservice_cmd="$(command -v rc-service 2> /dev/null)" +systemctl_cmd="$(command -v systemctl 2> /dev/null)" +service() { + + local cmd="${1}" action="${2}" + + if [ -n "${systemctl_cmd}" ]; then + run "${systemctl_cmd}" "${action}" "${cmd}" + return $? + elif [ -n "${service_cmd}" ]; then + run "${service_cmd}" "${cmd}" "${action}" + return $? + elif [ -n "${rcservice_cmd}" ]; then + run "${rcservice_cmd}" "${cmd}" "${action}" + return $? + fi + return 1 +} + +# ----------------------------------------------------------------------------- setup_terminal() { - TPUT_RESET="" - TPUT_YELLOW="" - TPUT_WHITE="" - TPUT_BGRED="" - TPUT_BGGREEN="" - TPUT_BOLD="" - TPUT_DIM="" - - # Is stderr on the terminal? If not, then fail - test -t 2 || return 1 - - if command -v tput 1>/dev/null 2>&1; then - if [ $(($(tput colors 2>/dev/null))) -ge 8 ]; then - # Enable colors - TPUT_RESET="$(tput sgr 0)" - TPUT_YELLOW="$(tput setaf 3)" - TPUT_WHITE="$(tput setaf 7)" - TPUT_BGRED="$(tput setab 1)" - TPUT_BGGREEN="$(tput setab 2)" - TPUT_BOLD="$(tput bold)" - TPUT_DIM="$(tput dim)" - fi - fi - - return 0 + TPUT_RESET="" + TPUT_YELLOW="" + TPUT_WHITE="" + TPUT_BGRED="" + TPUT_BGGREEN="" + TPUT_BOLD="" + TPUT_DIM="" + + # Is stderr on the terminal? If not, then fail + test -t 2 || return 1 + + if command -v tput 1> /dev/null 2>&1; then + if [ $(($(tput colors 2> /dev/null))) -ge 8 ]; then + # Enable colors + TPUT_RESET="$(tput sgr 0)" + TPUT_YELLOW="$(tput setaf 3)" + TPUT_WHITE="$(tput setaf 7)" + TPUT_BGRED="$(tput setab 1)" + TPUT_BGGREEN="$(tput setab 2)" + TPUT_BOLD="$(tput bold)" + TPUT_DIM="$(tput dim)" + fi + fi + + return 0 } -setup_terminal || echo >/dev/null +setup_terminal || echo > /dev/null run_ok() { - printf >&2 "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD} OK ${TPUT_RESET} ${*} \n\n" + printf >&2 "%s OK %s\n\n" "${TPUT_BGGREEN}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" } run_failed() { - printf >&2 "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD} FAILED ${TPUT_RESET} ${*} \n\n" + printf >&2 "%s FAILED %s\n\n" "${TPUT_BGRED}${TPUT_WHITE}${TPUT_BOLD}" "${TPUT_RESET}" } ESCAPED_PRINT_METHOD= -printf "%q " test >/dev/null 2>&1 -[ $? -eq 0 ] && ESCAPED_PRINT_METHOD="printfq" +if printf "%q " test > /dev/null 2>&1; then + ESCAPED_PRINT_METHOD="printfq" +fi escaped_print() { - if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then - printf "%q " "${@}" - else - printf "%s" "${*}" - fi - return 0 + if [ "${ESCAPED_PRINT_METHOD}" = "printfq" ]; then + printf "%q " "${@}" + else + printf "%s" "${*}" + fi + return 0 } run_logfile="/dev/null" run() { - local user="${USER--}" dir="${PWD}" info info_console - - if [ "${UID}" = "0" ]; then - info="[root ${dir}]# " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " - else - info="[${user} ${dir}]$ " - info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " - fi - - printf >>"${run_logfile}" "${info}" - escaped_print >>"${run_logfile}" "${@}" - printf >>"${run_logfile}" " ... " - - printf >&2 "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" - escaped_print >&2 "${@}" - printf >&2 "${TPUT_RESET}\n" - - "${@}" - - local ret=$? - if [ ${ret} -ne 0 ]; then - run_failed - printf >>"${run_logfile}" "FAILED with exit code ${ret}\n" - else - run_ok - printf >>"${run_logfile}" "OK\n" - fi - - return ${ret} + local user="${USER--}" dir="${PWD}" info info_console + + if [ "${UID}" = "0" ]; then + info="[root ${dir}]# " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]# " + else + info="[${user} ${dir}]$ " + info_console="[${TPUT_DIM}${dir}${TPUT_RESET}]$ " + fi + + { + printf "%s" "${info}" + escaped_print "${@}" + printf "%s" " ... " + } >> "${run_logfile}" + + printf "%s" "${info_console}${TPUT_BOLD}${TPUT_YELLOW}" >&2 + escaped_print >&2 "${@}" + printf "%s\n" "${TPUT_RESET}" >&2 + + "${@}" + + local ret=$? + if [ ${ret} -ne 0 ]; then + run_failed + printf >> "${run_logfile}" "FAILED with exit code %s\n" "${ret}" + else + run_ok + printf >> "${run_logfile}" "OK\n" + fi + + return ${ret} } portable_del_group() { - local groupname="${1}" - - # Check if group exist - echo >&2 "Removing ${groupname} user group ..." - - # Linux - if command -v groupdel 1>/dev/null 2>&1; then - if grep -q "${groupname}" /etc/group; then - run groupdel -f "${groupname}" && return 0 - else - echo >&2 "Group ${groupname} already removed in a previous step." - run_ok - fi - fi - - # mac OS - if command -v dseditgroup 1> /dev/null 2>&1; then - if dseditgroup -o read netdata 1> /dev/null 2>&1; then - run dseditgroup -o delete "${groupname}" && return 0 - else - echo >&2 "Could not find group ${groupname}, nothing to do" - fi - fi - - echo >&2 "Group ${groupname} was not automatically removed, you might have to remove it manually" - return 1 + local groupname="${1}" + + # Check if group exist + echo >&2 "Removing ${groupname} user group ..." + + # Linux + if command -v groupdel 1> /dev/null 2>&1; then + if grep -q "${groupname}" /etc/group; then + run groupdel "${groupname}" && return 0 + else + echo >&2 "Group ${groupname} already removed in a previous step." + return 0 + fi + fi + + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + if dseditgroup -o read netdata 1> /dev/null 2>&1; then + run dseditgroup -o delete "${groupname}" && return 0 + else + echo >&2 "Could not find group ${groupname}, nothing to do" + return 0 + fi + fi + + echo >&2 "Group ${groupname} was not automatically removed, you might have to remove it manually" + return 1 +} + +issystemd() { + local pids p myns ns systemctl + + # if the directory /lib/systemd/system OR /usr/lib/systemd/system (SLES 12.x) does not exit, it is not systemd + if [ ! -d /lib/systemd/system ] && [ ! -d /usr/lib/systemd/system ]; then + return 1 + fi + + # if there is no systemctl command, it is not systemd + systemctl=$(command -v systemctl 2> /dev/null) + if [ -z "${systemctl}" ] || [ ! -x "${systemctl}" ]; then + return 1 + fi + + # if pid 1 is systemd, it is systemd + [ "$(basename "$(readlink /proc/1/exe)" 2> /dev/null)" = "systemd" ] && return 0 + + # if systemd is not running, it is not systemd + pids=$(safe_pidof systemd 2> /dev/null) + [ -z "${pids}" ] && return 1 + + # check if the running systemd processes are not in our namespace + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + for p in ${pids}; do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + # if pid of systemd is in our namespace, it is systemd + [ -n "${myns}" ] && [ "${myns}" = "${ns}" ] && return 0 + done + + # else, it is not systemd + return 1 } portable_del_user() { - local username="${1}" - echo >&2 "Deleting ${username} user account ..." + local username="${1}" + echo >&2 "Deleting ${username} user account ..." - # Linux - if command -v userdel 1>/dev/null 2>&1; then - run userdel -f "${username}" && return 0 - fi + # Linux + if command -v userdel 1> /dev/null 2>&1; then + run userdel -f "${username}" && return 0 + fi - # mac OS - if command -v sysadminctl 1>/dev/null 2>&1; then - run sysadminctl -deleteUser "${username}" && return 0 - fi + # mac OS + if command -v sysadminctl 1> /dev/null 2>&1; then + run sysadminctl -deleteUser "${username}" && return 0 + fi - echo >&2 "User ${username} could not be deleted from system, you might have to remove it manually" - return 1 + echo >&2 "User ${username} could not be deleted from system, you might have to remove it manually" + return 1 } portable_del_user_from_group() { - local groupname="${1}" username="${2}" + local groupname="${1}" username="${2}" - # username is not in group - echo >&2 "Deleting ${username} user from ${groupname} group ..." + # username is not in group + echo >&2 "Deleting ${username} user from ${groupname} group ..." - # Linux - if command -v gpasswd 1>/dev/null 2>&1; then - run gpasswd -d "netdata" "${group}" && return 0 - fi + # Linux + if command -v gpasswd 1> /dev/null 2>&1; then + run gpasswd -d "netdata" "${group}" && return 0 + fi - # FreeBSD - if command -v pw 1>/dev/null 2>&1; then - run pw groupmod "${groupname}" -d "${username}" && return 0 - fi + # FreeBSD + if command -v pw 1> /dev/null 2>&1; then + run pw groupmod "${groupname}" -d "${username}" && return 0 + fi - # BusyBox - if command -v delgroup 1>/dev/null 2>&1; then - run delgroup "${username}" "${groupname}" && return 0 - fi + # BusyBox + if command -v delgroup 1> /dev/null 2>&1; then + run delgroup "${username}" "${groupname}" && return 0 + fi - # mac OS - if command -v dseditgroup 1> /dev/null 2>&1; then - run dseditgroup -o delete -u "${username}" "${groupname}" && return 0 - fi + # mac OS + if command -v dseditgroup 1> /dev/null 2>&1; then + run dseditgroup -o delete -u "${username}" "${groupname}" && return 0 + fi - echo >&2 "Failed to delete user ${username} from group ${groupname} !" - return 1 + echo >&2 "Failed to delete user ${username} from group ${groupname} !" + return 1 } quit_msg() { - echo - if [ "$FILE_REMOVAL_STATUS" -eq 0 ]; then - echo >&2 "Something went wrong :(" - else - echo >&2 "Netdata files were successfully removed from your system" - fi + echo + if [ "$FILE_REMOVAL_STATUS" -eq 0 ]; then + echo >&2 "Something went wrong :(" + else + echo >&2 "Netdata files were successfully removed from your system" + fi } user_input() { - TEXT="$1" - if [ "${INTERACTIVITY}" = "-i" ]; then - read -r -p "$TEXT" >&2 - fi + TEXT="$1" + if [ "${INTERACTIVITY}" = "-i" ]; then + read -r -p "$TEXT" >&2 + fi } rm_file() { - FILE="$1" - if [ -f "${FILE}" ]; then - run rm -v ${INTERACTIVITY} "${FILE}" - fi + FILE="$1" + if [ -f "${FILE}" ]; then + run rm -v ${INTERACTIVITY} "${FILE}" + fi } rm_dir() { - DIR="$1" - if [ -n "$DIR" ] && [ -d "$DIR" ]; then - user_input "Press ENTER to recursively delete directory '$DIR' > " - run rm -v -f -R "${DIR}" - fi + DIR="$1" + if [ -n "$DIR" ] && [ -d "$DIR" ]; then + user_input "Press ENTER to recursively delete directory '$DIR' > " + run rm -v -f -R "${DIR}" + fi +} + +safe_pidof() { + local pidof_cmd + pidof_cmd="$(command -v pidof 2> /dev/null)" + if [ -n "${pidof_cmd}" ]; then + ${pidof_cmd} "${@}" + return $? + else + ps -acxo pid,comm | + sed "s/^ *//g" | + grep netdata | + cut -d ' ' -f 1 + return $? + fi +} + +pidisnetdata() { + if [ -d /proc/self ]; then + if [ -z "$1" ] || [ ! -f "/proc/$1/stat" ]; then + return 1 + fi + [ "$(cut -d '(' -f 2 "/proc/$1/stat" | cut -d ')' -f 1)" = "netdata" ] && return 0 + return 1 + fi + return 0 +} + +stop_netdata_on_pid() { + local pid="${1}" ret=0 count=0 + + pidisnetdata "${pid}" || return 0 + + printf >&2 "Stopping netdata on pid %s ..." "${pid}" + while [ -n "$pid" ] && [ ${ret} -eq 0 ]; do + if [ ${count} -gt 24 ]; then + echo >&2 "Cannot stop the running netdata on pid ${pid}." + return 1 + fi + + count=$((count + 1)) + + pidisnetdata "${pid}" || ret=1 + if [ ${ret} -eq 1 ]; then + break + fi + + if [ ${count} -lt 12 ]; then + run kill "${pid}" 2> /dev/null + ret=$? + else + run kill -9 "${pid}" 2> /dev/null + ret=$? + fi + + test ${ret} -eq 0 && printf >&2 "." && sleep 5 + + done + + echo >&2 + if [ ${ret} -eq 0 ]; then + echo >&2 "SORRY! CANNOT STOP netdata ON PID ${pid} !" + return 1 + fi + + echo >&2 "netdata on pid ${pid} stopped." + return 0 } netdata_pids() { - local p myns ns - myns="$(readlink /proc/self/ns/pid 2>/dev/null)" - for p in \ - $(cat /var/run/netdata.pid 2>/dev/null) \ - $(cat /var/run/netdata/netdata.pid 2>/dev/null) \ - $(pidof netdata 2>/dev/null); do - - ns="$(readlink "/proc/${p}/ns/pid" 2>/dev/null)" - #shellcheck disable=SC2002 - if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then - name="$(cat "/proc/${p}/stat" 2>/dev/null | cut -d '(' -f 2 | cut -d ')' -f 1)" - if [ "${name}" = "netdata" ]; then - echo "${p}" - fi - fi - done + local p myns ns + + myns="$(readlink /proc/self/ns/pid 2> /dev/null)" + + for p in \ + $(cat /var/run/netdata.pid 2> /dev/null) \ + $(cat /var/run/netdata/netdata.pid 2> /dev/null) \ + $(safe_pidof netdata 2> /dev/null); do + ns="$(readlink "/proc/${p}/ns/pid" 2> /dev/null)" + + if [ -z "${myns}" ] || [ -z "${ns}" ] || [ "${myns}" = "${ns}" ]; then + pidisnetdata "${p}" && echo "${p}" + fi + done +} + +stop_all_netdata() { + local p + + if [ "${UID}" -eq 0 ]; then + uname="$(uname 2> /dev/null)" + + # Any of these may fail, but we need to not bail if they do. + if issystemd; then + if systemctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "Darwin" ]; then + if launchctl stop netdata; then + sleep 5 + fi + elif [ "${uname}" = "FreeBSD" ]; then + if /etc/rc.d/netdata stop; then + sleep 5 + fi + else + if service netdata stop; then + sleep 5 + fi + fi + fi + + if [ -n "$(netdata_pids)" ] && [ -n "$(builtin type -P netdatacli)" ]; then + netdatacli shutdown-agent + sleep 20 + fi + + for p in $(netdata_pids); do + # shellcheck disable=SC2086 + stop_netdata_on_pid ${p} + done } trap quit_msg EXIT @@ -284,50 +446,48 @@ source "${ENVIRONMENT_FILE}" || exit 1 #### STOP NETDATA echo >&2 "Stopping a possibly running netdata..." -for p in $(netdata_pids); do - i=0 - while kill "${p}" 2>/dev/null; do - if [ "$i" -gt 30 ]; then - echo >&2 "Forcefully stopping netdata with pid ${p}" - run kill -9 "${p}" - run sleep 2 - break - fi - sleep 1 - i=$((i + 1)) - done -done -sleep 2 +stop_all_netdata #### REMOVE NETDATA FILES rm_file /etc/logrotate.d/netdata rm_file /etc/systemd/system/netdata.service rm_file /lib/systemd/system/netdata.service rm_file /usr/lib/systemd/system/netdata.service +rm_file /etc/systemd/system/netdata-updater.service +rm_file /lib/systemd/system/netdata-updater.service +rm_file /usr/lib/systemd/system/netdata-updater.service +rm_file /etc/systemd/system/netdata-updater.timer +rm_file /lib/systemd/system/netdata-updater.timer +rm_file /usr/lib/systemd/system/netdata-updater.timer rm_file /etc/init.d/netdata rm_file /etc/periodic/daily/netdata-updater rm_file /etc/cron.daily/netdata-updater +rm_file /etc/cron.d/netdata-updater + if [ -n "${NETDATA_PREFIX}" ] && [ -d "${NETDATA_PREFIX}" ]; then - rm_dir "${NETDATA_PREFIX}" + rm_dir "${NETDATA_PREFIX}" else - rm_file "/usr/sbin/netdata" - rm_dir "/usr/share/netdata" - rm_dir "/usr/libexec/netdata" - rm_dir "/var/lib/netdata" - rm_dir "/var/cache/netdata" - rm_dir "/var/log/netdata" - rm_dir "/etc/netdata" + rm_file "/usr/sbin/netdata" + rm_file "/usr/sbin/netdatacli" + rm_file "/tmp/netdata-ipc" + rm_file "/usr/sbin/netdata-claim.sh" + rm_dir "/usr/share/netdata" + rm_dir "/usr/libexec/netdata" + rm_dir "/var/lib/netdata" + rm_dir "/var/cache/netdata" + rm_dir "/var/log/netdata" + rm_dir "/etc/netdata" fi FILE_REMOVAL_STATUS=1 #### REMOVE NETDATA USER FROM ADDED GROUPS if [ -n "$NETDATA_ADDED_TO_GROUPS" ]; then - user_input "Press ENTER to delete 'netdata' from following groups: '$NETDATA_ADDED_TO_GROUPS' > " - for group in $NETDATA_ADDED_TO_GROUPS; do - portable_del_user_from_group "${group}" "netdata" - done + user_input "Press ENTER to delete 'netdata' from following groups: '$NETDATA_ADDED_TO_GROUPS' > " + for group in $NETDATA_ADDED_TO_GROUPS; do + portable_del_user_from_group "${group}" "netdata" + done fi #### REMOVE USER diff --git a/packaging/installer/netdata-updater.sh b/packaging/installer/netdata-updater.sh index 6609edd5..10835cd0 100755 --- a/packaging/installer/netdata-updater.sh +++ b/packaging/installer/netdata-updater.sh @@ -1,162 +1,372 @@ #!/usr/bin/env bash -#shellcheck disable=SC2164 -# + # Netdata updater utility # # Variables needed by script: # - PATH # - CFLAGS +# - LDFLAGS +# - IS_NETDATA_STATIC_BINARY # - NETDATA_CONFIGURE_OPTIONS -# - REINSTALL_COMMAND +# - REINSTALL_OPTIONS # - NETDATA_TARBALL_URL # - NETDATA_TARBALL_CHECKSUM_URL # - NETDATA_TARBALL_CHECKSUM # - NETDATA_PREFIX / NETDATA_LIB_DIR (After 1.16.1 we will only depend on lib dir) # -# Copyright: SPDX-License-Identifier: GPL-3.0-or-later +# Optional environment options: +# +# - TMPDIR (set to a usable temporary directory) +# - NETDATA_NIGHTLIES_BASEURL (set the base url for downloading the dist tarball) +# +# Copyright: 2018-2020 Netdata Inc. +# SPDX-License-Identifier: GPL-3.0-or-later # # Author: Paweł Krupa <paulfantom@gmail.com> # Author: Pavlos Emm. Katsoulakis <paul@netdata.cloud> +# Author: Austin S. Hemmelgarn <austin@netdata.cloud> + +set -e + +script_dir="$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd -P)" + +if [ -x "${script_dir}/netdata-updater" ]; then + script_source="${script_dir}/netdata-updater" +else + script_source="${script_dir}/netdata-updater.sh" +fi info() { - echo >&3 "$(date) : INFO: " "${@}" + echo >&3 "$(date) : INFO: " "${@}" } error() { - echo >&3 "$(date) : ERROR: " "${@}" + echo >&3 "$(date) : ERROR: " "${@}" } +: "${ENVIRONMENT_FILE:=THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT}" + +if [ "${ENVIRONMENT_FILE}" == "THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT" ]; then + if [ -r "${script_dir}/../../../etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="${script_dir}/../../../etc/netdata/.environment" + elif [ -r "/etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="/etc/netdata/.environment" + elif [ -r "/opt/netdata/etc/netdata/.environment" ]; then + ENVIRONMENT_FILE="/opt/netdata/etc/netdata/.environment" + else + envpath="$(find / -type d \( -path /sys -o -path /proc -o -path /dev \) -prune -false -o -path '*netdata/.environment' -type f 2> /dev/null | head -n 1)" + if [ -r "${envpath}" ]; then + ENVIRONMENT_FILE="${envpath}" + else + error "Cannot find environment file, unable to update." + exit 1 + fi + fi +fi + safe_sha256sum() { - # Within the contexct of the installer, we only use -c option that is common between the two commands - # We will have to reconsider if we start non-common options - if command -v sha256sum >/dev/null 2>&1; then - sha256sum $@ - elif command -v shasum >/dev/null 2>&1; then - shasum -a 256 $@ - else - fatal "I could not find a suitable checksum binary to use" - fi + # Within the contexct of the installer, we only use -c option that is common between the two commands + # We will have to reconsider if we start non-common options + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$@" + elif command -v shasum > /dev/null 2>&1; then + shasum -a 256 "$@" + else + fatal "I could not find a suitable checksum binary to use" + fi } # this is what we will do if it fails (head-less only) fatal() { - error "FAILED TO UPDATE NETDATA : ${1}" + error "FAILED TO UPDATE NETDATA : ${1}" + exit 1 +} + +cleanup() { + if [ -n "${logfile}" ]; then + cat >&2 "${logfile}" + rm "${logfile}" + fi + + if [ -n "$ndtmpdir" ] && [ -d "$ndtmpdir" ]; then + rm -rf "$ndtmpdir" + fi +} - if [ -n "${logfile}" ]; then - cat >&2 "${logfile}" - rm "${logfile}" - fi - exit 1 +_cannot_use_tmpdir() { + local testfile ret + testfile="$(TMPDIR="${1}" mktemp -q -t netdata-test.XXXXXXXXXX)" + ret=0 + + if [ -z "${testfile}" ] ; then + return "${ret}" + fi + + if printf '#!/bin/sh\necho SUCCESS\n' > "${testfile}" ; then + if chmod +x "${testfile}" ; then + if [ "$("${testfile}")" = "SUCCESS" ] ; then + ret=1 + fi + fi + fi + + rm -f "${testfile}" + return "${ret}" } create_tmp_directory() { - # Check if tmp is mounted as noexec - if grep -Eq '^[^ ]+ /tmp [^ ]+ ([^ ]*,)?noexec[, ]' /proc/mounts; then - pattern="$(pwd)/netdata-updater-XXXXXX" - else - pattern="/tmp/netdata-updater-XXXXXX" - fi - - mktemp -d "$pattern" + if [ -n "${NETDATA_TMPDIR_PATH}" ]; then + TMPDIR="${NETDATA_TMPDIR_PATH}" + else + if [ -z "${NETDATA_TMPDIR}" ] || _cannot_use_tmpdir "${NETDATA_TMPDIR}" ; then + if [ -z "${TMPDIR}" ] || _cannot_use_tmpdir "${TMPDIR}" ; then + if _cannot_use_tmpdir /tmp ; then + if _cannot_use_tmpdir "${PWD}" ; then + echo >&2 + echo >&2 "Unable to find a usable temprorary directory. Please set \$TMPDIR to a path that is both writable and allows execution of files and try again." + exit 1 + else + TMPDIR="${PWD}" + fi + else + TMPDIR="/tmp" + fi + fi + else + TMPDIR="${NETDATA_TMPDIR}" + fi + + mktemp -d -t netdata-updater-XXXXXXXXXX + fi +} + +_safe_download() { + url="${1}" + dest="${2}" + if command -v curl > /dev/null 2>&1; then + curl -sSL --connect-timeout 10 --retry 3 "${url}" > "${dest}" + return $? + elif command -v wget > /dev/null 2>&1; then + wget -T 15 -O - "${url}" > "${dest}" + return $? + else + return 255 + fi } download() { - url="${1}" - dest="${2}" - if command -v curl >/dev/null 2>&1; then - curl -sSL --connect-timeout 10 --retry 3 "${url}" >"${dest}" || fatal "Cannot download ${url}" - elif command -v wget >/dev/null 2>&1; then - wget -T 15 -O - "${url}" >"${dest}" || fatal "Cannot download ${url}" - else - fatal "I need curl or wget to proceed, but neither is available on this system." - fi + url="${1}" + dest="${2}" + + _safe_download "${url}" "${dest}" + ret=$? + + if [ ${ret} -eq 0 ]; then + return 0 + elif [ ${ret} -eq 255 ]; then + fatal "I need curl or wget to proceed, but neither is available on this system." + else + fatal "Cannot download ${url}" + fi +} + +newer_commit_date() { + echo >&3 "Checking if a newer version of the updater script is available." + + if command -v jq > /dev/null 2>&1; then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | jq '.[0].commit.committer.date' | tr -d '"')" + elif command -v python > /dev/null 2>&1;then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | python -c 'from __future__ import print_function;import sys,json;print(json.load(sys.stdin)[0]["commit"]["committer"]["date"])')" + elif command -v python3 > /dev/null 2>&1;then + commit_date="$(_safe_download "https://api.github.com/repos/netdata/netdata/commits?path=packaging%2Finstaller%2Fnetdata-updater.sh&page=1&per_page=1" /dev/stdout | python3 -c 'from __future__ import print_function;import sys,json;print(json.load(sys.stdin)[0]["commit"]["committer"]["date"])')" + fi + + if [ -z "${commit_date}" ] ; then + commit_date="9999-12-31T23:59:59Z" + fi + + if [ -e "${script_source}" ]; then + script_date="$(date -r "${script_source}" +%s)" + else + script_date="$(date +%s)" + fi + + [ "$(date -d "${commit_date}" +%s)" -ge "${script_date}" ] +} + +self_update() { + if [ -z "${NETDATA_NO_UPDATER_SELF_UPDATE}" ] && newer_commit_date; then + echo >&3 "Downloading newest version of updater script." + + ndtmpdir=$(create_tmp_directory) + cd "$ndtmpdir" || exit 1 + + if _safe_download "https://raw.githubusercontent.com/netdata/netdata/master/packaging/installer/netdata-updater.sh" ./netdata-updater.sh; then + chmod +x ./netdata-updater.sh || exit 1 + export ENVIRONMENT_FILE="${ENVIRONMENT_FILE}" + exec ./netdata-updater.sh --not-running-from-cron --no-self-update --tmpdir-path "$(pwd)" + else + echo >&3 "Failed to download newest version of updater script, continuing with current version." + fi + fi +} + +parse_version() { + r="${1}" + if echo "${r}" | grep -q '^v.*'; then + # shellcheck disable=SC2001 + # XXX: Need a regex group subsitutation here. + r="$(echo "${r}" | sed -e 's/^v\(.*\)/\1/')" + fi + + read -r -a p <<< "$(echo "${r}" | tr '-' ' ')" + + v="${p[0]}" + b="${p[1]}" + _="${p[2]}" # ignore the SHA + + if [[ ! "${b}" =~ ^[0-9]+$ ]]; then + b="0" + fi + + read -r -a pp <<< "$(echo "${v}" | tr '.' ' ')" + printf "%03d%03d%03d%05d" "${pp[0]}" "${pp[1]}" "${pp[2]}" "${b}" +} + +get_latest_version() { + local latest + if [ "${RELEASE_CHANNEL}" == "stable" ]; then + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + else + latest="$(download "$NETDATA_NIGHTLIES_BASEURL/latest-version.txt" /dev/stdout)" + fi + parse_version "$latest" } set_tarball_urls() { - local extension="tar.gz" - - if [ ! -z "${NETDATA_LOCAL_TARBAL_OVERRIDE}" ]; then - info "Not fetching remote tarballs, local override was given" - return - fi - - if [ "$2" == "yes" ]; then - extension="gz.run" - fi - - if [ "$1" = "stable" ]; then - local latest - # Simple version - # latest="$(curl -sSL https://api.github.com/repos/netdata/netdata/releases/latest | grep tag_name | cut -d'"' -f4)" - latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" - export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.${extension}" - export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" - else - export NETDATA_TARBALL_URL="https://storage.googleapis.com/netdata-nightlies/netdata-latest.${extension}" - export NETDATA_TARBALL_CHECKSUM_URL="https://storage.googleapis.com/netdata-nightlies/sha256sums.txt" - fi + local extension="tar.gz" + + if [ "$2" == "yes" ]; then + extension="gz.run" + fi + + if [ "$1" = "stable" ]; then + local latest + # Simple version + latest="$(download "https://api.github.com/repos/netdata/netdata/releases/latest" /dev/stdout | grep tag_name | cut -d'"' -f4)" + export NETDATA_TARBALL_URL="https://github.com/netdata/netdata/releases/download/$latest/netdata-$latest.${extension}" + export NETDATA_TARBALL_CHECKSUM_URL="https://github.com/netdata/netdata/releases/download/$latest/sha256sums.txt" + else + export NETDATA_TARBALL_URL="$NETDATA_NIGHTLIES_BASEURL/netdata-latest.${extension}" + export NETDATA_TARBALL_CHECKSUM_URL="$NETDATA_NIGHTLIES_BASEURL/sha256sums.txt" + fi } update() { - [ -z "${logfile}" ] && info "Running on a terminal - (this script also supports running headless from crontab)" - - RUN_INSTALLER=0 - tmpdir=$(create_tmp_directory) - cd "$tmpdir" - - if [ -z "${NETDATA_LOCAL_TARBAL_OVERRIDE}" ]; then - download "${NETDATA_TARBALL_CHECKSUM_URL}" "${tmpdir}/sha256sum.txt" >&3 2>&3 - if [[ -n "${NETDATA_TARBALL_CHECKSUM}" ]] && grep "${NETDATA_TARBALL_CHECKSUM}" sha256sum.txt >&3 2>&3; then - info "Newest version is already installed" - else - download "${NETDATA_TARBALL_URL}" "${tmpdir}/netdata-latest.tar.gz" - if ! grep netdata-latest.tar.gz sha256sum.txt | safe_sha256sum -c - >&3 2>&3; then - fatal "Tarball checksum validation failed. Stopping netdata upgrade and leaving tarball in ${tmpdir}" - fi - NEW_CHECKSUM="$(safe_sha256sum netdata-latest.tar.gz 2>/dev/null| cut -d' ' -f1)" - tar -xf netdata-latest.tar.gz >&3 2>&3 - rm netdata-latest.tar.gz >&3 2>&3 - cd netdata-* - RUN_INSTALLER=1 - fi - else - info "!!Local tarball override detected!! - Entering directory ${NETDATA_LOCAL_TARBAL_OVERRIDE} for installation, not downloading anything" - RUN_INSTALLER=1 - cd ${NETDATA_LOCAL_TARBAL_OVERRIDE} - fi - - - # We got the sources, run the update now - if [ ${RUN_INSTALLER} -eq 1 ]; then - # signal netdata to start saving its database - # this is handy if your database is big - pids=$(pidof netdata) - do_not_start= - if [ -n "${pids}" ]; then - #shellcheck disable=SC2086 - kill -USR1 ${pids} - else - # netdata is currently not running, so do not start it after updating - do_not_start="--dont-start-it" - fi - - info "Re-installing netdata..." - eval "${REINSTALL_COMMAND} --dont-wait ${do_not_start}" >&3 2>&3 || fatal "FAILED TO COMPILE/INSTALL NETDATA" - - # We no longer store checksum info here. but leave this so that we clean up all environment files upon next update. - sed -i '/NETDATA_TARBALL/d' "${ENVIRONMENT_FILE}" - - info "Updating tarball checksum info" - echo "${NEW_CHECKSUM}" > "${NETDATA_LIB_DIR}/netdata.tarball.checksum" - fi - - rm -rf "${tmpdir}" >&3 2>&3 - [ -n "${logfile}" ] && rm "${logfile}" && logfile= - return + [ -z "${logfile}" ] && info "Running on a terminal - (this script also supports running headless from crontab)" + + RUN_INSTALLER=0 + ndtmpdir=$(create_tmp_directory) + cd "$ndtmpdir" || exit 1 + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" >&3 2>&3 + + current_version="$(command -v netdata > /dev/null && parse_version "$(netdata -v | cut -f 2 -d ' ')")" + latest_version="$(get_latest_version)" + + # If we can't get the current version for some reason assume `0` + current_version="${current_version:-0}" + + # If we can't get the latest version for some reason assume `0` + latest_version="${latest_version:-0}" + + info "Current Version: ${current_version}" + info "Latest Version: ${latest_version}" + + if [ "${latest_version}" -gt 0 ] && [ "${current_version}" -gt 0 ] && [ "${current_version}" -ge "${latest_version}" ]; then + info "Newest version (current=${current_version} >= latest=${latest_version}) is already installed" + elif [ -n "${NETDATA_TARBALL_CHECKSUM}" ] && grep "${NETDATA_TARBALL_CHECKSUM}" sha256sum.txt >&3 2>&3; then + info "Newest version is already installed" + else + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.tar.gz" + if ! grep netdata-latest.tar.gz sha256sum.txt | safe_sha256sum -c - >&3 2>&3; then + fatal "Tarball checksum validation failed. Stopping netdata upgrade and leaving tarball in ${ndtmpdir}\nUsually this is a result of an older copy of the tarball or checksum file being cached somewhere upstream and can be resolved by retrying in an hour." + fi + NEW_CHECKSUM="$(safe_sha256sum netdata-latest.tar.gz 2> /dev/null | cut -d' ' -f1)" + tar -xf netdata-latest.tar.gz >&3 2>&3 + rm netdata-latest.tar.gz >&3 2>&3 + cd netdata-* || exit 1 + RUN_INSTALLER=1 + cd "${NETDATA_LOCAL_TARBAL_OVERRIDE}" || exit 1 + fi + + # We got the sources, run the update now + if [ ${RUN_INSTALLER} -eq 1 ]; then + # signal netdata to start saving its database + # this is handy if your database is big + possible_pids=$(pidof netdata) + do_not_start= + if [ -n "${possible_pids}" ]; then + read -r -a pids_to_kill <<< "${possible_pids}" + kill -USR1 "${pids_to_kill[@]}" + else + # netdata is currently not running, so do not start it after updating + do_not_start="--dont-start-it" + fi + + if [ -n "${NETDATA_SELECTED_DASHBOARD}" ]; then + env="NETDATA_SELECTED_DASHBOARD=${NETDATA_SELECTED_DASHBOARD}" + fi + + if [ ! -x ./netdata-installer.sh ]; then + if [ "$(find . -mindepth 1 -maxdepth 1 -type d | wc -l)" -eq 1 ] && [ -x "$(find . -mindepth 1 -maxdepth 1 -type d)/netdata-installer.sh" ]; then + cd "$(find . -mindepth 1 -maxdepth 1 -type d)" || exit 1 + fi + fi + + info "Re-installing netdata..." + eval "${env} ./netdata-installer.sh ${REINSTALL_OPTIONS} --dont-wait ${do_not_start}" >&3 2>&3 || fatal "FAILED TO COMPILE/INSTALL NETDATA" + + # We no longer store checksum info here. but leave this so that we clean up all environment files upon next update. + sed -i '/NETDATA_TARBALL/d' "${ENVIRONMENT_FILE}" + + info "Updating tarball checksum info" + echo "${NEW_CHECKSUM}" > "${NETDATA_LIB_DIR}/netdata.tarball.checksum" + fi + + rm -rf "${ndtmpdir}" >&3 2>&3 + [ -n "${logfile}" ] && rm "${logfile}" && logfile= + + return 0 } -# Usually stored in /etc/netdata/.environment -: "${ENVIRONMENT_FILE:=THIS_SHOULD_BE_REPLACED_BY_INSTALLER_SCRIPT}" +logfile= +ndtmpdir= + +trap cleanup EXIT + +while [ -n "${1}" ]; do + if [ "${1}" = "--not-running-from-cron" ]; then + NETDATA_NOT_RUNNING_FROM_CRON=1 + shift 1 + elif [ "${1}" = "--no-updater-self-update" ]; then + NETDATA_NO_UPDATER_SELF_UPDATE=1 + shift 1 + elif [ "${1}" = "--tmpdir-path" ]; then + NETDATA_TMPDIR_PATH="${2}" + shift 2 + else + break + fi +done + +# Random sleep to aileviate stampede effect of Agents upgrading +# and disconnecting/reconnecting at the same time (or near to). +# But only we're not a controlling terminal (tty) +# Randomly sleep between 1s and 60m +if [ ! -t 1 ] && [ -z "${NETDATA_NOT_RUNNING_FROM_CRON}" ]; then + sleep $(((RANDOM % 3600) + 1)) +fi # shellcheck source=/dev/null source "${ENVIRONMENT_FILE}" || exit 1 @@ -167,51 +377,52 @@ export NETDATA_LIB_DIR="${NETDATA_LIB_DIR:-${NETDATA_PREFIX}/var/lib/netdata}" # Source the tarbal checksum, if not already available from environment (for existing installations with the old logic) [[ -z "${NETDATA_TARBALL_CHECKSUM}" ]] && [[ -f ${NETDATA_LIB_DIR}/netdata.tarball.checksum ]] && NETDATA_TARBALL_CHECKSUM="$(cat "${NETDATA_LIB_DIR}/netdata.tarball.checksum")" +# Grab the nightlies baseurl (defaulting to our Google Storage bucket) +export NETDATA_NIGHTLIES_BASEURL="${NETDATA_NIGHTLIES_BASEURL:-https://storage.googleapis.com/netdata-nightlies}" + if [ "${INSTALL_UID}" != "$(id -u)" ]; then - fatal "You are running this script as user with uid $(id -u). We recommend to run this script as root (user with uid 0)" + fatal "You are running this script as user with uid $(id -u). We recommend to run this script as root (user with uid 0)" fi -logfile= if [ -t 2 ]; then - # we are running on a terminal - # open fd 3 and send it to stderr - exec 3>&2 + # we are running on a terminal + # open fd 3 and send it to stderr + exec 3>&2 else - # we are headless - # create a temporary file for the log - logfile=$(mktemp ${logfile}/netdata-updater.log.XXXXXX) - # open fd 3 and send it to logfile - exec 3>"${logfile}" + # we are headless + # create a temporary file for the log + logfile="$(mktemp "${logfile}"/netdata-updater.log.XXXXXX)" + # open fd 3 and send it to logfile + exec 3> "${logfile}" fi +self_update + set_tarball_urls "${RELEASE_CHANNEL}" "${IS_NETDATA_STATIC_BINARY}" if [ "${IS_NETDATA_STATIC_BINARY}" == "yes" ]; then - TMPDIR="$(create_tmp_directory)" - PREVDIR="$(pwd)" - - echo >&2 "Entering ${TMPDIR}" - cd "${TMPDIR}" - - download "${NETDATA_TARBALL_CHECKSUM_URL}" "${TMPDIR}/sha256sum.txt" - download "${NETDATA_TARBALL_URL}" "${TMPDIR}/netdata-latest.gz.run" - if ! grep netdata-latest.gz.run "${TMPDIR}/sha256sum.txt" | safe_sha256sum -c - >/dev/null 2>&1; then - fatal "Static binary checksum validation failed. Stopping netdata installation and leaving binary in ${TMPDIR}" - fi - - # Do not pass any options other than the accept, for now - sh "${TMPDIR}/netdata-latest.gz.run" --accept - - #shellcheck disable=SC2181 - if [ $? -eq 0 ]; then - rm -r "${TMPDIR}" - else - echo >&2 "NOTE: did not remove: ${TMPDIR}" - fi - echo >&2 "Switching back to ${PREVDIR}" - cd "${PREVDIR}" + ndtmpdir="$(create_tmp_directory)" + PREVDIR="$(pwd)" + + echo >&2 "Entering ${ndtmpdir}" + cd "${ndtmpdir}" || exit 1 + + download "${NETDATA_TARBALL_CHECKSUM_URL}" "${ndtmpdir}/sha256sum.txt" + download "${NETDATA_TARBALL_URL}" "${ndtmpdir}/netdata-latest.gz.run" + if ! grep netdata-latest.gz.run "${ndtmpdir}/sha256sum.txt" | safe_sha256sum -c - > /dev/null 2>&1; then + fatal "Static binary checksum validation failed. Stopping netdata installation and leaving binary in ${ndtmpdir}\nUsually this is a result of an older copy of the file being cached somehere and can be resolved by simply retrying in an hour." + fi + + # Do not pass any options other than the accept, for now + # shellcheck disable=SC2086 + if sh "${ndtmpdir}/netdata-latest.gz.run" --accept -- ${REINSTALL_OPTIONS}; then + rm -r "${ndtmpdir}" + else + echo >&2 "NOTE: did not remove: ${ndtmpdir}" + fi + echo >&2 "Switching back to ${PREVDIR}" + cd "${PREVDIR}" || exit 1 else - # the installer updates this script - so we run and exit in a single line - update && exit 0 + # the installer updates this script - so we run and exit in a single line + update && exit 0 fi - |