summaryrefslogtreecommitdiffstats
path: root/packaging/docker/README.md
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2021-02-07 11:45:55 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2021-02-07 11:45:55 +0000
commita8220ab2d293bb7f4b014b79d16b2fb05090fa93 (patch)
tree77f0a30f016c0925cf7ee9292e644bba183c2774 /packaging/docker/README.md
parentAdding upstream version 1.19.0. (diff)
downloadnetdata-a8220ab2d293bb7f4b014b79d16b2fb05090fa93.tar.xz
netdata-a8220ab2d293bb7f4b014b79d16b2fb05090fa93.zip
Adding upstream version 1.29.0.upstream/1.29.0
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'packaging/docker/README.md')
-rw-r--r--packaging/docker/README.md461
1 files changed, 461 insertions, 0 deletions
diff --git a/packaging/docker/README.md b/packaging/docker/README.md
new file mode 100644
index 000000000..e97cb0c6f
--- /dev/null
+++ b/packaging/docker/README.md
@@ -0,0 +1,461 @@
+<!--
+title: "Install Netdata with Docker"
+date: 2020-04-23
+custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/docker/README.md
+-->
+
+# Install the Netdata Agent with Docker
+
+Running the Netdata Agent in a container works best for an internal network or to quickly analyze a host. Docker helps
+you get set up quickly, and doesn't install anything permanent on the system, which makes uninstalling the Agent easy.
+
+See our full list of Docker images at [Docker Hub](https://hub.docker.com/r/netdata/netdata).
+
+Starting with v1.12, Netdata collects anonymous usage information by default and sends it to Google Analytics. Read
+about the information collected, and learn how to-opt, on our [anonymous statistics](/docs/anonymous-statistics.md)
+page.
+
+The usage statistics are _vital_ for us, as we use them to discover bugs and prioritize new features. We thank you for
+_actively_ contributing to Netdata's future.
+
+## Limitations running the Agent in Docker
+
+For monitoring the whole host, running the Agent in a container can limit its capabilities. Some data, like the host OS
+performance or status, is not accessible or not as detailed in a container as when running the Agent directly on the
+host.
+
+A way around this is to provide special mounts to the Docker container so that the Agent can get visibility on host OS
+information like `/sys` and `/proc` folders or even `/etc/group` and shadow files.
+
+Also, we now ship Docker images using an [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
+directive, not a COMMAND directive. Please adapt your execution scripts accordingly. You can find more information about
+ENTRYPOINT vs COMMAND in the [Docker
+documentation](https://docs.docker.com/engine/reference/builder/#understand-how-cmd-and-entrypoint-interact).
+
+## Create a new Netdata Agent container
+
+You can create a new Agent container using either `docker run` or Docker Compose. After using either method, you can
+visit the Agent dashboard `http://NODE:19999`.
+
+Both methods create a [bind mount](https://docs.docker.com/storage/bind-mounts/) for Netdata's configuration files
+_within the container_ at `/etc/netdata`. See the [configuration section](#configure-agent-containers) for details. If
+you want to access the configuration files from your _host_ machine, see [host-editable
+configuration](#host-editable-configuration).
+
+**`docker run`**: Use the `docker run` command, along with the following options, to start a new container.
+
+```bash
+docker run -d --name=netdata \
+ -p 19999:19999 \
+ -v netdataconfig:/etc/netdata \
+ -v netdatalib:/var/lib/netdata \
+ -v netdatacache:/var/cache/netdata \
+ -v /etc/passwd:/host/etc/passwd:ro \
+ -v /etc/group:/host/etc/group:ro \
+ -v /proc:/host/proc:ro \
+ -v /sys:/host/sys:ro \
+ -v /etc/os-release:/host/etc/os-release:ro \
+ --restart unless-stopped \
+ --cap-add SYS_PTRACE \
+ --security-opt apparmor=unconfined \
+ netdata/netdata
+```
+
+**Docker Compose**: Copy the following code and paste into a new file called `docker-compose.yml`, then run
+`docker-compose up -d` in the same directory as the `docker-compose.yml` file to start the container.
+
+```yaml
+version: '3'
+services:
+ netdata:
+ image: netdata/netdata
+ container_name: netdata
+ hostname: example.com # set to fqdn of host
+ ports:
+ - 19999:19999
+ restart: unless-stopped
+ cap_add:
+ - SYS_PTRACE
+ security_opt:
+ - apparmor:unconfined
+ volumes:
+ - netdataconfig:/etc/netdata
+ - netdatalib:/var/lib/netdata
+ - netdatacache:/var/cache/netdata
+ - /etc/passwd:/host/etc/passwd:ro
+ - /etc/group:/host/etc/group:ro
+ - /proc:/host/proc:ro
+ - /sys:/host/sys:ro
+ - /etc/os-release:/host/etc/os-release:ro
+
+volumes:
+ netdataconfig:
+ netdatalib:
+ netdatacache:
+```
+
+## Health Checks
+
+Our Docker image provides integrated support for health checks through the standard Docker interfaces.
+
+You can control how the health checks run by using the environment variable `NETDATA_HEALTHCHECK_TARGET` as follows:
+
+- If left unset, the health check will attempt to access the
+ `/api/v1/info` endpoint of the agent.
+- If set to the exact value 'cli', the health check
+ script will use `netdatacli ping` to determine if the agent is running
+ correctly or not. This is sufficient to ensure that Netdata did not
+ hang during startup, but does not provide a rigorous verification
+ that the daemon is collecting data or is otherwise usable.
+- If set to anything else, the health check will treat the value as a
+ URL to check for a 200 status code on. In most cases, this should
+ start with `http://localhost:19999/` to check the agent running in
+ the container.
+
+In most cases, the default behavior of checking the `/api/v1/info`
+endpoint will be sufficient. If you are using a configuration which
+disables the web server or restricts access to certain API's, you will
+need to use a non-default configuration for health checks to work.
+
+## Configure Agent containers
+
+If you started an Agent container using one of the [recommended methods](#create-a-new-netdata-agent-container) and you
+want to edit Netdata's configuration, you must first use `docker exec` to attach to the container. Replace `netdata`
+with the name of your container.
+
+```bash
+docker exec -it netdata bash
+cd /etc/netdata
+./edit-config netdata.conf
+```
+
+You need to restart the Agent to apply changes. Exit the container if you haven't already, then use the `docker` command
+to restart the container: `docker restart netdata`.
+
+### Host-editable configuration
+
+If you want to make your container's configuration directory accessible from the host system, you need to use a
+[volume](https://docs.docker.com/storage/bind-mounts/) rather than a bind mount. The following commands create a
+temporary `netdata_tmp` container, which is used to populate a `netdataconfig` directory, which is then mounted inside
+the container at `/etc/netdata`.
+
+```bash
+mkdir netdataconfig
+docker run -d --name netdata_tmp netdata/netdata
+docker cp netdata_tmp:/etc/netdata netdataconfig/
+docker rm -f netdata_tmp
+```
+
+**`docker run`**: Use the `docker run` command, along with the following options, to start a new container. Note the
+changed `-v $(pwd)/netdataconfig/netdata:/etc/netdata:ro \` line from the recommended example above.
+
+```bash
+docker run -d --name=netdata \
+ -p 19999:19999 \
+ -v $(pwd)/netdataconfig/netdata:/etc/netdata:ro \
+ -v netdatalib:/var/lib/netdata \
+ -v netdatacache:/var/cache/netdata \
+ -v /etc/passwd:/host/etc/passwd:ro \
+ -v /etc/group:/host/etc/group:ro \
+ -v /proc:/host/proc:ro \
+ -v /sys:/host/sys:ro \
+ -v /etc/os-release:/host/etc/os-release:ro \
+ --restart unless-stopped \
+ --cap-add SYS_PTRACE \
+ --security-opt apparmor=unconfined \
+ netdata/netdata
+```
+
+**Docker Compose**: Copy the following code and paste into a new file called `docker-compose.yml`, then run
+`docker-compose up -d` in the same directory as the `docker-compose.yml` file to start the container. Note the changed
+`./netdataconfig/netdata:/etc/netdata:ro` line from the recommended example above.
+
+```yaml
+version: '3'
+services:
+ netdata:
+ image: netdata/netdata
+ container_name: netdata
+ hostname: example.com # set to fqdn of host
+ ports:
+ - 19999:19999
+ restart: unless-stopped
+ cap_add:
+ - SYS_PTRACE
+ security_opt:
+ - apparmor:unconfined
+ volumes:
+ - ./netdataconfig/netdata:/etc/netdata:ro
+ - netdatalib:/var/lib/netdata
+ - netdatacache:/var/cache/netdata
+ - /etc/passwd:/host/etc/passwd:ro
+ - /etc/group:/host/etc/group:ro
+ - /proc:/host/proc:ro
+ - /sys:/host/sys:ro
+ - /etc/os-release:/host/etc/os-release:ro
+
+volumes:
+ netdatalib:
+ netdatacache:
+```
+
+### Change the default hostname
+
+You can change the hostname of a Docker container, and thus the name that appears in the local dashboard and in Netdata
+Cloud, when creating a new container. If you want to change the hostname of a Netdata container _after_ you started it,
+you can safely stop and remove it. You configuration and metrics data reside in persistent volumes and are reattached to
+the recreated container.
+
+If you use `docker-run`, use the `--hostname` option with `docker run`.
+
+```bash
+docker run -d --name=netdata \
+ --hostname=my_docker_netdata
+```
+
+If you use `docker-compose`, add a `hostname:` key/value pair into your `docker-compose.yml` file, then create the
+container again using `docker-compose up -d`.
+
+```yaml
+version: '3'
+services:
+ netdata:
+ image: netdata/netdata
+ container_name: netdata
+ hostname: my_docker_compose_netdata
+ ...
+```
+
+If you don't want to destroy and recreate your container, you can edit the Agent's `netdata.conf` file directly. See the
+above section on [configuring Agent containers](#configure-agent-containers) to find the appropriate method based on
+how you created the container.
+
+### Add or remove other volumes
+
+Some of the volumes are optional depending on how you use Netdata:
+
+- If you don't want to use the apps.plugin functionality, you can remove the mounts of `/etc/passwd` and `/etc/group`
+ (they are used to get proper user and group names for the monitored host) to get slightly better security.
+- Most modern linux distros supply `/etc/os-release` although some older distros only supply `/etc/lsb-release`. If
+ this is the case you can change the line above that mounts the file inside the container to
+ `-v /etc/lsb-release:/host/etc/lsb-release:ro`.
+- If your host is virtualized then Netdata cannot detect it from inside the container and will output the wrong
+ metadata (e.g. on `/api/v1/info` queries). You can fix this by setting a variable that overrides the detection
+ using, e.g. `--env VIRTUALIZATION=$(systemd-detect-virt -v)`. If you are using a `docker-compose.yml` then add:
+
+```yaml
+ environment:
+ - VIRTUALIZATION=${VIRTUALIZATION}
+```
+
+This allows the information to be passed into `docker-compose` using:
+
+```bash
+VIRTUALIZATION=$(systemd-detect-virt -v) docker-compose up
+```
+
+#### Files inside systemd volumes
+
+If a volume is used by systemd service, some files can be removed during
+[reinitialization](https://github.com/netdata/netdata/issues/9916). To avoid this, you need to add
+`RuntimeDirectoryPreserve=yes` to the service file.
+
+### Docker container names resolution
+
+There are a few options for resolving container names within Netdata. Some methods of doing so will allow root access to
+your machine from within the container. Please read the following carefully.
+
+#### Docker socket proxy (safest option)
+
+Deploy a Docker socket proxy that accepts and filters out requests using something like
+[HAProxy](/docs/Running-behind-haproxy.md) so that it restricts connections to read-only access to the CONTAINERS
+endpoint.
+
+The reason it's safer to expose the socket to the proxy is because Netdata has a TCP port exposed outside the Docker
+network. Access to the proxy container is limited to only within the network.
+
+Below is [an example repository (and image)](https://github.com/Tecnativa/docker-socket-proxy) that provides a proxy to
+the socket.
+
+You run the Docker Socket Proxy in its own Docker Compose file and leave it on a private network that you can add to
+other services that require access.
+
+```yaml
+version: '3'
+services:
+ netdata:
+ image: netdata/netdata
+ # ... rest of your config ...
+ ports:
+ - 19999:19999
+ environment:
+ - DOCKER_HOST=proxy:2375
+ proxy:
+ image: tecnativa/docker-socket-proxy
+ volumes:
+ - /var/run/docker.sock:/var/run/docker.sock:ro
+ environment:
+ - CONTAINERS=1
+
+```
+**Note:** Replace `2375` with the port of your proxy.
+
+#### Giving group access to the Docker socket (less safe)
+
+**Important Note**: You should seriously consider the necessity of activating this option, as it grants to the `netdata`
+user access to the privileged socket connection of docker service and therefore your whole machine.
+
+If you want to have your container names resolved by Netdata, make the `netdata` user be part of the group that owns the
+socket.
+
+To achieve that just add environment variable `PGID=[GROUP NUMBER]` to the Netdata container, where `[GROUP NUMBER]` is
+practically the group id of the group assigned to the docker socket, on your host.
+
+This group number can be found by running the following (if socket group ownership is docker):
+
+```bash
+grep docker /etc/group | cut -d ':' -f 3
+```
+
+#### Running as root (unsafe)
+
+> You should seriously consider the necessity of activating this option, as it grants to the `netdata` user access to
+> the privileged socket connection of docker service, and therefore your whole machine.
+
+```yaml
+version: '3'
+services:
+ netdata:
+ image: netdata/netdata
+ # ... rest of your config ...
+ volumes:
+ # ... other volumes ...
+ - /var/run/docker.sock:/var/run/docker.sock:ro
+ environment:
+ - DOCKER_USR=root
+```
+
+### Pass command line options to Netdata
+
+Since we use an [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, you can provide
+[Netdata daemon command line options](https://learn.netdata.cloud/docs/agent/daemon/#command-line-options) such as the
+IP address Netdata will be running on, using the [command
+instruction](https://docs.docker.com/engine/reference/builder/#cmd).
+
+## Install the Agent using Docker Compose with SSL/TLS enabled HTTP Proxy
+
+For a permanent installation on a public server, you should [secure the Netdata
+instance](/docs/netdata-security.md). This section contains an example of how to install Netdata with an SSL
+reverse proxy and basic authentication.
+
+You can use the following `docker-compose.yml` and Caddyfile files to run Netdata with Docker. Replace the domains and
+email address for [Let's Encrypt](https://letsencrypt.org/) before starting.
+
+### Caddyfile
+
+This file needs to be placed in `/opt` with name `Caddyfile`. Here you customize your domain and you need to provide
+your email address to obtain a Let's Encrypt certificate. Certificate renewal will happen automatically and will be
+executed internally by the caddy server.
+
+```caddyfile
+netdata.example.org {
+ proxy / netdata:19999
+ tls admin@example.org
+}
+```
+
+### docker-compose.yml
+
+After setting Caddyfile run this with `docker-compose up -d` to have fully functioning Netdata setup behind HTTP reverse
+proxy.
+
+```yaml
+version: '3'
+volumes:
+ caddy:
+
+services:
+ caddy:
+ image: abiosoft/caddy
+ ports:
+ - 80:80
+ - 443:443
+ volumes:
+ - /opt/Caddyfile:/etc/Caddyfile
+ - $HOME/.caddy:/root/.caddy
+ environment:
+ ACME_AGREE: 'true'
+ netdata:
+ restart: always
+ hostname: netdata.example.org
+ image: netdata/netdata
+ cap_add:
+ - SYS_PTRACE
+ security_opt:
+ - apparmor:unconfined
+ volumes:
+ - netdatalib:/var/lib/netdata
+ - netdatacache:/var/cache/netdata
+ - /etc/passwd:/host/etc/passwd:ro
+ - /etc/group:/host/etc/group:ro
+ - /proc:/host/proc:ro
+ - /sys:/host/sys:ro
+ - /var/run/docker.sock:/var/run/docker.sock:ro
+
+volumes:
+ netdatalib:
+ netdatacache:
+```
+
+### Restrict access with basic auth
+
+You can restrict access by following [official caddy guide](https://caddyserver.com/docs/caddyfile/directives/basicauth#basicauth) and adding lines to
+Caddyfile.
+
+## Publish a test image to your own repository
+
+At Netdata, we provide multiple ways of testing your Docker images using your own repositories.
+You may either use the command line tools available or take advantage of our Travis CI infrastructure.
+
+### Inside Netdata organization, using Travis CI
+
+To enable Travis CI integration on your own repositories (Docker and Github), you need to be part of the Netdata
+organization.
+
+Once you have contacted the Netdata owners to setup you up on Github and Travis, execute the following steps
+
+- Preparation
+ - Have Netdata forked on your personal GitHub account
+ - Get a GitHub token: Go to **GitHub settings** -> **Developer Settings** -> **Personal access tokens**, and
+ generate a new token with full access to `repo_hook`, read-only access to `admin:org`, `public_repo`,
+ `repo_deployment`, `repo:status`, and `user:email` settings enabled. This will be your `GITHUB_TOKEN` that is
+ described later in the instructions, so keep it somewhere safe.
+ - Contact the Netdata team and seek for permissions on `https://scan.coverity.com` should you require Travis to be
+ able to push your forked code to coverity for analysis and report. Once you are setup, you should have your
+ email you used in coverity and a token from them. These will be your `COVERITY_SCAN_SUBMIT_EMAIL` and
+ `COVERITY_SCAN_TOKEN` that we will refer to later.
+ - Have a valid Docker hub account, the credentials from this account will be your `DOCKER_USERNAME` and
+ `DOCKER_PWD` mentioned later.
+
+- Setting up Travis CI for your own fork (Detailed instructions provided by Travis team [here](https://docs.travis-ci.com/user/tutorial/))
+ - Login to travis with your own GITHUB credentials (There is Open Auth access)
+ - Go to your profile settings, under [repositories](https://travis-ci.com/account/repositories) section and setup
+ your Netdata fork to be built by Travis CI.
+ - Once the repository has been setup, go to repository settings within Travis CI (usually under
+ `https://travis-ci.com/NETDATA_DEVELOPER/netdata/settings`, where `NETDATA_DEVELOPER` is your GitHub handle),
+ and select your desired settings.
+
+- While in Travis settings, under Netdata repository settings in the Environment Variables section, you need to add
+ the following:
+ - `DOCKER_USERNAME` and `DOCKER_PWD` variables so that Travis can login to your Docker Hub account and publish
+ Docker images there.
+ - `REPOSITORY` variable to `NETDATA_DEVELOPER/netdata`, where `NETDATA_DEVELOPER` is your GitHub handle again.
+ - `GITHUB_TOKEN` variable with the token generated on the preparation step, for Travis workflows to function
+ properly.
+ - `COVERITY_SCAN_SUBMIT_EMAIL` and `COVERITY_SCAN_TOKEN` variables to enable Travis to submit your code for
+ analysis to Coverity.
+
+Having followed these instructions, your forked repository should be all set up for integration with Travis CI. Happy
+testing!
+
+[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fpackaging%2Fdocker%2FREADME&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)