Adding upstream version 1.29.0.upstream/1.29.0

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 291 insertions, 0 deletions

diff --git a/packaging/installer/methods/source.md b/packaging/installer/methods/source.md
new file mode 100644
index 000000000..e0827fc17
--- /dev/null
+++ b/packaging/installer/methods/source.md
@@ -0,0 +1,291 @@
+<!--
+title: "Manually build Netdata from source"
+description: "Package maintainers and power users may be interested in manually building Netdata from source without using any of our installation scripts."
+custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/installer/methods/source.md
+-->
+
+# Manually build Netdata from source
+
+These instructions are for advanced users and distribution package
+maintainers. Unless this describes you, you almost certainly want
+to follow [our guide for manually installing Netdata from a git
+checkout](/packaging/installer/methods/manual.md) instead.
+
+## Required dependencies
+
+At a bare minimum, Netdata requires the following libraries and tools
+to build and run successfully:
+
+-   libuuid
+-   libuv version 1.0 or newer
+-   zlib
+-   GNU autoconf
+-   GNU automake
+-   GCC or Xcode (Clang is known to have issues in certain configurations, see [Using Clang](#using-clang))
+-   A version of `make` compatible with GNU automake
+-   Git (we use git in the build system to generate version info, don't need a full install, just a working `git show` command)
+
+Additionally, the following build time features require additional dependencies:
+
+-   TLS support for the web GUI:
+    -   OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer.
+-   dbengine metric storage:
+    -   liblz4 r129 or newer
+    -   OpenSSL 1.0 or newer (LibreSSL _amy_ work, but is largely untested).
+    -   [libJudy](http://judy.sourceforge.net/)
+-   Netdata Cloud support:
+    -   A working internet connection
+    -   A recent version of CMake
+    -   OpenSSL 1.0.2 or newer _or_ LibreSSL 3.0.0 or newer.
+    -   JSON-C (may be provided by the user as shown below, or by the system)
+
+## Preparing the source tree
+
+Certain features in Netdata require custom versions of specific libraries,
+which the the build system will link statically into Netdata. These
+libraries and their header files must be copied into specific locations
+in the source tree to be used.
+
+### Netdata cloud
+
+Netdata Cloud functionality requires custom builds of libmosquitto and
+libwebsockets.
+
+#### libmosquitto
+
+Netdata maintains a custom fork of libmosquitto at
+https://github.com/netdata/mosquitto with patches to allow for proper
+integration with libwebsockets, which is needed for correct operation of
+Netdata Cloud functionality. To prepare this library for the build system:
+
+1.  Verify the tag that Netdata expects to be used by checking the contents
+    of `packaging/mosquitto.version` in your Netdata sources.
+2.  Obtain the sources for that version by either:
+    -   Navigating to https://github.com/netdata/mosquitto/releases and
+        downloading and unpacking the source code archive for that release.
+    -   Cloning the repository with `git` and checking out the required tag.
+3.  If building on a platform other than Linux, prepare the mosquitto
+    sources by running `cmake -D WITH_STATIC_LIBRARIES:boolean=YES .` in
+    the mosquitto source directory.
+4.  Build mosquitto by running `make -C lib` in the mosquitto source directory.
+5.  In the Netdata source directory, create a directory called `externaldeps/mosquitto`.
+6.  Copy `lib/mosquitto.h` from the mosquitto source directory to
+    `externaldeps/mosquitto/mosquitto.h` in the Netdata source tree.
+7.  Copy `lib/libmosquitto.a` from the mosquitto source directory to
+    `externaldeps/mosquitto/libmosquitto.a` in the Netdata source tree. If
+    building on a platform other than Linux, the file that needs to be
+    copied will instead be named `lib/libmosquitto_static.a`, but it
+    still needs to be copied to `externaldeps/mosquitto/libmosquitto.a`.
+
+#### libwebsockets
+
+Netdata uses the standard upstream version of libwebsockets located at
+https://github.com/warmcat/libwebsockets, but requires a build with SOCKS5
+support, which is not enabled by most pre-built versions. Currently,
+we do not support using a system copy of libwebsockets. To prepare this
+library for the build system:
+
+1.  Verify the tag that Netdata expects to be used by checking the contents
+    of `packaging/libwebsockets.version` in your Netdata sources.
+2.  Obtain the sources for that version by either:
+    -   Navigating to https://github.com/warmcat/libwebsockets/releases and
+        downloading and unpacking the source code archive for that release.
+    -   Cloning the repository with `git` and checking out the required tag.
+3.  Prepare the libwebsockets sources by running `cmake -D
+    LWS_WITH_SOCKS5:bool=ON .` in the libwebsockets source directory.
+4.  Build libwebsockets by running `make` in the libwebsockets source
+    directory.
+5.  In the Netdata source directory, create a directory called
+    `externaldeps/libwebsockets`.
+6.  Copy `lib/libwebsockets.a` from the libwebsockets source directory to
+    `externaldeps/libwebsockets/libwebsockets.a` in the Netdata source tree.
+7.  Copy the entire contents of `lib/include` from the libwebsockets source
+    directory to `externaldeps/libwebsockets/include` in the Netdata source tree.
+
+#### JSON-C
+
+Netdata requires the use of JSON-C for JSON parsing when using Netdata
+Cloud. Netdata is able to use a system-provided copy of JSON-C, but
+some systems may not provide it. If your system does not provide JSON-C,
+you can do the following to prepare a copy for the build system:
+
+1.  Verify the tag that Netdata expects to be used by checking the contents
+    of `packaging/jsonc.version` in your Netdata sources.
+2.  Obtain the sources for that version by either:
+    -   Navigating to https://github.com/json-c/json-c and downloading
+        and unpacking the source code archive for that release.
+    -   Cloning the repository with `git` and checking out the required tag.
+3.  Prepare the JSON-C sources by running `cmake -DBUILD_SHARED_LIBS=OFF .`
+    in the JSON-C source directory.
+4.  Build JSON-C by running `make` in the JSON-C source directory.
+5.  In the Netdata source directory, create a directory called
+    `externaldeps/jsonc`.
+6.  Copy `libjson-c.a` fro the JSON-C source directory to
+    `externaldeps/jsonc/libjson-c.a` in the Netdata source tree.
+7.  Copy all of the header files (`*.h`) from the JSON-C source directory
+    to `externaldeps/jsonc/json-c` in the Netdata source tree.
+
+## Building Netdata
+
+Once the source tree has been prepared, Netdata is ready to be configured
+and built. Netdata currently uses GNU autotools as it's primary build
+system. To build Netdata this way:
+
+1.  Run `autoreconf -ivf` in the Netdata source tree.
+2.  Run `./configure` in the Netdata source tree.
+3.  Run `make` in the Netdata source tree.
+
+### Configure options
+
+Netdata provides a number of build time configure options. This section
+lists some of the ones you are most likely to need:
+
+-   `--prefix`: Specify the prefix under which Netdata will be installed.
+-   `--with-webdir`: Specify a path relative to the prefix in which to
+    install the web UI files.
+-   `--disable-cloud`: Disables all Netdata Cloud functionality for
+    this build.
+
+### Using Clang
+
+Netdata is primarily developed using GCC, but in most cases we also
+build just fine using Clang. Under some build configurations of Clang
+itself, you may see build failures with the linker reporting errors
+about `nonrepresentable section on output`. We currently do not have a
+conclusive fix for this issue (the obvious fix leads to other issues which
+we haven't been able to fix yet), and unfortunately the only workaround
+is to use a different build of Clang or to use GCC.
+
+### Linking errors relating to OpenSSL
+
+Netdata's build system currently does not reliably support building
+on systems which have multiple ABI incompatible versions of OpenSSL
+installed. In such situations, you may encounter linking errors due to
+Netdata trying to build against headers for one version but link to a
+different version.
+
+## Additional components
+
+A full featured install of Netdata requires some additional components
+which must be built and installed separately from the main Netdata
+agent. All of these should be handled _after_ installing Netdata itself.
+
+### React dashboard
+
+The above build steps include a deprecated web UI for Netdata that lacks
+support for Netdata Cloud. To get a fully featured dashboard, you must
+install our new React dashboard.
+
+#### Installing the pre-built React dashboard
+
+We provide pre-built archives of the React dashboard for each release
+(these are also used during our normal install process). To use one
+of these:
+
+1.  Verify the release version that Netdata expects to be used by checking
+    the contents of `packaging/dashboard.version` in your Netdata sources.
+2.  Go to https://github.com/netdata/dashboard/releases and download the
+    `dashboard.tar.gz` file for the required release.
+3.  Unpack the downloaded archive to a temporary directory.
+4.  Copy the contents of the `build` directory from the extracted
+    archive to `/usr/share/netdata/web` or the equivalent location for
+    your build of Netdata. This _will_ overwrite some files in the target
+    location.
+
+#### Building the React dashboard locally
+
+Alternatively, you may wish to build the React dashboard locally. Doing
+so requires a recent version of Node.JS with a working install of
+NPM. Once you have the required tools, do the following:
+
+1.  Verify the release version that Netdata expects to be used by checking
+    the contents of `packaging/dashboard.version` in your Netdata sources.
+2.  Obtain the sources for that version by either:
+    -   Navigating to https://github.com/netdata/dashboard and downloading
+        and unpacking the source code archive for that release.
+    -   Cloning the repository with `git` and checking out the required tag.
+3.  Run `npm install` in the dashboard source tree.
+4.  Run `npm run build` in the dashboard source tree.
+5.  Copy the contents of the `build` directory just like step 4 of
+    installing the pre-built React dashboard.
+
+### Go collectors
+
+A number of the collectors for Netdata are written in Go instead of C,
+and are developed in a separate repository from the mian Netdata code.
+An installation without these collectors is still usable, but will be
+unable to collect metrics for a number of network services the system
+may be providing. You can either install a pre-built copy of these
+collectors, or build them locally.
+
+#### Installing the pre-built Go collectors
+
+We provide pre-built binaries of the Go collectors for all the platforms
+we officially support. To use one of these:
+
+1.  Verify the release version that Netdata expects to be used by checking
+    the contents of `packaging/go.d.version` in your Netdata sources.
+2.  Go to https://github.com/netdata/go.d.plugin/releases, select the
+    required release, and download the `go.d.plugin-*.tar.gz` file
+    for your system type and CPu architecture and the `config.tar.gz`
+    configuration file archive.
+3.  Extract the `go.d.plugin-*.tar.gz` archive into a temporary
+    location, and then copy the single file in the archive to
+    `/usr/libexec/netdata/plugins.d` or the equivalent location for your
+    build of Netdata and rename it to `go.d.plugin`.
+4.  Extract the `config.tar.gz` archive to a temporarylocation and then
+    copy the contents of the archive to `/etc/netdata` or the equivalent
+    location for your build of Netdata.
+
+#### Building the Go collectors locally
+
+Alternatively, you may wish to build the Go collectors locally
+yourself. Doing so requires a working installation of Golang 1.13 or
+newer. Once you have the required tools, do the following:
+
+1.  Verify the release version that Netdata expects to be used by checking
+    the contents of `packaging/go.d.version` in your Netdata sources.
+2.  Obtain the sources for that version by either:
+    -   Navigating to https://github.com/netdata/go.d.plugin and downloading
+        and unpacking the source code archive for that release.
+    -   Cloning the repository with `git` and checking out the required tag.
+3.  Run `make` in the go.d.plugin source tree.
+4.  Copy `bin/godplugin` to `/usr/libexec/netdata/plugins.d` or th
+    equivalent location for your build of Netdata and rename it to
+    `go.d.plugin`.
+5.  Copy the contents of the `config` directory to `/etc/netdata` or the
+    equivalent location for your build of Netdata.
+
+### eBPF collector
+
+On Linux systems, Netdata has support for using the kernel's eBPF
+interface to monitor performance-related VFS, network, and process events,
+allowing for insights into process lifetimes and file access
+patterns. Using this functionality requires additional code managed in
+a separate repository from the core Netdata agent. You can either install
+a pre-built copy of the required code, or build it locally.
+
+#### Installing the pre-built eBPF code
+
+We provide pre-built copies of the eBPF code for 64-bit x86 systems
+using glibc or musl. To use one of these:
+
+1.  Verify the release version that Netdata expects to be used by checking
+    the contents of `packaging/ebpf.version` in your Netdata sources.
+2.  Go to https://github.com/netdata/kernel-collector/releases, select the
+    required release, and download the `netdata-kernel-collector-*.tar.xz`
+    file for the libc variant your system uses (either rmusl or glibc).
+3.  Extract the contents of the archive to a temporary location, and then
+    copy all of the `.o` and `.so.*` files and the contents of the `library/`
+    directory to `/usr/libexec/netdata/plugins.d` or the equivalent location
+    for your build of Netdata.
+
+#### Building the eBPF code locally
+
+Alternatively, you may wish to build the eBPF code locally yourself. For
+instructions, please consult [the README file for our kernel-collector
+repository](https://github.com/netdata/kernel-collector/blob/master/README.md),
+which outlines both the required dependencies, as well as multiple
+options for building the code.
+
+[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fpackaging%2Finstaller%2Fmethods%2Fsource&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)