diff options
Diffstat (limited to 'BUILD.md')
| -rw-r--r-- | BUILD.md | 365 |
1 files changed, 365 insertions, 0 deletions
diff --git a/BUILD.md b/BUILD.md new file mode 100644 index 00000000..049c86d3 --- /dev/null +++ b/BUILD.md @@ -0,0 +1,365 @@ +<!-- +title: "The build system" +custom_edit_url: https://github.com/netdata/netdata/edit/master/BUILD.md +--> + +# The build system + +We are currently migrating from `autotools` to `CMake` as a build-system. This document +currently describes how we intend to perform this migration, and will be updated after +the migration to explain how the new `CMake` configuration works. + +## Stages during the build + +1. The `netdata-installer.sh`, take in arguments and environment settings to control the + build. +2. The configure step: `autoreconf -ivf ; ./configure` passing arguments into the configure + script. This becomes `generation-time` in CMake. This includes package / system detection + and configuration resulting in the `config.h` in the source root. +3. The build step: recurse through the generated Makefiles and build the executable. +4. The first install step: calls `make install` to handle all the install steps put into + the Makefiles by the configure step (puts binaries / libraries / config into target + tree structure). +5. The second install step: the rest of the installer after the make install handles + system-level configuration (privilege setting, user / groups, fetch/build/install `go.d` + plugins, telemetry, installing service for startup, uninstaller, auto-updates. + +The ideal migration result is to replace all of this with the following steps: +``` +mkdir build ; cd build ; cmake .. -D... ; cmake --build . --target install +``` + +The `-D...` indicates where the command-line arguments for configuration are passed into +`CMake`. + +## CMake generation time + +At generation time we need to solve the following issues: + +### Feature flags + +Every command-line switch on the installer and the configure script needs to becomes an +argument to the CMake generation, we can do this with variables in the CMake cache: + +CMakeLists.txt: +``` +option(ENABLE_DBENGINE "Enable the dbengine storage" ON) +... +if(${ENABLE_DBENGINE}) +... +endif() +``` + +Command-line interface +``` +cmake -DENABLE_DBENGINE +``` + +### Dependency detection + +We have a mixture of soft- and hard-dependencies on libraries. For most of these we expect +`pkg-config` information, for some we manually probe for libraries and include files. We +should treat all of the external dependencies consistently: + +1. Default to autodetect using `pkg-config` (e.g. the standard `jemalloc` drops a `.pc` + into the system but we do not check for it. +2. If no `.pc` is found perform a manual search for libraries under known names, and + check for accessible symbols inside them. +3. Check that include paths work. +4. Allow a command-line override (e.g. `-DWITH_JEMALLOC=/...`). +5. If none of the above work then fail the install if the dependency is hard, otherwise + indicate it is not present in the `config.h`. + +Before doing any dependency detection we need to determine which search paths are +really in use for the current compiler, after the `project` declaration we can use: +``` +execute_process(COMMAND ${CMAKE_C_COMPILER} "--print-search-dirs" + COMMAND grep "^libraries:" + COMMAND sed "s/^libraries: =//" + COMMAND tr ":" " " + COMMAND tr -d "\n" + OUTPUT_VARIABLE CC_SEARCH_DIRS + RESULTS_VARIABLE CC_SEARCH_RES) +string(REGEX MATCH "^[0-9]+" CC_SEARCH_RES ${CC_SEARCH_RES}) +#string(STRIP "${CC_SEARCH_RES}" CC_SEARCH_RES) +if(0 LESS ${CC_SEARCH_RES}) + message(STATUS "Warning - cannot determine standard compiler library paths") + # Note: we will probably need a different method for Windows... +endif() + +``` + +The output format for this switch works on both `Clang` and `gcc`, it also includes +the include search path, which can be extracted in a similar way. Standard advice here +is to list the `ldconfig` cache or use the `-V` flag to check, but this does not work +consistently across platforms - in particular `gcc` will reconfigure `ld` when it is +called to gcc's internal view of search paths. During experiments each of these +alternative missed / added unused paths. Dumping the compiler's own estimate of the +search paths seems to work consistently across clang/gcc/linux/freebsd configurations. + +The default behaviour in CMake is to search across predefined paths (e.g. `CMAKE_LIBRARY_PATH`) +that are based on heuristics about the current platform. Most projects using CMake seem +to overwrite this with their own estimates. + +We can use the extracted paths as a base, add our own heuristics based on OS and then +`set(CMAKE_LIBRARY_PATH ${OUR_OWN_LIB_SEARCH})` to get the best results. Roughly we do +the following for each external dependency: +``` +set(WITH_JSONC "Detect" CACHE STRING "Manually set the path to a json-c installation") +... +if(${WITH_JSONC} STREQUAL "Detect") + pkg_check_modules(JSONC json-c) # Don't set the REQUIRED flag + if(JSONC_FOUND) + message(STATUS "libjsonc found through .pc -> ${JSONC_CFLAGS_OTHER} ${JSONC_LIBRARIES}") + # ... setup using JSONC_CFLAGS_OTHER JSONC_LIBRARIES and JSONC_INCLUDE_DIRS + else() + find_library(LIB_JSONC + NAMES json-c libjson-c + PATHS ${CMAKE_LIBRARY_PATH}) # Includes our additions by this point + if(${LIB_JSONC} STREQUAL "LIB_JSONC-NOTFOUND") + message(STATUS "Library json-c not installed, disabling") + else() + check_library_exists(${LIB_JSONC} json_object_get_type "" HAVE_JSONC) + # ... setup using heuristics for CFLAGS and check include files are available + endif() + endif() +else() + # ... use explicit path as base to check for library and includes ... +endif() + +``` + +For checking the include path we have two options, if we overwrite the `CMAKE_`... variables +to change the internal search path we can use: +``` +CHECK_INCLUDE_FILE(json/json.h HAVE_JSONC_H) +``` +Or we can build a custom search path and then use: +``` +find_file(HAVE_JSONC_H json/json.h PATHS ${OUR_INCLUDE_PATHS}) +``` + +Note: we may have cases where there is no `.pc` but we have access to a `.cmake` (e.g. AWS SDK, mongodb,cmocka) - these need to be checked / pulled inside the repo while building a prototype. + +### Compiler compatibility checks + +In CMakeLists.txt: + +``` +CHECK_INCLUDE_FILE(sys/prctl.h HAVE_PRCTL_H) +configure_file(cmake/config.in config.h) +``` + +In cmake/config.in: + +``` +#cmakedefine HAVE_PRCTL_H 1 +``` + +If we want to check explicitly if something compiles (e.g. the accept4 check, or the +`strerror_r` typing issue) then we set the `CMAKE_`... paths and then use: +``` +check_c_source_compiles( + " + #include <string.h> + int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; } + " + STRERROR_R_CHAR_P) + +``` +This produces a bool that we can use inside CMake or propagate into the `config.h`. + +We can handle the atomic checks with: +``` +check_c_source_compiles( + " + int main (int argc, char **argv) + { + volatile unsigned long ul1 = 1, ul2 = 0, ul3 = 2; + __atomic_load_n(&ul1, __ATOMIC_SEQ_CST); + __atomic_compare_exchange(&ul1, &ul2, &ul3, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); + __atomic_fetch_add(&ul1, 1, __ATOMIC_SEQ_CST); + __atomic_fetch_sub(&ul3, 1, __ATOMIC_SEQ_CST); + __atomic_or_fetch(&ul1, ul2, __ATOMIC_SEQ_CST); + __atomic_and_fetch(&ul1, ul2, __ATOMIC_SEQ_CST); + volatile unsigned long long ull1 = 1, ull2 = 0, ull3 = 2; + __atomic_load_n(&ull1, __ATOMIC_SEQ_CST); + __atomic_compare_exchange(&ull1, &ull2, &ull3, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); + __atomic_fetch_add(&ull1, 1, __ATOMIC_SEQ_CST); + __atomic_fetch_sub(&ull3, 1, __ATOMIC_SEQ_CST); + __atomic_or_fetch(&ull1, ull2, __ATOMIC_SEQ_CST); + __atomic_and_fetch(&ull1, ull2, __ATOMIC_SEQ_CST); + return 0; + } + " + HAVE_C__ATOMIC) +``` + +For the specific problem of getting the correct type signature in log.c for the `strerror_r` +calls we can replicate what we have now, or we can delete this code completely and use a +better solution that is documented [here](http://www.club.cc.cmu.edu/~cmccabe/blog_strerror.html). +To replicate what we have now: +``` +check_c_source_compiles( + " + #include <string.h> + int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; } + " + STRERROR_R_CHAR_P) + +check_c_source_compiles( + " + #include <string.h> + int main() { int x = strerror_r(0, &x, sizeof(x)); return 0; } + " + STRERROR_R_INT) + +if("${STRERROR_R_CHAR_P}" OR "${STRERROR_R_INT}") + set(HAVE_DECL_STRERROR_R 1) +endif() +message(STATUS "Result was ${HAVE_DECL_STRERROR_R}") + +``` + +Note: I did not find an explicit way to select compiler when both `clang` and `gcc` are +present. We might have an implicit way (like redirecting `cc`) but we should put one in. + + + +### Debugging problems in test compilations + +Test compilations attempt to feed a test-input into the targeted compiler and result +in a yes/no decision, this is similar to `AC_LANG_SOURCE(.... if test $ac_...` in .`m4`. +We have two techniques to use in CMake: +``` +cmake_minimum_required(VERSION 3.1.0) +include(CheckCCompilerFlag) +project(empty C) + +check_c_source_compiles( + " + #include <string.h> + int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; } + " + STRERROR_R_CHAR_P) + +try_compile(HAVE_JEMALLOC ${CMAKE_CURRENT_BINARY_DIR} + ${CMAKE_CURRENT_SOURCE_DIR}/quickdemo.c + LINK_LIBRARIES jemalloc) +``` + +The `check_c_source_compiles` is light-weight: + +* Inline source for the test, easy to follow. +* Build errors are reported in `CMakeFiles/CMakeErrors.log` + +But we cannot alter the include-paths / library-paths / compiler-flags specifically for +the test without overwriting the current CMake settings. The alternative approach is +slightly more heavy-weight: + +* Can't inline source for `try_compile` - it requires a `.c` file in the tree. +* Build errors are not shown, the recovery process for them is somewhat difficult. + +``` +rm -rf * && cmake .. --debug-trycompile +grep jemal CMakeFiles/CMakeTmp/CMakeFiles/*dir/* +cd CMakeFiles/CMakeTmp/CMakeFiles/cmTC_d6f0e.dir # for example +cmake --build ../.. +``` + +This implies that we can do this to diagnose problems / develop test-programs, but we +have to make them *bullet-proof* as we cannot expose this to end-users. This means that +the results of the compilation must be *crisp* - exactly yes/no if the feature we are +testing is supported. + +### System configuration checks + +For any system configuration checks that fall outside of the above scope (includes, libraries, +packages, test-compilation checks) we have a fall-back that we can use to glue any holes +that we need, e.g. to pull out the packaging strings, inside the `CMakeLists.h`: +``` +execute_process(COMMAND cat ${CMAKE_CURRENT_SOURCE_DIR}/packaging/version + COMMAND tr -d '\n' + OUTPUT_VARIABLE VERSION_FROM_FILE) +message(STATUS "Packaging version ${VERSION_FROM_FILE}") +``` +and this in the `config.h.in`: +``` +#define VERSION_FROM_FILE "@VERSION_FROM_FILE@" +``` + +## CMake build time + +We have a working definition of the targets that is in use with CLion and works on modern +CMake (3.15). It breaks on older CMake version (e.g. 3.7) with an error message (issue#7091). +No PoC yet to fix this, but it looks like changing the target properties should do it (in the +worst case we can drop the separate object completely and merge the sources directly into +the final target). + +Steps needed for building a prototype: + +1. Pick a reasonable configuration. +2. Use the PoC techniques above to do a full generation of `CMAKE_` variables in the cache + according to the feature options and dependencies. +3. Push these into the project variables. +4. Work on it until the build succeeds in at least one known configuration. +5. Smoke-test that the output is valid (i.e. the executable loads and runs, and we can + access the dashboard). +6. Do a full comparison of the `config.h` generated by autotools against the CMake version + and document / fix any deviations. + +## CMake install target + +I've only looked at this superficially as we do not have a prototype yet, but each of the +first-stage install steps (in `make install`) and the second-stage (in `netdata-installer.sh`) +look feasible. + +## General issues + +* We need to choose a minimum CMake version that is an available package across all of our + supported environments. There is currently a build issue #7091 that documents a problem + in the compilation phase (we cannot link in libnetdata as an object on old CMake versions + and need to find a different way to express this). + +* The default variable-expansion / comparisons in CMake are awkward, we need this to make it + sane: + ``` + cmake_policy(SET CMP0054 "NEW") + ``` +* Default paths for libs / includes are not comprehensive on most environments, we still need + some heuristics for common locations, e.g. `/usr/local` on FreeBSD. + +# Recommendations + +We should follow these steps: + +1. Build a prototype. +2. Build a test-environment to check the prototype against environments / configurations that + the team uses. +3. Perform an "internal" release - merge the new CMake into master, but not announce it or + offer to support it. +4. Check it works for the team internally. +5. Do a soft-release: offer it externally as a replacement option for autotools. +6. Gather feedback and usage reports on a wider range of configurations. +7. Do a hard-release: switch over the preferred build-system in the installation instructions. +8. Gather feedback and usage reports on a wider range of configurations (again). +9. Deprecate / remove the autotools build-system completely (so that we can support a single + build-system). + +Some smaller miscellaneous suggestions: + +1. Remove the `_Generic` / `strerror_r` config to make the system simpler (use the technique + on the blog post to make the standard version re-entrant so that it is thread-safe). +2. Pull in jemalloc by source into the repo if it is our preferred malloc implementation. + +# Background + +* [Stack overflow starting point](https://stackoverflow.com/questions/7132862/how-do-i-convert-an-autotools-project-to-a-cmake-project#7680240) +* [CMake wiki including previous autotools conversions](https://gitlab.kitware.com/cmake/community/wikis/Home) +* [Commands section in old CMake docs](https://cmake.org/cmake/help/v2.8.8/cmake.html#section_Commands) +* [try_compile in newer CMake docs](https://cmake.org/cmake/help/v3.7/command/try_compile.html) +* [configure_file in newer CMake docs](https://cmake.org/cmake/help/v3.7/command/configure_file.html?highlight=configure_file) +* [header checks in CMake](https://stackoverflow.com/questions/647892/how-to-check-header-files-and-library-functions-in-cmake-like-it-is-done-in-auto) +* [how to write platform checks](https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-To-Write-Platform-Checks) + +[](<>) |