diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-06 01:22:31 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-06 01:22:31 +0000 |
commit | 8d4f58e49b9dc7d3545651023a36729de773ad86 (patch) | |
tree | 7bc7be4a8e9e298daa1349348400aa2a653866f2 /libnetdata/procfile/README.md | |
parent | Initial commit. (diff) | |
download | netdata-8d4f58e49b9dc7d3545651023a36729de773ad86.tar.xz netdata-8d4f58e49b9dc7d3545651023a36729de773ad86.zip |
Adding upstream version 1.12.0.upstream/1.12.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'libnetdata/procfile/README.md')
-rw-r--r-- | libnetdata/procfile/README.md | 63 |
1 files changed, 63 insertions, 0 deletions
diff --git a/libnetdata/procfile/README.md b/libnetdata/procfile/README.md new file mode 100644 index 0000000..7037dc4 --- /dev/null +++ b/libnetdata/procfile/README.md @@ -0,0 +1,63 @@ + +# PROCFILE + +procfile is a library for reading text data files (i.e `/proc` files) in the fastest possible way. + +## How it works + +The library automatically adapts (through the iterations) its memory so that each file +is read with single `read()` call. + +Then the library splits the file into words, using the supplied separators. +The library also supported quoted words (i.e. strings within of which the separators are ignored). + +### Initialization + +Initially the caller: + +- calls `procfile_open()` to open the file and allocate the structures needed. + +### Iterations + +For each iteration, the caller: + +- calls `procfile_readall()` to read updated contents. + This call also rewinds (`lseek()` to 0) before reading it. + + For every file, a [BUFFER](../buffer/) is used that is automatically adjusted to fit + the entire file contents of the file. So the file is read with a single `read()` call + (providing atomicity / consistency when the data are read from the kernel). + + Once the data are read, 2 arrays of pointers are updated: + + - a `words` array, pointing to each word in the data read + - a `lines` array, pointing to the first word for each line + + This is highly optimized. Both arrays are automatically adjusted to + fit all contents and are updated in a single pass on the data. + + The library provides a number of macros: + + - `procfile_lines()` returns the # of lines read + - `procfile_linewords()` returns the # of words in the given line + - `procfile_word()` returns a pointer the given word # + - `procfile_line()` returns a pointer to the first word of the given line # + - `procfile_lineword()` returns a pointer to the given word # of the given line # + +### Cleanup + +When the caller exits: + +- calls `procfile_free()` to close the file and free all memory used. + +### Performance + +- a **raspberry Pi 1** (the oldest single core one) can process 5.000+ `/proc` files per second. +- a **J1900 Celeron** processor can process 23.000+ `/proc` files per second per core. + +To achieve this kind of performance, the library tries to work in batches so that the code +and the data are inside the processor's caches. + +This library is extensively used in netdata and its plugins. + +[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Flibnetdata%2Fprocfile%2FREADME&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)]() |