diff options
Diffstat (limited to '')
| -rw-r--r-- | src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md | 305 |
1 files changed, 305 insertions, 0 deletions
diff --git a/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md b/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md new file mode 100644 index 000000000..38a2c33a1 --- /dev/null +++ b/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md @@ -0,0 +1,305 @@ +<!-- +title: "How to write a Netdata collector in Go" +description: "This guide will walk you through the technical implementation of writing a new Netdata collector in Golang, with tips on interfaces, structure, configuration files, and more." +custom_edit_url: "https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md" +sidebar_label: "How to write a Netdata collector in Go" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "Developers/External plugins/go.d.plugin" +sidebar_position: 20 +--> + +# How to write a Netdata collector in Go + +## Prerequisites + +- Take a look at our [contributing guidelines](https://github.com/netdata/.github/blob/main/CONTRIBUTING.md). +- [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) this repository to your personal + GitHub account. +- [Clone](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository#:~:text=to%20GitHub%20Desktop-,On%20GitHub%2C%20navigate%20to%20the%20main%20page%20of%20the%20repository,Desktop%20to%20complete%20the%20clone.) + locally the **forked** repository (e.g `git clone https://github.com/odyslam/go.d.plugin`). +- Using a terminal, `cd` into the directory (e.g `cd go.d.plugin`) + +## Write and test a simple collector + +> :exclamation: You can skip most of these steps if you first experiment directy with the existing +> [example module](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/modules/example), which +> will +> give you an idea of how things work. + +Let's assume you want to write a collector named `example2`. + +The steps are: + +- Add the source code + to [`modules/example2/`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/modules). + - [module interface](#module-interface). + - [suggested module layout](#module-layout). + - [helper packages](#helper-packages). +- Add the configuration + to [`config/go.d/example2.conf`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/config/go.d). +- Add the module + to [`config/go.d.conf`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/config/go.d.conf). +- Import the module + in [`modules/init.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/init.go). +- Update + the [`available modules list`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin#available-modules). +- To build it, run `make` from the plugin root dir. This will create a new `go.d.plugin` binary that includes your newly + developed collector. It will be placed into the `bin` directory (e.g `go.d.plugin/bin`) +- Run it in the debug mode `bin/godplugin -d -m <MODULE_NAME>`. This will output the `STDOUT` of the collector, the same + output that is sent to the Netdata Agent and is transformed into charts. You can read more about this collector API in + our [documentation](https://github.com/netdata/netdata/blob/master/src/collectors/plugins.d/README.md#external-plugins-api). +- If you want to test the collector with the actual Netdata Agent, you need to replace the `go.d.plugin` binary that + exists in the Netdata Agent installation directory with the one you just compiled. Once + you [restart](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md#maintaining-a-netdata-agent-installation) + the Netdata Agent, it will detect and run + it, creating all the charts. It is advised not to remove the default `go.d.plugin` binary, but simply rename it + to `go.d.plugin.old` so that the Agent doesn't run it, but you can easily rename it back once you are done. +- Run `make clean` when you are done with testing. + +## Module Interface + +Every module should implement the following interface: + +``` +type Module interface { + Init() bool + Check() bool + Charts() *Charts + Collect() map[string]int64 + Cleanup() +} +``` + +### Init method + +- `Init` does module initialization. +- If it returns `false`, the job will be disabled. + +We propose to use the following template: + +``` +// example.go + +func (e *Example) Init() bool { + err := e.validateConfig() + if err != nil { + e.Errorf("config validation: %v", err) + return false + } + + someValue, err := e.initSomeValue() + if err != nil { + e.Errorf("someValue init: %v", err) + return false + } + e.someValue = someValue + + // ... + return true +} +``` + +Move specific initialization methods into the `init.go` file. See [suggested module layout](#module-Layout). + +### Check method + +- `Check` returns whether the job is able to collect metrics. +- Called after `Init` and only if `Init` returned `true`. +- If it returns `false`, the job will be disabled. + +The simplest way to implement `Check` is to see if we are getting any metrics from `Collect`. A lot of modules use such +approach. + +``` +// example.go + +func (e *Example) Check() bool { + return len(e.Collect()) > 0 +} +``` + +### Charts method + +:exclamation: Netdata module +produces [`charts`](https://github.com/netdata/netdata/blob/master/src/collectors/plugins.d/README.md#chart), not +raw metrics. + +Use [`agent/module`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/agent/module/charts.go) +package to create them, +it contains charts and dimensions structs. + +- `Charts` returns + the [charts](https://github.com/netdata/netdata/blob/master/src/collectors/plugins.d/README.md#chart) (`*module.Charts`). +- Called after `Check` and only if `Check` returned `true`. +- If it returns `nil`, the job will be disabled +- :warning: Make sure not to share returned value between module instances (jobs). + +Usually charts initialized in `Init` and `Chart` method just returns the charts instance: + +``` +// example.go + +func (e *Example) Charts() *Charts { + return e.charts +} +``` + +### Collect method + +- `Collect` collects metrics. +- Called only if `Check` returned `true`. +- Called every `update_every` seconds. +- `map[string]int64` keys are charts dimensions ids'. + +We propose to use the following template: + +``` +// example.go + +func (e *Example) Collect() map[string]int64 { + ms, err := e.collect() + if err != nil { + e.Error(err) + } + + if len(ms) == 0 { + return nil + } + return ms +} +``` + +Move metrics collection logic into the `collect.go` file. See [suggested module layout](#module-Layout). + +### Cleanup method + +- `Cleanup` performs the job cleanup/teardown. +- Called if `Init` or `Check` fails, or we want to stop the job after `Collect`. + +If you have nothing to clean up: + +``` +// example.go + +func (Example) Cleanup() {} +``` + +## Module Layout + +The general idea is to not put everything in a single file. + +We recommend using one file per logical area. This approach makes it easier to maintain the module. + +Suggested minimal layout: + +| Filename | Contains | +|---------------------------------------------------|--------------------------------------------------------| +| [`module_name.go`](#file-module_namego) | Module configuration, implementation and registration. | +| [`charts.go`](#file-chartsgo) | Charts, charts templates and constructor functions. | +| [`init.go`](#file-initgo) | Initialization methods. | +| [`collect.go`](#file-collectgo) | Metrics collection implementation. | +| [`module_name_test.go`](#file-module_name_testgo) | Public methods/functions tests. | +| [`testdata/`](#file-module_name_testgo) | Files containing sample data. | + +### File `module_name.go` + +> :exclamation: See the +> example [`example.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/example.go). + +Don't overload this file with the implementation details. + +Usually it contains only: + +- module registration. +- module configuration. +- [module interface implementation](#module-interface). + +### File `charts.go` + +> :exclamation: See the +> example: [`charts.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/charts.go). + +Put charts, charts templates and charts constructor functions in this file. + +### File `init.go` + +> :exclamation: See the +> example: [`init.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/init.go). + +All the module initialization details should go in this file. + +- make a function for each value that needs to be initialized. +- a function should return a value(s), not implicitly set/change any values in the main struct. + +``` +// init.go + +// Prefer this approach. +func (e Example) initSomeValue() (someValue, error) { + // ... + return someValue, nil +} + +// This approach is ok too, but we recommend to not use it. +func (e *Example) initSomeValue() error { + // ... + m.someValue = someValue + return nil +} +``` + +### File `collect.go` + +> :exclamation: See the +> example: [`collect.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/collect.go). + +This file is the entry point for the metrics collection. + +Feel free to split it into several files if you think it makes the code more readable. + +Use `collect_` prefix for the filenames: `collect_this.go`, `collect_that.go`, etc. + +``` +// collect.go + +func (e *Example) collect() (map[string]int64, error) { + collected := make(map[string])int64 + // ... + // ... + // ... + return collected, nil +} +``` + +### File `module_name_test.go` + +> :exclamation: See the +> example: [`example_test.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/example_test.go). + +> if you have no experience in testing we recommend starting +> with [testing package documentation](https://golang.org/pkg/testing/). + +> we use `assert` and `require` packages from [github.com/stretchr/testify](https://github.com/stretchr/testify) +> library, +> check [their documentation](https://pkg.go.dev/github.com/stretchr/testify). + +Testing is mandatory. + +- test only public functions and methods (`New`, `Init`, `Check`, `Charts`, `Cleanup`, `Collect`). +- do not create a test function per a case, use [table driven tests](https://github.com/golang/go/wiki/TableDrivenTests) + . Prefer `map[string]struct{ ... }` over `[]struct{ ... }`. +- use helper functions _to prepare_ test cases to keep them clean and readable. + +### Directory `testdata/` + +Put files with sample data in this directory if you need any. Its name should +be [`testdata`](https://golang.org/cmd/go/#hdr-Package_lists_and_patterns). + +> Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata". + +## Helper packages + +There are [some helper packages](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/pkg) for +writing a module. + |