path: root/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md

<!--
title: "How to write a Netdata collector in Go"
description: "This guide will walk you through the technical implementation of writing a new Netdata collector in Golang, with tips on interfaces, structure, configuration files, and more."
custom_edit_url: "/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md"
sidebar_label: "How to write a Netdata collector in Go"
learn_status: "Published"
learn_topic_type: "Tasks"
learn_rel_path: "Developers/External plugins/go.d.plugin"
sidebar_position: 20
-->

# How to write a Netdata collector in Go

## Prerequisites

- Take a look at our [contributing guidelines](https://github.com/netdata/.github/blob/main/CONTRIBUTING.md).
- [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) this repository to your personal
  GitHub account.
- [Clone](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository#:~:text=to%20GitHub%20Desktop-,On%20GitHub%2C%20navigate%20to%20the%20main%20page%20of%20the%20repository,Desktop%20to%20complete%20the%20clone.)
  locally the **forked** repository (e.g `git clone https://github.com/odyslam/go.d.plugin`).
- Using a terminal, `cd` into the directory (e.g `cd go.d.plugin`)

## Write and test a simple collector

> :exclamation: You can skip most of these steps if you first experiment directy with the existing
> [example module](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/modules/example), which
> will
> give you an idea of how things work.

Let's assume you want to write a collector named `example2`.

The steps are:

- Add the source code
  to [`modules/example2/`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/modules).
    - [module interface](#module-interface).
    - [suggested module layout](#module-layout).
    - [helper packages](#helper-packages).
- Add the configuration
  to [`config/go.d/example2.conf`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/config/go.d).
- Add the module
  to [`config/go.d.conf`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/config/go.d.conf).
- Import the module
  in [`modules/init.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/init.go).
- Update
  the [`available modules list`](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin#available-modules).
- To build it, run `make` from the plugin root dir. This will create a new `go.d.plugin` binary that includes your newly
  developed collector. It will be placed into the `bin` directory (e.g `go.d.plugin/bin`)
- Run it in the debug mode `bin/godplugin -d -m <MODULE_NAME>`. This will output the `STDOUT` of the collector, the same
  output that is sent to the Netdata Agent and is transformed into charts. You can read more about this collector API in
  our [documentation](/src/collectors/plugins.d/README.md#external-plugins-api).
- If you want to test the collector with the actual Netdata Agent, you need to replace the `go.d.plugin` binary that
  exists in the Netdata Agent installation directory with the one you just compiled. Once
  you [restart](/packaging/installer/README.md#maintaining-a-netdata-agent-installation)
  the Netdata Agent, it will detect and run
  it, creating all the charts. It is advised not to remove the default `go.d.plugin` binary, but simply rename it
  to `go.d.plugin.old` so that the Agent doesn't run it, but you can easily rename it back once you are done.
- Run `make clean` when you are done with testing.

## Module Interface

Every module should implement the following interface:

```
type Module interface {
    Init() bool
    Check() bool
    Charts() *Charts
    Collect() map[string]int64
    Cleanup()
}
```

### Init method

- `Init` does module initialization.
- If it returns `false`, the job will be disabled.

We propose to use the following template:

```
// example.go

func (e *Example) Init() bool {
    err := e.validateConfig()
    if err != nil {
        e.Errorf("config validation: %v", err)
        return false
    }

    someValue, err := e.initSomeValue()
    if err != nil {
        e.Errorf("someValue init: %v", err)
        return false
    }
    e.someValue = someValue

    // ...
    return true 
}
```

Move specific initialization methods into the `init.go` file. See [suggested module layout](#module-Layout).

### Check method

- `Check` returns whether the job is able to collect metrics.
- Called after `Init` and only if `Init` returned `true`.
- If it returns `false`, the job will be disabled.

The simplest way to implement `Check` is to see if we are getting any metrics from `Collect`. A lot of modules use such
approach.

```
// example.go

func (e *Example) Check() bool {
    return len(e.Collect()) > 0
}
```

### Charts method

:exclamation: Netdata module
produces [`charts`](/src/collectors/plugins.d/README.md#chart), not
raw metrics.

Use [`agent/module`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/agent/module/charts.go)
package to create them,
it contains charts and dimensions structs.

- `Charts` returns
  the [charts](/src/collectors/plugins.d/README.md#chart) (`*module.Charts`).
- Called after `Check` and only if `Check` returned `true`.
- If it returns `nil`, the job will be disabled
- :warning: Make sure not to share returned value between module instances (jobs).

Usually charts initialized in `Init` and `Chart` method just returns the charts instance:

```
// example.go

func (e *Example) Charts() *Charts {
    return e.charts
}
```

### Collect method

- `Collect` collects metrics.
- Called only if `Check` returned `true`.
- Called every `update_every` seconds.
- `map[string]int64` keys are charts dimensions ids'.

We propose to use the following template:

```
// example.go

func (e *Example) Collect() map[string]int64 {
    ms, err := e.collect()
    if err != nil {
        e.Error(err)
    }

    if len(ms) == 0 {
        return nil
    }
    return ms
}
```

Move metrics collection logic into the `collect.go` file. See [suggested module layout](#module-Layout).

### Cleanup method

- `Cleanup` performs the job cleanup/teardown.
- Called if `Init` or `Check` fails, or we want to stop the job after `Collect`.

If you have nothing to clean up:

```
// example.go

func (Example) Cleanup() {}
```

## Module Layout

The general idea is to not put everything in a single file.

We recommend using one file per logical area. This approach makes it easier to maintain the module.

Suggested minimal layout:

| Filename                                          | Contains                                               |
|---------------------------------------------------|--------------------------------------------------------|
| [`module_name.go`](#file-module_namego)           | Module configuration, implementation and registration. |
| [`charts.go`](#file-chartsgo)                     | Charts, charts templates and constructor functions.    |
| [`init.go`](#file-initgo)                         | Initialization methods.                                |
| [`collect.go`](#file-collectgo)                   | Metrics collection implementation.                     |
| [`module_name_test.go`](#file-module_name_testgo) | Public methods/functions tests.                        |
| [`testdata/`](#file-module_name_testgo)           | Files containing sample data.                          |

### File `module_name.go`

> :exclamation: See the
> example [`example.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/example.go).

Don't overload this file with the implementation details.

Usually it contains only:

- module registration.
- module configuration.
- [module interface implementation](#module-interface).

### File `charts.go`

> :exclamation: See the
> example: [`charts.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/charts.go).

Put charts, charts templates and charts constructor functions in this file.

### File `init.go`

> :exclamation: See the
> example: [`init.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/init.go).

All the module initialization details should go in this file.

- make a function for each value that needs to be initialized.
- a function should return a value(s), not implicitly set/change any values in the main struct.

```
// init.go

// Prefer this approach.
func (e Example) initSomeValue() (someValue, error) {
    // ...
    return someValue, nil 
}

// This approach is ok too, but we recommend to not use it.
func (e *Example) initSomeValue() error {
    // ...
    m.someValue = someValue
    return nil
}
```     

### File `collect.go`

> :exclamation: See the
> example: [`collect.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/collect.go).

This file is the entry point for the metrics collection.

Feel free to split it into several files if you think it makes the code more readable.

Use `collect_` prefix for the filenames: `collect_this.go`, `collect_that.go`, etc.

```
// collect.go

func (e *Example) collect() (map[string]int64, error) {
    collected := make(map[string])int64
    // ...
    // ...
    // ...
    return collected, nil
}
```

### File `module_name_test.go`

> :exclamation: See the
> example: [`example_test.go`](https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/modules/example/example_test.go).

> if you have no experience in testing we recommend starting
> with [testing package documentation](https://golang.org/pkg/testing/).

> we use `assert` and `require` packages from [github.com/stretchr/testify](https://github.com/stretchr/testify)
> library,
> check [their documentation](https://pkg.go.dev/github.com/stretchr/testify).

Testing is mandatory.

- test only public functions and methods (`New`, `Init`, `Check`, `Charts`, `Cleanup`, `Collect`).
- do not create a test function per a case, use [table driven tests](https://github.com/golang/go/wiki/TableDrivenTests)
  . Prefer `map[string]struct{ ... }` over `[]struct{ ... }`.
- use helper functions _to prepare_ test cases to keep them clean and readable.

### Directory `testdata/`

Put files with sample data in this directory if you need any. Its name should
be [`testdata`](https://golang.org/cmd/go/#hdr-Package_lists_and_patterns).

> Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

## Helper packages

There are [some helper packages](https://github.com/netdata/netdata/tree/master/src/go/collectors/go.d.plugin/pkg) for
writing a module.