diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2023-02-06 16:11:30 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2023-02-06 16:11:30 +0000 |
| commit | aa2fe8ccbfcb117efa207d10229eeeac5d0f97c7 (patch) | |
| tree | 941cbdd387b41c1a81587c20a6df9f0e5e0ff7ab /collectors | |
| parent | Adding upstream version 1.37.1. (diff) | |
| download | netdata-aa2fe8ccbfcb117efa207d10229eeeac5d0f97c7.tar.xz netdata-aa2fe8ccbfcb117efa207d10229eeeac5d0f97c7.zip | |
Adding upstream version 1.38.0.upstream/1.38.0
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'collectors')
202 files changed, 3799 insertions, 6465 deletions
diff --git a/collectors/COLLECTORS.md b/collectors/COLLECTORS.md index 7f66076ff..a61a32dd5 100644 --- a/collectors/COLLECTORS.md +++ b/collectors/COLLECTORS.md @@ -1,7 +1,11 @@ <!-- title: "Supported collectors list" description: "Netdata gathers real-time metrics from hundreds of data sources using collectors. Most require zero configuration and are pre-configured out of the box." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/COLLECTORS.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/COLLECTORS.md" +sidebar_label: "Supported collectors list" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "References/Collectors" --> # Supported collectors list @@ -10,16 +14,19 @@ Netdata uses collectors to help you gather metrics from your favorite applicatio real-time, interactive charts. The following list includes collectors for both external services/applications and internal system metrics. -Learn more about [how collectors work](/docs/collect/how-collectors-work.md), and then learn how to [enable or -configure](/docs/collect/enable-configure.md) any of the below collectors using the same process. +Learn more +about [how collectors work](https://github.com/netdata/netdata/blob/master/docs/collect/how-collectors-work.md), and +then learn how to [enable or +configure](https://github.com/netdata/netdata/blob/master/docs/collect/enable-configure.md) any of the below collectors using the same process. Some collectors have both Go and Python versions as we continue our effort to migrate all collectors to Go. In these cases, _Netdata always prioritizes the Go version_, and we highly recommend you use the Go versions for the best experience. -If you want to use a Python version of a collector, you need to explicitly [disable the Go -version](/docs/collect/enable-configure.md), and enable the Python version. Netdata then skips the Go version and -attempts to load the Python version and its accompanying configuration file. +If you want to use a Python version of a collector, you need to +explicitly [disable the Go version](https://github.com/netdata/netdata/blob/masterhttps://github.com/netdata/netdata/blob/master/docs/collect/enable-configure.md), +and enable the Python version. Netdata then skips the Go version and attempts to load the Python version and its +accompanying configuration file. If you don't see the app/service you'd like to monitor in this list: @@ -29,7 +36,7 @@ If you don't see the app/service you'd like to monitor in this list: a [feature request](https://github.com/netdata/netdata/issues/new/choose) on GitHub. - If you have basic software development skills, you can add your own plugin in [Go](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin#how-to-develop-a-collector) - or [Python](https://learn.netdata.cloud/guides/python-collector) + or [Python](https://github.com/netdata/netdata/blob/master/docs/guides/python-collector.md) Supported Collectors List: @@ -72,257 +79,300 @@ configure any of these collectors according to your setup and infrastructure. ### Generic -- [Prometheus endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus): Gathers +- [Prometheus endpoints](https://github.com/netdata/go.d.plugin/blob/master/modules/prometheus/README.md): Gathers metrics from any number of Prometheus endpoints, with support to autodetect more than 600 services and applications. -- [Pandas](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/pandas): A Python collector that gathers - metrics from a [pandas](https://pandas.pydata.org/) dataframe. Pandas is a high level data processing library in - Python that can read various formats of data from local files or web endpoints. Custom processing and transformation +- [Pandas](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/pandas/README.md): A Python + collector that gathers + metrics from a [pandas](https://pandas.pydata.org/) dataframe. Pandas is a high level data processing library in + Python that can read various formats of data from local files or web endpoints. Custom processing and transformation logic can also be expressed as part of the collector configuration. ### APM (application performance monitoring) -- [Go applications](/collectors/python.d.plugin/go_expvar/README.md): Monitor any Go application that exposes its +- [Go applications](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/go_expvar/README.md): + Monitor any Go application that exposes its metrics with the `expvar` package from the Go standard library. -- [Java Spring Boot 2 - applications](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/springboot2/) (Go version): +- [Java Spring Boot 2 applications](https://github.com/netdata/go.d.plugin/blob/master/modules/springboot2/README.md): Monitor running Java Spring Boot 2 applications that expose their metrics with the use of the Spring Boot Actuator. -- [Java Spring Boot 2 applications](/collectors/python.d.plugin/springboot/README.md) (Python version): Monitor - running Java Spring Boot applications that expose their metrics with the use of the Spring Boot Actuator. -- [statsd](/collectors/statsd.plugin/README.md): Implement a high performance `statsd` server for Netdata. -- [phpDaemon](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpdaemon/): Collect worker +- [statsd](https://github.com/netdata/netdata/blob/master/collectors/statsd.plugin/README.md): Implement a high + performance `statsd` server for Netdata. +- [phpDaemon](https://github.com/netdata/go.d.plugin/blob/master/modules/phpdaemon/README.md): Collect worker statistics (total, active, idle), and uptime for web and network applications. -- [uWSGI](/collectors/python.d.plugin/uwsgi/README.md): Monitor performance metrics exposed by the uWSGI Stats +- [uWSGI](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/uwsgi/README.md): Monitor + performance metrics exposed by the uWSGI Stats Server. ### Containers and VMs -- [Docker containers](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Docker - containers using the cgroups collector plugin. -- [DockerD](/collectors/python.d.plugin/dockerd/README.md): Collect container health statistics. -- [Docker Engine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/docker_engine/): Collect +- [Docker containers](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the + health and performance of individual Docker containers using the cgroups collector plugin. +- [DockerD](https://github.com/netdata/go.d.plugin/blob/master/modules/docker/README.md): Collect container health + statistics. +- [Docker Engine](https://github.com/netdata/go.d.plugin/blob/master/modules/docker_engine/README.md): Collect runtime statistics from the `docker` daemon using the `metrics-address` feature. -- [Docker Hub](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dockerhub/): Collect statistics +- [Docker Hub](https://github.com/netdata/go.d.plugin/blob/master/modules/dockerhub/README.md): Collect statistics about Docker repositories, such as pulls, starts, status, time since last update, and more. -- [Libvirt](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Libvirt containers +- [Libvirt](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the health and + performance of individual Libvirt containers using the cgroups collector plugin. -- [LXC](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXC containers using +- [LXC](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the health and + performance of individual LXC containers using the cgroups collector plugin. -- [LXD](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXD containers using +- [LXD](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the health and + performance of individual LXD containers using the cgroups collector plugin. -- [systemd-nspawn](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual +- [systemd-nspawn](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the + health and performance of individual systemd-nspawn containers using the cgroups collector plugin. -- [vCenter Server Appliance](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vcsa/): Monitor +- [vCenter Server Appliance](https://github.com/netdata/go.d.plugin/blob/master/modules/vcsa/README.md): Monitor appliance system, components, and software update health statuses via the Health API. -- [vSphere](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vsphere/): Collect host and virtual +- [vSphere](https://github.com/netdata/go.d.plugin/blob/master/modules/vsphere/README.md): Collect host and virtual machine performance metrics. -- [Xen/XCP-ng](/collectors/xenstat.plugin/README.md): Collect XenServer and XCP-ng metrics using `libxenstat`. +- [Xen/XCP-ng](https://github.com/netdata/netdata/blob/master/collectors/xenstat.plugin/README.md): Collect XenServer + and XCP-ng metrics using `libxenstat`. ### Data stores -- [CockroachDB](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/cockroachdb/): Monitor various +- [CockroachDB](https://github.com/netdata/go.d.plugin/blob/master/modules/cockroachdb/README.md): Monitor various database components using `_status/vars` endpoint. -- [Consul](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/consul/): Capture service and unbound +- [Consul](https://github.com/netdata/go.d.plugin/blob/master/modules/consul/README.md): Capture service and unbound checks status (passing, warning, critical, maintenance). -- [Couchbase](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/couchbase/): Gather per-bucket +- [Couchbase](https://github.com/netdata/go.d.plugin/blob/master/modules/couchbase/README.md): Gather per-bucket metrics from any number of instances of the distributed JSON document database. -- [CouchDB](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/couchdb): Monitor database health and +- [CouchDB](https://github.com/netdata/go.d.plugin/blob/master/modules/couchdb/README.md): Monitor database health and performance metrics (reads/writes, HTTP traffic, replication status, etc). -- [MongoDB](/collectors/python.d.plugin/mongodb/README.md): Collect memory-caching system performance metrics and - reads the server's response to `stats` command (stats interface). -- [MySQL](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/mysql/): Collect database global, +- [MongoDB](https://github.com/netdata/go.d.plugin/blob/master/modules/mongodb/README.md): Collect server, database, + replication and sharding performance and health metrics. +- [MySQL](https://github.com/netdata/go.d.plugin/blob/master/modules/mysql/README.md): Collect database global, replication and per user statistics. -- [OracleDB](/collectors/python.d.plugin/oracledb/README.md): Monitor database performance and health metrics. -- [Pika](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pika/): Gather metric, such as clients, +- [OracleDB](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/oracledb/README.md): Monitor + database performance and health metrics. +- [Pika](https://github.com/netdata/go.d.plugin/blob/master/modules/pika/README.md): Gather metric, such as clients, memory usage, queries, and more from the Redis interface-compatible database. -- [Postgres](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/postgres): Collect database health +- [Postgres](https://github.com/netdata/go.d.plugin/blob/master/modules/postgres/README.md): Collect database health and performance metrics. -- [ProxySQL](/collectors/python.d.plugin/proxysql/README.md): Monitor database backend and frontend performance - metrics. -- [Redis](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/redis/): Monitor status from any +- [ProxySQL](https://github.com/netdata/go.d.plugin/blob/master/modules/proxysql/README.md): Monitor database backend + and frontend performance metrics. +- [Redis](https://github.com/netdata/go.d.plugin/blob/master/modules/redis/README.md): Monitor status from any number of database instances by reading the server's response to the `INFO ALL` command. -- [RethinkDB](/collectors/python.d.plugin/rethinkdbs/README.md): Collect database server and cluster statistics. -- [Riak KV](/collectors/python.d.plugin/riakkv/README.md): Collect database stats from the `/stats` endpoint. -- [Zookeeper](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/zookeeper/): Monitor application +- [RethinkDB](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/rethinkdbs/README.md): Collect + database server and cluster statistics. +- [Riak KV](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/riakkv/README.md): Collect + database stats from the `/stats` endpoint. +- [Zookeeper](https://github.com/netdata/go.d.plugin/blob/master/modules/zookeeper/README.md): Monitor application health metrics reading the server's response to the `mntr` command. -- [Memcached](/collectors/python.d.plugin/memcached/README.md): Collect memory-caching system performance metrics. +- [Memcached](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/memcached/README.md): Collect + memory-caching system performance metrics. ### Distributed computing -- [BOINC](/collectors/python.d.plugin/boinc/README.md): Monitor the total number of tasks, open tasks, and task +- [BOINC](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/boinc/README.md): Monitor the total + number of tasks, open tasks, and task states for the distributed computing client. -- [Gearman](/collectors/python.d.plugin/gearman/README.md): Collect application summary (queued, running) and per-job +- [Gearman](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/gearman/README.md): Collect + application summary (queued, running) and per-job worker statistics (queued, idle, running). ### Email -- [Dovecot](/collectors/python.d.plugin/dovecot/README.md): Collect email server performance metrics by reading the +- [Dovecot](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/dovecot/README.md): Collect email + server performance metrics by reading the server's response to the `EXPORT global` command. -- [EXIM](/collectors/python.d.plugin/exim/README.md): Uses the `exim` tool to monitor the queue length of a +- [EXIM](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/exim/README.md): Uses the `exim` tool + to monitor the queue length of a mail/message transfer agent (MTA). -- [Postfix](/collectors/python.d.plugin/postfix/README.md): Uses the `postqueue` tool to monitor the queue length of a +- [Postfix](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/postfix/README.md): Uses + the `postqueue` tool to monitor the queue length of a mail/message transfer agent (MTA). ### Kubernetes -- [Kubelet](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubelet/): Monitor one or more +- [Kubelet](https://github.com/netdata/go.d.plugin/blob/master/modules/k8s_kubelet/README.md): Monitor one or more instances of the Kubelet agent and collects metrics on number of pods/containers running, volume of Docker operations, and more. -- [kube-proxy](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubeproxy/): Collect +- [kube-proxy](https://github.com/netdata/go.d.plugin/blob/master/modules/k8s_kubeproxy/README.md): Collect metrics, such as syncing proxy rules and REST client requests, from one or more instances of `kube-proxy`. -- [Service discovery](https://github.com/netdata/agent-service-discovery/): Find what services are running on a +- [Service discovery](https://github.com/netdata/agent-service-discovery/README.md): Find what services are running on a cluster's pods, converts that into configuration files, and exports them so they can be monitored by Netdata. ### Logs -- [Fluentd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/fluentd/): Gather application +- [Fluentd](https://github.com/netdata/go.d.plugin/blob/master/modules/fluentd/README.md): Gather application plugins metrics from an endpoint provided by `in_monitor plugin`. -- [Logstash](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/logstash/): Monitor JVM threads, +- [Logstash](https://github.com/netdata/go.d.plugin/blob/master/modules/logstash/README.md): Monitor JVM threads, memory usage, garbage collection statistics, and more. -- [OpenVPN status logs](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/openvpn_status_log): Parse +- [OpenVPN status logs](https://github.com/netdata/go.d.plugin/blob/master/modules/openvpn_status_log/README.md): Parse server log files and provide summary (client, traffic) metrics. -- [Squid web server logs](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/squidlog/): Tail Squid +- [Squid web server logs](https://github.com/netdata/go.d.plugin/blob/master/modules/squidlog/README.md): Tail Squid access logs to return the volume of requests, types of requests, bandwidth, and much more. - [Web server logs (Go version for Apache, - NGINX)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/): Tail access logs and provide + NGINX)](https://github.com/netdata/go.d.plugin/blob/master/modules/weblog/README.md/): Tail access logs and provide very detailed web server performance statistics. This module is able to parse 200k+ rows in less than half a second. -- [Web server logs (Apache, NGINX)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog): Tail +- [Web server logs (Apache, NGINX)](https://github.com/netdata/go.d.plugin/blob/master/modules/weblog/README.md): Tail access log file and collect web server/caching proxy metrics. ### Messaging -- [ActiveMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/activemq/): Collect message broker +- [ActiveMQ](https://github.com/netdata/go.d.plugin/blob/master/modules/activemq/README.md): Collect message broker queues and topics statistics using the ActiveMQ Console API. -- [Beanstalk](/collectors/python.d.plugin/beanstalk/README.md): Collect server and tube-level statistics, such as CPU +- [Beanstalk](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/beanstalk/README.md): Collect + server and tube-level statistics, such as CPU usage, jobs rates, commands, and more. -- [Pulsar](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pulsar/): Collect summary, +- [Pulsar](https://github.com/netdata/go.d.plugin/blob/master/modules/pulsar/README.md): Collect summary, namespaces, and topics performance statistics. -- [RabbitMQ (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/rabbitmq/): Collect message +- [RabbitMQ (Go)](https://github.com/netdata/go.d.plugin/blob/master/modules/rabbitmq/README.md): Collect message broker overview, system and per virtual host metrics. -- [RabbitMQ (Python)](/collectors/python.d.plugin/rabbitmq/README.md): Collect message broker global and per virtual +- [RabbitMQ (Python)](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/rabbitmq/README.md): + Collect message broker global and per virtual host metrics. -- [VerneMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vernemq/): Monitor MQTT broker +- [VerneMQ](https://github.com/netdata/go.d.plugin/blob/master/modules/vernemq/README.md): Monitor MQTT broker health and performance metrics. It collects all available info for both MQTTv3 and v5 communication ### Network -- [Bind 9](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/bind/): Collect nameserver summary +- [Bind 9](https://github.com/netdata/go.d.plugin/blob/master/modules/bind/README.md): Collect nameserver summary performance statistics via a web interface (`statistics-channels` feature). -- [Chrony](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/chrony): Monitor the precision and +- [Chrony](https://github.com/netdata/go.d.plugin/blob/master/modules/chrony/README.md): Monitor the precision and statistics of a local `chronyd` server. -- [CoreDNS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/coredns/): Measure DNS query round +- [CoreDNS](https://github.com/netdata/go.d.plugin/blob/master/modules/coredns/README.md): Measure DNS query round trip time. -- [Dnsmasq](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq_dhcp/): Automatically +- [Dnsmasq](https://github.com/netdata/go.d.plugin/blob/master/modules/dnsmasq_dhcp/README.md): Automatically detects all configured `Dnsmasq` DHCP ranges and Monitor their utilization. -- [DNSdist](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsdist/): Collect +- [DNSdist](https://github.com/netdata/go.d.plugin/blob/master/modules/dnsdist/README.md): Collect load-balancer performance and health metrics. -- [Dnsmasq DNS Forwarder](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq/): Gather +- [Dnsmasq DNS Forwarder](https://github.com/netdata/go.d.plugin/blob/master/modules/dnsmasq/README.md): Gather queries, entries, operations, and events for the lightweight DNS forwarder. -- [DNS Query Time](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsquery/): Monitor the round +- [DNS Query Time](https://github.com/netdata/go.d.plugin/blob/master/modules/dnsquery/README.md): Monitor the round trip time for DNS queries in milliseconds. -- [Freeradius](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/freeradius/): Collect +- [Freeradius](https://github.com/netdata/go.d.plugin/blob/master/modules/freeradius/README.md): Collect server authentication and accounting statistics from the `status server`. -- [Libreswan](/collectors/charts.d.plugin/libreswan/README.md): Collect bytes-in, bytes-out, and uptime metrics. -- [Icecast](/collectors/python.d.plugin/icecast/README.md): Monitor the number of listeners for active sources. -- [ISC Bind (RDNC)](/collectors/python.d.plugin/bind_rndc/README.md): Collect nameserver summary performance +- [Libreswan](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/libreswan/README.md): Collect + bytes-in, bytes-out, and uptime metrics. +- [Icecast](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/icecast/README.md): Monitor the + number of listeners for active sources. +- [ISC Bind (RDNC)](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/bind_rndc/README.md): + Collect nameserver summary performance statistics using the `rndc` tool. -- [ISC DHCP](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/isc_dhcpd): Reads a +- [ISC DHCP](https://github.com/netdata/go.d.plugin/blob/master/modules/isc_dhcpd/README.md): Reads a `dhcpd.leases` file and collects metrics on total active leases, pool active leases, and pool utilization. -- [OpenLDAP](/collectors/python.d.plugin/openldap/README.md): Provides statistics information from the OpenLDAP +- [OpenLDAP](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/openldap/README.md): Provides + statistics information from the OpenLDAP (`slapd`) server. -- [NSD](/collectors/python.d.plugin/nsd/README.md): Monitor nameserver performance metrics using the `nsd-control` +- [NSD](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/nsd/README.md): Monitor nameserver + performance metrics using the `nsd-control` tool. -- [NTP daemon](/collectors/python.d.plugin/ntpd/README.md): Monitor the system variables of the local `ntpd` daemon - (optionally including variables of the polled peers) using the NTP Control Message Protocol via a UDP socket. -- [OpenSIPS](/collectors/charts.d.plugin/opensips/README.md): Collect server health and performance metrics using the +- [NTP daemon](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/ntpd): Monitor the system variables + of the local `ntpd` daemon (optionally including variables of the polled peers) using the NTP Control Message Protocol + via a UDP socket. +- [OpenSIPS](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/opensips/README.md): Collect + server health and performance metrics using the `opensipsctl` tool. -- [OpenVPN](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/openvpn/): Gather server summary +- [OpenVPN](https://github.com/netdata/go.d.plugin/blob/master/modules/openvpn/README.md): Gather server summary (client, traffic) and per user metrics (traffic, connection time) stats using `management-interface`. -- [Pi-hole](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pihole/): Monitor basic (DNS +- [Pi-hole](https://github.com/netdata/go.d.plugin/blob/master/modules/pihole/README.md): Monitor basic (DNS queries, clients, blocklist) and extended (top clients, top permitted, and blocked domains) statistics using the PHP API. -- [PowerDNS Authoritative Server](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns): +- [PowerDNS Authoritative Server](https://github.com/netdata/go.d.plugin/blob/master/modules/powerdns/README.md): Monitor one or more instances of the nameserver software to collect questions, events, and latency metrics. -- [PowerDNS Recursor](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns_recursor): +- [PowerDNS Recursor](https://github.com/netdata/go.d.plugin/blob/master/modules/powerdns/README.md_recursor): Gather incoming/outgoing questions, drops, timeouts, and cache usage from any number of DNS recursor instances. -- [RetroShare](/collectors/python.d.plugin/retroshare/README.md): Monitor application bandwidth, peers, and DHT +- [RetroShare](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/retroshare/README.md): Monitor + application bandwidth, peers, and DHT metrics. -- [Tor](/collectors/python.d.plugin/tor/README.md): Capture traffic usage statistics using the Tor control port. -- [Unbound](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/unbound/): Collect DNS resolver +- [Tor](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/tor/README.md): Capture traffic usage + statistics using the Tor control port. +- [Unbound](https://github.com/netdata/go.d.plugin/blob/master/modules/unbound/README.md): Collect DNS resolver summary and extended system and per thread metrics via the `remote-control` interface. ### Provisioning -- [Puppet](/collectors/python.d.plugin/puppet/README.md): Monitor the status of Puppet Server and Puppet DB. +- [Puppet](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/puppet/README.md): Monitor the + status of Puppet Server and Puppet DB. ### Remote devices -- [AM2320](/collectors/python.d.plugin/am2320/README.md): Monitor sensor temperature and humidity. -- [Access point](/collectors/charts.d.plugin/ap/README.md): Monitor client, traffic and signal metrics using the `aw` +- [AM2320](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/am2320/README.md): Monitor sensor + temperature and humidity. +- [Access point](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/ap/README.md): Monitor + client, traffic and signal metrics using the `aw` tool. -- [APC UPS](/collectors/charts.d.plugin/apcupsd/README.md): Capture status information using the `apcaccess` tool. -- [Energi Core](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/energid): Monitor +- [APC UPS](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/apcupsd/README.md): Capture status + information using the `apcaccess` tool. +- [Energi Core](https://github.com/netdata/go.d.plugin/blob/master/modules/energid/README.md): Monitor blockchain indexes, memory usage, network usage, and transactions of wallet instances. -- [UPS/PDU](/collectors/charts.d.plugin/nut/README.md): Read the status of UPS/PDU devices using the `upsc` tool. -- [SNMP devices](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/snmp): Gather data using the SNMP +- [UPS/PDU](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/nut/README.md): Read the status of + UPS/PDU devices using the `upsc` tool. +- [SNMP devices](https://github.com/netdata/go.d.plugin/blob/master/modules/snmp/README.md): Gather data using the SNMP protocol. -- [1-Wire sensors](/collectors/python.d.plugin/w1sensor/README.md): Monitor sensor temperature. +- [1-Wire sensors](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/w1sensor/README.md): + Monitor sensor temperature. ### Search -- [Elasticsearch](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/elasticsearch): Collect +- [Elasticsearch](https://github.com/netdata/go.d.plugin/blob/master/modules/elasticsearch/README.md): Collect dozens of metrics on search engine performance from local nodes and local indices. Includes cluster health and statistics. -- [Solr](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/solr/): Collect application search +- [Solr](https://github.com/netdata/go.d.plugin/blob/master/modules/solr/README.md): Collect application search requests, search errors, update requests, and update errors statistics. ### Storage -- [Ceph](/collectors/python.d.plugin/ceph/README.md): Monitor the Ceph cluster usage and server data consumption. -- [HDFS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/hdfs/): Monitor health and performance +- [Ceph](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/ceph/README.md): Monitor the Ceph + cluster usage and server data consumption. +- [HDFS](https://github.com/netdata/go.d.plugin/blob/master/modules/hdfs/README.md): Monitor health and performance metrics for filesystem datanodes and namenodes. -- [IPFS](/collectors/python.d.plugin/ipfs/README.md): Collect file system bandwidth, peers, and repo metrics. -- [Scaleio](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/scaleio/): Monitor storage system, +- [IPFS](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/ipfs/README.md): Collect file system + bandwidth, peers, and repo metrics. +- [Scaleio](https://github.com/netdata/go.d.plugin/blob/master/modules/scaleio/README.md): Monitor storage system, storage pools, and SDCS health and performance metrics via VxFlex OS Gateway API. -- [Samba](/collectors/python.d.plugin/samba/README.md): Collect file sharing metrics using the `smbstatus` tool. +- [Samba](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/samba/README.md): Collect file + sharing metrics using the `smbstatus` tool. ### Web -- [Apache](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/apache/): Collect Apache web +- [Apache](https://github.com/netdata/go.d.plugin/blob/master/modules/apache/README.md): Collect Apache web server performance metrics via the `server-status?auto` endpoint. -- [HAProxy](/collectors/python.d.plugin/haproxy/README.md): Collect frontend, backend, and health metrics. -- [HTTP endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/httpcheck/): Monitor +- [HAProxy](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/haproxy/README.md): Collect + frontend, backend, and health metrics. +- [HTTP endpoints](https://github.com/netdata/go.d.plugin/blob/master/modules/httpcheck/README.md): Monitor any HTTP endpoint's availability and response time. -- [Lighttpd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd/): Collect web server +- [Lighttpd](https://github.com/netdata/go.d.plugin/blob/master/modules/lighttpd/README.md): Collect web server performance metrics using the `server-status?auto` endpoint. -- [Lighttpd2](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd2/): Collect web server +- [Lighttpd2](https://github.com/netdata/go.d.plugin/blob/master/modules/lighttpd2/README.md): Collect web server performance metrics using the `server-status?format=plain` endpoint. -- [Litespeed](/collectors/python.d.plugin/litespeed/README.md): Collect web server data (network, connection, +- [Litespeed](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/litespeed/README.md): Collect + web server data (network, connection, requests, cache) by reading `.rtreport*` files. -- [Nginx](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginx/): Monitor web server +- [Nginx](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md): Monitor web server status information by gathering metrics via `ngx_http_stub_status_module`. -- [Nginx VTS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginxvts/): Gathers metrics from +- [Nginx VTS](https://github.com/netdata/go.d.plugin/blob/master/modules/nginxvts/README.md): Gathers metrics from any Nginx deployment with the _virtual host traffic status module_ enabled, including metrics on uptime, memory usage, and cache, and more. -- [PHP-FPM](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpfpm/): Collect application +- [PHP-FPM](https://github.com/netdata/go.d.plugin/blob/master/modules/phpfpm/README.md): Collect application summary and processes health metrics by scraping the status page (`/status?full`). -- [TCP endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/portcheck/): Monitor any +- [TCP endpoints](https://github.com/netdata/go.d.plugin/blob/master/modules/portcheck/README.md): Monitor any TCP endpoint's availability and response time. -- [Spigot Minecraft servers](/collectors/python.d.plugin/spigotmc/README.md): Monitor average ticket rate and number +- [Spigot Minecraft servers](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/spigotmc/README.md): + Monitor average ticket rate and number of users. -- [Squid](/collectors/python.d.plugin/squid/README.md): Monitor client and server bandwidth/requests by gathering +- [Squid](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/squid/README.md): Monitor client and + server bandwidth/requests by gathering data from the Cache Manager component. -- [Tengine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/tengine/): Monitor web server +- [Tengine](https://github.com/netdata/go.d.plugin/blob/master/modules/tengine/README.md): Monitor web server statistics using information provided by `ngx_http_reqstat_module`. -- [Tomcat](/collectors/python.d.plugin/tomcat/README.md): Collect web server performance metrics from the Manager App +- [Tomcat](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/tomcat/README.md): Collect web + server performance metrics from the Manager App (`/manager/status?XML=true`). -- [Traefik](/collectors/python.d.plugin/traefik/README.md): Uses Traefik's Health API to provide statistics. -- [Varnish](/collectors/python.d.plugin/varnish/README.md): Provides HTTP accelerator global, backends (VBE), and +- [Traefik](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/traefik/README.md): Uses Traefik's + Health API to provide statistics. +- [Varnish](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/varnish/README.md): Provides HTTP + accelerator global, backends (VBE), and disks (SMF) statistics using the `varnishstat` tool. -- [x509 check](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/x509check/): Monitor certificate +- [x509 check](https://github.com/netdata/go.d.plugin/blob/master/modules/x509check/README.md): Monitor certificate expiration time. -- [Whois domain expiry](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/whoisquery/): Checks the +- [Whois domain expiry](https://github.com/netdata/go.d.plugin/blob/master/modules/whoisquery/README.md): Checks the remaining time until a given domain is expired. ## System collectors @@ -332,139 +382,198 @@ The Netdata Agent can collect these system- and hardware-level metrics using a v ### Applications -- [Fail2ban](/collectors/python.d.plugin/fail2ban/README.md): Parses configuration files to detect all jails, then +- [Fail2ban](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/fail2ban/README.md): Parses + configuration files to detect all jails, then uses log files to report ban rates and volume of banned IPs. -- [Monit](/collectors/python.d.plugin/monit/README.md): Monitor statuses of targets (service-checks) using the XML +- [Monit](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/monit/README.md): Monitor statuses + of targets (service-checks) using the XML stats interface. - [WMI (Windows Management Instrumentation) - exporter](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/wmi/): Collect CPU, memory, + exporter](https://github.com/netdata/go.d.plugin/blob/master/modules/wmi/README.md): Collect CPU, memory, network, disk, OS, system, and log-in metrics scraping `wmi_exporter`. ### Disks and filesystems -- [BCACHE](/collectors/proc.plugin/README.md): Monitor BCACHE statistics with the the `proc.plugin` collector. -- [Block devices](/collectors/proc.plugin/README.md): Gather metrics about the health and performance of block +- [BCACHE](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor BCACHE statistics + with the the `proc.plugin` collector. +- [Block devices](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather metrics about + the health and performance of block devices using the the `proc.plugin` collector. -- [Btrfs](/collectors/proc.plugin/README.md): Monitors Btrfs filesystems with the the `proc.plugin` collector. -- [Device mapper](/collectors/proc.plugin/README.md): Gather metrics about the Linux device mapper with the proc +- [Btrfs](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitors Btrfs filesystems + with the the `proc.plugin` collector. +- [Device mapper](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather metrics about + the Linux device mapper with the proc collector. -- [Disk space](/collectors/diskspace.plugin/README.md): Collect disk space usage metrics on Linux mount points. -- [Clock synchronization](/collectors/timex.plugin/README.md): Collect the system clock synchronization status on Linux. -- [Files and directories](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/filecheck): Gather +- [Disk space](https://github.com/netdata/netdata/blob/master/collectors/diskspace.plugin/README.md): Collect disk space + usage metrics on Linux mount points. +- [Clock synchronization](https://github.com/netdata/netdata/blob/master/collectors/timex.plugin/README.md): Collect the + system clock synchronization status on Linux. +- [Files and directories](https://github.com/netdata/go.d.plugin/blob/master/modules/filecheck/README.md): Gather metrics about the existence, modification time, and size of files or directories. -- [ioping.plugin](/collectors/ioping.plugin/README.md): Measure disk read/write latency. -- [NFS file servers and clients](/collectors/proc.plugin/README.md): Gather operations, utilization, and space usage +- [ioping.plugin](https://github.com/netdata/netdata/blob/master/collectors/ioping.plugin/README.md): Measure disk + read/write latency. +- [NFS file servers and clients](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): + Gather operations, utilization, and space usage using the the `proc.plugin` collector. -- [RAID arrays](/collectors/proc.plugin/README.md): Collect health, disk status, operation status, and more with the +- [RAID arrays](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect health, disk + status, operation status, and more with the the `proc.plugin` collector. -- [Veritas Volume Manager](/collectors/proc.plugin/README.md): Gather metrics about the Veritas Volume Manager (VVM). -- [ZFS](/collectors/proc.plugin/README.md): Monitor bandwidth and utilization of ZFS disks/partitions using the proc +- [Veritas Volume Manager](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather + metrics about the Veritas Volume Manager (VVM). +- [ZFS](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor bandwidth and + utilization of ZFS disks/partitions using the proc collector. ### eBPF -- [Files](/collectors/ebpf.plugin/README.md): Provides information about how often a system calls kernel +- [Files](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md): Provides information about + how often a system calls kernel functions related to file descriptors using the eBPF collector. -- [Virtual file system (VFS)](/collectors/ebpf.plugin/README.md): Monitor IO, errors, deleted objects, and +- [Virtual file system (VFS)](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md): Monitor + IO, errors, deleted objects, and more for kernel virtual file systems (VFS) using the eBPF collector. -- [Processes](/collectors/ebpf.plugin/README.md): Monitor threads, task exits, and errors using the eBPF collector. +- [Processes](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md): Monitor threads, task + exits, and errors using the eBPF collector. ### Hardware -- [Adaptec RAID](/collectors/python.d.plugin/adaptec_raid/README.md): Monitor logical and physical devices health +- [Adaptec RAID](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/adaptec_raid/README.md): + Monitor logical and physical devices health metrics using the `arcconf` tool. -- [CUPS](/collectors/cups.plugin/README.md): Monitor CUPS. -- [FreeIPMI](/collectors/freeipmi.plugin/README.md): Uses `libipmimonitoring-dev` or `libipmimonitoring-devel` to +- [CUPS](https://github.com/netdata/netdata/blob/master/collectors/cups.plugin/README.md): Monitor CUPS. +- [FreeIPMI](https://github.com/netdata/netdata/blob/master/collectors/freeipmi.plugin/README.md): + Uses `libipmimonitoring-dev` or `libipmimonitoring-devel` to monitor the number of sensors, temperatures, voltages, currents, and more. -- [Hard drive temperature](/collectors/python.d.plugin/hddtemp/README.md): Monitor the temperature of storage +- [Hard drive temperature](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/hddtemp/README.md): + Monitor the temperature of storage devices. -- [HP Smart Storage Arrays](/collectors/python.d.plugin/hpssa/README.md): Monitor controller, cache module, logical +- [HP Smart Storage Arrays](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/hpssa/README.md): + Monitor controller, cache module, logical and physical drive state, and temperature using the `ssacli` tool. -- [MegaRAID controllers](/collectors/python.d.plugin/megacli/README.md): Collect adapter, physical drives, and +- [MegaRAID controllers](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/megacli/README.md): + Collect adapter, physical drives, and battery stats using the `megacli` tool. -- [NVIDIA GPU](/collectors/python.d.plugin/nvidia_smi/README.md): Monitor performance metrics (memory usage, fan +- [NVIDIA GPU](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/nvidia_smi/README.md): Monitor + performance metrics (memory usage, fan speed, pcie bandwidth utilization, temperature, and more) using the `nvidia-smi` tool. -- [Sensors](/collectors/python.d.plugin/sensors/README.md): Reads system sensors information (temperature, voltage, +- [Sensors](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/sensors/README.md): Reads system + sensors information (temperature, voltage, electric current, power, and more) from `/sys/devices/`. -- [S.M.A.R.T](/collectors/python.d.plugin/smartd_log/README.md): Reads SMART Disk Monitoring daemon logs. +- [S.M.A.R.T](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/smartd_log/README.md): Reads + SMART Disk Monitoring daemon logs. ### Memory -- [Available memory](/collectors/proc.plugin/README.md): Tracks changes in available RAM using the the `proc.plugin` +- [Available memory](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Tracks changes in + available RAM using the the `proc.plugin` collector. -- [Committed memory](/collectors/proc.plugin/README.md): Monitor committed memory using the `proc.plugin` collector. -- [Huge pages](/collectors/proc.plugin/README.md): Gather metrics about huge pages in Linux and FreeBSD with the +- [Committed memory](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor committed + memory using the `proc.plugin` collector. +- [Huge pages](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather metrics about + huge pages in Linux and FreeBSD with the `proc.plugin` collector. -- [KSM](/collectors/proc.plugin/README.md): Measure the amount of merging, savings, and effectiveness using the +- [KSM](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Measure the amount of merging, + savings, and effectiveness using the `proc.plugin` collector. -- [Numa](/collectors/proc.plugin/README.md): Gather metrics on the number of non-uniform memory access (NUMA) events +- [Numa](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather metrics on the number + of non-uniform memory access (NUMA) events every second using the `proc.plugin` collector. -- [Page faults](/collectors/proc.plugin/README.md): Collect the number of memory page faults per second using the +- [Page faults](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect the number of + memory page faults per second using the `proc.plugin` collector. -- [RAM](/collectors/proc.plugin/README.md): Collect metrics on system RAM, available RAM, and more using the +- [RAM](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect metrics on system RAM, + available RAM, and more using the `proc.plugin` collector. -- [SLAB](/collectors/slabinfo.plugin/README.md): Collect kernel SLAB details on Linux systems. -- [swap](/collectors/proc.plugin/README.md): Monitor the amount of free and used swap at every second using the +- [SLAB](https://github.com/netdata/netdata/blob/master/collectors/slabinfo.plugin/README.md): Collect kernel SLAB + details on Linux systems. +- [swap](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor the amount of free + and used swap at every second using the `proc.plugin` collector. -- [Writeback memory](/collectors/proc.plugin/README.md): Collect how much memory is actively being written to disk at +- [Writeback memory](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect how much + memory is actively being written to disk at every second using the `proc.plugin` collector. ### Networks -- [Access points](/collectors/charts.d.plugin/ap/README.md): Visualizes data related to access points. -- [fping.plugin](fping.plugin/README.md): Measure network latency, jitter and packet loss between the monitored node +- [Access points](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/ap/README.md): Visualizes + data related to access points. +- [Ping](https://github.com/netdata/go.d.plugin/blob/master/modules/ping/README.md): Measure network latency, jitter and + packet loss between the monitored node and any number of remote network end points. -- [Netfilter](/collectors/nfacct.plugin/README.md): Collect netfilter firewall, connection tracker, and accounting +- [Netfilter](https://github.com/netdata/netdata/blob/master/collectors/nfacct.plugin/README.md): Collect netfilter + firewall, connection tracker, and accounting metrics using `libmnl` and `libnetfilter_acct`. -- [Network stack](/collectors/proc.plugin/README.md): Monitor the networking stack for errors, TCP connection aborts, +- [Network stack](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor the + networking stack for errors, TCP connection aborts, bandwidth, and more. -- [Network QoS](/collectors/tc.plugin/README.md): Collect traffic QoS metrics (`tc`) of Linux network interfaces. -- [SYNPROXY](/collectors/proc.plugin/README.md): Monitor entries uses, SYN packets received, TCP cookies, and more. +- [Network QoS](https://github.com/netdata/netdata/blob/master/collectors/tc.plugin/README.md): Collect traffic QoS + metrics (`tc`) of Linux network interfaces. +- [SYNPROXY](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor entries uses, SYN + packets received, TCP cookies, and more. ### Operating systems -- [freebsd.plugin](freebsd.plugin/README.md): Collect resource usage and performance data on FreeBSD systems. -- [macOS](/collectors/macos.plugin/README.md): Collect resource usage and performance data on macOS systems. +- [freebsd.plugin](https://github.com/netdata/netdata/blob/master/collectors/freebsd.plugin/README.md): Collect resource + usage and performance data on FreeBSD systems. +- [macOS](https://github.com/netdata/netdata/blob/master/collectors/macos.plugin/README.md): Collect resource usage and + performance data on macOS systems. ### Processes -- [Applications](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, eBPF, and other metrics per +- [Applications](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md): Gather CPU, disk, + memory, network, eBPF, and other metrics per application using the `apps.plugin` collector. -- [systemd](/collectors/cgroups.plugin/README.md): Monitor the CPU and memory usage of systemd services using the +- [systemd](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md): Monitor the CPU and + memory usage of systemd services using the `cgroups.plugin` collector. -- [systemd unit states](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/systemdunits): See the +- [systemd unit states](https://github.com/netdata/go.d.plugin/blob/master/modules/systemdunits/README.md): See the state (active, inactive, activating, deactivating, failed) of various systemd unit types. -- [System processes](/collectors/proc.plugin/README.md): Collect metrics on system load and total processes running +- [System processes](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect metrics + on system load and total processes running using `/proc/loadavg` and the `proc.plugin` collector. -- [Uptime](/collectors/proc.plugin/README.md): Monitor the uptime of a system using the `proc.plugin` collector. +- [Uptime](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor the uptime of a + system using the `proc.plugin` collector. ### Resources -- [CPU frequency](/collectors/proc.plugin/README.md): Monitor CPU frequency, as set by the `cpufreq` kernel module, +- [CPU frequency](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor CPU + frequency, as set by the `cpufreq` kernel module, using the `proc.plugin` collector. -- [CPU idle](/collectors/proc.plugin/README.md): Measure CPU idle every second using the `proc.plugin` collector. -- [CPU performance](/collectors/perf.plugin/README.md): Collect CPU performance metrics using performance monitoring +- [CPU idle](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Measure CPU idle every + second using the `proc.plugin` collector. +- [CPU performance](https://github.com/netdata/netdata/blob/master/collectors/perf.plugin/README.md): Collect CPU + performance metrics using performance monitoring units (PMU). -- [CPU throttling](/collectors/proc.plugin/README.md): Gather metrics about thermal throttling using the `/proc/stat` +- [CPU throttling](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Gather metrics + about thermal throttling using the `/proc/stat` module and the `proc.plugin` collector. -- [CPU utilization](/collectors/proc.plugin/README.md): Capture CPU utilization, both system-wide and per-core, using +- [CPU utilization](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Capture CPU + utilization, both system-wide and per-core, using the `/proc/stat` module and the `proc.plugin` collector. -- [Entropy](/collectors/proc.plugin/README.md): Monitor the available entropy on a system using the `proc.plugin` +- [Entropy](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor the available + entropy on a system using the `proc.plugin` collector. -- [Interprocess Communication (IPC)](/collectors/proc.plugin/README.md): Monitor IPC semaphores and shared memory +- [Interprocess Communication (IPC)](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): + Monitor IPC semaphores and shared memory using the `proc.plugin` collector. -- [Interrupts](/collectors/proc.plugin/README.md): Monitor interrupts per second using the `proc.plugin` collector. -- [IdleJitter](/collectors/idlejitter.plugin/README.md): Measure CPU latency and jitter on all operating systems. -- [SoftIRQs](/collectors/proc.plugin/README.md): Collect metrics on SoftIRQs, both system-wide and per-core, using the +- [Interrupts](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Monitor interrupts per + second using the `proc.plugin` collector. +- [IdleJitter](https://github.com/netdata/netdata/blob/master/collectors/idlejitter.plugin/README.md): Measure CPU + latency and jitter on all operating systems. +- [SoftIRQs](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Collect metrics on + SoftIRQs, both system-wide and per-core, using the `proc.plugin` collector. -- [SoftNet](/collectors/proc.plugin/README.md): Capture SoftNet events per second, both system-wide and per-core, +- [SoftNet](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md): Capture SoftNet events per + second, both system-wide and per-core, using the `proc.plugin` collector. ### Users -- [systemd-logind](/collectors/python.d.plugin/logind/README.md): Monitor active sessions, users, and seats tracked +- [systemd-logind](https://github.com/netdata/go.d.plugin/blob/master/modules/logind/README.md): Monitor active + sessions, users, and seats tracked by `systemd-logind` or `elogind`. -- [User/group usage](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, and other metrics per user +- [User/group usage](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md): Gather CPU, disk, + memory, network, and other metrics per user and user group using the `apps.plugin` collector. ## Netdata collectors @@ -473,13 +582,18 @@ These collectors are recursive in nature, in that they monitor some function of collectors are described only in code and associated charts in Netdata dashboards. - [ACLK (code only)](https://github.com/netdata/netdata/blob/master/aclk/legacy/aclk_stats.c): View whether a Netdata - Agent is connected to Netdata Cloud via the [ACLK](/aclk/README.md), the volume of queries, process times, and more. -- [Alarms](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/alarms): This collector creates an + Agent is connected to Netdata Cloud via the [ACLK](https://github.com/netdata/netdata/blob/master/aclk/README.md), the + volume of queries, process times, and more. +- [Alarms](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/alarms/README.md): This collector + creates an **Alarms** menu with one line plot showing the alarm states of a Netdata Agent over time. -- [Anomalies](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/anomalies): This collector uses the +- [Anomalies](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/anomalies/README.md): This + collector uses the Python PyOD library to perform unsupervised anomaly detection on your Netdata charts and/or dimensions. - [Exporting (code only)](https://github.com/netdata/netdata/blob/master/exporting/send_internal_metrics.c): Gather - metrics on CPU utilization for the [exporting engine](/exporting/README.md), and specific metrics for each enabled + metrics on CPU utilization for + the [exporting engine](https://github.com/netdata/netdata/blob/master/exporting/README.md), and specific metrics for + each enabled exporting connector. - [Global statistics (code only)](https://github.com/netdata/netdata/blob/master/daemon/global_statistics.c): See metrics on the CPU utilization, network traffic, volume of web clients, API responses, database engine usage, and @@ -493,14 +607,54 @@ If you're interested in developing a new collector that you'd like to contribute the `go.d.plugin`. - [go.d.plugin](https://github.com/netdata/go.d.plugin): An orchestrator for data collection modules written in `go`. -- [python.d.plugin](python.d.plugin/README.md): An orchestrator for data collection modules written in `python` v2/v3. -- [charts.d.plugin](charts.d.plugin/README.md): An orchestrator for data collection modules written in `bash` v4+. +- [python.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md): An + orchestrator for data collection modules written in `python` v2/v3. +- [charts.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md): An + orchestrator for data collection modules written in `bash` v4+. ## Third-party collectors These collectors are developed and maintained by third parties and, unlike the other collectors, are not installed by default. To use a third-party collector, visit their GitHub/documentation page and follow their installation procedures. +<details> +<summary>Typical third party Python collector installation instructions</summary> + +In general the below steps should be sufficient to use a third party collector. + +1. Download collector code file + into [folder expected by Netdata](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md#environment-variables). +2. Download default collector configuration file + into [folder expected by Netdata](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md#environment-variables). +3. [Edit configuration file](https://github.com/netdata/netdata/blob/master/docs/collect/enable-configure#configure-a-collector) + from step 2 if required. +4. [Enable collector](https://github.com/netdata/netdata/blob/master/docs/collect/enable-configure#enable-a-collector-or-its-orchestrator). +5. [Restart Netdata](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) + +For example below are the steps to enable +the [Python ClickHouse collector](https://github.com/netdata/community/tree/main/collectors/python.d.plugin/clickhouse). + +```bash +# download python collector script to /usr/libexec/netdata/python.d/ +$ sudo wget https://raw.githubusercontent.com/netdata/community/main/collectors/python.d.plugin/clickhouse/clickhouse.chart.py -O /usr/libexec/netdata/python.d/clickhouse.chart.py + +# (optional) download default .conf to /etc/netdata/python.d/ +$ sudo wget https://raw.githubusercontent.com/netdata/community/main/collectors/python.d.plugin/clickhouse/clickhouse.conf -O /etc/netdata/python.d/clickhouse.conf + +# enable collector by adding line a new line with "clickhouse: yes" to /etc/netdata/python.d.conf file +# this will append to the file if it already exists or create it if not +$ sudo echo "clickhouse: yes" >> /etc/netdata/python.d.conf + +# (optional) edit clickhouse.conf if needed +$ sudo vi /etc/netdata/python.d/clickhouse.conf + +# restart netdata +# see docs for more information: https://learn.netdata.cloud/docs/configure/start-stop-restart +$ sudo systemctl restart netdata +``` + +</details> + - [CyberPower UPS](https://github.com/HawtDogFlvrWtr/netdata_cyberpwrups_plugin): Polls CyberPower UPS data using PowerPanel® Personal Linux. - [Logged-in users](https://github.com/veksh/netdata-numsessions): Collect the number of currently logged-on users. @@ -511,8 +665,12 @@ default. To use a third-party collector, visit their GitHub/documentation page a - [Teamspeak 3](https://github.com/coraxx/netdata_ts3_plugin): Pulls active users and bandwidth from TeamSpeak 3 servers. - [SSH](https://github.com/Yaser-Amiri/netdata-ssh-module): Monitor failed authentication requests of an SSH server. +- [ClickHouse](https://github.com/netdata/community/tree/main/collectors/python.d.plugin/clickhouse): + Monitor [ClickHouse](https://clickhouse.com/) database. ## Etc -- [charts.d example](charts.d.plugin/example/README.md): An example `charts.d` collector. -- [python.d example](python.d.plugin/example/README.md): An example `python.d` collector. +- [charts.d example](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/example/README.md): An + example `charts.d` collector. +- [python.d example](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/example/README.md): An + example `python.d` collector. diff --git a/collectors/Makefile.am b/collectors/Makefile.am index 9f8bf5280..24e4c3f09 100644 --- a/collectors/Makefile.am +++ b/collectors/Makefile.am @@ -10,7 +10,6 @@ SUBDIRS = \ cups.plugin \ diskspace.plugin \ timex.plugin \ - fping.plugin \ ioping.plugin \ freebsd.plugin \ freeipmi.plugin \ diff --git a/collectors/README.md b/collectors/README.md index de46a72a4..91a4eeb44 100644 --- a/collectors/README.md +++ b/collectors/README.md @@ -1,48 +1,54 @@ <!-- title: "Collecting metrics" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/README.md" id: "collectors-ref" +sidebar_label: "Plugins Reference" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "References/Collectors" --> # Collecting metrics Netdata can collect metrics from hundreds of different sources, be they internal data created by the system itself, or -external data created by services or applications. To see _all_ of the sources Netdata collects from, view our [list of -supported collectors](/collectors/COLLECTORS.md). +external data created by services or applications. To see _all_ of the sources Netdata collects from, view our +[list of supported collectors](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md). There are two essential points to understand about how collecting metrics works in Netdata: -- All collectors are **installed by default** with every installation of Netdata. You do not need to install - collectors manually to collect metrics from new sources. -- Upon startup, Netdata will **auto-detect** any application or service that has a - [collector](/collectors/COLLECTORS.md), as long as both the collector and the app/service are configured correctly. +- All collectors are **installed by default** with every installation of Netdata. You do not need to install + collectors manually to collect metrics from new sources. +- Upon startup, Netdata will **auto-detect** any application or service that has a + [collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md), as long as both the collector + and the app/service are configured correctly. Most users will want to enable a new Netdata collector for their app/service. For those details, see -our [collectors' configuration reference](/collectors/REFERENCE.md). +our [collectors' configuration reference](https://github.com/netdata/netdata/blob/master/collectors/REFERENCE.md). ## Take your next steps with collectors -[Supported collectors list](/collectors/COLLECTORS.md) +[Supported collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) -[Collectors configuration reference](/collectors/REFERENCE.md) +[Collectors configuration reference](https://github.com/netdata/netdata/blob/master/collectors/REFERENCE.md) ## Guides -[Monitor Nginx or Apache web server log files with Netdata](/docs/guides/collect-apache-nginx-web-logs.md) +[Monitor Nginx or Apache web server log files with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/collect-apache-nginx-web-logs.md) -[Monitor CockroachDB metrics with Netdata](/docs/guides/monitor-cockroachdb.md) +[Monitor CockroachDB metrics with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/monitor-cockroachdb.md) -[Monitor Unbound DNS servers with Netdata](/docs/guides/collect-unbound-metrics.md) +[Monitor Unbound DNS servers with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/collect-unbound-metrics.md) -[Monitor a Hadoop cluster with Netdata](/docs/guides/monitor-hadoop-cluster.md) +[Monitor a Hadoop cluster with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/monitor-hadoop-cluster.md) ## Related features -**[Dashboards](/web/README.md)**: Visualize your newly-collect metrics in real-time using Netdata's [built-in -dashboard](/web/gui/README.md). +**[Dashboards](https://github.com/netdata/netdata/blob/master/web/README.md)**: Visualize your newly-collect metrics in +real-time using Netdata's [built-in dashboard](https://github.com/netdata/netdata/blob/master/web/gui/README.md). -**[Exporting](/exporting/README.md)**: Extend our built-in [database engine](/database/engine/README.md), which supports -long-term metrics storage, by archiving metrics to external databases like Graphite, Prometheus, MongoDB, TimescaleDB, and more. -It can export metrics to multiple databases simultaneously. +**[Exporting](https://github.com/netdata/netdata/blob/master/exporting/README.md)**: Extend our +built-in [database engine](https://github.com/netdata/netdata/blob/master/database/engine/README.md), which supports +long-term metrics storage, by archiving metrics to external databases like Graphite, Prometheus, MongoDB, TimescaleDB, +and more. It can export metrics to multiple databases simultaneously. diff --git a/collectors/REFERENCE.md b/collectors/REFERENCE.md index 939b189ee..270dded29 100644 --- a/collectors/REFERENCE.md +++ b/collectors/REFERENCE.md @@ -1,6 +1,10 @@ <!-- title: "Collectors configuration reference" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/REFERENCE.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/REFERENCE.md" +sidebar_label: "Collectors configuration" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "Setup" --> # Collectors configuration reference @@ -19,7 +23,7 @@ independent processes in a variety of programming languages based on their purpo MySQL database, among many others. For most users, enabling individual collectors for the application/service you're interested in is far more important -than knowing which plugin it uses. See our [collectors list](/collectors/COLLECTORS.md) to see whether your favorite app/service has +than knowing which plugin it uses. See our [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) to see whether your favorite app/service has a collector, and then read the documentation for that specific collector to figure out how to enable it. There are three types of plugins: @@ -31,7 +35,7 @@ There are three types of plugins: independent processes. They communicate with the daemon via pipes. - **Plugin orchestrators**, which are external plugins that instead support a number of **modules**. Modules are a type of collector. We have a few plugin orchestrators available for those who want to develop their own collectors, - but focus most of our efforts on the [Go plugin](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/). + but focus most of our efforts on the [Go plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md). ## Enable, configure, and disable modules @@ -57,7 +61,7 @@ sudo su -s /bin/bash netdata The next step is based on the collector's orchestrator. You can figure out which orchestrator the collector uses by uses either -by viewing the [collectors list](COLLECTORS.md) and referencing the _configuration file_ field. For example, if that +by viewing the [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) and referencing the _configuration file_ field. For example, if that field contains `go.d`, that collector uses the Go orchestrator. ```bash @@ -93,7 +97,6 @@ This section features a list of Netdata's plugins, with a boolean setting to ena # enable running new plugins = yes # check for new plugins every = 60 # slabinfo = no - # fping = yes # ioping = yes # python.d = yes # go.d = yes @@ -105,7 +108,7 @@ This section features a list of Netdata's plugins, with a boolean setting to ena By default, most plugins are enabled, so you don't need to enable them explicitly to use their collectors. To enable or disable any specific plugin, remove the comment (`#`) and change the boolean setting to `yes` or `no`. -All **external plugins** are managed by [plugins.d](plugins.d/README.md), which provides additional management options. +All **external plugins** are managed by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md), which provides additional management options. ## Internal plugins @@ -162,9 +165,10 @@ through this, is to examine what other similar plugins do. ## External Plugins -**External plugins** use the API and are managed by [plugins.d](plugins.d/README.md). +**External plugins** use the API and are managed +by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md). ## Write a custom collector -You can add custom collectors by following the [external plugins documentation](/collectors/plugins.d/README.md). +You can add custom collectors by following the [external plugins documentation](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md). diff --git a/collectors/all.h b/collectors/all.h index 85a7ac8b2..74fdde3f5 100644 --- a/collectors/all.h +++ b/collectors/all.h @@ -363,5 +363,30 @@ #define NETDATA_CHART_PRIO_NETDATA_TIMEX 132030 #define NETDATA_CHART_PRIO_NETDATA_TC_TIME 1000100 +// NETDATA ML CHARTS + +// [ml] charts +#define ML_CHART_PRIO_DIMENSIONS 39181 +#define ML_CHART_PRIO_ANOMALY_RATE 39182 +#define ML_CHART_PRIO_DETECTOR_EVENTS 39183 + +// [netdata.ml] charts +#define NETDATA_ML_CHART_PRIO_MACHINE_LEARNING_STATUS 890001 +#define NETDATA_ML_CHART_PRIO_METRIC_TYPES 890002 +#define NETDATA_ML_CHART_PRIO_TRAINING_STATUS 890003 + +#define NETDATA_ML_CHART_PRIO_PREDICTION_USAGE 890004 +#define NETDATA_ML_CHART_PRIO_TRAINING_USAGE 890005 + +#define NETDATA_ML_CHART_PRIO_QUEUE_STATS 890006 +#define NETDATA_ML_CHART_PRIO_TRAINING_TIME_STATS 890007 +#define NETDATA_ML_CHART_PRIO_TRAINING_RESULTS 890008 + +#define NETDATA_ML_CHART_FAMILY "machine learning" +#define NETDATA_ML_PLUGIN "ml.plugin" +#define NETDATA_ML_MODULE_TRAINING "training" +#define NETDATA_ML_MODULE_DETECTION "detection" +#define NETDATA_ML_MODULE_PREDICTION "prediction" + #endif //NETDATA_ALL_H diff --git a/collectors/apps.plugin/README.md b/collectors/apps.plugin/README.md index 150889d45..ac0d349a2 100644 --- a/collectors/apps.plugin/README.md +++ b/collectors/apps.plugin/README.md @@ -1,7 +1,10 @@ <!-- -title: "apps.plugin" -sidebar_label: "Application monitoring (apps.plugin)" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/apps.plugin/README.md +title: "Application monitoring (apps.plugin)" +sidebar_label: "Application monitoring " +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/apps.plugin/README.md" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # apps.plugin @@ -63,8 +66,8 @@ Each of these sections provides the same number of charts: - Network - Sockets open (`apps.sockets`) -In addition, if the [eBPF collector](/collectors/ebpf.plugin/README.md) is running, your dashboard will also show an -additional [list of charts](/collectors/ebpf.plugin/README.md#integration-with-appsplugin) using low-level Linux +In addition, if the [eBPF collector](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md) is running, your dashboard will also show an +additional [list of charts](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md#integration-with-appsplugin) using low-level Linux metrics. The above are reported: @@ -160,10 +163,10 @@ There are a few command line options you can pass to `apps.plugin`. The list of ### Integration with eBPF If you don't see charts under the **eBPF syscall** or **eBPF net** sections, you should edit your -[`ebpf.d.conf`](/collectors/ebpf.plugin/README.md#configure-the-ebpf-collector) file to ensure the eBPF program is enabled. +[`ebpf.d.conf`](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md#configure-the-ebpf-collector) file to ensure the eBPF program is enabled. Also see our [guide on troubleshooting apps with eBPF -metrics](/docs/guides/troubleshoot/monitor-debug-applications-ebpf.md) for ideas on how to interpret these charts in a +metrics](https://github.com/netdata/netdata/blob/master/docs/guides/troubleshoot/monitor-debug-applications-ebpf.md) for ideas on how to interpret these charts in a few scenarios. ## Permissions @@ -234,7 +237,7 @@ Examples below for process group `sql`: - Open Pipes  - Open Sockets  -For more information about badges check [Generating Badges](/web/api/badges/README.md) +For more information about badges check [Generating Badges](https://github.com/netdata/netdata/blob/master/web/api/badges/README.md) ## Comparison with console tools diff --git a/collectors/apps.plugin/apps_groups.conf b/collectors/apps.plugin/apps_groups.conf index dd45c5a5a..fdb048609 100644 --- a/collectors/apps.plugin/apps_groups.conf +++ b/collectors/apps.plugin/apps_groups.conf @@ -170,6 +170,7 @@ google-agent: *google_guest_agent* *google_osconfig_agent* nvidia-smi: nvidia-smi htop: htop watchdog: watchdog +telegraf: telegraf # ----------------------------------------------------------------------------- # storage, file systems and file servers @@ -378,7 +379,7 @@ sidekiq: *sidekiq* java: java ipfs: ipfs -node: node* +node: node factorio: factorio p4: p4* diff --git a/collectors/apps.plugin/apps_plugin.c b/collectors/apps.plugin/apps_plugin.c index 89b83332b..84506c8e1 100644 --- a/collectors/apps.plugin/apps_plugin.c +++ b/collectors/apps.plugin/apps_plugin.c @@ -10,8 +10,10 @@ #include "libnetdata/libnetdata.h" #include "libnetdata/required_dummies.h" +#define APPS_PLUGIN_PROCESSES_FUNCTION_DESCRIPTION "Detailed information on the currently running processes." + #define APPS_PLUGIN_FUNCTIONS() do { \ - fprintf(stdout, PLUGINSD_KEYWORD_FUNCTION " \"processes\" 10 \"Detailed information on the currently running processes on this node\"\n"); \ + fprintf(stdout, PLUGINSD_KEYWORD_FUNCTION " \"processes\" 10 \"%s\"\n", APPS_PLUGIN_PROCESSES_FUNCTION_DESCRIPTION); \ } while(0) @@ -420,6 +422,7 @@ struct pid_stat { int sortlist; // higher numbers = top on the process tree // each process gets a unique number + bool matched_by_config; struct target *target; // app_groups.conf targets struct target *user_target; // uid based targets struct target *group_target; // gid based targets @@ -974,7 +977,7 @@ static inline struct pid_stat *get_pid_entry(pid_t pid) { init_pid_fds(p, 0, p->fds_size); p->pid = pid; - DOUBLE_LINKED_LIST_APPEND_UNSAFE(root_of_pids, p, prev, next); + DOUBLE_LINKED_LIST_APPEND_ITEM_UNSAFE(root_of_pids, p, prev, next); all_pids[pid] = p; all_pids_count++; @@ -992,7 +995,7 @@ static inline void del_pid_entry(pid_t pid) { debug_log("process %d %s exited, deleting it.", pid, p->comm); - DOUBLE_LINKED_LIST_REMOVE_UNSAFE(root_of_pids, p, prev, next); + DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(root_of_pids, p, prev, next); // free the filename #ifndef __FreeBSD__ @@ -1103,6 +1106,7 @@ static inline void assign_target_to_pid(struct pid_stat *p) { || (proc_pid_cmdline_is_needed && w->starts_with && w->ends_with && p->cmdline && strstr(p->cmdline, w->compare)) ))) { + p->matched_by_config = true; if(w->target) p->target = w->target; else p->target = w; @@ -2832,11 +2836,11 @@ static void apply_apps_groups_targets_inheritance(void) { } // init goes always to default target - if(all_pids[INIT_PID]) + if(all_pids[INIT_PID] && !all_pids[INIT_PID]->matched_by_config) all_pids[INIT_PID]->target = apps_groups_default_target; // pid 0 goes always to default target - if(all_pids[0]) + if(all_pids[0] && !all_pids[INIT_PID]->matched_by_config) all_pids[0]->target = apps_groups_default_target; // give a default target on all top level processes @@ -3359,7 +3363,7 @@ static void normalize_utilization(struct target *root) { // here we try to eliminate them by disabling childs processing either for specific dimensions // or entirely. Of course, either way, we disable it just a single iteration. - kernel_uint_t max_time = processors * time_factor * RATES_DETAIL; + kernel_uint_t max_time = get_system_cpus() * time_factor * RATES_DETAIL; kernel_uint_t utime = 0, cutime = 0, stime = 0, cstime = 0, gtime = 0, cgtime = 0, minflt = 0, cminflt = 0, majflt = 0, cmajflt = 0; if(global_utime > max_time) global_utime = max_time; @@ -3589,6 +3593,13 @@ static void send_collected_data_to_netdata(struct target *root, const char *type } send_END(); + send_BEGIN(type, "rss", dt); + for (w = root; w ; w = w->next) { + if(unlikely(w->exposed && w->processes)) + send_SET(w->name, w->status_vmrss); + } + send_END(); + send_BEGIN(type, "vmem", dt); for (w = root; w ; w = w->next) { if(unlikely(w->exposed && w->processes)) @@ -3728,6 +3739,12 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type } APPS_PLUGIN_FUNCTIONS(); + fprintf(stdout, "CHART %s.rss '' '%s Resident Set Size (w/shared)' 'MiB' mem %s.rss stacked 20004 %d\n", type, title, type, update_every); + for (w = root; w ; w = w->next) { + if(unlikely(w->exposed)) + fprintf(stdout, "DIMENSION %s '' absolute %ld %ld\n", w->name, 1L, 1024L); + } + APPS_PLUGIN_FUNCTIONS(); fprintf(stdout, "CHART %s.vmem '' '%s Virtual Memory Size' 'MiB' mem %s.vmem stacked 20005 %d\n", type, title, type, update_every); for (w = root; w ; w = w->next) { @@ -4271,7 +4288,7 @@ static void apps_plugin_function_processes_help(const char *transaction) { pluginsd_function_result_end_to_stdout(); } -#define add_table_field(wb, key, name, visible, type, units, max, sort, sortable, sticky, unique_key, pointer_to, summary) do { \ +#define add_table_field(wb, key, name, visible, type, visualization, transform, decimal_points, units, max, sort, sortable, sticky, unique_key, pointer_to, summary, range) do { \ if(fields_added) buffer_strcat(wb, ","); \ buffer_sprintf(wb, "\n \"%s\": {", key); \ buffer_sprintf(wb, "\n \"index\":%d,", fields_added); \ @@ -4281,6 +4298,13 @@ static void apps_plugin_function_processes_help(const char *transaction) { buffer_sprintf(wb, "\n \"type\":\"%s\",", type); \ if(units) \ buffer_sprintf(wb, "\n \"units\":\"%s\",", (char*)(units)); \ + buffer_sprintf(wb, "\n \"visualization\":\"%s\",", visualization); \ + buffer_sprintf(wb, "\n \"value_options\":{"); \ + if(units) \ + buffer_sprintf(wb, "\n \"units\":\"%s\",", (char*)(units)); \ + buffer_sprintf(wb, "\n \"transform\":\"%s\",", transform); \ + buffer_sprintf(wb, "\n \"decimal_points\":%d", decimal_points); \ + buffer_sprintf(wb, "\n },"); \ if(!isnan((NETDATA_DOUBLE)(max))) \ buffer_sprintf(wb, "\n \"max\":%f,", (NETDATA_DOUBLE)(max)); \ if(pointer_to) \ @@ -4288,7 +4312,8 @@ static void apps_plugin_function_processes_help(const char *transaction) { buffer_sprintf(wb, "\n \"sort\":\"%s\",", sort); \ buffer_sprintf(wb, "\n \"sortable\":%s,", (sortable)?"true":"false"); \ buffer_sprintf(wb, "\n \"sticky\":%s,", (sticky)?"true":"false"); \ - buffer_sprintf(wb, "\n \"summary\":\"%s\"", summary); \ + buffer_sprintf(wb, "\n \"summary\":\"%s\",", summary); \ + buffer_sprintf(wb, "\n \"filter\":\"%s\"", (range)?"range":"multiselect"); \ buffer_sprintf(wb, "\n }"); \ fields_added++; \ } while(0) @@ -4380,16 +4405,18 @@ static void apps_plugin_function_processes(const char *transaction, char *functi unsigned int memory_divisor = 1024; unsigned int io_divisor = 1024 * RATES_DETAIL; - BUFFER *wb = buffer_create(PLUGINSD_LINE_MAX); + BUFFER *wb = buffer_create(PLUGINSD_LINE_MAX, NULL); buffer_sprintf(wb, "{" "\n \"status\":%d" ",\n \"type\":\"table\"" ",\n \"update_every\":%d" + ",\n \"help\":\"%s\"" ",\n \"data\":[" "\n" , HTTP_RESP_OK , update_every + , APPS_PLUGIN_PROCESSES_FUNCTION_DESCRIPTION ); NETDATA_DOUBLE @@ -4404,7 +4431,7 @@ static void apps_plugin_function_processes(const char *transaction, char *functi , RSS_max = 0.0 , Shared_max = 0.0 , Swap_max = 0.0 - , MemPcnt_max = 0.0 + , Memory_max = 0.0 ; unsigned long long @@ -4513,42 +4540,27 @@ static void apps_plugin_function_processes(const char *transaction, char *functi // gid buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->gid); - // procs - add_value_field_llu_with_max(wb, Processes, p->children_count); - - // threads - add_value_field_llu_with_max(wb, Threads, p->num_threads); - - // uptime - add_value_field_llu_with_max(wb, Uptime, p->uptime); - - // minor page faults - add_value_field_llu_with_max(wb, MinFlt, p->minflt / RATES_DETAIL); - add_value_field_llu_with_max(wb, CMinFlt, p->cminflt / RATES_DETAIL); - add_value_field_llu_with_max(wb, TMinFlt, (p->minflt + p->cminflt) / RATES_DETAIL); - - // major page faults - add_value_field_llu_with_max(wb, MajFlt, p->majflt / RATES_DETAIL); - add_value_field_llu_with_max(wb, CMajFlt, p->cmajflt / RATES_DETAIL); - add_value_field_llu_with_max(wb, TMajFlt, (p->majflt + p->cmajflt) / RATES_DETAIL); - // CPU utilization % + add_value_field_ndd_with_max(wb, CPU, (NETDATA_DOUBLE)(p->utime + p->stime + p->gtime + p->cutime + p->cstime + p->cgtime) / cpu_divisor); add_value_field_ndd_with_max(wb, UserCPU, (NETDATA_DOUBLE)(p->utime) / cpu_divisor); add_value_field_ndd_with_max(wb, SysCPU, (NETDATA_DOUBLE)(p->stime) / cpu_divisor); add_value_field_ndd_with_max(wb, GuestCPU, (NETDATA_DOUBLE)(p->gtime) / cpu_divisor); add_value_field_ndd_with_max(wb, CUserCPU, (NETDATA_DOUBLE)(p->cutime) / cpu_divisor); add_value_field_ndd_with_max(wb, CSysCPU, (NETDATA_DOUBLE)(p->cstime) / cpu_divisor); add_value_field_ndd_with_max(wb, CGuestCPU, (NETDATA_DOUBLE)(p->cgtime) / cpu_divisor); - add_value_field_ndd_with_max(wb, CPU, (NETDATA_DOUBLE)(p->utime + p->stime + p->gtime + p->cutime + p->cstime + p->cgtime) / cpu_divisor); // memory MiB - add_value_field_ndd_with_max(wb, VMSize, (NETDATA_DOUBLE)p->status_vmsize / memory_divisor); + if(MemTotal) + add_value_field_ndd_with_max(wb, Memory, (NETDATA_DOUBLE)p->status_vmrss * 100.0 / (NETDATA_DOUBLE)MemTotal); + add_value_field_ndd_with_max(wb, RSS, (NETDATA_DOUBLE)p->status_vmrss / memory_divisor); add_value_field_ndd_with_max(wb, Shared, (NETDATA_DOUBLE)p->status_vmshared / memory_divisor); + add_value_field_ndd_with_max(wb, VMSize, (NETDATA_DOUBLE)p->status_vmsize / memory_divisor); add_value_field_ndd_with_max(wb, Swap, (NETDATA_DOUBLE)p->status_vmswap / memory_divisor); - if(MemTotal) - add_value_field_ndd_with_max(wb, MemPcnt, (NETDATA_DOUBLE)p->status_vmrss * 100.0 / (NETDATA_DOUBLE)MemTotal); + // Physical I/O + add_value_field_llu_with_max(wb, PReads, p->io_storage_bytes_read / io_divisor); + add_value_field_llu_with_max(wb, PWrites, p->io_storage_bytes_written / io_divisor); // Logical I/O #ifndef __FreeBSD__ @@ -4556,15 +4568,22 @@ static void apps_plugin_function_processes(const char *transaction, char *functi add_value_field_llu_with_max(wb, LWrites, p->io_logical_bytes_written / io_divisor); #endif - // Physical I/O - add_value_field_llu_with_max(wb, PReads, p->io_storage_bytes_read / io_divisor); - add_value_field_llu_with_max(wb, PWrites, p->io_storage_bytes_written / io_divisor); - // I/O calls add_value_field_llu_with_max(wb, RCalls, p->io_read_calls / RATES_DETAIL); add_value_field_llu_with_max(wb, WCalls, p->io_write_calls / RATES_DETAIL); + // minor page faults + add_value_field_llu_with_max(wb, MinFlt, p->minflt / RATES_DETAIL); + add_value_field_llu_with_max(wb, CMinFlt, p->cminflt / RATES_DETAIL); + add_value_field_llu_with_max(wb, TMinFlt, (p->minflt + p->cminflt) / RATES_DETAIL); + + // major page faults + add_value_field_llu_with_max(wb, MajFlt, p->majflt / RATES_DETAIL); + add_value_field_llu_with_max(wb, CMajFlt, p->cmajflt / RATES_DETAIL); + add_value_field_llu_with_max(wb, TMajFlt, (p->majflt + p->cmajflt) / RATES_DETAIL); + // open file descriptors + add_value_field_llu_with_max(wb, FDs, p->openfds.files + p->openfds.pipes + p->openfds.sockets + p->openfds.inotifies + p->openfds.eventfds + p->openfds.timerfds + p->openfds.signalfds + p->openfds.eventpolls + p->openfds.other); add_value_field_llu_with_max(wb, Files, p->openfds.files); add_value_field_llu_with_max(wb, Pipes, p->openfds.pipes); add_value_field_llu_with_max(wb, Sockets, p->openfds.sockets); @@ -4574,7 +4593,11 @@ static void apps_plugin_function_processes(const char *transaction, char *functi add_value_field_llu_with_max(wb, SigFDs, p->openfds.signalfds); add_value_field_llu_with_max(wb, EvPollFDs, p->openfds.eventpolls); add_value_field_llu_with_max(wb, OtherFDs, p->openfds.other); - add_value_field_llu_with_max(wb, FDs, p->openfds.files + p->openfds.pipes + p->openfds.sockets + p->openfds.inotifies + p->openfds.eventfds + p->openfds.timerfds + p->openfds.signalfds + p->openfds.eventpolls + p->openfds.other); + + // processes, threads, uptime + add_value_field_llu_with_max(wb, Processes, p->children_count); + add_value_field_llu_with_max(wb, Threads, p->num_threads); + add_value_field_llu_with_max(wb, Uptime, p->uptime); buffer_fast_strcat(wb, "]", 1); @@ -4590,75 +4613,77 @@ static void apps_plugin_function_processes(const char *transaction, char *functi // IMPORTANT! // THE ORDER SHOULD BE THE SAME WITH THE VALUES! - add_table_field(wb, "Pid", "Process ID", true, "integer", NULL, NAN, "ascending", true, true, true, NULL, "count_unique"); - add_table_field(wb, "Cmd", "Process Name", true, "string", NULL, NAN, "ascending", true, true, false, NULL, "count_unique"); + add_table_field(wb, "PID", "Process ID", true, "integer", "value", "number", 0, NULL, NAN, "ascending", true, true, true, NULL, "count_unique", false); + add_table_field(wb, "Cmd", "Process Name", true, "string", "value", "none", 0, NULL, NAN, "ascending", true, true, false, NULL, "count_unique", false); #ifdef NETDATA_DEV_MODE - add_table_field(wb, "CmdLine", "Command Line", false, "detail-string:Cmd", NULL, NAN, "ascending", true, false, false, NULL, "count_unique"); + add_table_field(wb, "CmdLine", "Command Line", false, "detail-string:Cmd", "value", "none", 0, NULL, NAN, "ascending", true, false, false, NULL, "count_unique", false); #endif - add_table_field(wb, "PPid", "Parent Process ID", false, "integer", NULL, NAN, "ascending", true, false, false, "Pid", "count_unique"); - add_table_field(wb, "Category", "Category (apps_groups.conf)", true, "string", NULL, NAN, "ascending", true, true, false, NULL, "count_unique"); - add_table_field(wb, "User", "User Owner", true, "string", NULL, NAN, "ascending", true, false, false, NULL, "count_unique"); - add_table_field(wb, "Uid", "User ID", false, "integer", NULL, NAN, "ascending", true, false, false, NULL, "count_unique"); - add_table_field(wb, "Group", "Group Owner", false, "string", NULL, NAN, "ascending", true, false, false, NULL, "count_unique"); - add_table_field(wb, "Gid", "Group ID", false, "integer", NULL, NAN, "ascending", true, false, false, NULL, "count_unique"); - add_table_field(wb, "Processes", "Processes", true, "bar-with-integer", "processes", Processes_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Threads", "Threads", true, "bar-with-integer", "threads", Threads_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Uptime", "Uptime in seconds", true, "duration", "seconds", Uptime_max, "descending", true, false, false, NULL, "max"); - - // minor page faults - add_table_field(wb, "MinFlt", "Minor Page Faults/s", false, "bar", "pgflts/s", MinFlt_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CMinFlt", "Children Minor Page Faults/s", false, "bar", "pgflts/s", CMinFlt_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "TMinFlt", "Total Minor Page Faults/s", false, "bar", "pgflts/s", TMinFlt_max, "descending", true, false, false, NULL, "sum"); - - // major page faults - add_table_field(wb, "MajFlt", "Major Page Faults/s", false, "bar", "pgflts/s", MajFlt_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CMajFlt", "Children Major Page Faults/s", false, "bar", "pgflts/s", CMajFlt_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "TMajFlt", "Total Major Page Faults/s", true, "bar", "pgflts/s", TMajFlt_max, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "PPID", "Parent Process ID", false, "integer", "value", "number", 0, NULL, NAN, "ascending", true, false, false, "PID", "count_unique", false); + add_table_field(wb, "Category", "Category (apps_groups.conf)", true, "string", "value", "none", 0, NULL, NAN, "ascending", true, true, false, NULL, "count_unique", false); + add_table_field(wb, "User", "User Owner", true, "string", "value", "none", 0, NULL, NAN, "ascending", true, false, false, NULL, "count_unique", false); + add_table_field(wb, "Uid", "User ID", false, "integer", "value", "number", 0, NULL, NAN, "ascending", true, false, false, NULL, "count_unique", false); + add_table_field(wb, "Group", "Group Owner", false, "string", "value", "none", 0, NULL, NAN, "ascending", true, false, false, NULL, "count_unique", false); + add_table_field(wb, "Gid", "Group ID", false, "integer", "value", "number", 0, NULL, NAN, "ascending", true, false, false, NULL, "count_unique", false); // CPU utilization - add_table_field(wb, "UserCPU", "User CPU time", false, "bar", "%", UserCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "SysCPU", "System CPU Time", false, "bar", "%", SysCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "GuestCPU", "Guest CPU Time", false, "bar", "%", GuestCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CUserCPU", "Children User CPU Time", false, "bar", "%", CUserCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CSysCPU", "Children System CPU Time", false, "bar", "%", CSysCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CGuestCPU", "Children Guest CPU Time", false, "bar", "%", CGuestCPU_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "CPU", "Total CPU Time", true, "bar", "%", CPU_max, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "CPU", "Total CPU Time (100% = 1 core)", true, "bar-with-integer", "bar", "number", 2, "%", CPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "UserCPU", "User CPU time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", UserCPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "SysCPU", "System CPU Time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", SysCPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "GuestCPU", "Guest CPU Time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", GuestCPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "CUserCPU", "Children User CPU Time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", CUserCPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "CSysCPU", "Children System CPU Time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", CSysCPU_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "CGuestCPU", "Children Guest CPU Time (100% = 1 core)", false, "bar-with-integer", "bar", "number", 2, "%", CGuestCPU_max, "descending", true, false, false, NULL, "sum", true); // memory - add_table_field(wb, "VMSize", "Virtual Memory Size", false, "bar", "MiB", VMSize_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "RSS", "Resident Set Size", MemTotal ? false : true, "bar", "MiB", RSS_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Shared", "Shared Pages", false, "bar", "MiB", Shared_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Swap", "Swap Memory", false, "bar", "MiB", Swap_max, "descending", true, false, false, NULL, "sum"); - if(MemTotal) - add_table_field(wb, "MemPcnt", "Memory Percentage", true, "bar", "%", 100.0, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "Memory", "Memory Percentage", true, "bar-with-integer", "bar", "number", 2, "%", 100.0, "descending", true, false, false, NULL, "sum", true); - // Logical I/O -#ifndef __FreeBSD__ - add_table_field(wb, "LReads", "Logical I/O Reads", false, "bar", "KiB/s", LReads_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "LWrites", "Logical I/O Writes", false, "bar", "KiB/s", LWrites_max, "descending", true, false, false, NULL, "sum"); -#endif + add_table_field(wb, "Resident", "Resident Set Size", true, "bar-with-integer", "bar", "number", 2, "MiB", RSS_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Shared", "Shared Pages", true, "bar-with-integer", "bar", "number", 2, "MiB", Shared_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Virtual", "Virtual Memory Size", true, "bar-with-integer", "bar", "number", 2, "MiB", VMSize_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Swap", "Swap Memory", false, "bar-with-integer", "bar", "number", 2, "MiB", Swap_max, "descending", true, false, false, NULL, "sum", true); // Physical I/O - add_table_field(wb, "PReads", "Physical I/O Reads", true, "bar", "KiB/s", PReads_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "PWrites", "Physical I/O Writes", true, "bar", "KiB/s", PWrites_max, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "PReads", "Physical I/O Reads", true, "bar-with-integer", "bar", "number", 2, "KiB/s", PReads_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "PWrites", "Physical I/O Writes", true, "bar-with-integer", "bar", "number", 2, "KiB/s", PWrites_max, "descending", true, false, false, NULL, "sum", true); + + // Logical I/O +#ifndef __FreeBSD__ + add_table_field(wb, "LReads", "Logical I/O Reads", true, "bar-with-integer", "bar", "number", 2, "KiB/s", LReads_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "LWrites", "Logical I/O Writes", true, "bar-with-integer", "bar", "number", 2, "KiB/s", LWrites_max, "descending", true, false, false, NULL, "sum", true); +#endif // I/O calls - add_table_field(wb, "RCalls", "I/O Read Calls", false, "bar", "calls/s", RCalls_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "WCalls", "I/O Write Calls", false, "bar", "calls/s", WCalls_max, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "RCalls", "I/O Read Calls", true, "bar-with-integer", "bar", "number", 2, "calls/s", RCalls_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "WCalls", "I/O Write Calls", true, "bar-with-integer", "bar", "number", 2, "calls/s", WCalls_max, "descending", true, false, false, NULL, "sum", true); + + // minor page faults + add_table_field(wb, "MinFlt", "Minor Page Faults/s", false, "bar-with-integer", "bar", "number", 2, "pgflts/s", MinFlt_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "CMinFlt", "Children Minor Page Faults/s", false, "bar-with-integer", "bar", "number", 2, "pgflts/s", CMinFlt_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "TMinFlt", "Total Minor Page Faults/s", false, "bar-with-integer", "bar", "number", 2, "pgflts/s", TMinFlt_max, "descending", true, false, false, NULL, "sum", true); + + // major page faults + add_table_field(wb, "MajFlt", "Major Page Faults/s", false, "bar-with-integer", "bar", "number", 2, "pgflts/s", MajFlt_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "CMajFlt", "Children Major Page Faults/s", false, "bar-with-integer", "bar", "number", 2, "pgflts/s", CMajFlt_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "TMajFlt", "Total Major Page Faults/s", true, "bar-with-integer", "bar", "number", 2, "pgflts/s", TMajFlt_max, "descending", true, false, false, NULL, "sum", true); // open file descriptors - add_table_field(wb, "Files", "Open Files", false, "bar", "fds", Files_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Pipes", "Open Pipes", false, "bar", "fds", Pipes_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "Sockets", "Open Sockets", false, "bar", "fds", Sockets_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "iNotiFDs", "Open iNotify Descriptors", false, "bar", "fds", iNotiFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "EventFDs", "Open Event Descriptors", false, "bar", "fds", EventFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "TimerFDs", "Open Timer Descriptors", false, "bar", "fds", TimerFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "SigFDs", "Open Signal Descriptors", false, "bar", "fds", SigFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "EvPollFDs", "Open Event Poll Descriptors", false, "bar", "fds", EvPollFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "OtherFDs", "Other Open Descriptors", false, "bar", "fds", OtherFDs_max, "descending", true, false, false, NULL, "sum"); - add_table_field(wb, "FDs", "All Open File Descriptors", true, "bar", "fds", FDs_max, "descending", true, false, false, NULL, "sum"); + add_table_field(wb, "FDs", "All Open File Descriptors", true, "bar-with-integer", "bar", "number", 0, "fds", FDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Files", "Open Files", true, "bar-with-integer", "bar", "number", 0, "fds", Files_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Pipes", "Open Pipes", true, "bar-with-integer", "bar", "number", 0, "fds", Pipes_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Sockets", "Open Sockets", true, "bar-with-integer", "bar", "number", 0, "fds", Sockets_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "iNotiFDs", "Open iNotify Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", iNotiFDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "EventFDs", "Open Event Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", EventFDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "TimerFDs", "Open Timer Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", TimerFDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "SigFDs", "Open Signal Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", SigFDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "EvPollFDs", "Open Event Poll Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", EvPollFDs_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "OtherFDs", "Other Open Descriptors", false, "bar-with-integer", "bar", "number", 0, "fds", OtherFDs_max, "descending", true, false, false, NULL, "sum", true); + + // processes, threads, uptime + add_table_field(wb, "Processes", "Processes", true, "bar-with-integer", "bar", "number", 0, "processes", Processes_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Threads", "Threads", true, "bar-with-integer", "bar", "number", 0, "threads", Threads_max, "descending", true, false, false, NULL, "sum", true); + add_table_field(wb, "Uptime", "Uptime in seconds", true, "duration", "bar", "duration", 2, "seconds", Uptime_max, "descending", true, false, false, NULL, "max", true); buffer_strcat( wb, @@ -4674,7 +4699,7 @@ static void apps_plugin_function_processes(const char *transaction, char *functi "\n \"Memory\": {" "\n \"name\":\"Memory\"," "\n \"type\":\"stacked-bar\"," - "\n \"columns\": [ \"VMSize\", \"RSS\", \"Shared\", \"Swap\" ]" + "\n \"columns\": [ \"Virtual\", \"Resident\", \"Shared\", \"Swap\" ]" "\n }," ); @@ -4685,7 +4710,7 @@ static void apps_plugin_function_processes(const char *transaction, char *functi "\n \"MemoryPercent\": {" "\n \"name\":\"Memory Percentage\"," "\n \"type\":\"stacked-bar\"," - "\n \"columns\": [ \"MemPcnt\" ]" + "\n \"columns\": [ \"Memory\" ]" "\n }," ); @@ -4747,19 +4772,19 @@ static void apps_plugin_function_processes(const char *transaction, char *functi "\n \"group_by\": {" "\n \"pid\": {" "\n \"name\":\"Process Tree by PID\"," - "\n \"columns\":[ \"PPid\" ]" + "\n \"columns\":[ \"PPID\" ]" "\n }," "\n \"category\": {" "\n \"name\":\"Process Tree by Category\"," - "\n \"columns\":[ \"Category\", \"PPid\" ]" + "\n \"columns\":[ \"Category\", \"PPID\" ]" "\n }," "\n \"user\": {" "\n \"name\":\"Process Tree by User\"," - "\n \"columns\":[ \"User\", \"PPid\" ]" + "\n \"columns\":[ \"User\", \"PPID\" ]" "\n }," "\n \"group\": {" "\n \"name\":\"Process Tree by Group\"," - "\n \"columns\":[ \"Group\", \"PPid\" ]" + "\n \"columns\":[ \"Group\", \"PPID\" ]" "\n }" "\n }" ); @@ -4834,6 +4859,7 @@ void *reader_main(void *arg __maybe_unused) { int main(int argc, char **argv) { // debug_flags = D_PROCFILE; + stderror = stderr; clocks_init(); @@ -4897,7 +4923,7 @@ int main(int argc, char **argv) { #endif get_system_pid_max(); - get_system_cpus(); + get_system_cpus_uncached(); parse_args(argc, argv); diff --git a/collectors/cgroups.plugin/README.md b/collectors/cgroups.plugin/README.md index d0f822e66..e58f1ba04 100644 --- a/collectors/cgroups.plugin/README.md +++ b/collectors/cgroups.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "cgroups.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/cgroups.plugin/README.md +title: "Monitor Cgroups (cgroups.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/cgroups.plugin/README.md" +sidebar_label: "Monitor Cgroups" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Virtualized environments/Containers" --> # cgroups.plugin @@ -74,7 +78,7 @@ currently unsupported when using unified cgroups. ### enabled cgroups To provide a sane default, Netdata uses the -following [pattern list](https://learn.netdata.cloud/docs/agent/libnetdata/simple_pattern): +following [pattern list](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md): - checks the pattern against the path of the cgroup @@ -305,4 +309,4 @@ cannot find, but immediately: - I/O full pressure Network interfaces are monitored by means of -the [proc plugin](/collectors/proc.plugin/README.md#monitored-network-interface-metrics). +the [proc plugin](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md#monitored-network-interface-metrics). diff --git a/collectors/cgroups.plugin/cgroup-network.c b/collectors/cgroups.plugin/cgroup-network.c index 0b66ea475..a490df394 100644 --- a/collectors/cgroups.plugin/cgroup-network.c +++ b/collectors/cgroups.plugin/cgroup-network.c @@ -43,7 +43,7 @@ unsigned int read_iface_iflink(const char *prefix, const char *iface) { unsigned long long iflink = 0; int ret = read_single_number_file(filename, &iflink); - if(ret) error("Cannot read '%s'.", filename); + if(ret) collector_error("Cannot read '%s'.", filename); return (unsigned int)iflink; } @@ -56,7 +56,7 @@ unsigned int read_iface_ifindex(const char *prefix, const char *iface) { unsigned long long ifindex = 0; int ret = read_single_number_file(filename, &ifindex); - if(ret) error("Cannot read '%s'.", filename); + if(ret) collector_error("Cannot read '%s'.", filename); return (unsigned int)ifindex; } @@ -70,18 +70,18 @@ struct iface *read_proc_net_dev(const char *scope __maybe_unused, const char *pr snprintfz(filename, FILENAME_MAX, "%s%s", prefix, (*prefix)?"/proc/1/net/dev":"/proc/net/dev"); #ifdef NETDATA_INTERNAL_CHECKS - info("parsing '%s'", filename); + collector_info("parsing '%s'", filename); #endif ff = procfile_open(filename, " \t,:|", PROCFILE_FLAG_DEFAULT); if(unlikely(!ff)) { - error("Cannot open file '%s'", filename); + collector_error("Cannot open file '%s'", filename); return NULL; } ff = procfile_readall(ff); if(unlikely(!ff)) { - error("Cannot read file '%s'", filename); + collector_error("Cannot read file '%s'", filename); return NULL; } @@ -99,7 +99,7 @@ struct iface *read_proc_net_dev(const char *scope __maybe_unused, const char *pr root = t; #ifdef NETDATA_INTERNAL_CHECKS - info("added %s interface '%s', ifindex %u, iflink %u", scope, t->device, t->ifindex, t->iflink); + collector_info("added %s interface '%s', ifindex %u, iflink %u", scope, t->device, t->ifindex, t->iflink); #endif } @@ -145,7 +145,7 @@ static void continue_as_child(void) { pid_t ret; if (child < 0) - error("fork() failed"); + collector_error("fork() failed"); /* Only the child returns */ if (child == 0) @@ -180,7 +180,7 @@ int proc_pid_fd(const char *prefix, const char *ns, pid_t pid) { int fd = open(filename, O_RDONLY); if(fd == -1) - error("Cannot open proc_pid_fd() file '%s'", filename); + collector_error("Cannot open proc_pid_fd() file '%s'", filename); return fd; } @@ -230,7 +230,7 @@ int switch_namespace(const char *prefix, pid_t pid) { if(setns(all_ns[i].fd, all_ns[i].nstype) == -1) { if(pass == 1) { all_ns[i].status = 0; - error("Cannot switch to %s namespace of pid %d", all_ns[i].name, (int) pid); + collector_error("Cannot switch to %s namespace of pid %d", all_ns[i].name, (int) pid); } } else @@ -243,17 +243,17 @@ int switch_namespace(const char *prefix, pid_t pid) { if(root_fd != -1) { if(fchdir(root_fd) < 0) - error("Cannot fchdir() to pid %d root directory", (int)pid); + collector_error("Cannot fchdir() to pid %d root directory", (int)pid); if(chroot(".") < 0) - error("Cannot chroot() to pid %d root directory", (int)pid); + collector_error("Cannot chroot() to pid %d root directory", (int)pid); close(root_fd); } if(cwd_fd != -1) { if(fchdir(cwd_fd) < 0) - error("Cannot fchdir() to pid %d current working directory", (int)pid); + collector_error("Cannot fchdir() to pid %d current working directory", (int)pid); close(cwd_fd); } @@ -277,7 +277,7 @@ int switch_namespace(const char *prefix, pid_t pid) { #else errno = ENOSYS; - error("setns() is missing on this system."); + collector_error("setns() is missing on this system."); return 1; #endif @@ -286,13 +286,13 @@ int switch_namespace(const char *prefix, pid_t pid) { pid_t read_pid_from_cgroup_file(const char *filename) { int fd = open(filename, procfile_open_flags); if(fd == -1) { - error("Cannot open pid_from_cgroup() file '%s'.", filename); + collector_error("Cannot open pid_from_cgroup() file '%s'.", filename); return 0; } FILE *fp = fdopen(fd, "r"); if(!fp) { - error("Cannot upgrade fd to fp for file '%s'.", filename); + collector_error("Cannot upgrade fd to fp for file '%s'.", filename); return 0; } @@ -308,7 +308,7 @@ pid_t read_pid_from_cgroup_file(const char *filename) { fclose(fp); #ifdef NETDATA_INTERNAL_CHECKS - if(pid > 0) info("found pid %d on file '%s'", pid, filename); + if(pid > 0) collector_info("found pid %d on file '%s'", pid, filename); #endif return pid; @@ -331,7 +331,7 @@ pid_t read_pid_from_cgroup(const char *path) { DIR *dir = opendir(path); if (!dir) { - error("cannot read directory '%s'", path); + collector_error("cannot read directory '%s'", path); return 0; } @@ -369,7 +369,7 @@ struct found_device { void add_device(const char *host, const char *guest) { #ifdef NETDATA_INTERNAL_CHECKS - info("adding device with host '%s', guest '%s'", host, guest); + collector_info("adding device with host '%s', guest '%s'", host, guest); #endif uint32_t hash = simple_hash(host); @@ -422,36 +422,36 @@ void detect_veth_interfaces(pid_t pid) { host = read_proc_net_dev("host", netdata_configured_host_prefix); if(!host) { errno = 0; - error("cannot read host interface list."); + collector_error("cannot read host interface list."); goto cleanup; } if(!eligible_ifaces(host)) { errno = 0; - info("there are no double-linked host interfaces available."); + collector_info("there are no double-linked host interfaces available."); goto cleanup; } if(switch_namespace(netdata_configured_host_prefix, pid)) { errno = 0; - error("cannot switch to the namespace of pid %u", (unsigned int) pid); + collector_error("cannot switch to the namespace of pid %u", (unsigned int) pid); goto cleanup; } #ifdef NETDATA_INTERNAL_CHECKS - info("switched to namespaces of pid %d", pid); + collector_info("switched to namespaces of pid %d", pid); #endif cgroup = read_proc_net_dev("cgroup", NULL); if(!cgroup) { errno = 0; - error("cannot read cgroup interface list."); + collector_error("cannot read cgroup interface list."); goto cleanup; } if(!eligible_ifaces(cgroup)) { errno = 0; - error("there are not double-linked cgroup interfaces available."); + collector_error("there are not double-linked cgroup interfaces available."); goto cleanup; } @@ -495,7 +495,7 @@ cleanup: #define CGROUP_NETWORK_INTERFACE_MAX_LINE 2048 void call_the_helper(pid_t pid, const char *cgroup) { if(setresuid(0, 0, 0) == -1) - error("setresuid(0, 0, 0) failed."); + collector_error("setresuid(0, 0, 0) failed."); char command[CGROUP_NETWORK_INTERFACE_MAX_LINE + 1]; if(cgroup) @@ -503,7 +503,7 @@ void call_the_helper(pid_t pid, const char *cgroup) { else snprintfz(command, CGROUP_NETWORK_INTERFACE_MAX_LINE, "exec " PLUGINS_DIR "/cgroup-network-helper.sh --pid %d", pid); - info("running: %s", command); + collector_info("running: %s", command); pid_t cgroup_pid; FILE *fp_child_input, *fp_child_output; @@ -539,7 +539,7 @@ void call_the_helper(pid_t pid, const char *cgroup) { netdata_pclose(fp_child_input, fp_child_output, cgroup_pid); } else - error("cannot execute cgroup-network helper script: %s", command); + collector_error("cannot execute cgroup-network helper script: %s", command); } int is_valid_path_symbol(char c) { @@ -570,33 +570,33 @@ int verify_path(const char *path) { const char *s = path; while((c = *s++)) { if(!( isalnum(c) || is_valid_path_symbol(c) )) { - error("invalid character in path '%s'", path); + collector_error("invalid character in path '%s'", path); return -1; } } if(strstr(path, "\\") && !strstr(path, "\\x")) { - error("invalid escape sequence in path '%s'", path); + collector_error("invalid escape sequence in path '%s'", path); return 1; } if(strstr(path, "/../")) { - error("invalid parent path sequence detected in '%s'", path); + collector_error("invalid parent path sequence detected in '%s'", path); return 1; } if(path[0] != '/') { - error("only absolute path names are supported - invalid path '%s'", path); + collector_error("only absolute path names are supported - invalid path '%s'", path); return -1; } if (stat(path, &sb) == -1) { - error("cannot stat() path '%s'", path); + collector_error("cannot stat() path '%s'", path); return -1; } if((sb.st_mode & S_IFMT) != S_IFDIR) { - error("path '%s' is not a directory", path); + collector_error("path '%s' is not a directory", path); return -1; } @@ -618,10 +618,10 @@ char *fix_path_variable(void) { char *s = strsep(&ptr, ":"); if(s && *s) { if(verify_path(s) == -1) { - error("the PATH variable includes an invalid path '%s' - removed it.", s); + collector_error("the PATH variable includes an invalid path '%s' - removed it.", s); } else { - info("the PATH variable includes a valid path '%s'.", s); + collector_info("the PATH variable includes a valid path '%s'.", s); if(added) strcat(safe_path, ":"); strcat(safe_path, s); added++; @@ -629,8 +629,8 @@ char *fix_path_variable(void) { } } - info("unsafe PATH: '%s'.", path); - info(" safe PATH: '%s'.", safe_path); + collector_info("unsafe PATH: '%s'.", path); + collector_info(" safe PATH: '%s'.", safe_path); freez(p); return safe_path; @@ -646,6 +646,7 @@ void usage(void) { } int main(int argc, char **argv) { + stderror = stderr; pid_t pid = 0; program_name = argv[0]; @@ -690,7 +691,7 @@ int main(int argc, char **argv) { if(pid <= 0) { errno = 0; - error("Invalid pid %d given", (int) pid); + collector_error("Invalid pid %d given", (int) pid); return 2; } @@ -699,7 +700,7 @@ int main(int argc, char **argv) { else if(!strcmp(argv[arg], "--cgroup")) { char *cgroup = argv[arg+1]; if(verify_path(cgroup) == -1) { - error("cgroup '%s' does not exist or is not valid.", cgroup); + collector_error("cgroup '%s' does not exist or is not valid.", cgroup); return 1; } @@ -708,7 +709,7 @@ int main(int argc, char **argv) { if(pid <= 0 && !detected_devices) { errno = 0; - error("Cannot find a cgroup PID from cgroup '%s'", cgroup); + collector_error("Cannot find a cgroup PID from cgroup '%s'", cgroup); } } else diff --git a/collectors/cgroups.plugin/sys_fs_cgroup.c b/collectors/cgroups.plugin/sys_fs_cgroup.c index 8f7548286..66db0b728 100644 --- a/collectors/cgroups.plugin/sys_fs_cgroup.c +++ b/collectors/cgroups.plugin/sys_fs_cgroup.c @@ -174,9 +174,9 @@ static enum cgroups_systemd_setting cgroups_detect_systemd(const char *exec) } if (ret == -1) { - error("Failed to get the output of \"%s\"", exec); + collector_error("Failed to get the output of \"%s\"", exec); } else if (ret == 0) { - info("Cannot get the output of \"%s\" within %"PRId64" seconds", exec, (int64_t)timeout.tv_sec); + collector_info("Cannot get the output of \"%s\" within %"PRId64" seconds", exec, (int64_t)timeout.tv_sec); } else { while (fgets(buf, MAXSIZE_PROC_CMDLINE, fp_child_output) != NULL) { if ((begin = strstr(buf, SYSTEMD_HIERARCHY_STRING))) { @@ -214,7 +214,7 @@ static enum cgroups_type cgroups_try_detect_version() FILE *fp_child_input; FILE *fp_child_output = netdata_popen("grep cgroup /proc/filesystems", &command_pid, &fp_child_input); if (!fp_child_output) { - error("popen failed"); + collector_error("popen failed"); return CGROUPS_AUTODETECT_FAIL; } while (fgets(buf, MAXSIZE_PROC_CMDLINE, fp_child_output) != NULL) { @@ -258,12 +258,12 @@ static enum cgroups_type cgroups_try_detect_version() // check kernel command line flag that can override that setting FILE *fp = fopen("/proc/cmdline", "r"); if (!fp) { - error("Error reading kernel boot commandline parameters"); + collector_error("Error reading kernel boot commandline parameters"); return CGROUPS_AUTODETECT_FAIL; } if (!fgets(buf, MAXSIZE_PROC_CMDLINE, fp)) { - error("couldn't read all cmdline params into buffer"); + collector_error("couldn't read all cmdline params into buffer"); fclose(fp); return CGROUPS_AUTODETECT_FAIL; } @@ -271,7 +271,7 @@ static enum cgroups_type cgroups_try_detect_version() fclose(fp); if (strstr(buf, "systemd.unified_cgroup_hierarchy=0")) { - info("cgroups v2 (unified cgroups) is available but are disabled on this system."); + collector_info("cgroups v2 (unified cgroups) is available but are disabled on this system."); return CGROUPS_V1; } return CGROUPS_V2; @@ -311,7 +311,7 @@ void read_cgroup_plugin_configuration() { if(cgroup_use_unified_cgroups == CONFIG_BOOLEAN_AUTO) cgroup_use_unified_cgroups = (cgroups_try_detect_version() == CGROUPS_V2); - info("use unified cgroups %s", cgroup_use_unified_cgroups ? "true" : "false"); + collector_info("use unified cgroups %s", cgroup_use_unified_cgroups ? "true" : "false"); cgroup_containers_chart_priority = (int)config_get_number("plugin:cgroups", "containers priority", cgroup_containers_chart_priority); if(cgroup_containers_chart_priority < 1) @@ -361,7 +361,7 @@ void read_cgroup_plugin_configuration() { mi = mountinfo_find_by_filesystem_super_option(root, "cgroup", "cpuacct"); if(!mi) mi = mountinfo_find_by_filesystem_mount_source(root, "cgroup", "cpuacct"); if(!mi) { - error("CGROUP: cannot find cpuacct mountinfo. Assuming default: /sys/fs/cgroup/cpuacct"); + collector_error("CGROUP: cannot find cpuacct mountinfo. Assuming default: /sys/fs/cgroup/cpuacct"); s = "/sys/fs/cgroup/cpuacct"; } else s = mi->mount_point; @@ -371,7 +371,7 @@ void read_cgroup_plugin_configuration() { mi = mountinfo_find_by_filesystem_super_option(root, "cgroup", "cpuset"); if(!mi) mi = mountinfo_find_by_filesystem_mount_source(root, "cgroup", "cpuset"); if(!mi) { - error("CGROUP: cannot find cpuset mountinfo. Assuming default: /sys/fs/cgroup/cpuset"); + collector_error("CGROUP: cannot find cpuset mountinfo. Assuming default: /sys/fs/cgroup/cpuset"); s = "/sys/fs/cgroup/cpuset"; } else s = mi->mount_point; @@ -381,7 +381,7 @@ void read_cgroup_plugin_configuration() { mi = mountinfo_find_by_filesystem_super_option(root, "cgroup", "blkio"); if(!mi) mi = mountinfo_find_by_filesystem_mount_source(root, "cgroup", "blkio"); if(!mi) { - error("CGROUP: cannot find blkio mountinfo. Assuming default: /sys/fs/cgroup/blkio"); + collector_error("CGROUP: cannot find blkio mountinfo. Assuming default: /sys/fs/cgroup/blkio"); s = "/sys/fs/cgroup/blkio"; } else s = mi->mount_point; @@ -391,7 +391,7 @@ void read_cgroup_plugin_configuration() { mi = mountinfo_find_by_filesystem_super_option(root, "cgroup", "memory"); if(!mi) mi = mountinfo_find_by_filesystem_mount_source(root, "cgroup", "memory"); if(!mi) { - error("CGROUP: cannot find memory mountinfo. Assuming default: /sys/fs/cgroup/memory"); + collector_error("CGROUP: cannot find memory mountinfo. Assuming default: /sys/fs/cgroup/memory"); s = "/sys/fs/cgroup/memory"; } else s = mi->mount_point; @@ -401,7 +401,7 @@ void read_cgroup_plugin_configuration() { mi = mountinfo_find_by_filesystem_super_option(root, "cgroup", "devices"); if(!mi) mi = mountinfo_find_by_filesystem_mount_source(root, "cgroup", "devices"); if(!mi) { - error("CGROUP: cannot find devices mountinfo. Assuming default: /sys/fs/cgroup/devices"); + collector_error("CGROUP: cannot find devices mountinfo. Assuming default: /sys/fs/cgroup/devices"); s = "/sys/fs/cgroup/devices"; } else s = mi->mount_point; @@ -433,7 +433,7 @@ void read_cgroup_plugin_configuration() { if(mi) debug(D_CGROUP, "found unified cgroup root using mountsource info, with path: '%s'", mi->mount_point); } if(!mi) { - error("CGROUP: cannot find cgroup2 mountinfo. Assuming default: /sys/fs/cgroup"); + collector_error("CGROUP: cannot find cgroup2 mountinfo. Assuming default: /sys/fs/cgroup"); s = "/sys/fs/cgroup"; } else s = mi->mount_point; @@ -575,13 +575,13 @@ void netdata_cgroup_ebpf_initialize_shm() { shm_fd_cgroup_ebpf = shm_open(NETDATA_SHARED_MEMORY_EBPF_CGROUP_NAME, O_CREAT | O_RDWR, 0660); if (shm_fd_cgroup_ebpf < 0) { - error("Cannot initialize shared memory used by cgroup and eBPF, integration won't happen."); + collector_error("Cannot initialize shared memory used by cgroup and eBPF, integration won't happen."); return; } size_t length = sizeof(netdata_ebpf_cgroup_shm_header_t) + cgroup_root_max * sizeof(netdata_ebpf_cgroup_shm_body_t); if (ftruncate(shm_fd_cgroup_ebpf, length)) { - error("Cannot set size for shared memory."); + collector_error("Cannot set size for shared memory."); goto end_init_shm; } @@ -590,7 +590,7 @@ void netdata_cgroup_ebpf_initialize_shm() shm_fd_cgroup_ebpf, 0); if (!shm_cgroup_ebpf.header) { - error("Cannot map shared memory used between cgroup and eBPF, integration won't happen"); + collector_error("Cannot map shared memory used between cgroup and eBPF, integration won't happen"); goto end_init_shm; } shm_cgroup_ebpf.body = (netdata_ebpf_cgroup_shm_body_t *) ((char *)shm_cgroup_ebpf.header + @@ -604,7 +604,7 @@ void netdata_cgroup_ebpf_initialize_shm() return; } - error("Cannot create semaphore, integration between eBPF and cgroup won't happen"); + collector_error("Cannot create semaphore, integration between eBPF and cgroup won't happen"); munmap(shm_cgroup_ebpf.header, length); end_init_shm: @@ -1077,7 +1077,7 @@ static inline void cgroup_read_cpuacct_stat(struct cpuacct_stat *cp) { unsigned long i, lines = procfile_lines(ff); if(unlikely(lines < 1)) { - error("CGROUP: file '%s' should have 1+ lines.", cp->filename); + collector_error("CGROUP: file '%s' should have 1+ lines.", cp->filename); cp->updated = 0; return; } @@ -1123,7 +1123,7 @@ static inline void cgroup_read_cpuacct_cpu_stat(struct cpuacct_cpu_throttling *c unsigned long lines = procfile_lines(ff); if (unlikely(lines < 3)) { - error("CGROUP: file '%s' should have 3 lines.", cp->filename); + collector_error("CGROUP: file '%s' should have 3 lines.", cp->filename); cp->updated = 0; return; } @@ -1180,7 +1180,7 @@ static inline void cgroup2_read_cpuacct_cpu_stat(struct cpuacct_stat *cp, struct unsigned long lines = procfile_lines(ff); if (unlikely(lines < 3)) { - error("CGROUP: file '%s' should have at least 3 lines.", cp->filename); + collector_error("CGROUP: file '%s' should have at least 3 lines.", cp->filename); cp->updated = 0; return; } @@ -1261,7 +1261,7 @@ static inline void cgroup_read_cpuacct_usage(struct cpuacct_usage *ca) { } if(unlikely(procfile_lines(ff) < 1)) { - error("CGROUP: file '%s' should have 1+ lines but has %zu.", ca->filename, procfile_lines(ff)); + collector_error("CGROUP: file '%s' should have 1+ lines but has %zu.", ca->filename, procfile_lines(ff)); ca->updated = 0; return; } @@ -1326,7 +1326,7 @@ static inline void cgroup_read_blkio(struct blkio *io) { unsigned long i, lines = procfile_lines(ff); if(unlikely(lines < 1)) { - error("CGROUP: file '%s' should have 1+ lines.", io->filename); + collector_error("CGROUP: file '%s' should have 1+ lines.", io->filename); io->updated = 0; return; } @@ -1398,7 +1398,7 @@ static inline void cgroup2_read_blkio(struct blkio *io, unsigned int word_offset unsigned long i, lines = procfile_lines(ff); if (unlikely(lines < 1)) { - error("CGROUP: file '%s' should have 1+ lines.", io->filename); + collector_error("CGROUP: file '%s' should have 1+ lines.", io->filename); io->updated = 0; return; } @@ -1442,7 +1442,7 @@ static inline void cgroup2_read_pressure(struct pressure *res) { size_t lines = procfile_lines(ff); if (lines < 1) { - error("CGROUP: file '%s' should have 1+ lines.", res->filename); + collector_error("CGROUP: file '%s' should have 1+ lines.", res->filename); res->updated = 0; return; } @@ -1456,7 +1456,7 @@ static inline void cgroup2_read_pressure(struct pressure *res) { res->full.share_time.value10 = strtod(procfile_lineword(ff, 1, 2), NULL); res->full.share_time.value60 = strtod(procfile_lineword(ff, 1, 4), NULL); res->full.share_time.value300 = strtod(procfile_lineword(ff, 1, 6), NULL); - res->full.total_time.value_total = str2ull(procfile_lineword(ff, 0, 8)) / 1000; // us->ms + res->full.total_time.value_total = str2ull(procfile_lineword(ff, 1, 8)) / 1000; // us->ms } res->updated = 1; @@ -1499,7 +1499,7 @@ static inline void cgroup_read_memory(struct memory *mem, char parent_cg_is_unif unsigned long i, lines = procfile_lines(ff); if(unlikely(lines < 1)) { - error("CGROUP: file '%s' should have 1+ lines.", mem->filename_detailed); + collector_error("CGROUP: file '%s' should have 1+ lines.", mem->filename_detailed); mem->updated_detailed = 0; goto memory_next; } @@ -1669,7 +1669,7 @@ static inline void read_cgroup_network_interfaces(struct cgroup *cg) { FILE *fp_child_input, *fp_child_output; (void)netdata_popen_raw_default_flags_and_environment(&cgroup_pid, &fp_child_input, &fp_child_output, cgroups_network_interface_script, "--cgroup", cgroup_identifier); if(!fp_child_output) { - error("CGROUP: cannot popen(%s --cgroup \"%s\", \"r\").", cgroups_network_interface_script, cgroup_identifier); + collector_error("CGROUP: cannot popen(%s --cgroup \"%s\", \"r\").", cgroups_network_interface_script, cgroup_identifier); return; } @@ -1687,12 +1687,12 @@ static inline void read_cgroup_network_interfaces(struct cgroup *cg) { } if(!*s) { - error("CGROUP: empty host interface returned by script"); + collector_error("CGROUP: empty host interface returned by script"); continue; } if(!*t) { - error("CGROUP: empty guest interface returned by script"); + collector_error("CGROUP: empty guest interface returned by script"); continue; } @@ -1702,7 +1702,7 @@ static inline void read_cgroup_network_interfaces(struct cgroup *cg) { i->next = cg->interfaces; cg->interfaces = i; - info("CGROUP: cgroup '%s' has network interface '%s' as '%s'", cg->id, i->host_device, i->container_device); + collector_info("CGROUP: cgroup '%s' has network interface '%s' as '%s'", cg->id, i->host_device, i->container_device); // register a device rename to proc_net_dev.c netdev_rename_device_add( @@ -1875,7 +1875,7 @@ static inline void discovery_rename_cgroup(struct cgroup *cg) { FILE *fp_child_input, *fp_child_output; (void)netdata_popen_raw_default_flags_and_environment(&cgroup_pid, &fp_child_input, &fp_child_output, cgroups_rename_script, cg->id, cg->intermediate_id); if (!fp_child_output) { - error("CGROUP: cannot popen(%s \"%s\", \"r\").", cgroups_rename_script, cg->intermediate_id); + collector_error("CGROUP: cannot popen(%s \"%s\", \"r\").", cgroups_rename_script, cg->intermediate_id); cg->pending_renames = 0; cg->processed = 1; return; @@ -2034,14 +2034,14 @@ static inline void discovery_find_cgroup_in_dir_callback(const char *dir) { } if (cgroup_root_count >= cgroup_root_max) { - info("CGROUP: maximum number of cgroups reached (%d). Not adding cgroup '%s'", cgroup_root_count, dir); + collector_info("CGROUP: maximum number of cgroups reached (%d). Not adding cgroup '%s'", cgroup_root_count, dir); return; } if (cgroup_max_depth > 0) { int depth = calc_cgroup_depth(dir); if (depth > cgroup_max_depth) { - info("CGROUP: '%s' is too deep (%d, while max is %d)", dir, depth, cgroup_max_depth); + collector_info("CGROUP: '%s' is too deep (%d, while max is %d)", dir, depth, cgroup_max_depth); return; } } @@ -2066,7 +2066,7 @@ static inline int discovery_find_dir_in_subdirs(const char *base, const char *th DIR *dir = opendir(this); if(!dir) { - error("CGROUP: cannot read directory '%s'", base); + collector_error("CGROUP: cannot read directory '%s'", base); return ret; } ret = 1; @@ -2550,7 +2550,7 @@ static inline void discovery_find_all_cgroups_v1() { if (cgroup_enable_cpuacct_stat || cgroup_enable_cpuacct_usage) { if (discovery_find_dir_in_subdirs(cgroup_cpuacct_base, NULL, discovery_find_cgroup_in_dir_callback) == -1) { cgroup_enable_cpuacct_stat = cgroup_enable_cpuacct_usage = CONFIG_BOOLEAN_NO; - error("CGROUP: disabled cpu statistics."); + collector_error("CGROUP: disabled cpu statistics."); } } @@ -2560,7 +2560,7 @@ static inline void discovery_find_all_cgroups_v1() { cgroup_enable_blkio_io = cgroup_enable_blkio_ops = cgroup_enable_blkio_throttle_io = cgroup_enable_blkio_throttle_ops = cgroup_enable_blkio_merged_ops = cgroup_enable_blkio_queued_ops = CONFIG_BOOLEAN_NO; - error("CGROUP: disabled blkio statistics."); + collector_error("CGROUP: disabled blkio statistics."); } } @@ -2568,14 +2568,14 @@ static inline void discovery_find_all_cgroups_v1() { if (discovery_find_dir_in_subdirs(cgroup_memory_base, NULL, discovery_find_cgroup_in_dir_callback) == -1) { cgroup_enable_memory = cgroup_enable_detailed_memory = cgroup_enable_swap = cgroup_enable_memory_failcnt = CONFIG_BOOLEAN_NO; - error("CGROUP: disabled memory statistics."); + collector_error("CGROUP: disabled memory statistics."); } } if (cgroup_search_in_devices) { if (discovery_find_dir_in_subdirs(cgroup_devices_base, NULL, discovery_find_cgroup_in_dir_callback) == -1) { cgroup_search_in_devices = 0; - error("CGROUP: disabled devices statistics."); + collector_error("CGROUP: disabled devices statistics."); } } } @@ -2583,7 +2583,7 @@ static inline void discovery_find_all_cgroups_v1() { static inline void discovery_find_all_cgroups_v2() { if (discovery_find_dir_in_subdirs(cgroup_unified_base, NULL, discovery_find_cgroup_in_dir_callback) == -1) { cgroup_unified_exist = CONFIG_BOOLEAN_NO; - error("CGROUP: disabled unified cgroups statistics."); + collector_error("CGROUP: disabled unified cgroups statistics."); } } @@ -2651,7 +2651,7 @@ static int discovery_is_cgroup_duplicate(struct cgroup *cg) { struct cgroup *c; for (c = discovered_cgroup_root; c; c = c->discovered_next) { if (c != cg && c->enabled && c->hash_chart == cg->hash_chart && !strcmp(c->chart_id, cg->chart_id)) { - error("CGROUP: chart id '%s' already exists with id '%s' and is enabled and available. Disabling cgroup with id '%s'.", cg->chart_id, c->id, cg->id); + collector_error("CGROUP: chart id '%s' already exists with id '%s' and is enabled and available. Disabling cgroup with id '%s'.", cg->chart_id, c->id, cg->id); return 1; } } @@ -2686,7 +2686,7 @@ static inline void discovery_process_cgroup(struct cgroup *cg) { cg->processed = 1; if ((strlen(cg->chart_id) + strlen(cgroup_chart_id_prefix)) >= RRD_ID_LENGTH_MAX) { - info("cgroup '%s' (chart id '%s') disabled because chart_id exceeds the limit (RRD_ID_LENGTH_MAX)", cg->id, cg->chart_id); + collector_info("cgroup '%s' (chart id '%s') disabled because chart_id exceeds the limit (RRD_ID_LENGTH_MAX)", cg->id, cg->chart_id); return; } @@ -2754,10 +2754,20 @@ static inline void discovery_find_all_cgroups() { debug(D_CGROUP, "done searching for cgroups"); } +static void cgroup_discovery_cleanup(void *ptr) { + UNUSED(ptr); + + discovery_thread.exited = 1; + worker_unregister(); + service_exits(); +} + void cgroup_discovery_worker(void *ptr) { UNUSED(ptr); + netdata_thread_cleanup_push(cgroup_discovery_cleanup, ptr); + worker_register("CGROUPSDISC"); worker_register_job_name(WORKER_DISCOVERY_INIT, "init"); worker_register_job_name(WORKER_DISCOVERY_FIND, "find"); @@ -2777,24 +2787,23 @@ void cgroup_discovery_worker(void *ptr) NULL, SIMPLE_PATTERN_EXACT); - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { worker_is_idle(); uv_mutex_lock(&discovery_thread.mutex); - while (!discovery_thread.start_discovery) + while (!discovery_thread.start_discovery && service_running(SERVICE_COLLECTORS)) uv_cond_wait(&discovery_thread.cond_var, &discovery_thread.mutex); discovery_thread.start_discovery = 0; uv_mutex_unlock(&discovery_thread.mutex); - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; discovery_find_all_cgroups(); } - discovery_thread.exited = 1; - worker_unregister(); -} + netdata_thread_cleanup_pop(1); +} // ---------------------------------------------------------------------------- // generate charts @@ -3507,52 +3516,15 @@ static inline char *cgroup_chart_type(char *buffer, const char *id, size_t len) return buffer; } -static inline unsigned long long cpuset_str2ull(char **s) { - unsigned long long n = 0; - char c; - for(c = **s; c >= '0' && c <= '9' ; c = *(++*s)) { - n *= 10; - n += c - '0'; - } - return n; -} - static inline void update_cpu_limits(char **filename, unsigned long long *value, struct cgroup *cg) { if(*filename) { int ret = -1; if(value == &cg->cpuset_cpus) { - static char *buf = NULL; - static size_t buf_size = 0; - - if(!buf) { - buf_size = 100U + 6 * get_system_cpus(); // taken from kernel/cgroup/cpuset.c - buf = mallocz(buf_size + 1); - } - - ret = read_file(*filename, buf, buf_size); - - if(!ret) { - char *s = buf; - unsigned long long ncpus = 0; - - // parse the cpuset string and calculate the number of cpus the cgroup is allowed to use - while(*s) { - unsigned long long n = cpuset_str2ull(&s); - ncpus++; - if(*s == ',') { - s++; - continue; - } - if(*s == '-') { - s++; - unsigned long long m = cpuset_str2ull(&s); - ncpus += m - n; // calculate the number of cpus in the region - } - s++; - } - - if(likely(ncpus)) *value = ncpus; + unsigned long ncpus = read_cpuset_cpus(*filename, get_system_cpus()); + if(ncpus) { + *value = ncpus; + ret = 0; } } else if(value == &cg->cpu_cfs_period) { @@ -3564,7 +3536,7 @@ static inline void update_cpu_limits(char **filename, unsigned long long *value, else ret = -1; if(ret) { - error("Cannot refresh cgroup %s cpu limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); + collector_error("Cannot refresh cgroup %s cpu limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); freez(*filename); *filename = NULL; } @@ -3588,7 +3560,7 @@ static inline void update_cpu_limits2(struct cgroup *cg) { unsigned long lines = procfile_lines(ff); if (unlikely(lines < 1)) { - error("CGROUP: file '%s' should have 1 lines.", cg->filename_cpu_cfs_quota); + collector_error("CGROUP: file '%s' should have 1 lines.", cg->filename_cpu_cfs_quota); return; } @@ -3605,7 +3577,7 @@ static inline void update_cpu_limits2(struct cgroup *cg) { return; cpu_limits2_err: - error("Cannot refresh cgroup %s cpu limit by reading '%s'. Will not update its limit anymore.", cg->id, cg->filename_cpu_cfs_quota); + collector_error("Cannot refresh cgroup %s cpu limit by reading '%s'. Will not update its limit anymore.", cg->id, cg->filename_cpu_cfs_quota); freez(cg->filename_cpu_cfs_quota); cg->filename_cpu_cfs_quota = NULL; @@ -3617,7 +3589,7 @@ static inline int update_memory_limits(char **filename, const RRDSETVAR_ACQUIRED if(unlikely(!*chart_var)) { *chart_var = rrdsetvar_custom_chart_variable_add_and_acquire(cg->st_mem_usage, chart_var_name); if(!*chart_var) { - error("Cannot create cgroup %s chart variable '%s'. Will not update its limit anymore.", cg->id, chart_var_name); + collector_error("Cannot create cgroup %s chart variable '%s'. Will not update its limit anymore.", cg->id, chart_var_name); freez(*filename); *filename = NULL; } @@ -3626,7 +3598,7 @@ static inline int update_memory_limits(char **filename, const RRDSETVAR_ACQUIRED if(*filename && *chart_var) { if(!(cg->options & CGROUP_OPTIONS_IS_UNIFIED)) { if(read_single_number_file(*filename, value)) { - error("Cannot refresh cgroup %s memory limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); + collector_error("Cannot refresh cgroup %s memory limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); freez(*filename); *filename = NULL; } @@ -3638,7 +3610,7 @@ static inline int update_memory_limits(char **filename, const RRDSETVAR_ACQUIRED char buffer[30 + 1]; int ret = read_file(*filename, buffer, 30); if(ret) { - error("Cannot refresh cgroup %s memory limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); + collector_error("Cannot refresh cgroup %s memory limit by reading '%s'. Will not update its limit anymore.", cg->id, *filename); freez(*filename); *filename = NULL; return 0; @@ -3747,7 +3719,7 @@ void update_cgroup_charts(int update_every) { if(unlikely(!cg->chart_var_cpu_limit)) { cg->chart_var_cpu_limit = rrdsetvar_custom_chart_variable_add_and_acquire(cg->st_cpu, "cpu_limit"); if(!cg->chart_var_cpu_limit) { - error("Cannot create cgroup %s chart variable 'cpu_limit'. Will not update its limit anymore.", cg->id); + collector_error("Cannot create cgroup %s chart variable 'cpu_limit'. Will not update its limit anymore.", cg->id); if(cg->filename_cpuset_cpus) freez(cg->filename_cpuset_cpus); cg->filename_cpuset_cpus = NULL; if(cg->filename_cpu_cfs_period) freez(cg->filename_cpu_cfs_period); @@ -4139,7 +4111,7 @@ void update_cgroup_charts(int update_every) { if(likely(ff && procfile_lines(ff) && !strncmp(procfile_word(ff, 0), "MemTotal", 8))) ram_total = str2ull(procfile_word(ff, 1)) * 1024; else { - error("Cannot read file %s. Will not update cgroup %s RAM limit anymore.", filename, cg->id); + collector_error("Cannot read file %s. Will not update cgroup %s RAM limit anymore.", filename, cg->id); freez(cg->filename_memory_limit); cg->filename_memory_limit = NULL; } @@ -4773,19 +4745,19 @@ static void cgroup_main_cleanup(void *ptr) { struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); usec_t max = 2 * USEC_PER_SEC, step = 50000; if (!discovery_thread.exited) { - info("stopping discovery thread worker"); + collector_info("stopping discovery thread worker"); uv_mutex_lock(&discovery_thread.mutex); discovery_thread.start_discovery = 1; uv_cond_signal(&discovery_thread.cond_var); uv_mutex_unlock(&discovery_thread.mutex); } - info("waiting for discovery thread to finish..."); + collector_info("waiting for discovery thread to finish..."); while (!discovery_thread.exited && max > 0) { max -= step; @@ -4824,7 +4796,7 @@ void *cgroups_main(void *ptr) { netdata_cgroup_ebpf_initialize_shm(); if (uv_mutex_init(&cgroup_root_mutex)) { - error("CGROUP: cannot initialize mutex for the main cgroup list"); + collector_error("CGROUP: cannot initialize mutex for the main cgroup list"); goto exit; } @@ -4833,17 +4805,17 @@ void *cgroups_main(void *ptr) { discovery_thread.exited = 0; if (uv_mutex_init(&discovery_thread.mutex)) { - error("CGROUP: cannot initialize mutex for discovery thread"); + collector_error("CGROUP: cannot initialize mutex for discovery thread"); goto exit; } if (uv_cond_init(&discovery_thread.cond_var)) { - error("CGROUP: cannot initialize conditional variable for discovery thread"); + collector_error("CGROUP: cannot initialize conditional variable for discovery thread"); goto exit; } int error = uv_thread_create(&discovery_thread.thread, cgroup_discovery_worker, NULL); if (error) { - error("CGROUP: cannot create thread worker. uv_thread_create(): %s", uv_strerror(error)); + collector_error("CGROUP: cannot create thread worker. uv_thread_create(): %s", uv_strerror(error)); goto exit; } uv_thread_set_name_np(discovery_thread.thread, "PLUGIN[cgroups]"); @@ -4853,11 +4825,11 @@ void *cgroups_main(void *ptr) { usec_t step = cgroup_update_every * USEC_PER_SEC; usec_t find_every = cgroup_check_for_new_every * USEC_PER_SEC, find_dt = 0; - while(!netdata_exit) { + while(service_running(SERVICE_COLLECTORS)) { worker_is_idle(); usec_t hb_dt = heartbeat_next(&hb, step); - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; find_dt += hb_dt; if (unlikely(find_dt >= find_every || (!is_inside_k8s && cgroups_check))) { @@ -4872,9 +4844,11 @@ void *cgroups_main(void *ptr) { worker_is_busy(WORKER_CGROUPS_READ); read_all_discovered_cgroups(cgroup_root); + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; worker_is_busy(WORKER_CGROUPS_CHART); update_cgroup_charts(cgroup_update_every); + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; worker_is_idle(); uv_mutex_unlock(&cgroup_root_mutex); diff --git a/collectors/charts.d.plugin/README.md b/collectors/charts.d.plugin/README.md index 06f4af1ec..092a3f027 100644 --- a/collectors/charts.d.plugin/README.md +++ b/collectors/charts.d.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "charts.d.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/README.md" +sidebar_label: "charts.d.plugin" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "Developers/Collectors" --> # charts.d.plugin @@ -60,11 +64,11 @@ For a module called `X`, the following criteria must be met: the collector cannot be used). - `X_create()` - creates the Netdata charts, following the standard Netdata plugin guides as described in - **[External Plugins](/collectors/plugins.d/README.md)** (commands `CHART` and `DIMENSION`). + **[External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md)** (commands `CHART` and `DIMENSION`). The return value does matter: 0 = OK, 1 = FAILED. - `X_update()` - collects the values for the defined charts, following the standard Netdata plugin guides - as described in **[External Plugins](/collectors/plugins.d/README.md)** (commands `BEGIN`, `SET`, `END`). + as described in **[External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md)** (commands `BEGIN`, `SET`, `END`). The return value also matters: 0 = OK, 1 = FAILED. 5. The following global variables are available to be set: @@ -72,7 +76,7 @@ For a module called `X`, the following criteria must be met: The module script may use more functions or variables. But all of them must begin with `X_`. -The standard Netdata plugin variables are also available (check **[External Plugins](/collectors/plugins.d/README.md)**). +The standard Netdata plugin variables are also available (check **[External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md)**). ### X_check() @@ -86,7 +90,7 @@ connect to a local mysql database to find out if it can read the values it needs ### X_create() The purpose of the BASH function `X_create()` is to create the charts and dimensions using the standard Netdata -plugin guides (**[External Plugins](/collectors/plugins.d/README.md)**). +plugin guides (**[External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md)**). `X_create()` will be called just once and only after `X_check()` was successful. You can however call it yourself when there is need for it (for example to add a new dimension to an existing chart). @@ -96,7 +100,7 @@ A non-zero return value will disable the collector. ### X_update() `X_update()` will be called repeatedly every `X_update_every` seconds, to collect new values and send them to Netdata, -following the Netdata plugin guides (**[External Plugins](/collectors/plugins.d/README.md)**). +following the Netdata plugin guides (**[External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md)**). The function will be called with one parameter: microseconds since the last time it was run. This value should be appended to the `BEGIN` statement of every chart updated by the collector script. diff --git a/collectors/charts.d.plugin/ap/README.md b/collectors/charts.d.plugin/ap/README.md index a7953a541..03ab6d13e 100644 --- a/collectors/charts.d.plugin/ap/README.md +++ b/collectors/charts.d.plugin/ap/README.md @@ -1,7 +1,10 @@ <!-- title: "Access point monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/ap/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/ap/README.md" sidebar_label: "Access points" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # Access point monitoring with Netdata @@ -83,7 +86,7 @@ Station 40:b8:37:5a:ed:5e (on wlan0) ## Configuration Edit the `charts.d/ap.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/charts.d.plugin/apcupsd/README.md b/collectors/charts.d.plugin/apcupsd/README.md index f1aebf97c..602977be1 100644 --- a/collectors/charts.d.plugin/apcupsd/README.md +++ b/collectors/charts.d.plugin/apcupsd/README.md @@ -1,7 +1,10 @@ <!-- title: "APC UPS monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/apcupsd/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/apcupsd/README.md" sidebar_label: "APC UPS" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # APC UPS monitoring with Netdata @@ -11,7 +14,7 @@ Monitors different APC UPS models and retrieves status information using `apcacc ## Configuration Edit the `charts.d/apcupsd.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/charts.d.plugin/example/README.md b/collectors/charts.d.plugin/example/README.md index 77446b270..d5faaabf4 100644 --- a/collectors/charts.d.plugin/example/README.md +++ b/collectors/charts.d.plugin/example/README.md @@ -1,6 +1,10 @@ <!-- title: "Example" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/example/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/example/README.md" +sidebar_label: "example-charts.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Mock Collectors" --> # Example diff --git a/collectors/charts.d.plugin/libreswan/README.md b/collectors/charts.d.plugin/libreswan/README.md index 41c4e24c9..7c4eabcf9 100644 --- a/collectors/charts.d.plugin/libreswan/README.md +++ b/collectors/charts.d.plugin/libreswan/README.md @@ -1,7 +1,10 @@ <!-- title: "Libreswan IPSec tunnel monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/libreswan/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/libreswan/README.md" sidebar_label: "Libreswan IPSec tunnels" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # Libreswan IPSec tunnel monitoring with Netdata @@ -22,7 +25,7 @@ The following charts are created, **per tunnel**: ## Configuration Edit the `charts.d/libreswan.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/charts.d.plugin/nut/README.md b/collectors/charts.d.plugin/nut/README.md index 69d7622cd..7bb8a5507 100644 --- a/collectors/charts.d.plugin/nut/README.md +++ b/collectors/charts.d.plugin/nut/README.md @@ -1,7 +1,10 @@ <!-- title: "UPS/PDU monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/nut/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/nut/README.md" sidebar_label: "UPS/PDU" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # UPS/PDU monitoring with Netdata @@ -51,7 +54,7 @@ The following charts will be created: ## Configuration Edit the `charts.d/nut.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/charts.d.plugin/nut/nut.chart.sh b/collectors/charts.d.plugin/nut/nut.chart.sh index 2f7e3f336..7c32b6dde 100644 --- a/collectors/charts.d.plugin/nut/nut.chart.sh +++ b/collectors/charts.d.plugin/nut/nut.chart.sh @@ -81,43 +81,46 @@ nut_create() { for x in "${nut_ids[@]}"; do cat << EOF -CHART nut_$x.charge '' "UPS Charge" "percentage" ups nut.charge area $((nut_priority + 1)) $nut_update_every +CHART nut_$x.charge '' "UPS Charge" "percentage" ups nut.charge area $((nut_priority + 2)) $nut_update_every DIMENSION battery_charge charge absolute 1 100 -CHART nut_$x.runtime '' "UPS Runtime" "seconds" ups nut.runtime area $((nut_priority + 2)) $nut_update_every +CHART nut_$x.runtime '' "UPS Runtime" "seconds" ups nut.runtime area $((nut_priority + 3)) $nut_update_every DIMENSION battery_runtime runtime absolute 1 100 -CHART nut_$x.battery_voltage '' "UPS Battery Voltage" "Volts" ups nut.battery.voltage line $((nut_priority + 3)) $nut_update_every +CHART nut_$x.battery_voltage '' "UPS Battery Voltage" "Volts" ups nut.battery.voltage line $((nut_priority + 4)) $nut_update_every DIMENSION battery_voltage voltage absolute 1 100 DIMENSION battery_voltage_high high absolute 1 100 DIMENSION battery_voltage_low low absolute 1 100 DIMENSION battery_voltage_nominal nominal absolute 1 100 -CHART nut_$x.input_voltage '' "UPS Input Voltage" "Volts" input nut.input.voltage line $((nut_priority + 4)) $nut_update_every +CHART nut_$x.input_voltage '' "UPS Input Voltage" "Volts" input nut.input.voltage line $((nut_priority + 5)) $nut_update_every DIMENSION input_voltage voltage absolute 1 100 DIMENSION input_voltage_fault fault absolute 1 100 DIMENSION input_voltage_nominal nominal absolute 1 100 -CHART nut_$x.input_current '' "UPS Input Current" "Ampere" input nut.input.current line $((nut_priority + 5)) $nut_update_every +CHART nut_$x.input_current '' "UPS Input Current" "Ampere" input nut.input.current line $((nut_priority + 6)) $nut_update_every DIMENSION input_current_nominal nominal absolute 1 100 -CHART nut_$x.input_frequency '' "UPS Input Frequency" "Hz" input nut.input.frequency line $((nut_priority + 6)) $nut_update_every +CHART nut_$x.input_frequency '' "UPS Input Frequency" "Hz" input nut.input.frequency line $((nut_priority + 7)) $nut_update_every DIMENSION input_frequency frequency absolute 1 100 DIMENSION input_frequency_nominal nominal absolute 1 100 -CHART nut_$x.output_voltage '' "UPS Output Voltage" "Volts" output nut.output.voltage line $((nut_priority + 7)) $nut_update_every +CHART nut_$x.output_voltage '' "UPS Output Voltage" "Volts" output nut.output.voltage line $((nut_priority + 8)) $nut_update_every DIMENSION output_voltage voltage absolute 1 100 CHART nut_$x.load '' "UPS Load" "percentage" ups nut.load area $((nut_priority)) $nut_update_every DIMENSION load load absolute 1 100 -CHART nut_$x.temp '' "UPS Temperature" "temperature" ups nut.temperature line $((nut_priority + 8)) $nut_update_every +CHART nut_$x.load_usage '' "UPS Load Usage" "Watts" ups nut.load_usage area $((nut_priority + 1)) $nut_update_every +DIMENSION load_usage load_usage absolute 1 100 + +CHART nut_$x.temp '' "UPS Temperature" "temperature" ups nut.temperature line $((nut_priority + 9)) $nut_update_every DIMENSION temp temp absolute 1 100 EOF if [ "${nut_clients_chart}" = "1" ]; then cat << EOF2 -CHART nut_$x.clients '' "UPS Connected Clients" "clients" ups nut.clients area $((nut_priority + 9)) $nut_update_every +CHART nut_$x.clients '' "UPS Connected Clients" "clients" ups nut.clients area $((nut_priority + 10)) $nut_update_every DIMENSION clients '' absolute 1 1 EOF2 fi @@ -154,6 +157,8 @@ BEGIN { input_frequency_nominal = 0; output_voltage = 0; load = 0; + load_usage = 0; + nompower = 0; temp = 0; client = 0; do_clients = ${nut_clients_chart}; @@ -172,16 +177,19 @@ BEGIN { /^input.frequency.nominal: .*/ { input_frequency_nominal = \$2 * 100 }; /^output.voltage: .*/ { output_voltage = \$2 * 100 }; /^ups.load: .*/ { load = \$2 * 100 }; +/^ups.realpower.nominal: .*/ { nompower = \$2 }; /^ups.temperature: .*/ { temp = \$2 * 100 }; /^ups.connected_clients: .*/ { clients = \$2 }; END { + { load_usage = nompower * load / 100 }; + print \"BEGIN nut_$x.charge $1\"; print \"SET battery_charge = \" battery_charge; print \"END\" - print \"BEGIN nut_$x.runtime $1\"; - print \"SET battery_runtime = \" battery_runtime; - print \"END\" + print \"BEGIN nut_$x.runtime $1\"; + print \"SET battery_runtime = \" battery_runtime; + print \"END\" print \"BEGIN nut_$x.battery_voltage $1\"; print \"SET battery_voltage = \" battery_voltage; @@ -213,6 +221,10 @@ END { print \"SET load = \" load; print \"END\" + print \"BEGIN nut_$x.load_usage $1\"; + print \"SET load_usage = \" load_usage; + print \"END\" + print \"BEGIN nut_$x.temp $1\"; print \"SET temp = \" temp; print \"END\" diff --git a/collectors/charts.d.plugin/opensips/README.md b/collectors/charts.d.plugin/opensips/README.md index b08d19232..74624c7f1 100644 --- a/collectors/charts.d.plugin/opensips/README.md +++ b/collectors/charts.d.plugin/opensips/README.md @@ -1,7 +1,10 @@ <!-- title: "OpenSIPS monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/opensips/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/opensips/README.md" sidebar_label: "OpenSIPS" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # OpenSIPS monitoring with Netdata @@ -9,7 +12,7 @@ sidebar_label: "OpenSIPS" ## Configuration Edit the `charts.d/opensips.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/charts.d.plugin/sensors/README.md b/collectors/charts.d.plugin/sensors/README.md index 1b98b1a70..142ae14aa 100644 --- a/collectors/charts.d.plugin/sensors/README.md +++ b/collectors/charts.d.plugin/sensors/README.md @@ -1,6 +1,10 @@ <!-- title: "Linux machine sensors monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/sensors/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/charts.d.plugin/sensors/README.md" +sidebar_label: "lm-sensors" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # Linux machine sensors monitoring with Netdata @@ -27,7 +31,7 @@ One chart for every sensor chip found and each of the above will be created. ## Enable the collector The `sensors` collector is disabled by default. To enable it, edit the `charts.d.conf` file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -44,7 +48,7 @@ sensors=force ## Configuration Edit the `charts.d/sensors.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/checks.plugin/README.md b/collectors/checks.plugin/README.md new file mode 100644 index 000000000..801f27752 --- /dev/null +++ b/collectors/checks.plugin/README.md @@ -0,0 +1,12 @@ +<!-- +title: "checks.plugin" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/checks.plugin/README.md" +sidebar_label: "checks.plugin" +learn_status: "Unpublished" +--> + +# checks.plugin + +A debugging plugin (by default it is disabled) + + diff --git a/collectors/cups.plugin/README.md b/collectors/cups.plugin/README.md index f3b2a28d1..0658cc8b3 100644 --- a/collectors/cups.plugin/README.md +++ b/collectors/cups.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "cups.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/cups.plugin/README.md +title: "Printers (cups.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/cups.plugin/README.md" +sidebar_label: "cups.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # cups.plugin @@ -11,7 +15,7 @@ custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/cups. This plugin needs a running local CUPS daemon (`cupsd`). This plugin does not need any configuration. Supports cups since version 1.7. -If you installed Netdata using our native packages, you will have to additionaly install `netdata-plugin-cups` to use this plugin for data collection. It is not installed by default due to the large number of dependencies it requires. +If you installed Netdata using our native packages, you will have to additionally install `netdata-plugin-cups` to use this plugin for data collection. It is not installed by default due to the large number of dependencies it requires. ## Charts diff --git a/collectors/cups.plugin/cups_plugin.c b/collectors/cups.plugin/cups_plugin.c index 9a200c31d..b9d91c851 100644 --- a/collectors/cups.plugin/cups_plugin.c +++ b/collectors/cups.plugin/cups_plugin.c @@ -222,6 +222,7 @@ void reset_metrics() { } int main(int argc, char **argv) { + stderror = stderr; clocks_init(); // ------------------------------------------------------------------------ diff --git a/collectors/diskspace.plugin/README.md b/collectors/diskspace.plugin/README.md index c037a0b16..6d1ec7ca2 100644 --- a/collectors/diskspace.plugin/README.md +++ b/collectors/diskspace.plugin/README.md @@ -1,7 +1,11 @@ <!-- -title: "diskspace.plugin" +title: "Monitor disk (diskspace.plugin)" description: "Monitor the disk usage space of mounted disks in real-time with the Netdata Agent, plus preconfigured alarms for disks at risk of filling up." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/diskspace.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/diskspace.plugin/README.md" +sidebar_label: "Disks" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # diskspace.plugin @@ -38,6 +42,6 @@ Charts can be enabled/disabled for every mount separately: # inodes usage = auto ``` -> for disks performance monitoring, see the `proc` plugin, [here](/collectors/proc.plugin/README.md#monitoring-disks) +> for disks performance monitoring, see the `proc` plugin, [here](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md#monitoring-disks) diff --git a/collectors/diskspace.plugin/plugin_diskspace.c b/collectors/diskspace.plugin/plugin_diskspace.c index e806a3360..743612ffb 100644 --- a/collectors/diskspace.plugin/plugin_diskspace.c +++ b/collectors/diskspace.plugin/plugin_diskspace.c @@ -3,7 +3,6 @@ #include "../proc.plugin/plugin_proc.h" #define PLUGIN_DISKSPACE_NAME "diskspace.plugin" -#define THREAD_DISKSPACE_SLOW_NAME "PLUGIN[diskspace slow]" #define DEFAULT_EXCLUDED_PATHS "/proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/*" #define DEFAULT_EXCLUDED_FILESYSTEMS "*gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs" @@ -182,7 +181,7 @@ static void calculate_values_and_show_charts( #ifdef NETDATA_INTERNAL_CHECKS if(unlikely(btotal != bavail + breserved_root + bused)) - error("DISKSPACE: disk block statistics for '%s' (disk '%s') do not sum up: total = %llu, available = %llu, reserved = %llu, used = %llu", mi->mount_point, disk, (unsigned long long)btotal, (unsigned long long)bavail, (unsigned long long)breserved_root, (unsigned long long)bused); + collector_error("DISKSPACE: disk block statistics for '%s' (disk '%s') do not sum up: total = %llu, available = %llu, reserved = %llu, used = %llu", mi->mount_point, disk, (unsigned long long)btotal, (unsigned long long)bavail, (unsigned long long)breserved_root, (unsigned long long)bused); #endif // -------------------------------------------------------------------------- @@ -201,7 +200,7 @@ static void calculate_values_and_show_charts( #ifdef NETDATA_INTERNAL_CHECKS if(unlikely(btotal != bavail + breserved_root + bused)) - error("DISKSPACE: disk inode statistics for '%s' (disk '%s') do not sum up: total = %llu, available = %llu, reserved = %llu, used = %llu", mi->mount_point, disk, (unsigned long long)ftotal, (unsigned long long)favail, (unsigned long long)freserved_root, (unsigned long long)fused); + collector_error("DISKSPACE: disk inode statistics for '%s' (disk '%s') do not sum up: total = %llu, available = %llu, reserved = %llu, used = %llu", mi->mount_point, disk, (unsigned long long)ftotal, (unsigned long long)favail, (unsigned long long)freserved_root, (unsigned long long)fused); #endif int rendered = 0; @@ -320,7 +319,7 @@ static inline void do_disk_space_stats(struct mountinfo *mi, int update_every) { , SIMPLE_PATTERN_EXACT ); - dict_mountpoints = dictionary_create(DICT_OPTION_NONE); + dict_mountpoints = dictionary_create_advanced(DICT_OPTION_NONE, &dictionary_stats_category_collectors, 0); } struct mount_point_metadata *m = dictionary_get(dict_mountpoints, mi->mount_point); @@ -349,23 +348,23 @@ static inline void do_disk_space_stats(struct mountinfo *mi, int update_every) { struct stat bs; if(stat(mi->mount_point, &bs) == -1) { - error("DISKSPACE: Cannot stat() mount point '%s' (disk '%s', filesystem '%s', root '%s')." - , mi->mount_point - , disk - , mi->filesystem?mi->filesystem:"" - , mi->root?mi->root:"" - ); + collector_error("DISKSPACE: Cannot stat() mount point '%s' (disk '%s', filesystem '%s', root '%s')." + , mi->mount_point + , disk + , mi->filesystem?mi->filesystem:"" + , mi->root?mi->root:"" + ); def_space = CONFIG_BOOLEAN_NO; def_inodes = CONFIG_BOOLEAN_NO; } else { if((bs.st_mode & S_IFMT) != S_IFDIR) { - error("DISKSPACE: Mount point '%s' (disk '%s', filesystem '%s', root '%s') is not a directory." - , mi->mount_point - , disk - , mi->filesystem?mi->filesystem:"" - , mi->root?mi->root:"" - ); + collector_error("DISKSPACE: Mount point '%s' (disk '%s', filesystem '%s', root '%s') is not a directory." + , mi->mount_point + , disk + , mi->filesystem?mi->filesystem:"" + , mi->root?mi->root:"" + ); def_space = CONFIG_BOOLEAN_NO; def_inodes = CONFIG_BOOLEAN_NO; } @@ -431,12 +430,12 @@ static inline void do_disk_space_stats(struct mountinfo *mi, int update_every) { if (statvfs(mi->mount_point, &buff_statvfs) < 0) { if(!m->shown_error) { - error("DISKSPACE: failed to statvfs() mount point '%s' (disk '%s', filesystem '%s', root '%s')" - , mi->mount_point - , disk - , mi->filesystem?mi->filesystem:"" - , mi->root?mi->root:"" - ); + collector_error("DISKSPACE: failed to statvfs() mount point '%s' (disk '%s', filesystem '%s', root '%s')" + , mi->mount_point + , disk + , mi->filesystem?mi->filesystem:"" + , mi->root?mi->root:"" + ); m->shown_error = 1; } return; @@ -464,12 +463,12 @@ static inline void do_slow_disk_space_stats(struct basic_mountinfo *mi, int upda struct statvfs buff_statvfs; if (statvfs(mi->mount_point, &buff_statvfs) < 0) { if(!m->shown_error) { - error("DISKSPACE: failed to statvfs() mount point '%s' (disk '%s', filesystem '%s', root '%s')" - , mi->mount_point - , mi->persistent_id - , mi->filesystem?mi->filesystem:"" - , mi->root?mi->root:"" - ); + collector_error("DISKSPACE: failed to statvfs() mount point '%s' (disk '%s', filesystem '%s', root '%s')" + , mi->mount_point + , mi->persistent_id + , mi->filesystem?mi->filesystem:"" + , mi->root?mi->root:"" + ); m->shown_error = 1; } return; @@ -483,7 +482,7 @@ static void diskspace_slow_worker_cleanup(void *ptr) { UNUSED(ptr); - info("cleaning up..."); + collector_info("cleaning up..."); worker_unregister(); } @@ -515,7 +514,7 @@ void *diskspace_slow_worker(void *ptr) heartbeat_t hb; heartbeat_init(&hb); - while(!netdata_exit) { + while(service_running(SERVICE_COLLECTORS)) { worker_is_idle(); heartbeat_next(&hb, USEC_PER_SEC); @@ -530,7 +529,7 @@ void *diskspace_slow_worker(void *ptr) if (!dict_mountpoints) continue; - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; // -------------------------------------------------------------------------- // disk space metrics @@ -547,10 +546,10 @@ void *diskspace_slow_worker(void *ptr) for(bmi = slow_mountinfo_root; bmi; bmi = bmi->next) { do_slow_disk_space_stats(bmi, slow_update_every); - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; } - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; worker_is_busy(WORKER_JOB_SLOW_CLEANUP); @@ -584,7 +583,7 @@ static void diskspace_main_cleanup(void *ptr) { struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); if (diskspace_slow_thread) { netdata_thread_join(*diskspace_slow_thread, NULL); @@ -632,7 +631,7 @@ void *diskspace_main(void *ptr) { netdata_thread_create( diskspace_slow_thread, - THREAD_DISKSPACE_SLOW_NAME, + "P[diskspace slow]", NETDATA_THREAD_OPTION_JOINABLE, diskspace_slow_worker, &slow_worker_data); @@ -640,11 +639,11 @@ void *diskspace_main(void *ptr) { usec_t step = update_every * USEC_PER_SEC; heartbeat_t hb; heartbeat_init(&hb); - while(!netdata_exit) { + while(service_running(SERVICE_COLLECTORS)) { worker_is_idle(); /* usec_t hb_dt = */ heartbeat_next(&hb, step); - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; // -------------------------------------------------------------------------- // this is smart enough not to reload it every time @@ -671,11 +670,11 @@ void *diskspace_main(void *ptr) { worker_is_busy(WORKER_JOB_MOUNTPOINT); do_disk_space_stats(mi, update_every); - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; } netdata_mutex_unlock(&slow_mountinfo_mutex); - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; if(dict_mountpoints) { worker_is_busy(WORKER_JOB_CLEANUP); diff --git a/collectors/ebpf.plugin/README.md b/collectors/ebpf.plugin/README.md index 7762ed34f..deedf4d79 100644 --- a/collectors/ebpf.plugin/README.md +++ b/collectors/ebpf.plugin/README.md @@ -1,9 +1,11 @@ <!-- -title: "eBPF monitoring with Netdata" -description: "Use Netdata's extended Berkeley Packet Filter (eBPF) collector to monitor kernel-level metrics about your -complex applications with per-second granularity." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/ebpf.plugin/README.md -sidebar_label: "eBPF" +title: "Kernel traces/metrics (eBPF) monitoring with Netdata" +description: "Use Netdata's extended Berkeley Packet Filter (eBPF) collector to monitor kernel-level metrics about yourcomplex applications with per-second granularity." +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/ebpf.plugin/README.md" +sidebar_label: "Kernel traces/metrics (eBPF)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # eBPF monitoring with Netdata @@ -13,7 +15,7 @@ The Netdata Agent provides many [eBPF](https://ebpf.io/what-is-ebpf/) programs t > ❗ eBPF monitoring only works on Linux systems and with specific Linux kernels, including all kernels newer than `4.11.0`, and all kernels on CentOS 7.6 or later. For kernels older than `4.11.0`, improved support is in active development. This document provides comprehensive details about the `ebpf.plugin`. -For hands-on configuration and troubleshooting tips see our [tutorial on troubleshooting apps with eBPF metrics](/docs/guides/troubleshoot/monitor-debug-applications-ebpf.md). +For hands-on configuration and troubleshooting tips see our [tutorial on troubleshooting apps with eBPF metrics](https://github.com/netdata/netdata/blob/master/docs/guides/troubleshoot/monitor-debug-applications-ebpf.md). <figure> <img src="https://user-images.githubusercontent.com/1153921/74746434-ad6a1e00-5222-11ea-858a-a7882617ae02.png" alt="An example of VFS charts, made possible by the eBPF collector plugin" /> @@ -42,12 +44,12 @@ If your Agent is v1.22 or older, you may to enable the collector yourself. To enable or disable the entire eBPF collector: -1. Navigate to the [Netdata config directory](/docs/configure/nodes.md#the-netdata-config-directory). +1. Navigate to the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). ```bash cd /etc/netdata ``` -2. Use the [`edit-config`](/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit `netdata.conf`. +2. Use the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit `netdata.conf`. ```bash ./edit-config netdata.conf @@ -67,11 +69,11 @@ You can configure the eBPF collector's behavior to fine-tune which metrics you r To edit the `ebpf.d.conf`: -1. Navigate to the [Netdata config directory](/docs/configure/nodes.md#the-netdata-config-directory). +1. Navigate to the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). ```bash cd /etc/netdata ``` -2. Use the [`edit-config`](/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit [`ebpf.d.conf`](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/ebpf.d.conf). +2. Use the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit [`ebpf.d.conf`](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/ebpf.d.conf). ```bash ./edit-config ebpf.d.conf @@ -103,11 +105,10 @@ accepts the following values: #### Integration with `apps.plugin` The eBPF collector also creates charts for each running application through an integration with the -[`apps.plugin`](/collectors/apps.plugin/README.md). This integration helps you understand how specific applications +[`apps.plugin`](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md). This integration helps you understand how specific applications interact with the Linux kernel. -If you want to _disable_ the integration with `apps.plugin` along with the above charts, change the setting `apps` to -`no`. +If you want to enable `apps.plugin` integration, change the "apps" setting to "yes". ```conf [global] @@ -122,7 +123,7 @@ it runs. #### Integration with `cgroups.plugin` The eBPF collector also creates charts for each cgroup through an integration with the -[`cgroups.plugin`](/collectors/cgroups.plugin/README.md). This integration helps you understand how a specific cgroup +[`cgroups.plugin`](https://github.com/netdata/netdata/blob/master/collectors/cgroups.plugin/README.md). This integration helps you understand how a specific cgroup interacts with the Linux kernel. The integration with `cgroups.plugin` is disabled by default to avoid creating overhead on your system. If you want to @@ -244,7 +245,7 @@ The eBPF collector enables and runs the following eBPF programs by default: You can also enable the following eBPF programs: - `cachestat`: Netdata's eBPF data collector creates charts about the memory page cache. When the integration with - [`apps.plugin`](/collectors/apps.plugin/README.md) is enabled, this collector creates charts for the whole host _and_ + [`apps.plugin`](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md) is enabled, this collector creates charts for the whole host _and_ for each application. - `dcstat` : This eBPF program creates charts that show information about file access using directory cache. It appends `kprobes` for `lookup_fast()` and `d_lookup()` to identify if files are inside directory cache, outside and files are @@ -261,11 +262,11 @@ You can configure each thread of the eBPF data collector. This allows you to ove To configure an eBPF thread: -1. Navigate to the [Netdata config directory](/docs/configure/nodes.md#the-netdata-config-directory). +1. Navigate to the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). ```bash cd /etc/netdata ``` -2. Use the [`edit-config`](/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit a thread configuration file. The following configuration files are available: +2. Use the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) script to edit a thread configuration file. The following configuration files are available: - `network.conf`: Configuration for the [`network` thread](#network-configuration). This config file overwrites the global options and also lets you specify which network the eBPF collector monitors. @@ -304,7 +305,7 @@ You can configure the information shown on `outbound` and `inbound` charts with When you define a `ports` setting, Netdata will collect network metrics for that specific port. For example, if you write `ports = 19999`, Netdata will collect only connections for itself. The `hostnames` setting accepts -[simple patterns](/libnetdata/simple_pattern/README.md). The `ports`, and `ips` settings accept negation (`!`) to deny +[simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md). The `ports`, and `ips` settings accept negation (`!`) to deny specific values or asterisk alone to define all values. In the above example, Netdata will collect metrics for all ports between 1 and 443, with the exception of 53 (domain) @@ -881,7 +882,7 @@ significantly increases kernel memory usage by several hundred MB. If your node is experiencing high memory usage and there is no obvious culprit to be found in the `apps.mem` chart, consider testing for high kernel memory usage by [disabling eBPF monitoring](#configuring-ebpfplugin). Next, -[restart Netdata](/docs/configure/start-stop-restart.md) with `sudo systemctl restart netdata` to see if system memory +[restart Netdata](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) with `sudo systemctl restart netdata` to see if system memory usage (see the `system.ram` chart) has dropped significantly. Beginning with `v1.31`, kernel memory usage is configurable via the [`pid table size` setting](#ebpf-load-mode) diff --git a/collectors/ebpf.plugin/ebpf.c b/collectors/ebpf.plugin/ebpf.c index 00b53a57d..67fe477c2 100644 --- a/collectors/ebpf.plugin/ebpf.c +++ b/collectors/ebpf.plugin/ebpf.c @@ -483,6 +483,16 @@ static void ebpf_exit() if (unlink(filename)) error("Cannot remove PID file %s", filename); +#ifdef NETDATA_INTERNAL_CHECKS + error("Good bye world! I was PID %d", main_thread_id); +#endif + printf("DISABLE\n"); + + if (shm_ebpf_cgroup.header) { + munmap(shm_ebpf_cgroup.header, shm_ebpf_cgroup.header->body_length); + shm_unlink(NETDATA_SHARED_MEMORY_EBPF_CGROUP_NAME); + } + exit(0); } @@ -534,7 +544,7 @@ static void ebpf_stop_threads(int sig) pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_exit_plugin = 1; - usec_t max = 3 * USEC_PER_SEC, step = 100000; + usec_t max = USEC_PER_SEC, step = 100000; while (i && max) { max -= step; sleep_usec(step); @@ -548,32 +558,35 @@ static void ebpf_stop_threads(int sig) pthread_mutex_unlock(&ebpf_exit_cleanup); } - //Unload threads(except sync and filesystem) - pthread_mutex_lock(&ebpf_exit_cleanup); - for (i = 0; ebpf_threads[i].name != NULL; i++) { - if (ebpf_threads[i].enabled == NETDATA_THREAD_EBPF_STOPPED && i != EBPF_MODULE_FILESYSTEM_IDX && - i != EBPF_MODULE_SYNC_IDX) - ebpf_unload_legacy_code(ebpf_modules[i].objects, ebpf_modules[i].probe_links); - } - pthread_mutex_unlock(&ebpf_exit_cleanup); + if (!i) { + //Unload threads(except sync and filesystem) + pthread_mutex_lock(&ebpf_exit_cleanup); + for (i = 0; ebpf_threads[i].name != NULL; i++) { + if (ebpf_threads[i].enabled == NETDATA_THREAD_EBPF_STOPPED && i != EBPF_MODULE_FILESYSTEM_IDX && + i != EBPF_MODULE_SYNC_IDX) + ebpf_unload_legacy_code(ebpf_modules[i].objects, ebpf_modules[i].probe_links); + } + pthread_mutex_unlock(&ebpf_exit_cleanup); - //Unload filesystem - pthread_mutex_lock(&ebpf_exit_cleanup); - if (ebpf_threads[EBPF_MODULE_FILESYSTEM_IDX].enabled == NETDATA_THREAD_EBPF_STOPPED) { - for (i = 0; localfs[i].filesystem != NULL; i++) { - ebpf_unload_legacy_code(localfs[i].objects, localfs[i].probe_links); + //Unload filesystem + pthread_mutex_lock(&ebpf_exit_cleanup); + if (ebpf_threads[EBPF_MODULE_FILESYSTEM_IDX].enabled == NETDATA_THREAD_EBPF_STOPPED) { + for (i = 0; localfs[i].filesystem != NULL; i++) { + ebpf_unload_legacy_code(localfs[i].objects, localfs[i].probe_links); + } } - } - pthread_mutex_unlock(&ebpf_exit_cleanup); + pthread_mutex_unlock(&ebpf_exit_cleanup); - //Unload Sync - pthread_mutex_lock(&ebpf_exit_cleanup); - if (ebpf_threads[EBPF_MODULE_SYNC_IDX].enabled == NETDATA_THREAD_EBPF_STOPPED) { - for (i = 0; local_syscalls[i].syscall != NULL; i++) { - ebpf_unload_legacy_code(local_syscalls[i].objects, local_syscalls[i].probe_links); + //Unload Sync + pthread_mutex_lock(&ebpf_exit_cleanup); + if (ebpf_threads[EBPF_MODULE_SYNC_IDX].enabled == NETDATA_THREAD_EBPF_STOPPED) { + for (i = 0; local_syscalls[i].syscall != NULL; i++) { + ebpf_unload_legacy_code(local_syscalls[i].objects, local_syscalls[i].probe_links); + } } + pthread_mutex_unlock(&ebpf_exit_cleanup); + } - pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_exit(); } @@ -1317,7 +1330,7 @@ static void read_local_addresses() } } - fill_ip_list((family == AF_INET)?&network_viewer_opt.ipv4_local_ip:&network_viewer_opt.ipv6_local_ip, + ebpf_fill_ip_list((family == AF_INET)?&network_viewer_opt.ipv4_local_ip:&network_viewer_opt.ipv6_local_ip, w, "selector"); } @@ -1520,13 +1533,8 @@ static void read_collector_values(int *disable_apps, int *disable_cgroups, enabled = appconfig_get_boolean(&collector_config, EBPF_PROGRAMS_SECTION, ebpf_modules[EBPF_MODULE_SOCKET_IDX].config_name, CONFIG_BOOLEAN_NO); - if (enabled) { ebpf_enable_chart(EBPF_MODULE_SOCKET_IDX, *disable_apps, *disable_cgroups); - // Read network viewer section if network viewer is enabled - // This is kept here to keep backward compatibility - parse_network_viewer_section(&collector_config); - parse_service_name_section(&collector_config); started++; } @@ -1536,7 +1544,17 @@ static void read_collector_values(int *disable_apps, int *disable_cgroups, if (!enabled) enabled = appconfig_get_boolean(&collector_config, EBPF_PROGRAMS_SECTION, "network connections", CONFIG_BOOLEAN_NO); - ebpf_modules[EBPF_MODULE_SOCKET_IDX].optional = (int)enabled; + network_viewer_opt.enabled = enabled; + if (enabled) { + if (!ebpf_modules[EBPF_MODULE_SOCKET_IDX].enabled) + ebpf_enable_chart(EBPF_MODULE_SOCKET_IDX, *disable_apps, *disable_cgroups); + + // Read network viewer section if network viewer is enabled + // This is kept here to keep backward compatibility + parse_network_viewer_section(&collector_config); + parse_service_name_section(&collector_config); + started++; + } enabled = appconfig_get_boolean(&collector_config, EBPF_PROGRAMS_SECTION, "cachestat", CONFIG_BOOLEAN_NO); @@ -1642,8 +1660,10 @@ static void read_collector_values(int *disable_apps, int *disable_cgroups, ebpf_enable_all_charts(*disable_apps, *disable_cgroups); // Read network viewer section // This is kept here to keep backward compatibility - parse_network_viewer_section(&collector_config); - parse_service_name_section(&collector_config); + if (network_viewer_opt.enabled) { + parse_network_viewer_section(&collector_config); + parse_service_name_section(&collector_config); + } } } @@ -2158,6 +2178,7 @@ static void ebpf_manage_pid(pid_t pid) */ int main(int argc, char **argv) { + stderror = stderr; clocks_init(); main_thread_id = gettid(); @@ -2237,13 +2258,26 @@ int main(int argc, char **argv) } } - usec_t step = EBPF_DEFAULT_UPDATE_EVERY * USEC_PER_SEC; + usec_t step = USEC_PER_SEC; + int counter = NETDATA_EBPF_CGROUP_UPDATE - 1; heartbeat_t hb; heartbeat_init(&hb); //Plugin will be killed when it receives a signal while (!ebpf_exit_plugin) { (void)heartbeat_next(&hb, step); + + // We are using a small heartbeat time to wake up thread, + // but we should not update so frequently the shared memory data + if (++counter >= NETDATA_EBPF_CGROUP_UPDATE) { + counter = 0; + if (!shm_ebpf_cgroup.header) + ebpf_map_cgroup_shared_memory(); + + ebpf_parse_cgroup_shm_data(); + } } + ebpf_stop_threads(0); + return 0; } diff --git a/collectors/ebpf.plugin/ebpf.d.conf b/collectors/ebpf.plugin/ebpf.d.conf index cf5c740fc..112df275d 100644 --- a/collectors/ebpf.plugin/ebpf.d.conf +++ b/collectors/ebpf.plugin/ebpf.d.conf @@ -17,7 +17,7 @@ # [global] ebpf load mode = entry - apps = yes + apps = no cgroups = no update every = 5 pid table size = 32768 @@ -50,7 +50,7 @@ # When plugin detects that system has support to BTF, it enables integration with apps.plugin. # [ebpf programs] - cachestat = no + cachestat = yes dcstat = no disk = no fd = yes @@ -60,10 +60,10 @@ mount = yes oomkill = yes process = yes - shm = no - socket = yes + shm = yes + socket = no softirq = yes sync = yes - swap = no - vfs = yes + swap = yes + vfs = no network connections = no diff --git a/collectors/ebpf.plugin/ebpf.h b/collectors/ebpf.plugin/ebpf.h index 28b04ce48..16e62498c 100644 --- a/collectors/ebpf.plugin/ebpf.h +++ b/collectors/ebpf.plugin/ebpf.h @@ -123,6 +123,9 @@ enum ebpf_threads_status { #endif #endif +// Messages +#define NETDATA_EBPF_DEFAULT_FNT_NOT_FOUND "Cannot find the necessary functions to monitor" + // Chart definitions #define NETDATA_EBPF_FAMILY "ebpf" #define NETDATA_EBPF_IP_FAMILY "ip" diff --git a/collectors/ebpf.plugin/ebpf_cachestat.c b/collectors/ebpf.plugin/ebpf_cachestat.c index 4c410647d..b21cc6103 100644 --- a/collectors/ebpf.plugin/ebpf_cachestat.c +++ b/collectors/ebpf.plugin/ebpf_cachestat.c @@ -15,15 +15,6 @@ netdata_cachestat_pid_t *cachestat_vector = NULL; static netdata_idx_t cachestat_hash_values[NETDATA_CACHESTAT_END]; static netdata_idx_t *cachestat_values = NULL; -struct netdata_static_thread cachestat_threads = {.name = "CACHESTAT KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL}; - ebpf_local_maps_t cachestat_maps[] = {{.name = "cstat_global", .internal_input = NETDATA_CACHESTAT_END, .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC, .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}, @@ -51,6 +42,9 @@ netdata_ebpf_targets_t cachestat_targets[] = { {.name = "add_to_page_cache_lru", {.name = "mark_buffer_dirty", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}}; +static char *account_page[NETDATA_CACHESTAT_ACCOUNT_DIRTY_END] ={ "account_page_dirtied", + "__set_page_dirty", "__folio_mark_dirty" }; + #ifdef LIBBPF_MAJOR_VERSION #include "includes/cachestat.skel.h" // BTF code @@ -83,10 +77,12 @@ static void ebpf_cachestat_disable_probe(struct cachestat_bpf *obj) */ static void ebpf_cachestat_disable_specific_probe(struct cachestat_bpf *obj) { - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_16) { + if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_FOLIO_DIRTY])) { bpf_program__set_autoload(obj->progs.netdata_account_page_dirtied_kprobe, false); bpf_program__set_autoload(obj->progs.netdata_set_page_dirty_kprobe, false); - } else if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_15) { + } else if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_SET_PAGE_DIRTY])) { bpf_program__set_autoload(obj->progs.netdata_folio_mark_dirty_kprobe, false); bpf_program__set_autoload(obj->progs.netdata_account_page_dirtied_kprobe, false); } else { @@ -122,10 +118,12 @@ static void ebpf_cachestat_disable_trampoline(struct cachestat_bpf *obj) */ static void ebpf_cachestat_disable_specific_trampoline(struct cachestat_bpf *obj) { - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_16) { + if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_FOLIO_DIRTY])) { bpf_program__set_autoload(obj->progs.netdata_account_page_dirtied_fentry, false); bpf_program__set_autoload(obj->progs.netdata_set_page_dirty_fentry, false); - } else if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_15) { + } else if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_SET_PAGE_DIRTY])) { bpf_program__set_autoload(obj->progs.netdata_folio_mark_dirty_fentry, false); bpf_program__set_autoload(obj->progs.netdata_account_page_dirtied_fentry, false); } else { @@ -149,10 +147,12 @@ static inline void netdata_set_trampoline_target(struct cachestat_bpf *obj) bpf_program__set_attach_target(obj->progs.netdata_mark_page_accessed_fentry, 0, cachestat_targets[NETDATA_KEY_CALLS_MARK_PAGE_ACCESSED].name); - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_16) { + if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_FOLIO_DIRTY])) { bpf_program__set_attach_target(obj->progs.netdata_folio_mark_dirty_fentry, 0, cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name); - } else if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_15) { + } else if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_SET_PAGE_DIRTY])) { bpf_program__set_attach_target(obj->progs.netdata_set_page_dirty_fentry, 0, cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name); } else { @@ -192,12 +192,14 @@ static int ebpf_cachestat_attach_probe(struct cachestat_bpf *obj) if (ret) return -1; - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_16) { + if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_FOLIO_DIRTY])) { obj->links.netdata_folio_mark_dirty_kprobe = bpf_program__attach_kprobe(obj->progs.netdata_folio_mark_dirty_kprobe, false, cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name); ret = libbpf_get_error(obj->links.netdata_folio_mark_dirty_kprobe); - } else if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_15) { + } else if (!strcmp(cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name, + account_page[NETDATA_CACHESTAT_SET_PAGE_DIRTY])) { obj->links.netdata_set_page_dirty_kprobe = bpf_program__attach_kprobe(obj->progs.netdata_set_page_dirty_kprobe, false, cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name); @@ -278,7 +280,7 @@ static void ebpf_cachestat_disable_release_task(struct cachestat_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_cachestat_load_and_attach(struct cachestat_bpf *obj, ebpf_module_t *em) { @@ -331,18 +333,13 @@ static inline int ebpf_cachestat_load_and_attach(struct cachestat_bpf *obj, ebpf static void ebpf_cachestat_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_cleanup_publish_syscall(cachestat_counter_publish_aggregated); freez(cachestat_vector); freez(cachestat_values); - freez(cachestat_threads.thread); #ifdef LIBBPF_MAJOR_VERSION if (bpf_obj) @@ -363,20 +360,7 @@ static void ebpf_cachestat_free(ebpf_module_t *em) static void ebpf_cachestat_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*cachestat_threads.thread); - ebpf_cachestat_free(em); -} -/** - * Cachestat cleanup - * - * Clean up allocated addresses. - * - * @param ptr thread data. - */ -static void ebpf_cachestat_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_cachestat_free(em); } @@ -656,7 +640,7 @@ void ebpf_cachestat_create_apps_charts(struct ebpf_module *em, void *ptr) * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_cachestat_read_global_table() { uint32_t idx; netdata_idx_t *val = cachestat_hash_values; @@ -677,35 +661,6 @@ static void read_global_table() } /** - * Socket read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data on very busy socket. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_cachestat_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_cachestat_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_LATENCY_CACHESTAT_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Send global * * Send global charts to Netdata @@ -1106,26 +1061,23 @@ void ebpf_cachestat_send_cgroup_data(int update_every) */ static void cachestat_collector(ebpf_module_t *em) { - cachestat_threads.thread = callocz(1, sizeof(netdata_thread_t)); - cachestat_threads.start_routine = ebpf_cachestat_read_hash; - - netdata_thread_create(cachestat_threads.thread, cachestat_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_cachestat_read_hash, em); - netdata_publish_cachestat_t publish; memset(&publish, 0, sizeof(publish)); int cgroups = em->cgroup_charts; int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; //This will be cancelled by its parent while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_cachestat_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) read_apps_table(); @@ -1237,16 +1189,28 @@ static void ebpf_cachestat_allocate_global_vectors(int apps) * Update Internal value * * Update values used during runtime. + * + * @return It returns 0 when one of the functions is present and -1 otherwise. */ -static void ebpf_cachestat_set_internal_value() +static int ebpf_cachestat_set_internal_value() { - static char *account_page[] = { "account_page_dirtied", "__set_page_dirty", "__folio_mark_dirty" }; - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_16) - cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name = account_page[NETDATA_CACHESTAT_FOLIO_DIRTY]; - else if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_15) - cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name = account_page[NETDATA_CACHESTAT_SET_PAGE_DIRTY]; - else - cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name = account_page[NETDATA_CACHESTAT_ACCOUNT_PAGE_DIRTY]; + ebpf_addresses_t address = {.function = NULL, .hash = 0, .addr = 0}; + int i; + for (i = 0; i < NETDATA_CACHESTAT_ACCOUNT_DIRTY_END ; i++) { + address.function = account_page[i]; + ebpf_load_addresses(&address, -1); + if (address.addr) + break; + } + + if (!address.addr) { + error("%s cachestat.", NETDATA_EBPF_DEFAULT_FNT_NOT_FOUND); + return -1; + } + + cachestat_targets[NETDATA_KEY_CALLS_ACCOUNT_PAGE_DIRTIED].name = address.function; + + return 0; } /* @@ -1300,7 +1264,10 @@ void *ebpf_cachestat_thread(void *ptr) ebpf_update_pid_table(&cachestat_maps[NETDATA_CACHESTAT_PID_STATS], em); - ebpf_cachestat_set_internal_value(); + if (ebpf_cachestat_set_internal_value()) { + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; + goto endcachestat; + } #ifdef LIBBPF_MAJOR_VERSION ebpf_adjust_thread_load(em, default_btf); diff --git a/collectors/ebpf.plugin/ebpf_cachestat.h b/collectors/ebpf.plugin/ebpf_cachestat.h index 07f0745d4..15b06511e 100644 --- a/collectors/ebpf.plugin/ebpf_cachestat.h +++ b/collectors/ebpf.plugin/ebpf_cachestat.h @@ -19,8 +19,6 @@ #define EBPF_CACHESTAT_DIMENSION_HITS "hits/s" #define EBPF_CACHESTAT_DIMENSION_MISSES "misses/s" -#define NETDATA_LATENCY_CACHESTAT_SLEEP_MS 600000ULL - // configuration file #define NETDATA_CACHESTAT_CONFIG_FILE "cachestat.conf" @@ -48,7 +46,9 @@ enum cachestat_counters { enum cachestat_account_dirty_pages { NETDATA_CACHESTAT_ACCOUNT_PAGE_DIRTY, NETDATA_CACHESTAT_SET_PAGE_DIRTY, - NETDATA_CACHESTAT_FOLIO_DIRTY + NETDATA_CACHESTAT_FOLIO_DIRTY, + + NETDATA_CACHESTAT_ACCOUNT_DIRTY_END }; enum cachestat_indexes { diff --git a/collectors/ebpf.plugin/ebpf_dcstat.c b/collectors/ebpf.plugin/ebpf_dcstat.c index 71169e153..75e83214a 100644 --- a/collectors/ebpf.plugin/ebpf_dcstat.c +++ b/collectors/ebpf.plugin/ebpf_dcstat.c @@ -19,15 +19,6 @@ struct config dcstat_config = { .first_section = NULL, .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare }, .rwlock = AVL_LOCK_INITIALIZER } }; -struct netdata_static_thread dcstat_threads = {"DCSTAT KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL}; - ebpf_local_maps_t dcstat_maps[] = {{.name = "dcstat_global", .internal_input = NETDATA_DIRECTORY_CACHE_END, .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC, .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}, @@ -216,7 +207,7 @@ static void ebpf_dc_disable_release_task(struct dc_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_dc_load_and_attach(struct dc_bpf *obj, ebpf_module_t *em) { @@ -303,16 +294,11 @@ void ebpf_dcstat_clean_names() static void ebpf_dcstat_free(ebpf_module_t *em ) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); freez(dcstat_vector); freez(dcstat_values); - freez(dcstat_threads.thread); ebpf_cleanup_publish_syscall(dcstat_counter_publish_aggregated); @@ -338,18 +324,6 @@ static void ebpf_dcstat_free(ebpf_module_t *em ) static void ebpf_dcstat_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*dcstat_threads.thread); - ebpf_dcstat_free(em); -} - -/** - * Clean up the main thread. - * - * @param ptr thread data. - */ -static void ebpf_dcstat_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_dcstat_free(em); } @@ -538,7 +512,7 @@ static void ebpf_update_dc_cgroup() * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_dc_read_global_table() { uint32_t idx; netdata_idx_t *val = dcstat_hash_values; @@ -559,35 +533,6 @@ static void read_global_table() } /** - * DCstat read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_dcstat_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_dcstat_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_LATENCY_DCSTAT_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Cachestat sum PIDs * * Sum values for all PIDs associated to a group @@ -1034,25 +979,22 @@ void ebpf_dc_send_cgroup_data(int update_every) */ static void dcstat_collector(ebpf_module_t *em) { - dcstat_threads.thread = mallocz(sizeof(netdata_thread_t)); - dcstat_threads.start_routine = ebpf_dcstat_read_hash; - - netdata_thread_create(dcstat_threads.thread, dcstat_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_dcstat_read_hash, em); - netdata_publish_dcstat_t publish; memset(&publish, 0, sizeof(publish)); int cgroups = em->cgroup_charts; int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_dc_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) read_apps_table(); diff --git a/collectors/ebpf.plugin/ebpf_dcstat.h b/collectors/ebpf.plugin/ebpf_dcstat.h index d8687f968..201fc8a02 100644 --- a/collectors/ebpf.plugin/ebpf_dcstat.h +++ b/collectors/ebpf.plugin/ebpf_dcstat.h @@ -28,8 +28,6 @@ #define NETDATA_SYSTEMD_DC_NOT_CACHE_CONTEXT "services.dc_not_cache" #define NETDATA_SYSTEMD_DC_NOT_FOUND_CONTEXT "services.dc_not_found" -#define NETDATA_LATENCY_DCSTAT_SLEEP_MS 700000ULL - enum directory_cache_indexes { NETDATA_DCSTAT_IDX_RATIO, NETDATA_DCSTAT_IDX_REFERENCE, diff --git a/collectors/ebpf.plugin/ebpf_disk.c b/collectors/ebpf.plugin/ebpf_disk.c index a27bd81e3..5e7e2599d 100644 --- a/collectors/ebpf.plugin/ebpf_disk.c +++ b/collectors/ebpf.plugin/ebpf_disk.c @@ -33,16 +33,6 @@ static netdata_syscall_stat_t disk_aggregated_data[NETDATA_EBPF_HIST_MAX_BINS]; static netdata_publish_syscall_t disk_publish_aggregated[NETDATA_EBPF_HIST_MAX_BINS]; static netdata_idx_t *disk_hash_values = NULL; -static struct netdata_static_thread disk_threads = { - .name = "DISK KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; ebpf_publish_disk_t *plot_disks = NULL; pthread_mutex_t plot_mutex; @@ -439,11 +429,7 @@ static void ebpf_cleanup_disk_list() static void ebpf_disk_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_disk_disable_tracepoints(); @@ -452,7 +438,6 @@ static void ebpf_disk_free(ebpf_module_t *em) ebpf_histogram_dimension_cleanup(dimensions, NETDATA_EBPF_HIST_MAX_BINS); freez(disk_hash_values); - freez(disk_threads.thread); pthread_mutex_destroy(&plot_mutex); ebpf_cleanup_plot_disks(); @@ -473,20 +458,6 @@ static void ebpf_disk_free(ebpf_module_t *em) static void ebpf_disk_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*disk_threads.thread); - ebpf_disk_free(em); -} - -/** - * Disk Cleanup - * - * Clean up allocated memory. - * - * @param ptr thread data. - */ -static void ebpf_disk_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_disk_free(em); } @@ -592,35 +563,6 @@ static void read_hard_disk_tables(int table) } /** - * Disk read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data on very busy socket. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_disk_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_disk_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_LATENCY_DISK_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_hard_disk_tables(disk_maps[NETDATA_DISK_READ].map_fd); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Obsolete Hard Disk charts * * Make Hard disk charts and fill chart name @@ -743,21 +685,19 @@ static void ebpf_latency_send_hd_data(int update_every) static void disk_collector(ebpf_module_t *em) { disk_hash_values = callocz(ebpf_nprocs, sizeof(netdata_idx_t)); - disk_threads.thread = mallocz(sizeof(netdata_thread_t)); - disk_threads.start_routine = ebpf_disk_read_hash; - - netdata_thread_create(disk_threads.thread, disk_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_disk_read_hash, em); int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + read_hard_disk_tables(disk_maps[NETDATA_DISK_READ].map_fd); pthread_mutex_lock(&lock); ebpf_remove_pointer_from_plot_disk(em); ebpf_latency_send_hd_data(update_every); diff --git a/collectors/ebpf.plugin/ebpf_disk.h b/collectors/ebpf.plugin/ebpf_disk.h index c14b887f8..c606d6594 100644 --- a/collectors/ebpf.plugin/ebpf_disk.h +++ b/collectors/ebpf.plugin/ebpf_disk.h @@ -11,8 +11,6 @@ #define NETDATA_EBPF_PROC_PARTITIONS "/proc/partitions" -#define NETDATA_LATENCY_DISK_SLEEP_MS 650000ULL - // Process configuration name #define NETDATA_DISK_CONFIG_FILE "disk.conf" diff --git a/collectors/ebpf.plugin/ebpf_fd.c b/collectors/ebpf.plugin/ebpf_fd.c index 30b7f22ce..79537066c 100644 --- a/collectors/ebpf.plugin/ebpf_fd.c +++ b/collectors/ebpf.plugin/ebpf_fd.c @@ -6,6 +6,9 @@ static char *fd_dimension_names[NETDATA_FD_SYSCALL_END] = { "open", "close" }; static char *fd_id_names[NETDATA_FD_SYSCALL_END] = { "do_sys_open", "__close_fd" }; +static char *close_targets[NETDATA_EBPF_MAX_FD_TARGETS] = {"close_fd", "__close_fd"}; +static char *open_targets[NETDATA_EBPF_MAX_FD_TARGETS] = {"do_sys_openat2", "do_sys_open"}; + static netdata_syscall_stat_t fd_aggregated_data[NETDATA_FD_SYSCALL_END]; static netdata_publish_syscall_t fd_publish_aggregated[NETDATA_FD_SYSCALL_END]; @@ -29,15 +32,6 @@ struct config fd_config = { .first_section = NULL, .last_section = NULL, .mutex .index = {.avl_tree = { .root = NULL, .compar = appconfig_section_compare }, .rwlock = AVL_LOCK_INITIALIZER } }; -struct netdata_static_thread fd_thread = {"FD KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL}; - static netdata_idx_t fd_hash_values[NETDATA_FD_COUNTER]; static netdata_idx_t *fd_values = NULL; @@ -65,7 +59,7 @@ static inline void ebpf_fd_disable_probes(struct fd_bpf *obj) bpf_program__set_autoload(obj->progs.netdata_sys_open_kprobe, false); bpf_program__set_autoload(obj->progs.netdata_sys_open_kretprobe, false); bpf_program__set_autoload(obj->progs.netdata_release_task_fd_kprobe, false); - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { + if (!strcmp(fd_targets[NETDATA_FD_SYSCALL_CLOSE].name, close_targets[NETDATA_FD_CLOSE_FD])) { bpf_program__set_autoload(obj->progs.netdata___close_fd_kretprobe, false); bpf_program__set_autoload(obj->progs.netdata___close_fd_kprobe, false); bpf_program__set_autoload(obj->progs.netdata_close_fd_kprobe, false); @@ -85,7 +79,7 @@ static inline void ebpf_fd_disable_probes(struct fd_bpf *obj) */ static inline void ebpf_disable_specific_probes(struct fd_bpf *obj) { - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { + if (!strcmp(fd_targets[NETDATA_FD_SYSCALL_CLOSE].name, close_targets[NETDATA_FD_CLOSE_FD])) { bpf_program__set_autoload(obj->progs.netdata___close_fd_kretprobe, false); bpf_program__set_autoload(obj->progs.netdata___close_fd_kprobe, false); } else { @@ -121,7 +115,7 @@ static inline void ebpf_disable_trampoline(struct fd_bpf *obj) */ static inline void ebpf_disable_specific_trampoline(struct fd_bpf *obj) { - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { + if (!strcmp(fd_targets[NETDATA_FD_SYSCALL_CLOSE].name, close_targets[NETDATA_FD_CLOSE_FD])) { bpf_program__set_autoload(obj->progs.netdata___close_fd_fentry, false); bpf_program__set_autoload(obj->progs.netdata___close_fd_fexit, false); } else { @@ -143,7 +137,7 @@ static void ebpf_set_trampoline_target(struct fd_bpf *obj) bpf_program__set_attach_target(obj->progs.netdata_sys_open_fexit, 0, fd_targets[NETDATA_FD_SYSCALL_OPEN].name); bpf_program__set_attach_target(obj->progs.netdata_release_task_fd_fentry, 0, EBPF_COMMON_FNCT_CLEAN_UP); - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { + if (!strcmp(fd_targets[NETDATA_FD_SYSCALL_CLOSE].name, close_targets[NETDATA_FD_CLOSE_FD])) { bpf_program__set_attach_target( obj->progs.netdata_close_fd_fentry, 0, fd_targets[NETDATA_FD_SYSCALL_CLOSE].name); bpf_program__set_attach_target(obj->progs.netdata_close_fd_fexit, 0, fd_targets[NETDATA_FD_SYSCALL_CLOSE].name); @@ -185,7 +179,7 @@ static int ebpf_fd_attach_probe(struct fd_bpf *obj) if (ret) return -1; - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { + if (!strcmp(fd_targets[NETDATA_FD_SYSCALL_CLOSE].name, close_targets[NETDATA_FD_CLOSE_FD])) { obj->links.netdata_close_fd_kretprobe = bpf_program__attach_kprobe(obj->progs.netdata_close_fd_kretprobe, true, fd_targets[NETDATA_FD_SYSCALL_CLOSE].name); ret = libbpf_get_error(obj->links.netdata_close_fd_kretprobe); @@ -217,24 +211,50 @@ static int ebpf_fd_attach_probe(struct fd_bpf *obj) } /** - * Set target values + * FD Fill Address * - * Set pointers used to laod data. + * Fill address value used to load probes/trampoline. */ -static void ebpf_fd_set_target_values() +static inline void ebpf_fd_fill_address(ebpf_addresses_t *address, char **targets) { - static char *close_targets[] = {"close_fd", "__close_fd"}; - static char *open_targets[] = {"do_sys_openat2", "do_sys_open"}; - if (running_on_kernel >= NETDATA_EBPF_KERNEL_5_11) { - fd_targets[NETDATA_FD_SYSCALL_OPEN].name = open_targets[0]; - fd_targets[NETDATA_FD_SYSCALL_CLOSE].name = close_targets[0]; - } else { - fd_targets[NETDATA_FD_SYSCALL_OPEN].name = open_targets[1]; - fd_targets[NETDATA_FD_SYSCALL_CLOSE].name = close_targets[1]; + int i; + for (i = 0; i < NETDATA_EBPF_MAX_FD_TARGETS; i++) { + address->function = targets[i]; + ebpf_load_addresses(address, -1); + if (address->addr) + break; } } /** + * Set target values + * + * Set pointers used to load data. + * + * @return It returns 0 on success and -1 otherwise. + */ +static int ebpf_fd_set_target_values() +{ + ebpf_addresses_t address = {.function = NULL, .hash = 0, .addr = 0}; + ebpf_fd_fill_address(&address, close_targets); + + if (!address.addr) + return -1; + + fd_targets[NETDATA_FD_SYSCALL_CLOSE].name = address.function; + + address.addr = 0; + ebpf_fd_fill_address(&address, open_targets); + + if (!address.addr) + return -1; + + fd_targets[NETDATA_FD_SYSCALL_OPEN].name = address.function; + + return 0; +} + +/** * Set hash tables * * Set the values for maps according the value given by kernel. @@ -283,14 +303,18 @@ static void ebpf_fd_disable_release_task(struct fd_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_fd_load_and_attach(struct fd_bpf *obj, ebpf_module_t *em) { netdata_ebpf_targets_t *mt = em->targets; netdata_ebpf_program_loaded_t test = mt[NETDATA_FD_SYSCALL_OPEN].mode; - ebpf_fd_set_target_values(); + if (ebpf_fd_set_target_values()) { + error("%s file descriptor.", NETDATA_EBPF_DEFAULT_FNT_NOT_FOUND); + return -1; + } + if (test == EBPF_LOAD_TRAMPOLINE) { ebpf_fd_disable_probes(obj); ebpf_disable_specific_trampoline(obj); @@ -340,15 +364,10 @@ static inline int ebpf_fd_load_and_attach(struct fd_bpf *obj, ebpf_module_t *em) static void ebpf_fd_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_cleanup_publish_syscall(fd_publish_aggregated); - freez(fd_thread.thread); freez(fd_values); freez(fd_vector); @@ -372,18 +391,6 @@ static void ebpf_fd_free(ebpf_module_t *em) static void ebpf_fd_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*fd_thread.thread); - ebpf_fd_free(em); -} - -/** - * Clean up the main thread. - * - * @param ptr thread data. - */ -static void ebpf_fd_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_fd_free(em); } @@ -420,7 +427,7 @@ static void ebpf_fd_send_data(ebpf_module_t *em) * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_fd_read_global_table() { uint32_t idx; netdata_idx_t *val = fd_hash_values; @@ -441,34 +448,6 @@ static void read_global_table() } /** - * File descriptor read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_fd_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_fd_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - usec_t step = NETDATA_FD_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Apps Accumulator * * Sum all values read from kernel and store in the first address. @@ -942,22 +921,20 @@ static void ebpf_fd_send_cgroup_data(ebpf_module_t *em) */ static void fd_collector(ebpf_module_t *em) { - fd_thread.thread = mallocz(sizeof(netdata_thread_t)); - fd_thread.start_routine = ebpf_fd_read_hash; - - netdata_thread_create(fd_thread.thread, fd_thread.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_fd_read_hash, em); - int cgroups = em->cgroup_charts; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_fd_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) read_apps_table(); diff --git a/collectors/ebpf.plugin/ebpf_fd.h b/collectors/ebpf.plugin/ebpf_fd.h index 914a34b98..e6545d79c 100644 --- a/collectors/ebpf.plugin/ebpf_fd.h +++ b/collectors/ebpf.plugin/ebpf_fd.h @@ -6,8 +6,6 @@ // Module name #define NETDATA_EBPF_MODULE_NAME_FD "filedescriptor" -#define NETDATA_FD_SLEEP_MS 850000ULL - // Menu group #define NETDATA_FILE_GROUP "file_access" @@ -36,9 +34,6 @@ #define NETDATA_SYSTEMD_FD_CLOSE_ERR_CONTEXT "services.fd_close_error" typedef struct netdata_fd_stat { - uint64_t pid_tgid; // Unique identifier - uint32_t pid; // Process ID - uint32_t open_call; // Open syscalls (open and openat) uint32_t close_call; // Close syscall (close) @@ -74,6 +69,14 @@ enum fd_syscalls { NETDATA_FD_SYSCALL_END }; +enum fd_close_syscall { + NETDATA_FD_CLOSE_FD, + NETDATA_FD___CLOSE_FD, + + NETDATA_FD_CLOSE_END +}; + +#define NETDATA_EBPF_MAX_FD_TARGETS 2 void *ebpf_fd_thread(void *ptr); void ebpf_fd_create_apps_charts(struct ebpf_module *em, void *ptr); diff --git a/collectors/ebpf.plugin/ebpf_filesystem.c b/collectors/ebpf.plugin/ebpf_filesystem.c index 7dbec7410..5250ed8af 100644 --- a/collectors/ebpf.plugin/ebpf_filesystem.c +++ b/collectors/ebpf.plugin/ebpf_filesystem.c @@ -30,17 +30,6 @@ static ebpf_local_maps_t fs_maps[] = {{.name = "tbl_ext4", .internal_input = NET .type = NETDATA_EBPF_MAP_CONTROLLER, .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}}; -struct netdata_static_thread filesystem_threads = { - .name = "EBPF FS READ", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - static netdata_syscall_stat_t filesystem_aggregated_data[NETDATA_EBPF_HIST_MAX_BINS]; static netdata_publish_syscall_t filesystem_publish_aggregated[NETDATA_EBPF_HIST_MAX_BINS]; @@ -337,14 +326,9 @@ void ebpf_filesystem_cleanup_ebpf_data() static void ebpf_filesystem_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); - freez(filesystem_threads.thread); ebpf_cleanup_publish_syscall(filesystem_publish_aggregated); ebpf_filesystem_cleanup_ebpf_data(); @@ -367,20 +351,6 @@ static void ebpf_filesystem_free(ebpf_module_t *em) static void ebpf_filesystem_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*filesystem_threads.thread); - ebpf_filesystem_free(em); -} - -/** - * File system cleanup - * - * Clean up allocated thread. - * - * @param ptr thread data. - */ -static void ebpf_filesystem_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_filesystem_free(em); } @@ -483,30 +453,16 @@ static void read_filesystem_tables() * * @return It always returns NULL. */ -void *ebpf_filesystem_read_hash(void *ptr) +void ebpf_filesystem_read_hash(ebpf_module_t *em) { - netdata_thread_cleanup_push(ebpf_filesystem_cleanup, ptr); - ebpf_module_t *em = (ebpf_module_t *)ptr; - - heartbeat_t hb; - heartbeat_init(&hb); - usec_t step = NETDATA_FILESYSTEM_READ_SLEEP_MS * em->update_every; - int update_every = em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); + ebpf_obsolete_fs_charts(em->update_every); - (void) ebpf_update_partitions(em); - ebpf_obsolete_fs_charts(update_every); + (void) ebpf_update_partitions(em); - // No more partitions, it is not necessary to read tables - if (em->optional) - continue; - - read_filesystem_tables(); - } + if (em->optional) + return; - netdata_thread_cleanup_pop(1); - return NULL; + read_filesystem_tables(); } /** @@ -543,21 +499,18 @@ static void ebpf_histogram_send_data() */ static void filesystem_collector(ebpf_module_t *em) { - filesystem_threads.thread = mallocz(sizeof(netdata_thread_t)); - filesystem_threads.start_routine = ebpf_filesystem_read_hash; - - netdata_thread_create(filesystem_threads.thread, filesystem_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, ebpf_filesystem_read_hash, em); - int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + ebpf_filesystem_read_hash(em); pthread_mutex_lock(&lock); ebpf_create_fs_charts(update_every); diff --git a/collectors/ebpf.plugin/ebpf_filesystem.h b/collectors/ebpf.plugin/ebpf_filesystem.h index 0d558df7d..cf19b253e 100644 --- a/collectors/ebpf.plugin/ebpf_filesystem.h +++ b/collectors/ebpf.plugin/ebpf_filesystem.h @@ -11,7 +11,6 @@ #define NETDATA_FS_MAX_DIST_NAME 64UL #define NETDATA_FILESYSTEM_CONFIG_NAME "filesystem" -#define NETDATA_FILESYSTEM_READ_SLEEP_MS 600000ULL // Process configuration name #define NETDATA_FILESYSTEM_CONFIG_FILE "filesystem.conf" diff --git a/collectors/ebpf.plugin/ebpf_hardirq.c b/collectors/ebpf.plugin/ebpf_hardirq.c index b07dd24ca..20c4b9d05 100644 --- a/collectors/ebpf.plugin/ebpf_hardirq.c +++ b/collectors/ebpf.plugin/ebpf_hardirq.c @@ -135,17 +135,6 @@ static hardirq_ebpf_val_t *hardirq_ebpf_vals = NULL; // tmp store for static hard IRQ values we get from a per-CPU eBPF map. static hardirq_ebpf_static_val_t *hardirq_ebpf_static_vals = NULL; -static struct netdata_static_thread hardirq_threads = { - .name = "HARDIRQ KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - /** * Hardirq Free * @@ -156,21 +145,18 @@ static struct netdata_static_thread hardirq_threads = { static void ebpf_hardirq_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); - freez(hardirq_threads.thread); for (int i = 0; hardirq_tracepoints[i].class != NULL; i++) { ebpf_disable_tracepoint(&hardirq_tracepoints[i]); } freez(hardirq_ebpf_vals); freez(hardirq_ebpf_static_vals); + pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; + pthread_mutex_unlock(&ebpf_exit_cleanup); } /** @@ -183,20 +169,6 @@ static void ebpf_hardirq_free(ebpf_module_t *em) static void hardirq_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*hardirq_threads.thread); - ebpf_hardirq_free(em); -} - -/** - * Hardirq clean up - * - * Clean up allocated memory. - * - * @param ptr thread data. - */ -static void hardirq_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_hardirq_free(em); } @@ -331,24 +303,10 @@ static void hardirq_read_latency_static_map(int mapfd) /** * Read eBPF maps for hard IRQ. */ -static void *hardirq_reader(void *ptr) +static void hardirq_reader() { - netdata_thread_cleanup_push(hardirq_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_HARDIRQ_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - hardirq_read_latency_map(hardirq_maps[HARDIRQ_MAP_LATENCY].map_fd); - hardirq_read_latency_static_map(hardirq_maps[HARDIRQ_MAP_LATENCY_STATIC].map_fd); - } - - netdata_thread_cleanup_pop(1); - return NULL; + hardirq_read_latency_map(hardirq_maps[HARDIRQ_MAP_LATENCY].map_fd); + hardirq_read_latency_static_map(hardirq_maps[HARDIRQ_MAP_LATENCY_STATIC].map_fd); } static void hardirq_create_charts(int update_every) @@ -428,17 +386,6 @@ static void hardirq_collector(ebpf_module_t *em) avl_init_lock(&hardirq_pub, hardirq_val_cmp); - // create reader thread. - hardirq_threads.thread = mallocz(sizeof(netdata_thread_t)); - hardirq_threads.start_routine = hardirq_reader; - netdata_thread_create( - hardirq_threads.thread, - hardirq_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, - hardirq_reader, - em - ); - // create chart and static dims. pthread_mutex_lock(&lock); hardirq_create_charts(em->update_every); @@ -449,13 +396,17 @@ static void hardirq_collector(ebpf_module_t *em) // loop and read from published data until ebpf plugin is closed. heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; //This will be cancelled by its parent while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + hardirq_reader(); pthread_mutex_lock(&lock); // write dims now for all hitherto discovered IRQs. diff --git a/collectors/ebpf.plugin/ebpf_hardirq.h b/collectors/ebpf.plugin/ebpf_hardirq.h index 381da57d8..fe38b1bb1 100644 --- a/collectors/ebpf.plugin/ebpf_hardirq.h +++ b/collectors/ebpf.plugin/ebpf_hardirq.h @@ -47,7 +47,6 @@ typedef struct hardirq_ebpf_static_val { *****************************************************************/ #define NETDATA_EBPF_MODULE_NAME_HARDIRQ "hardirq" -#define NETDATA_HARDIRQ_SLEEP_MS 650000ULL #define NETDATA_HARDIRQ_CONFIG_FILE "hardirq.conf" typedef struct hardirq_val { diff --git a/collectors/ebpf.plugin/ebpf_mdflush.c b/collectors/ebpf.plugin/ebpf_mdflush.c index dc805da23..1a5a7731e 100644 --- a/collectors/ebpf.plugin/ebpf_mdflush.c +++ b/collectors/ebpf.plugin/ebpf_mdflush.c @@ -35,17 +35,6 @@ static avl_tree_lock mdflush_pub; // tmp store for mdflush values we get from a per-CPU eBPF map. static mdflush_ebpf_val_t *mdflush_ebpf_vals = NULL; -static struct netdata_static_thread mdflush_threads = { - .name = "MDFLUSH KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - /** * MDflush Free * @@ -55,18 +44,10 @@ static struct netdata_static_thread mdflush_threads = { */ static void ebpf_mdflush_free(ebpf_module_t *em) { - pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } - pthread_mutex_unlock(&ebpf_exit_cleanup); - freez(mdflush_ebpf_vals); - freez(mdflush_threads.thread); - + pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; + pthread_mutex_unlock(&ebpf_exit_cleanup); } /** @@ -83,20 +64,6 @@ static void mdflush_exit(void *ptr) } /** - * CLeanup - * - * Clean allocated memory. - * - * @param ptr thread data. - */ -static void mdflush_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*mdflush_threads.thread); - ebpf_mdflush_free(em); -} - -/** * Compare mdflush values. * * @param a `netdata_mdflush_t *`. @@ -188,28 +155,6 @@ static void mdflush_read_count_map() } } -/** - * Read eBPF maps for mdflush. - */ -static void *mdflush_reader(void *ptr) -{ - netdata_thread_cleanup_push(mdflush_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_MDFLUSH_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - mdflush_read_count_map(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - static void mdflush_create_charts(int update_every) { ebpf_create_chart( @@ -256,34 +201,27 @@ static void mdflush_collector(ebpf_module_t *em) { mdflush_ebpf_vals = callocz(ebpf_nprocs, sizeof(mdflush_ebpf_val_t)); + int update_every = em->update_every; avl_init_lock(&mdflush_pub, mdflush_val_cmp); - // create reader thread. - mdflush_threads.thread = mallocz(sizeof(netdata_thread_t)); - mdflush_threads.start_routine = mdflush_reader; - netdata_thread_create( - mdflush_threads.thread, - mdflush_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, - mdflush_reader, - em - ); - // create chart and static dims. pthread_mutex_lock(&lock); - mdflush_create_charts(em->update_every); + mdflush_create_charts(update_every); ebpf_update_stats(&plugin_statistics, em); pthread_mutex_unlock(&lock); // loop and read from published data until ebpf plugin is closed. heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + + counter = 0; + mdflush_read_count_map(); // write dims now for all hitherto discovered devices. write_begin_chart("mdstat", "mdstat_flush"); avl_traverse_lock(&mdflush_pub, mdflush_write_dims, NULL); diff --git a/collectors/ebpf.plugin/ebpf_mdflush.h b/collectors/ebpf.plugin/ebpf_mdflush.h index b04eefd28..4913ad019 100644 --- a/collectors/ebpf.plugin/ebpf_mdflush.h +++ b/collectors/ebpf.plugin/ebpf_mdflush.h @@ -6,8 +6,6 @@ // Module name #define NETDATA_EBPF_MODULE_NAME_MDFLUSH "mdflush" -#define NETDATA_MDFLUSH_SLEEP_MS 850000ULL - // charts #define NETDATA_MDFLUSH_GLOBAL_CHART "mdflush" diff --git a/collectors/ebpf.plugin/ebpf_mount.c b/collectors/ebpf.plugin/ebpf_mount.c index ec1f07a65..e06010b5b 100644 --- a/collectors/ebpf.plugin/ebpf_mount.c +++ b/collectors/ebpf.plugin/ebpf_mount.c @@ -22,17 +22,6 @@ static netdata_idx_t *mount_values = NULL; static netdata_idx_t mount_hash_values[NETDATA_MOUNT_END]; -struct netdata_static_thread mount_thread = { - .name = "MOUNT KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - netdata_ebpf_targets_t mount_targets[] = { {.name = "mount", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "umount", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}}; @@ -187,7 +176,7 @@ static void ebpf_mount_set_hash_tables(struct mount_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_mount_load_and_attach(struct mount_bpf *obj, ebpf_module_t *em) { @@ -239,14 +228,9 @@ static inline int ebpf_mount_load_and_attach(struct mount_bpf *obj, ebpf_module_ static void ebpf_mount_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); - freez(mount_thread.thread); freez(mount_values); #ifdef LIBBPF_MAJOR_VERSION @@ -269,20 +253,6 @@ static void ebpf_mount_free(ebpf_module_t *em) static void ebpf_mount_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*mount_thread.thread); - ebpf_mount_free(em); -} - -/** - * Mount cleanup - * - * Clean up allocated memory. - * - * @param ptr thread data. - */ -static void ebpf_mount_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_mount_free(em); } @@ -297,7 +267,7 @@ static void ebpf_mount_cleanup(void *ptr) * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_mount_read_global_table() { uint32_t idx; netdata_idx_t *val = mount_hash_values; @@ -318,36 +288,6 @@ static void read_global_table() } /** - * Mount read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_mount_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_mount_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_LATENCY_MOUNT_SLEEP_MS * em->update_every; - //This will be cancelled by its parent - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Send data to Netdata calling auxiliary functions. */ static void ebpf_mount_send_data() @@ -371,23 +311,20 @@ static void ebpf_mount_send_data() */ static void mount_collector(ebpf_module_t *em) { - mount_thread.thread = mallocz(sizeof(netdata_thread_t)); - mount_thread.start_routine = ebpf_mount_read_hash; - memset(mount_hash_values, 0, sizeof(mount_hash_values)); - mount_values = callocz((size_t)ebpf_nprocs, sizeof(netdata_idx_t)); - - netdata_thread_create(mount_thread.thread, mount_thread.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_mount_read_hash, em); + memset(mount_hash_values, 0, sizeof(mount_hash_values)); heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + ebpf_mount_read_global_table(); pthread_mutex_lock(&lock); ebpf_mount_send_data(); diff --git a/collectors/ebpf.plugin/ebpf_mount.h b/collectors/ebpf.plugin/ebpf_mount.h index 5a8d11a59..11b21f832 100644 --- a/collectors/ebpf.plugin/ebpf_mount.h +++ b/collectors/ebpf.plugin/ebpf_mount.h @@ -8,8 +8,6 @@ #define NETDATA_EBPF_MOUNT_SYSCALL 2 -#define NETDATA_LATENCY_MOUNT_SLEEP_MS 700000ULL - #define NETDATA_EBPF_MOUNT_CALLS "call" #define NETDATA_EBPF_MOUNT_ERRORS "error" #define NETDATA_EBPF_MOUNT_FAMILY "mount (eBPF)" diff --git a/collectors/ebpf.plugin/ebpf_oomkill.c b/collectors/ebpf.plugin/ebpf_oomkill.c index d93e4159e..82420d54e 100644 --- a/collectors/ebpf.plugin/ebpf_oomkill.c +++ b/collectors/ebpf.plugin/ebpf_oomkill.c @@ -46,7 +46,9 @@ static netdata_publish_syscall_t oomkill_publish_aggregated = {.name = "oomkill" static void oomkill_cleanup(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; + pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; + pthread_mutex_unlock(&ebpf_exit_cleanup); } static void oomkill_write_data(int32_t *keys, uint32_t total) @@ -294,12 +296,13 @@ static void oomkill_collector(ebpf_module_t *em) // loop and read until ebpf plugin is closed. heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (!ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; pthread_mutex_lock(&collect_data_mutex); pthread_mutex_lock(&lock); diff --git a/collectors/ebpf.plugin/ebpf_oomkill.h b/collectors/ebpf.plugin/ebpf_oomkill.h index 786086384..f921f9d87 100644 --- a/collectors/ebpf.plugin/ebpf_oomkill.h +++ b/collectors/ebpf.plugin/ebpf_oomkill.h @@ -17,7 +17,6 @@ typedef uint8_t oomkill_ebpf_val_t; *****************************************************************/ #define NETDATA_EBPF_MODULE_NAME_OOMKILL "oomkill" -#define NETDATA_OOMKILL_SLEEP_MS 650000ULL #define NETDATA_OOMKILL_CONFIG_FILE "oomkill.conf" #define NETDATA_OOMKILL_CHART "oomkills" diff --git a/collectors/ebpf.plugin/ebpf_process.c b/collectors/ebpf.plugin/ebpf_process.c index 682577da7..9a191d391 100644 --- a/collectors/ebpf.plugin/ebpf_process.c +++ b/collectors/ebpf.plugin/ebpf_process.c @@ -57,17 +57,6 @@ struct config process_config = { .first_section = NULL, static char *threads_stat[NETDATA_EBPF_THREAD_STAT_END] = {"total", "running"}; static char *load_event_stat[NETDATA_EBPF_LOAD_STAT_END] = {"legacy", "co-re"}; -static struct netdata_static_thread cgroup_thread = { - .name = "EBPF CGROUP", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - /***************************************************************** * * PROCESS DATA AND SEND TO NETDATA @@ -327,55 +316,6 @@ static void ebpf_process_update_apps_data() } /** - * Cgroup Exit - * - * Function used with netdata_thread_clean_push - * - * @param ptr unused argument - */ -static void ebpf_cgroup_exit(void *ptr) -{ - UNUSED(ptr); -} - -/** - * Cgroup update shm - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data from shared memory. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_cgroup_update_shm(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_cgroup_exit, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - usec_t step = 3 * USEC_PER_SEC; - int counter = NETDATA_EBPF_CGROUP_UPDATE - 1; - //This will be cancelled by its parent - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - // We are using a small heartbeat time to wake up thread, - // but we should not update so frequently the shared memory data - if (++counter >= NETDATA_EBPF_CGROUP_UPDATE) { - counter = 0; - if (!shm_ebpf_cgroup.header) - ebpf_map_cgroup_shared_memory(); - - ebpf_parse_cgroup_shm_data(); - } - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Update cgroup * * Update cgroup data based in @@ -745,7 +685,6 @@ static void ebpf_process_exit(void *ptr) pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; pthread_mutex_unlock(&ebpf_exit_cleanup); - pthread_cancel(*cgroup_thread.thread); } /***************************************************************** @@ -1104,13 +1043,6 @@ void ebpf_send_statistic_data() */ static void process_collector(ebpf_module_t *em) { - // Start cgroup integration before other threads - cgroup_thread.thread = mallocz(sizeof(netdata_thread_t)); - cgroup_thread.start_routine = ebpf_cgroup_update_shm; - - netdata_thread_create(cgroup_thread.thread, cgroup_thread.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_cgroup_update_shm, NULL); - heartbeat_t hb; heartbeat_init(&hb); int publish_global = em->global_charts; @@ -1152,7 +1084,7 @@ static void process_collector(ebpf_module_t *em) ebpf_process_update_apps_data(); } - if (cgroups) { + if (cgroups && shm_ebpf_cgroup.header) { ebpf_update_process_cgroup(); } } @@ -1169,7 +1101,7 @@ static void process_collector(ebpf_module_t *em) ebpf_process_send_apps_data(apps_groups_root_target, em); } - if (cgroups) { + if (cgroups && shm_ebpf_cgroup.header) { ebpf_process_send_cgroup_data(em); } } diff --git a/collectors/ebpf.plugin/ebpf_process.h b/collectors/ebpf.plugin/ebpf_process.h index 43df34d48..6fded16fc 100644 --- a/collectors/ebpf.plugin/ebpf_process.h +++ b/collectors/ebpf.plugin/ebpf_process.h @@ -39,7 +39,7 @@ #define NETDATA_SYSTEMD_PROCESS_EXIT_CONTEXT "services.task_exit" #define NETDATA_SYSTEMD_PROCESS_ERROR_CONTEXT "services.task_error" -#define NETDATA_EBPF_CGROUP_UPDATE 10 +#define NETDATA_EBPF_CGROUP_UPDATE 30 // Statistical information enum netdata_ebpf_thread_stats{ diff --git a/collectors/ebpf.plugin/ebpf_shm.c b/collectors/ebpf.plugin/ebpf_shm.c index f81287d82..4057eff7f 100644 --- a/collectors/ebpf.plugin/ebpf_shm.c +++ b/collectors/ebpf.plugin/ebpf_shm.c @@ -34,17 +34,6 @@ static ebpf_local_maps_t shm_maps[] = {{.name = "tbl_pid_shm", .internal_input = .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}, {.name = NULL, .internal_input = 0, .user_input = 0}}; -struct netdata_static_thread shm_threads = { - .name = "SHM KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - netdata_ebpf_targets_t shm_targets[] = { {.name = "shmget", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "shmat", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "shmdt", .mode = EBPF_LOAD_TRAMPOLINE}, @@ -246,7 +235,7 @@ static void ebpf_shm_adjust_map_size(struct shm_bpf *obj, ebpf_module_t *em) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_shm_load_and_attach(struct shm_bpf *obj, ebpf_module_t *em) { @@ -299,11 +288,7 @@ static inline int ebpf_shm_load_and_attach(struct shm_bpf *obj, ebpf_module_t *e static void ebpf_shm_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_cleanup_publish_syscall(shm_publish_aggregated); @@ -316,7 +301,9 @@ static void ebpf_shm_free(ebpf_module_t *em) shm_bpf__destroy(bpf_obj); #endif + pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; + pthread_mutex_unlock(&ebpf_exit_cleanup); } /** @@ -329,20 +316,6 @@ static void ebpf_shm_free(ebpf_module_t *em) static void ebpf_shm_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*shm_threads.thread); - ebpf_shm_free(em); -} - -/** - * SHM Cleanup - * - * Clean up allocated memory. - * - * @param ptr thread data. - */ -static void ebpf_shm_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_shm_free(em); } @@ -491,7 +464,7 @@ static void shm_send_global() * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_shm_read_global_table() { netdata_idx_t *stored = shm_values; netdata_idx_t *val = shm_hash_values; @@ -512,30 +485,6 @@ static void read_global_table() } /** - * Shared memory reader thread. - * - * @param ptr It is a NULL value for this thread. - * @return It always returns NULL. - */ -void *ebpf_shm_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_shm_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - usec_t step = NETDATA_SHM_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Sum values for all targets. */ static void ebpf_shm_sum_pids(netdata_publish_shm_t *shm, struct pid_on_target *root) @@ -894,28 +843,19 @@ void ebpf_shm_send_cgroup_data(int update_every) */ static void shm_collector(ebpf_module_t *em) { - shm_threads.thread = mallocz(sizeof(netdata_thread_t)); - shm_threads.start_routine = ebpf_shm_read_hash; - - netdata_thread_create( - shm_threads.thread, - shm_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, - ebpf_shm_read_hash, - em - ); - int cgroups = em->cgroup_charts; int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_shm_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) { read_apps_table(); diff --git a/collectors/ebpf.plugin/ebpf_shm.h b/collectors/ebpf.plugin/ebpf_shm.h index 4e068819b..b06a4a5d1 100644 --- a/collectors/ebpf.plugin/ebpf_shm.h +++ b/collectors/ebpf.plugin/ebpf_shm.h @@ -6,8 +6,6 @@ // Module name #define NETDATA_EBPF_MODULE_NAME_SHM "shm" -#define NETDATA_SHM_SLEEP_MS 850000ULL - // charts #define NETDATA_SHM_GLOBAL_CHART "shared_memory_calls" #define NETDATA_SHMGET_CHART "shmget_call" diff --git a/collectors/ebpf.plugin/ebpf_socket.c b/collectors/ebpf.plugin/ebpf_socket.c index 3a023e4a4..1954be714 100644 --- a/collectors/ebpf.plugin/ebpf_socket.c +++ b/collectors/ebpf.plugin/ebpf_socket.c @@ -62,8 +62,6 @@ ebpf_socket_publish_apps_t **socket_bandwidth_curr = NULL; static ebpf_bandwidth_t *bandwidth_vector = NULL; pthread_mutex_t nv_mutex; -int wait_to_plot = 0; - netdata_vector_plot_t inbound_vectors = { .plot = NULL, .next = 0, .last = 0 }; netdata_vector_plot_t outbound_vectors = { .plot = NULL, .next = 0, .last = 0 }; netdata_socket_t *socket_values; @@ -389,7 +387,7 @@ static void ebpf_socket_adjust_map_size(struct socket_bpf *obj, ebpf_module_t *e * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_socket_load_and_attach(struct socket_bpf *obj, ebpf_module_t *em) { @@ -459,6 +457,9 @@ static inline void clean_internal_socket_plot(netdata_socket_plot_t *ptr) */ static void clean_allocated_socket_plot() { + if (!network_viewer_opt.enabled) + return; + uint32_t i; uint32_t end = inbound_vectors.last; netdata_socket_plot_t *plot = inbound_vectors.plot; @@ -647,7 +648,8 @@ static void ebpf_socket_free(ebpf_module_t *em ) static void ebpf_socket_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*socket_threads.thread); + if (socket_threads.thread) + netdata_thread_cancel(*socket_threads.thread); ebpf_socket_free(em); } @@ -724,7 +726,7 @@ static void ebpf_update_global_publish( */ static inline void update_nv_plot_data(netdata_plot_values_t *plot, netdata_socket_t *sock) { - if (sock->ct > plot->last_time) { + if (sock->ct != plot->last_time) { plot->last_time = sock->ct; plot->plot_recv_packets = sock->recv_packets; plot->plot_sent_packets = sock->sent_packets; @@ -747,6 +749,7 @@ static inline void update_nv_plot_data(netdata_plot_values_t *plot, netdata_sock */ static inline void calculate_nv_plot() { + pthread_mutex_lock(&nv_mutex); uint32_t i; uint32_t end = inbound_vectors.next; for (i = 0; i < end; i++) { @@ -764,9 +767,12 @@ static inline void calculate_nv_plot() } outbound_vectors.max_plot = end; + /* // The 'Other' dimension is always calculated for the chart to have at least one dimension update_nv_plot_data(&outbound_vectors.plot[outbound_vectors.last].plot, &outbound_vectors.plot[outbound_vectors.last].sock); + */ + pthread_mutex_unlock(&nv_mutex); } /** @@ -1441,17 +1447,17 @@ static void ebpf_socket_create_nv_charts(netdata_vector_plot_t *ptr, int update_ * * @return It returns 1 if the IP is inside the range and 0 otherwise */ -static int is_specific_ip_inside_range(union netdata_ip_t *cmp, int family) +static int ebpf_is_specific_ip_inside_range(union netdata_ip_t *cmp, int family) { if (!network_viewer_opt.excluded_ips && !network_viewer_opt.included_ips) return 1; - uint32_t ipv4_test = ntohl(cmp->addr32[0]); + uint32_t ipv4_test = htonl(cmp->addr32[0]); ebpf_network_viewer_ip_list_t *move = network_viewer_opt.excluded_ips; while (move) { if (family == AF_INET) { - if (ntohl(move->first.addr32[0]) <= ipv4_test && - ipv4_test <= ntohl(move->last.addr32[0]) ) + if (move->first.addr32[0] <= ipv4_test && + ipv4_test <= move->last.addr32[0]) return 0; } else { if (memcmp(move->first.addr8, cmp->addr8, sizeof(union netdata_ip_t)) <= 0 && @@ -1464,12 +1470,13 @@ static int is_specific_ip_inside_range(union netdata_ip_t *cmp, int family) move = network_viewer_opt.included_ips; while (move) { - if (family == AF_INET) { - if (ntohl(move->first.addr32[0]) <= ipv4_test && - ntohl(move->last.addr32[0]) >= ipv4_test) + if (family == AF_INET && move->ver == AF_INET) { + if (move->first.addr32[0] <= ipv4_test && + move->last.addr32[0] >= ipv4_test) return 1; } else { - if (memcmp(move->first.addr8, cmp->addr8, sizeof(union netdata_ip_t)) <= 0 && + if (move->ver == AF_INET6 && + memcmp(move->first.addr8, cmp->addr8, sizeof(union netdata_ip_t)) <= 0 && memcmp(move->last.addr8, cmp->addr8, sizeof(union netdata_ip_t)) >= 0) { return 1; } @@ -1565,7 +1572,7 @@ int is_socket_allowed(netdata_socket_idx_t *key, int family) if (!is_port_inside_range(key->dport)) return 0; - return is_specific_ip_inside_range(&key->daddr, family); + return ebpf_is_specific_ip_inside_range(&key->daddr, family); } /** @@ -1580,38 +1587,26 @@ int is_socket_allowed(netdata_socket_idx_t *key, int family) * * @return It returns 0 case the values are equal, 1 case a is bigger than b and -1 case a is smaller than b. */ -static int compare_sockets(void *a, void *b) +static int ebpf_compare_sockets(void *a, void *b) { struct netdata_socket_plot *val1 = a; struct netdata_socket_plot *val2 = b; - int cmp; + int cmp = 0; // We do not need to compare val2 family, because data inside hash table is always from the same family if (val1->family == AF_INET) { //IPV4 - if (val1->flags & NETDATA_INBOUND_DIRECTION) { - if (val1->index.sport == val2->index.sport) - cmp = 0; - else { - cmp = (val1->index.sport > val2->index.sport)?1:-1; - } - } else { + if (network_viewer_opt.included_port || network_viewer_opt.excluded_port) cmp = memcmp(&val1->index.dport, &val2->index.dport, sizeof(uint16_t)); - if (!cmp) { - cmp = memcmp(&val1->index.daddr.addr32[0], &val2->index.daddr.addr32[0], sizeof(uint32_t)); - } + + if (!cmp) { + cmp = memcmp(&val1->index.daddr.addr32[0], &val2->index.daddr.addr32[0], sizeof(uint32_t)); } } else { - if (val1->flags & NETDATA_INBOUND_DIRECTION) { - if (val1->index.sport == val2->index.sport) - cmp = 0; - else { - cmp = (val1->index.sport > val2->index.sport)?1:-1; - } - } else { + if (network_viewer_opt.included_port || network_viewer_opt.excluded_port) cmp = memcmp(&val1->index.dport, &val2->index.dport, sizeof(uint16_t)); - if (!cmp) { - cmp = memcmp(&val1->index.daddr.addr32, &val2->index.daddr.addr32, 4*sizeof(uint32_t)); - } + + if (!cmp) { + cmp = memcmp(&val1->index.daddr.addr32, &val2->index.daddr.addr32, 4*sizeof(uint32_t)); } } @@ -1631,12 +1626,15 @@ static int compare_sockets(void *a, void *b) * * @return it returns the size of the data copied on success and -1 otherwise. */ -static inline int build_outbound_dimension_name(char *dimname, char *hostname, char *service_name, - char *proto, int family) +static inline int ebpf_build_outbound_dimension_name(char *dimname, char *hostname, char *service_name, + char *proto, int family) { - return snprintf(dimname, CONFIG_MAX_NAME - 7, (family == AF_INET)?"%s:%s:%s_":"%s:%s:[%s]_", - service_name, proto, - hostname); + if (network_viewer_opt.included_port || network_viewer_opt.excluded_port) + return snprintf(dimname, CONFIG_MAX_NAME - 7, (family == AF_INET)?"%s:%s:%s_":"%s:%s:[%s]_", + service_name, proto, hostname); + + return snprintf(dimname, CONFIG_MAX_NAME - 7, (family == AF_INET)?"%s:%s_":"%s:[%s]_", + proto, hostname); } /** @@ -1692,7 +1690,7 @@ static inline void fill_resolved_name(netdata_socket_plot_t *ptr, char *hostname } if (is_outbound) - size = build_outbound_dimension_name(dimname, hostname, service_name, protocol, ptr->family); + size = ebpf_build_outbound_dimension_name(dimname, hostname, service_name, protocol, ptr->family); else size = build_inbound_dimension_name(dimname,service_name, protocol); @@ -1850,14 +1848,12 @@ static void fill_last_nv_dimension(netdata_socket_plot_t *ptr, int is_outbound) */ static inline void update_socket_data(netdata_socket_t *sock, netdata_socket_t *lvalues) { - sock->recv_packets += lvalues->recv_packets; - sock->sent_packets += lvalues->sent_packets; - sock->recv_bytes += lvalues->recv_bytes; - sock->sent_bytes += lvalues->sent_bytes; - sock->retransmit += lvalues->retransmit; - - if (lvalues->ct > sock->ct) - sock->ct = lvalues->ct; + sock->recv_packets = lvalues->recv_packets; + sock->sent_packets = lvalues->sent_packets; + sock->recv_bytes = lvalues->recv_bytes; + sock->sent_bytes = lvalues->sent_bytes; + sock->retransmit = lvalues->retransmit; + sock->ct = lvalues->ct; } /** @@ -1881,7 +1877,7 @@ static void store_socket_inside_avl(netdata_vector_plot_t *out, netdata_socket_t ret = (netdata_socket_plot_t *) avl_search_lock(&out->tree, (avl_t *)&test); if (ret) { - if (lvalues->ct > ret->plot.last_time) { + if (lvalues->ct != ret->plot.last_time) { update_socket_data(&ret->sock, lvalues); } } else { @@ -1892,7 +1888,7 @@ static void store_socket_inside_avl(netdata_vector_plot_t *out, netdata_socket_t int resolved; if (curr == last) { - if (lvalues->ct > w->plot.last_time) { + if (lvalues->ct != w->plot.last_time) { update_socket_data(&w->sock, lvalues); } return; @@ -1977,6 +1973,9 @@ netdata_vector_plot_t * select_vector_to_store(uint32_t *direction, netdata_sock */ static void hash_accumulator(netdata_socket_t *values, netdata_socket_idx_t *key, int family, int end) { + if (!network_viewer_opt.enabled || !is_socket_allowed(key, family)) + return; + uint64_t bsent = 0, brecv = 0, psent = 0, precv = 0; uint16_t retransmit = 0; int i; @@ -1994,7 +1993,7 @@ static void hash_accumulator(netdata_socket_t *values, netdata_socket_idx_t *key if (!protocol) protocol = w->protocol; - if (w->ct > ct) + if (w->ct != ct) ct = w->ct; } @@ -2006,11 +2005,9 @@ static void hash_accumulator(netdata_socket_t *values, netdata_socket_idx_t *key values[0].protocol = (!protocol)?IPPROTO_TCP:protocol; values[0].ct = ct; - if (is_socket_allowed(key, family)) { - uint32_t dir; - netdata_vector_plot_t *table = select_vector_to_store(&dir, key, protocol); - store_socket_inside_avl(table, &values[0], key, family, dir); - } + uint32_t dir; + netdata_vector_plot_t *table = select_vector_to_store(&dir, key, protocol); + store_socket_inside_avl(table, &values[0], key, family, dir); } /** @@ -2018,16 +2015,13 @@ static void hash_accumulator(netdata_socket_t *values, netdata_socket_idx_t *key * * Read data from hash tables created on kernel ring. * - * @param fd the hash table with data. - * @param family the family associated to the hash table + * @param fd the hash table with data. + * @param family the family associated to the hash table * * @return it returns 0 on success and -1 otherwise. */ -static void read_socket_hash_table(int fd, int family, int network_connection) +static void ebpf_read_socket_hash_table(int fd, int family) { - if (wait_to_plot) - return; - netdata_socket_idx_t key = {}; netdata_socket_idx_t next_key = {}; @@ -2046,9 +2040,7 @@ static void read_socket_hash_table(int fd, int family, int network_connection) continue; } - if (network_connection) { - hash_accumulator(values, &key, family, end); - } + hash_accumulator(values, &key, family, end); key = next_key; } @@ -2057,12 +2049,12 @@ static void read_socket_hash_table(int fd, int family, int network_connection) /** * Fill Network Viewer Port list * - * Fill the strcture with values read from /proc or hash table. + * Fill the structure with values read from /proc or hash table. * * @param out the structure where we will store data. * @param value the ports we are listen to. * @param proto the protocol used for this connection. - * @param in the strcuture with values read form different sources. + * @param in the structure with values read form different sources. */ static inline void fill_nv_port_list(ebpf_network_viewer_port_list_t *out, uint16_t value, uint16_t proto, netdata_passive_connection_t *in) @@ -2081,7 +2073,7 @@ static inline void fill_nv_port_list(ebpf_network_viewer_port_list_t *out, uint1 * * @param value the ports we are listen to. * @param proto the protocol used with port connection. - * @param in the strcuture with values read form different sources. + * @param in the structure with values read form different sources. */ void update_listen_table(uint16_t value, uint16_t proto, netdata_passive_connection_t *in) { @@ -2159,22 +2151,19 @@ static void read_listen_table() void *ebpf_socket_read_hash(void *ptr) { netdata_thread_cleanup_push(ebpf_socket_cleanup, ptr); - ebpf_module_t *em = (ebpf_module_t *)ptr; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = NETDATA_SOCKET_READ_SLEEP_MS * em->update_every; int fd_ipv4 = socket_maps[NETDATA_SOCKET_TABLE_IPV4].map_fd; int fd_ipv6 = socket_maps[NETDATA_SOCKET_TABLE_IPV6].map_fd; - int network_connection = em->optional; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin) + continue; pthread_mutex_lock(&nv_mutex); - read_listen_table(); - read_socket_hash_table(fd_ipv4, AF_INET, network_connection); - read_socket_hash_table(fd_ipv6, AF_INET6, network_connection); - wait_to_plot = 1; + ebpf_read_socket_hash_table(fd_ipv4, AF_INET); + ebpf_read_socket_hash_table(fd_ipv6, AF_INET6); pthread_mutex_unlock(&nv_mutex); } @@ -2863,44 +2852,50 @@ static void ebpf_socket_send_cgroup_data(int update_every) /** * Main loop for this collector. * - * @param step the number of microseconds used with heart beat * @param em the structure with thread information */ -static void socket_collector(usec_t step, ebpf_module_t *em) +static void socket_collector(ebpf_module_t *em) { heartbeat_t hb; heartbeat_init(&hb); + uint32_t network_connection = network_viewer_opt.enabled; - socket_threads.thread = mallocz(sizeof(netdata_thread_t)); - socket_threads.start_routine = ebpf_socket_read_hash; + if (network_connection) { + socket_threads.thread = mallocz(sizeof(netdata_thread_t)); + socket_threads.start_routine = ebpf_socket_read_hash; - netdata_thread_create(socket_threads.thread, socket_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, ebpf_socket_read_hash, em); + netdata_thread_create(socket_threads.thread, socket_threads.name, + NETDATA_THREAD_OPTION_DEFAULT, ebpf_socket_read_hash, em); + } int cgroups = em->cgroup_charts; if (cgroups) ebpf_socket_update_cgroup_algorithm(); int socket_global_enabled = em->global_charts; - int network_connection = em->optional; int update_every = em->update_every; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t socket_apps_enabled = em->apps_charts; - pthread_mutex_lock(&collect_data_mutex); - if (socket_global_enabled) + if (socket_global_enabled) { + read_listen_table(); read_hash_global_tables(); + } + pthread_mutex_lock(&collect_data_mutex); if (socket_apps_enabled) ebpf_socket_update_apps_data(); if (cgroups) ebpf_update_socket_cgroup(); - calculate_nv_plot(); + if (network_connection) + calculate_nv_plot(); pthread_mutex_lock(&lock); if (socket_global_enabled) @@ -2925,7 +2920,6 @@ static void socket_collector(usec_t step, ebpf_module_t *em) ebpf_socket_create_nv_charts(&outbound_vectors, update_every); fflush(stdout); ebpf_socket_send_nv_data(&outbound_vectors); - wait_to_plot = 0; pthread_mutex_unlock(&nv_mutex); } @@ -2959,8 +2953,10 @@ static void ebpf_socket_allocate_global_vectors(int apps) bandwidth_vector = callocz((size_t)ebpf_nprocs, sizeof(ebpf_bandwidth_t)); socket_values = callocz((size_t)ebpf_nprocs, sizeof(netdata_socket_t)); - inbound_vectors.plot = callocz(network_viewer_opt.max_dim, sizeof(netdata_socket_plot_t)); - outbound_vectors.plot = callocz(network_viewer_opt.max_dim, sizeof(netdata_socket_plot_t)); + if (network_viewer_opt.enabled) { + inbound_vectors.plot = callocz(network_viewer_opt.max_dim, sizeof(netdata_socket_plot_t)); + outbound_vectors.plot = callocz(network_viewer_opt.max_dim, sizeof(netdata_socket_plot_t)); + } } /** @@ -3219,12 +3215,11 @@ static void get_ipv6_first_addr(union netdata_ip_t *out, union netdata_ip_t *in, * * @return It returns 1 if the IP is inside the range and 0 otherwise */ -static int is_ip_inside_range(union netdata_ip_t *rfirst, union netdata_ip_t *rlast, - union netdata_ip_t *cmpfirst, union netdata_ip_t *cmplast, int family) +static int ebpf_is_ip_inside_range(union netdata_ip_t *rfirst, union netdata_ip_t *rlast, + union netdata_ip_t *cmpfirst, union netdata_ip_t *cmplast, int family) { if (family == AF_INET) { - if (ntohl(rfirst->addr32[0]) <= ntohl(cmpfirst->addr32[0]) && - ntohl(rlast->addr32[0]) >= ntohl(cmplast->addr32[0])) + if ((rfirst->addr32[0] <= cmpfirst->addr32[0]) && (rlast->addr32[0] >= cmplast->addr32[0])) return 1; } else { if (memcmp(rfirst->addr8, cmpfirst->addr8, sizeof(union netdata_ip_t)) <= 0 && @@ -3241,16 +3236,22 @@ static int is_ip_inside_range(union netdata_ip_t *rfirst, union netdata_ip_t *rl * * @param out a pointer to the link list. * @param in the structure that will be linked. + * @param table the modified table. */ -void fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_list_t *in, char *table) +void ebpf_fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_list_t *in, char *table) { #ifndef NETDATA_INTERNAL_CHECKS UNUSED(table); #endif + if (in->ver == AF_INET) { // It is simpler to compare using host order + in->first.addr32[0] = ntohl(in->first.addr32[0]); + in->last.addr32[0] = ntohl(in->last.addr32[0]); + } if (likely(*out)) { ebpf_network_viewer_ip_list_t *move = *out, *store = *out; while (move) { - if (in->ver == move->ver && is_ip_inside_range(&move->first, &move->last, &in->first, &in->last, in->ver)) { + if (in->ver == move->ver && + ebpf_is_ip_inside_range(&move->first, &move->last, &in->first, &in->last, in->ver)) { info("The range/value (%s) is inside the range/value (%s) already inserted, it will be ignored.", in->value, move->value); freez(in->value); @@ -3267,14 +3268,12 @@ void fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_li } #ifdef NETDATA_INTERNAL_CHECKS - char first[512], last[512]; + char first[256], last[512]; if (in->ver == AF_INET) { - if (inet_ntop(AF_INET, in->first.addr8, first, INET_ADDRSTRLEN) && - inet_ntop(AF_INET, in->last.addr8, last, INET_ADDRSTRLEN)) - info("Adding values %s - %s to %s IP list \"%s\" used on network viewer", - first, last, - (*out == network_viewer_opt.included_ips)?"included":"excluded", - table); + info("Adding values %s: (%u - %u) to %s IP list \"%s\" used on network viewer", + in->value, in->first.addr32[0], in->last.addr32[0], + (*out == network_viewer_opt.included_ips)?"included":"excluded", + table); } else { if (inet_ntop(AF_INET6, in->first.addr8, first, INET6_ADDRSTRLEN) && inet_ntop(AF_INET6, in->last.addr8, last, INET6_ADDRSTRLEN)) @@ -3294,7 +3293,7 @@ void fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_li * @param out a pointer to store the link list * @param ip the value given as parameter */ -static void parse_ip_list(void **out, char *ip) +static void ebpf_parse_ip_list(void **out, char *ip) { ebpf_network_viewer_ip_list_t **list = (ebpf_network_viewer_ip_list_t **)out; @@ -3442,7 +3441,7 @@ static void parse_ip_list(void **out, char *ip) ebpf_network_viewer_ip_list_t *store; - storethisip: +storethisip: store = callocz(1, sizeof(ebpf_network_viewer_ip_list_t)); store->value = ipdup; store->hash = simple_hash(ipdup); @@ -3450,7 +3449,7 @@ static void parse_ip_list(void **out, char *ip) memcpy(store->first.addr8, first.addr8, sizeof(first.addr8)); memcpy(store->last.addr8, last.addr8, sizeof(last.addr8)); - fill_ip_list(list, store, "socket"); + ebpf_fill_ip_list(list, store, "socket"); return; cleanipdup: @@ -3464,7 +3463,7 @@ cleanipdup: * * @param ptr is a pointer with the text to parse. */ -static void parse_ips(char *ptr) +static void ebpf_parse_ips(char *ptr) { // No value if (unlikely(!ptr)) @@ -3491,8 +3490,9 @@ static void parse_ips(char *ptr) } if (isascii(*ptr)) { // Parse port - parse_ip_list((!neg)?(void **)&network_viewer_opt.included_ips:(void **)&network_viewer_opt.excluded_ips, - ptr); + ebpf_parse_ip_list((!neg)?(void **)&network_viewer_opt.included_ips: + (void **)&network_viewer_opt.excluded_ips, + ptr); } ptr = end; @@ -3762,7 +3762,7 @@ void parse_network_viewer_section(struct config *cfg) value = appconfig_get(cfg, EBPF_NETWORK_VIEWER_SECTION, "ips", "!127.0.0.1/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 !::1/128"); - parse_ips(value); + ebpf_parse_ips(value); } /** @@ -3916,16 +3916,9 @@ void *ebpf_socket_thread(void *ptr) { netdata_thread_cleanup_push(ebpf_socket_exit, ptr); - memset(&inbound_vectors.tree, 0, sizeof(avl_tree_lock)); - memset(&outbound_vectors.tree, 0, sizeof(avl_tree_lock)); - avl_init_lock(&inbound_vectors.tree, compare_sockets); - avl_init_lock(&outbound_vectors.tree, compare_sockets); - ebpf_module_t *em = (ebpf_module_t *)ptr; em->maps = socket_maps; - parse_network_viewer_section(&socket_config); - parse_service_name_section(&socket_config); parse_table_size_options(&socket_config); if (pthread_mutex_init(&nv_mutex, NULL)) { @@ -3935,7 +3928,15 @@ void *ebpf_socket_thread(void *ptr) } ebpf_socket_allocate_global_vectors(em->apps_charts); - initialize_inbound_outbound(); + + if (network_viewer_opt.enabled) { + memset(&inbound_vectors.tree, 0, sizeof(avl_tree_lock)); + memset(&outbound_vectors.tree, 0, sizeof(avl_tree_lock)); + avl_init_lock(&inbound_vectors.tree, ebpf_compare_sockets); + avl_init_lock(&outbound_vectors.tree, ebpf_compare_sockets); + + initialize_inbound_outbound(); + } if (running_on_kernel < NETDATA_EBPF_KERNEL_5_0) em->mode = MODE_ENTRY; @@ -3966,7 +3967,7 @@ void *ebpf_socket_thread(void *ptr) pthread_mutex_unlock(&lock); - socket_collector((usec_t)(em->update_every * USEC_PER_SEC), em); + socket_collector(em); endsocket: ebpf_update_disabled_plugin_stats(em); diff --git a/collectors/ebpf.plugin/ebpf_socket.h b/collectors/ebpf.plugin/ebpf_socket.h index ca6b193f0..63b1e107b 100644 --- a/collectors/ebpf.plugin/ebpf_socket.h +++ b/collectors/ebpf.plugin/ebpf_socket.h @@ -244,6 +244,7 @@ typedef struct ebpf_network_viewer_hostname_list { #define NETDATA_NV_CAP_VALUE 50L typedef struct ebpf_network_viewer_options { + uint32_t enabled; uint32_t max_dim; // Store value read from 'maximum dimensions' uint32_t hostname_resolution_enabled; @@ -360,7 +361,7 @@ void clean_port_structure(ebpf_network_viewer_port_list_t **clean); extern ebpf_network_viewer_port_list_t *listen_ports; void update_listen_table(uint16_t value, uint16_t proto, netdata_passive_connection_t *values); void parse_network_viewer_section(struct config *cfg); -void fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_list_t *in, char *table); +void ebpf_fill_ip_list(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_list_t *in, char *table); void parse_service_name_section(struct config *cfg); extern ebpf_socket_publish_apps_t **socket_bandwidth_curr; diff --git a/collectors/ebpf.plugin/ebpf_softirq.c b/collectors/ebpf.plugin/ebpf_softirq.c index 3b5d15921..49e9c3051 100644 --- a/collectors/ebpf.plugin/ebpf_softirq.c +++ b/collectors/ebpf.plugin/ebpf_softirq.c @@ -54,17 +54,6 @@ static softirq_val_t softirq_vals[] = { // tmp store for soft IRQ values we get from a per-CPU eBPF map. static softirq_ebpf_val_t *softirq_ebpf_vals = NULL; -static struct netdata_static_thread softirq_threads = { - .name = "SOFTIRQ KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - /** * Cachestat Free * @@ -75,40 +64,20 @@ static struct netdata_static_thread softirq_threads = { static void ebpf_softirq_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); - freez(softirq_threads.thread); - for (int i = 0; softirq_tracepoints[i].class != NULL; i++) { ebpf_disable_tracepoint(&softirq_tracepoints[i]); } freez(softirq_ebpf_vals); pthread_mutex_lock(&ebpf_exit_cleanup); - em->thread->enabled = NETDATA_MAIN_THREAD_EXITED; + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; pthread_mutex_unlock(&ebpf_exit_cleanup); } /** - * Exit - * - * Cancel thread. - * - * @param ptr thread data. - */ -static void softirq_exit(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*softirq_threads.thread); - ebpf_softirq_free(em); -} - -/** * Cleanup * * Clean up allocated memory. @@ -146,28 +115,6 @@ static void softirq_read_latency_map() } } -/** - * Read eBPF maps for soft IRQ. - */ -static void *softirq_reader(void *ptr) -{ - netdata_thread_cleanup_push(softirq_exit, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_SOFTIRQ_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - softirq_read_latency_map(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - static void softirq_create_charts(int update_every) { ebpf_create_chart( @@ -212,17 +159,6 @@ static void softirq_collector(ebpf_module_t *em) { softirq_ebpf_vals = callocz(ebpf_nprocs, sizeof(softirq_ebpf_val_t)); - // create reader thread. - softirq_threads.thread = mallocz(sizeof(netdata_thread_t)); - softirq_threads.start_routine = softirq_reader; - netdata_thread_create( - softirq_threads.thread, - softirq_threads.name, - NETDATA_THREAD_OPTION_DEFAULT, - softirq_reader, - em - ); - // create chart and static dims. pthread_mutex_lock(&lock); softirq_create_charts(em->update_every); @@ -233,13 +169,16 @@ static void softirq_collector(ebpf_module_t *em) // loop and read from published data until ebpf plugin is closed. heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; //This will be cancelled by its parent while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + softirq_read_latency_map(); pthread_mutex_lock(&lock); // write dims now for all hitherto discovered IRQs. diff --git a/collectors/ebpf.plugin/ebpf_softirq.h b/collectors/ebpf.plugin/ebpf_softirq.h index 7dcddbb49..eea2a1841 100644 --- a/collectors/ebpf.plugin/ebpf_softirq.h +++ b/collectors/ebpf.plugin/ebpf_softirq.h @@ -20,7 +20,6 @@ typedef struct softirq_ebpf_val { *****************************************************************/ #define NETDATA_EBPF_MODULE_NAME_SOFTIRQ "softirq" -#define NETDATA_SOFTIRQ_SLEEP_MS 650000ULL #define NETDATA_SOFTIRQ_CONFIG_FILE "softirq.conf" typedef struct sofirq_val { diff --git a/collectors/ebpf.plugin/ebpf_swap.c b/collectors/ebpf.plugin/ebpf_swap.c index 8199573a9..059efb63b 100644 --- a/collectors/ebpf.plugin/ebpf_swap.c +++ b/collectors/ebpf.plugin/ebpf_swap.c @@ -34,17 +34,6 @@ static ebpf_local_maps_t swap_maps[] = {{.name = "tbl_pid_swap", .internal_input .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}, {.name = NULL, .internal_input = 0, .user_input = 0}}; -struct netdata_static_thread swap_threads = { - .name = "SWAP KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - netdata_ebpf_targets_t swap_targets[] = { {.name = "swap_readpage", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "swap_writepage", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = NULL, .mode = EBPF_LOAD_TRAMPOLINE}}; @@ -184,7 +173,7 @@ static void ebpf_swap_disable_release_task(struct swap_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_swap_load_and_attach(struct swap_bpf *obj, ebpf_module_t *em) { @@ -236,18 +225,13 @@ static inline int ebpf_swap_load_and_attach(struct swap_bpf *obj, ebpf_module_t static void ebpf_swap_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); ebpf_cleanup_publish_syscall(swap_publish_aggregated); freez(swap_vector); freez(swap_values); - freez(swap_threads.thread); #ifdef LIBBPF_MAJOR_VERSION if (bpf_obj) @@ -268,20 +252,6 @@ static void ebpf_swap_free(ebpf_module_t *em) static void ebpf_swap_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*swap_threads.thread); - ebpf_swap_free(em); -} - -/** - * Swap cleanup - * - * Clean up allocated memory. - * - * @param ptr thread data. - */ -static void ebpf_swap_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_swap_free(em); } @@ -412,7 +382,7 @@ static void swap_send_global() * * Read the table with number of calls to all functions */ -static void read_global_table() +static void ebpf_swap_read_global_table() { netdata_idx_t *stored = swap_values; netdata_idx_t *val = swap_hash_values; @@ -433,33 +403,6 @@ static void read_global_table() } /** - * Swap read hash - * - * This is the thread callback. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_swap_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_swap_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - usec_t step = NETDATA_SWAP_SLEEP_MS * em->update_every; - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Sum PIDs * * Sum values for all targets. @@ -714,23 +657,19 @@ void ebpf_swap_send_cgroup_data(int update_every) */ static void swap_collector(ebpf_module_t *em) { - swap_threads.thread = mallocz(sizeof(netdata_thread_t)); - swap_threads.start_routine = ebpf_swap_read_hash; - - netdata_thread_create(swap_threads.thread, swap_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_swap_read_hash, em); - int cgroup = em->cgroup_charts; int update_every = em->update_every; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = update_every * USEC_PER_SEC; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_swap_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) read_apps_table(); diff --git a/collectors/ebpf.plugin/ebpf_sync.c b/collectors/ebpf.plugin/ebpf_sync.c index 840497533..7c81c1df3 100644 --- a/collectors/ebpf.plugin/ebpf_sync.c +++ b/collectors/ebpf.plugin/ebpf_sync.c @@ -10,17 +10,6 @@ static netdata_publish_syscall_t sync_counter_publish_aggregated[NETDATA_SYNC_ID static netdata_idx_t sync_hash_values[NETDATA_SYNC_IDX_END]; -struct netdata_static_thread sync_threads = { - .name = "SYNC KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - static ebpf_local_maps_t sync_maps[] = {{.name = "tbl_sync", .internal_input = NETDATA_SYNC_END, .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC, .map_fd = ND_EBPF_MAP_FD_NOT_INITIALIZED}, @@ -77,7 +66,7 @@ static inline void ebpf_sync_disable_probe(struct sync_bpf *obj) } /** - * Disable tramppoline + * Disable trampoline * * Disable trampoline to use another method. * @@ -140,7 +129,7 @@ static void ebpf_sync_set_hash_tables(struct sync_bpf *obj, sync_syscalls_index_ * @param target the syscall that we are attaching a tracer. * @param idx the index for the main structure * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_sync_load_and_attach(struct sync_bpf *obj, ebpf_module_t *em, char *target, sync_syscalls_index_t idx) @@ -216,17 +205,12 @@ void ebpf_sync_cleanup_objects() static void ebpf_sync_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); #ifdef LIBBPF_MAJOR_VERSION ebpf_sync_cleanup_objects(); #endif - freez(sync_threads.thread); pthread_mutex_lock(&ebpf_exit_cleanup); em->thread->enabled = NETDATA_THREAD_EBPF_STOPPED; @@ -243,18 +227,6 @@ static void ebpf_sync_free(ebpf_module_t *em) static void ebpf_sync_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*sync_threads.thread); - ebpf_sync_free(em); -} - -/** - * Clean up the main thread. - * - * @param ptr thread data. - */ -static void ebpf_sync_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_sync_free(em); } @@ -350,7 +322,7 @@ static int ebpf_sync_initialize_syscall(ebpf_module_t *em) * * Read the table with number of calls for all functions */ -static void read_global_table() +static void ebpf_sync_read_global_table() { netdata_idx_t stored; uint32_t idx = NETDATA_SYNC_CALL; @@ -366,34 +338,6 @@ static void read_global_table() } /** - * Sync read hash - * - * This is the thread callback. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_sync_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_sync_cleanup, ptr); - ebpf_module_t *em = (ebpf_module_t *)ptr; - - heartbeat_t hb; - heartbeat_init(&hb); - usec_t step = NETDATA_EBPF_SYNC_SLEEP_MS * em->update_every; - - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Create Sync charts * * Create charts and dimensions according user input. @@ -452,20 +396,17 @@ static void sync_send_data() */ static void sync_collector(ebpf_module_t *em) { - sync_threads.thread = mallocz(sizeof(netdata_thread_t)); - sync_threads.start_routine = ebpf_sync_read_hash; - - netdata_thread_create(sync_threads.thread, sync_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_sync_read_hash, em); - heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; + ebpf_sync_read_global_table(); pthread_mutex_lock(&lock); sync_send_data(); diff --git a/collectors/ebpf.plugin/ebpf_vfs.c b/collectors/ebpf.plugin/ebpf_vfs.c index ad6de4a07..b3c0ba45d 100644 --- a/collectors/ebpf.plugin/ebpf_vfs.c +++ b/collectors/ebpf.plugin/ebpf_vfs.c @@ -34,17 +34,6 @@ struct config vfs_config = { .first_section = NULL, .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare }, .rwlock = AVL_LOCK_INITIALIZER } }; -struct netdata_static_thread vfs_threads = { - .name = "VFS KERNEL", - .config_section = NULL, - .config_name = NULL, - .env_name = NULL, - .enabled = 1, - .thread = NULL, - .init_routine = NULL, - .start_routine = NULL -}; - netdata_ebpf_targets_t vfs_targets[] = { {.name = "vfs_write", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "vfs_writev", .mode = EBPF_LOAD_TRAMPOLINE}, {.name = "vfs_read", .mode = EBPF_LOAD_TRAMPOLINE}, @@ -357,7 +346,7 @@ static void ebpf_vfs_disable_release_task(struct vfs_bpf *obj) * @param obj is the main structure for bpf objects. * @param em structure with configuration * - * @return it returns 0 on succes and -1 otherwise + * @return it returns 0 on success and -1 otherwise */ static inline int ebpf_vfs_load_and_attach(struct vfs_bpf *obj, ebpf_module_t *em) { @@ -409,16 +398,11 @@ static inline int ebpf_vfs_load_and_attach(struct vfs_bpf *obj, ebpf_module_t *e static void ebpf_vfs_free(ebpf_module_t *em) { pthread_mutex_lock(&ebpf_exit_cleanup); - if (em->thread->enabled == NETDATA_THREAD_EBPF_RUNNING) { - em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; - pthread_mutex_unlock(&ebpf_exit_cleanup); - return; - } + em->thread->enabled = NETDATA_THREAD_EBPF_STOPPING; pthread_mutex_unlock(&ebpf_exit_cleanup); freez(vfs_hash_values); freez(vfs_vector); - freez(vfs_threads.thread); #ifdef LIBBPF_MAJOR_VERSION if (bpf_obj) @@ -440,18 +424,6 @@ static void ebpf_vfs_free(ebpf_module_t *em) static void ebpf_vfs_exit(void *ptr) { ebpf_module_t *em = (ebpf_module_t *)ptr; - netdata_thread_cancel(*vfs_threads.thread); - ebpf_vfs_free(em); -} - -/** -* Clean up the main thread. -* -* @param ptr thread data. -**/ -static void ebpf_vfs_cleanup(void *ptr) -{ - ebpf_module_t *em = (ebpf_module_t *)ptr; ebpf_vfs_free(em); } @@ -518,7 +490,7 @@ static void ebpf_vfs_send_data(ebpf_module_t *em) /** * Read the hash table and store data to allocated vectors. */ -static void read_global_table() +static void ebpf_vfs_read_global_table() { uint64_t idx; netdata_idx_t res[NETDATA_VFS_COUNTER]; @@ -874,36 +846,6 @@ static void read_update_vfs_cgroup() } /** - * VFS read hash - * - * This is the thread callback. - * This thread is necessary, because we cannot freeze the whole plugin to read the data. - * - * @param ptr It is a NULL value for this thread. - * - * @return It always returns NULL. - */ -void *ebpf_vfs_read_hash(void *ptr) -{ - netdata_thread_cleanup_push(ebpf_vfs_cleanup, ptr); - heartbeat_t hb; - heartbeat_init(&hb); - - ebpf_module_t *em = (ebpf_module_t *)ptr; - - usec_t step = NETDATA_LATENCY_VFS_SLEEP_MS * em->update_every; - //This will be cancelled by its parent - while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - - read_global_table(); - } - - netdata_thread_cleanup_pop(1); - return NULL; -} - -/** * Sum PIDs * * Sum values for all targets. @@ -1525,22 +1467,19 @@ static void ebpf_vfs_send_cgroup_data(ebpf_module_t *em) */ static void vfs_collector(ebpf_module_t *em) { - vfs_threads.thread = mallocz(sizeof(netdata_thread_t)); - vfs_threads.start_routine = ebpf_vfs_read_hash; - - netdata_thread_create(vfs_threads.thread, vfs_threads.name, NETDATA_THREAD_OPTION_DEFAULT, - ebpf_vfs_read_hash, em); - int cgroups = em->cgroup_charts; heartbeat_t hb; heartbeat_init(&hb); - usec_t step = em->update_every * USEC_PER_SEC; + int update_every = em->update_every; + int counter = update_every - 1; while (!ebpf_exit_plugin) { - (void)heartbeat_next(&hb, step); - if (ebpf_exit_plugin) - break; + (void)heartbeat_next(&hb, USEC_PER_SEC); + if (ebpf_exit_plugin || ++counter != update_every) + continue; + counter = 0; netdata_apps_integration_flags_t apps = em->apps_charts; + ebpf_vfs_read_global_table(); pthread_mutex_lock(&collect_data_mutex); if (apps) ebpf_vfs_read_apps(); diff --git a/collectors/ebpf.plugin/ebpf_vfs.h b/collectors/ebpf.plugin/ebpf_vfs.h index 2e3c7cc29..d7fc2672f 100644 --- a/collectors/ebpf.plugin/ebpf_vfs.h +++ b/collectors/ebpf.plugin/ebpf_vfs.h @@ -8,8 +8,6 @@ #define NETDATA_DIRECTORY_VFS_CONFIG_FILE "vfs.conf" -#define NETDATA_LATENCY_VFS_SLEEP_MS 750000ULL - // Global chart name #define NETDATA_VFS_FILE_CLEAN_COUNT "vfs_deleted_objects" #define NETDATA_VFS_FILE_IO_COUNT "vfs_io" diff --git a/collectors/fping.plugin/Makefile.am b/collectors/fping.plugin/Makefile.am deleted file mode 100644 index 90654832b..000000000 --- a/collectors/fping.plugin/Makefile.am +++ /dev/null @@ -1,24 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-or-later - -AUTOMAKE_OPTIONS = subdir-objects -MAINTAINERCLEANFILES = $(srcdir)/Makefile.in - -CLEANFILES = \ - fping.plugin \ - $(NULL) - -include $(top_srcdir)/build/subst.inc -SUFFIXES = .in - -dist_plugins_SCRIPTS = \ - fping.plugin \ - $(NULL) - -dist_noinst_DATA = \ - fping.plugin.in \ - README.md \ - $(NULL) - -dist_libconfig_DATA = \ - fping.conf \ - $(NULL) diff --git a/collectors/fping.plugin/README.md b/collectors/fping.plugin/README.md deleted file mode 100644 index e32d3911b..000000000 --- a/collectors/fping.plugin/README.md +++ /dev/null @@ -1,110 +0,0 @@ -<!-- -title: "fping.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/fping.plugin/README.md ---> - -# fping.plugin - -The fping plugin supports monitoring latency, packet loss and uptime of any number of network end points, -by pinging them with `fping`. - -This plugin requires version 5.1 or newer of `fping` (earlier versions may or may not work). Our static builds and -Docker images come bundled with a known working version of `fping`. Native packages and local builds will need to -have a working version installed before the plugin is usable. - -## Installing fping locally - -If your distribution’s repositories do not include a working version of `fping`, the supplied plugin can install -it, by running: - -```sh -/usr/libexec/netdata/plugins.d/fping.plugin install -``` - -The above will download, build and install the right version as `/usr/local/bin/fping`. This requires a working C -compiler, GNU autotools (at least autoconf and automake), and GNU make. On Debian or Ubuntu, you can pull in most -of the required tools by installing the `build-essential` package (this should include everything except automake -and autoconf). - -## Configuration - -Then you need to edit `/etc/netdata/fping.conf` (to edit it on your system run -`/etc/netdata/edit-config fping.conf`) like this: - -```sh -# set here all the hosts you need to ping -# I suggest to use hostnames and put their IPs in /etc/hosts -hosts="host1 host2 host3" - -# override the chart update frequency - the default is inherited from Netdata -update_every=1 - -# time in milliseconds (1 sec = 1000 ms) to ping the hosts -# 200 = 5 pings per second -ping_every=200 - -# other fping options - these are the defaults -fping_opts="-R -b 56 -i 1 -r 0 -t 5000" -``` - -## alarms - -Netdata will automatically attach a few alarms for each host. -Check the [latest versions of the fping alarms](https://raw.githubusercontent.com/netdata/netdata/master/health/health.d/fping.conf) - -## Additional Tips - -### Customizing Amount of Pings Per Second - -For example, to update the chart every 10 seconds and use 2 pings every 10 seconds, use this: - -```sh -# Chart Update Frequency (Time in Seconds) -update_every=10 - -# Time in Milliseconds (1 sec = 1000 ms) to Ping the Hosts -# The Following Example Sends 1 Ping Every 5000 ms -# Calculation Formula: ping_every = (update_every * 1000 ) / 2 -ping_every=5000 -``` - -### Multiple fping Plugins With Different Settings - -You may need to run multiple fping plugins with different settings for different end points. -For example, you may need to ping a few hosts 10 times per second, and others once per second. - -Netdata allows you to add as many `fping` plugins as you like. - -Follow this procedure: - -**1. Create New fping Configuration File** - -```sh -# Step Into Configuration Directory -cd /etc/netdata - -# Copy Original fping Configuration File To New Configuration File -cp fping.conf fping2.conf -``` - -Edit `fping2.conf` and set the settings and the hosts you need for the seconds instance. - -**2. Soft Link Original fping Plugin to New Plugin File** - -```sh -# Become root (If The Step Step Is Performed As Non-Root User) -sudo su - -# Step Into The Plugins Directory -cd /usr/libexec/netdata/plugins.d - -# Link fping.plugin to fping2.plugin -ln -s fping.plugin fping2.plugin -``` - -That's it. Netdata will detect the new plugin and start it. - -You can name the new plugin any name you like. -Just make sure the plugin and the configuration file have the same name. - - diff --git a/collectors/fping.plugin/fping.conf b/collectors/fping.plugin/fping.conf deleted file mode 100644 index 63a7f7acd..000000000 --- a/collectors/fping.plugin/fping.conf +++ /dev/null @@ -1,44 +0,0 @@ -# no need for shebang - this file is sourced from fping.plugin - -# fping.plugin requires a recent version of fping. -# -# You can get it on your system, by running: -# -# /usr/libexec/netdata/plugins.d/fping.plugin install - -# ----------------------------------------------------------------------------- -# configuration options - -# The fping binary to use. We need one that can output netdata friendly info -# (supporting: -N). If you have multiple versions, put here the full filename -# of the right one - -#fping="/usr/local/bin/fping" - - -# a space separated list of hosts to fping -# we suggest to put names here and the IPs of these names in /etc/hosts - -hosts="" - - -# The update frequency of the chart - the default is inherited from netdata - -#update_every=2 - - -# The time in milliseconds (1 sec = 1000 ms) to ping the hosts -# by default 5 pings per host per iteration -# fping will not allow this to be below 20ms - -#ping_every="200" - - -# other fping options - defaults: -# -R = send packets with random data -# -b 56 = the number of bytes per packet -# -i 1 = 1 ms when sending packets to others hosts (switching hosts) -# -r 0 = never retry packets -# -t 5000 = per packet timeout at 5000 ms - -#fping_opts="-R -b 56 -i 1 -r 0 -t 5000" diff --git a/collectors/fping.plugin/fping.plugin.in b/collectors/fping.plugin/fping.plugin.in deleted file mode 100755 index 4b3d1d126..000000000 --- a/collectors/fping.plugin/fping.plugin.in +++ /dev/null @@ -1,202 +0,0 @@ -#!/usr/bin/env bash -# SPDX-License-Identifier: GPL-3.0-or-later - -# netdata -# real-time performance and health monitoring, done right! -# (C) 2017 Costa Tsaousis <costa@tsaousis.gr> -# GPL v3+ -# -# This plugin requires a latest version of fping. -# You can compile it from source, by running me with option: install - -export PATH="${PATH}:/sbin:/usr/sbin:/usr/local/sbin" -export LC_ALL=C - -if [ "${1}" = "install" ] - then - [ "${UID}" != 0 ] && echo >&2 "Please run me as root. This will install a single binary file: /usr/local/bin/fping." && exit 1 - - [ -z "${2}" ] && fping_version="5.1" || fping_version="${2}" - - run() { - printf >&2 " > " - printf >&2 "%q " "${@}" - printf >&2 "\n" - "${@}" || exit 1 - } - - download() { - local curl="$(which curl 2>/dev/null || command -v curl 2>/dev/null)" - [ ! -z "${curl}" ] && run curl -s -L "${1}" && return 0 - - local wget="$(which wget 2>/dev/null || command -v wget 2>/dev/null)" - [ ! -z "${wget}" ] && run wget -q -O - "${1}" && return 0 - - echo >&2 "Cannot find 'curl' or 'wget' in this system." && exit 1 - } - - [ ! -d /usr/src ] && run mkdir -p /usr/src - [ ! -d /usr/local/bin ] && run mkdir -p /usr/local/bin - - run cd /usr/src - - if [ -d "fping-${fping_version}" ] - then - run rm -rf "fping-${fping_version}" || exit 1 - fi - - download "https://github.com/schweikert/fping/releases/download/v${fping_version}/fping-${fping_version}.tar.gz" | run tar -zxvpf - - [ $? -ne 0 ] && exit 1 - run cd "fping-${fping_version}" || exit 1 - - run ./configure --prefix=/usr/local - run make clean - run make - if [ -f /usr/local/bin/fping ] - then - run mv -f /usr/local/bin/fping /usr/local/bin/fping.old - fi - run mv src/fping /usr/local/bin/fping - run chown root:root /usr/local/bin/fping - run chmod 4755 /usr/local/bin/fping - echo >&2 - echo >&2 "All done, you have a compatible fping now at /usr/local/bin/fping." - echo >&2 - - fping="$(which fping 2>/dev/null || command -v fping 2>/dev/null)" - if [ "${fping}" != "/usr/local/bin/fping" ] - then - echo >&2 "You have another fping installed at: ${fping}." - echo >&2 "Please set:" - echo >&2 - echo >&2 " fping=\"/usr/local/bin/fping\"" - echo >&2 - echo >&2 "at /etc/netdata/fping.conf" - echo >&2 - fi - exit 0 -fi - -# ----------------------------------------------------------------------------- - -PROGRAM_NAME="$(basename "${0}")" - -logdate() { - date "+%Y-%m-%d %H:%M:%S" -} - -log() { - local status="${1}" - shift - - echo >&2 "$(logdate): ${PROGRAM_NAME}: ${status}: ${*}" - -} - -warning() { - log WARNING "${@}" -} - -error() { - log ERROR "${@}" -} - -info() { - log INFO "${@}" -} - -fatal() { - log FATAL "${@}" - echo "DISABLE" - exit 1 -} - -debug=0 -debug() { - [ $debug -eq 1 ] && log DEBUG "${@}" -} - -# ----------------------------------------------------------------------------- - -# store in ${plugin} the name we run under -# this allows us to copy/link fping.plugin under a different name -# to have multiple fping plugins running with different settings -plugin="${PROGRAM_NAME/.plugin/}" - - -# ----------------------------------------------------------------------------- - -# the frequency to send info to netdata -# passed by netdata as the first parameter -update_every="${1-1}" - -# the netdata configuration directory -# passed by netdata as an environment variable -[ -z "${NETDATA_USER_CONFIG_DIR}" ] && NETDATA_USER_CONFIG_DIR="@configdir_POST@" -[ -z "${NETDATA_STOCK_CONFIG_DIR}" ] && NETDATA_STOCK_CONFIG_DIR="@libconfigdir_POST@" - -# ----------------------------------------------------------------------------- -# configuration options -# can be overwritten at /etc/netdata/fping.conf - -# the fping binary to use -# we need one that can output netdata friendly info (supporting: -N) -# if you have multiple versions, put here the full filename of the right one -fping="$( which fping 2>/dev/null || command -v fping 2>/dev/null )" - -# a space separated list of hosts to fping -# we suggest to put names here and the IPs of these names in /etc/hosts -hosts="" - -# the time in milliseconds (1 sec = 1000 ms) -# to ping the hosts - by default 5 pings per host per iteration -ping_every="$((update_every * 1000 / 5))" - -# fping options -fping_opts="-R -b 56 -i 1 -r 0 -t 5000" - -# ----------------------------------------------------------------------------- -# load the configuration files - -for CONFIG in "${NETDATA_STOCK_CONFIG_DIR}/${plugin}.conf" "${NETDATA_USER_CONFIG_DIR}/${plugin}.conf" -do - if [ -f "${CONFIG}" ] - then - info "Loading config file '${CONFIG}'..." - source "${CONFIG}" - [ $? -ne 0 ] && error "Failed to load config file '${CONFIG}'." - else - warning "Cannot find file '${CONFIG}'." - fi -done - -if [ -z "${hosts}" ] -then - fatal "no hosts configured - nothing to do." -fi - -if [ -z "${fping}" ] -then - fatal "fping command is not found. Please set its full path in '${NETDATA_USER_CONFIG_DIR}/${plugin}.conf'" -fi - -if [ ! -x "${fping}" ] -then - fatal "fping command '${fping}' is not executable - cannot proceed." -fi - -if [ ${ping_every} -lt 20 ] - then - warning "ping every was set to ${ping_every} but 20 is the minimum for non-root users. Setting it to 20 ms." - ping_every=20 -fi - -# the fping options we will use -options=( -N -l -Q ${update_every} -p ${ping_every} ${fping_opts} ${hosts} ) - -# execute fping -info "starting fping: ${fping} ${options[*]}" -exec "${fping}" "${options[@]}" - -# if we cannot execute fping, stop -fatal "command '${fping} ${options[*]}' failed to be executed (returned code $?)." diff --git a/collectors/freebsd.plugin/README.md b/collectors/freebsd.plugin/README.md index 9a97a7ece..3d37a41f7 100644 --- a/collectors/freebsd.plugin/README.md +++ b/collectors/freebsd.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "freebsd.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/freebsd.plugin/README.md +title: "FreeBSD system metrics (freebsd.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/freebsd.plugin/README.md" +sidebar_label: "FreeBSD system metrics (freebsd.plugin)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # freebsd.plugin diff --git a/collectors/freebsd.plugin/freebsd_devstat.c b/collectors/freebsd.plugin/freebsd_devstat.c index 0f037741a..d4180d33b 100644 --- a/collectors/freebsd.plugin/freebsd_devstat.c +++ b/collectors/freebsd.plugin/freebsd_devstat.c @@ -116,7 +116,7 @@ static void disks_cleanup() { struct disk *dm = disks_root, *last = NULL; while(dm) { if (unlikely(!dm->updated)) { - // info("Removing disk '%s', linked after '%s'", dm->name, last?last->name:"ROOT"); + // collector_info("Removing disk '%s', linked after '%s'", dm->name, last?last->name:"ROOT"); if (disks_last_used == dm) disks_last_used = last; @@ -728,28 +728,28 @@ int do_kern_devstat(int update_every, usec_t dt) { if (unlikely(common_error)) { do_system_io = 0; - error("DISABLED: system.io chart"); + collector_error("DISABLED: system.io chart"); do_io = 0; - error("DISABLED: disk.* charts"); + collector_error("DISABLED: disk.* charts"); do_ops = 0; - error("DISABLED: disk_ops.* charts"); + collector_error("DISABLED: disk_ops.* charts"); do_qops = 0; - error("DISABLED: disk_qops.* charts"); + collector_error("DISABLED: disk_qops.* charts"); do_util = 0; - error("DISABLED: disk_util.* charts"); + collector_error("DISABLED: disk_util.* charts"); do_iotime = 0; - error("DISABLED: disk_iotime.* charts"); + collector_error("DISABLED: disk_iotime.* charts"); do_await = 0; - error("DISABLED: disk_await.* charts"); + collector_error("DISABLED: disk_await.* charts"); do_avagsz = 0; - error("DISABLED: disk_avgsz.* charts"); + collector_error("DISABLED: disk_avgsz.* charts"); do_svctm = 0; - error("DISABLED: disk_svctm.* charts"); - error("DISABLED: kern.devstat module"); + collector_error("DISABLED: disk_svctm.* charts"); + collector_error("DISABLED: kern.devstat module"); return 1; } } else { - error("DISABLED: kern.devstat module"); + collector_error("DISABLED: kern.devstat module"); return 1; } diff --git a/collectors/freebsd.plugin/freebsd_getifaddrs.c b/collectors/freebsd.plugin/freebsd_getifaddrs.c index 1e870c0db..f1e67088e 100644 --- a/collectors/freebsd.plugin/freebsd_getifaddrs.c +++ b/collectors/freebsd.plugin/freebsd_getifaddrs.c @@ -73,7 +73,7 @@ static void network_interfaces_cleanup() { struct cgroup_network_interface *ifm = network_interfaces_root, *last = NULL; while(ifm) { if (unlikely(!ifm->updated)) { - // info("Removing network interface '%s', linked after '%s'", ifm->name, last?last->name:"ROOT"); + // collector_info("Removing network interface '%s', linked after '%s'", ifm->name, last?last->name:"ROOT"); if (network_interfaces_last_used == ifm) network_interfaces_last_used = last; @@ -193,26 +193,26 @@ int do_getifaddrs(int update_every, usec_t dt) { struct ifaddrs *ifap; if (unlikely(getifaddrs(&ifap))) { - error("FREEBSD: getifaddrs() failed"); + collector_error("FREEBSD: getifaddrs() failed"); do_bandwidth_net = 0; - error("DISABLED: system.net chart"); + collector_error("DISABLED: system.net chart"); do_packets_net = 0; - error("DISABLED: system.packets chart"); + collector_error("DISABLED: system.packets chart"); do_bandwidth_ipv4 = 0; - error("DISABLED: system.ipv4 chart"); + collector_error("DISABLED: system.ipv4 chart"); do_bandwidth_ipv6 = 0; - error("DISABLED: system.ipv6 chart"); + collector_error("DISABLED: system.ipv6 chart"); do_bandwidth = 0; - error("DISABLED: net.* charts"); + collector_error("DISABLED: net.* charts"); do_packets = 0; - error("DISABLED: net_packets.* charts"); + collector_error("DISABLED: net_packets.* charts"); do_errors = 0; - error("DISABLED: net_errors.* charts"); + collector_error("DISABLED: net_errors.* charts"); do_drops = 0; - error("DISABLED: net_drops.* charts"); + collector_error("DISABLED: net_drops.* charts"); do_events = 0; - error("DISABLED: net_events.* charts"); - error("DISABLED: getifaddrs module"); + collector_error("DISABLED: net_events.* charts"); + collector_error("DISABLED: getifaddrs module"); return 1; } else { #define IFA_DATA(s) (((struct if_data *)ifa->ifa_data)->ifi_ ## s) @@ -589,7 +589,7 @@ int do_getifaddrs(int update_every, usec_t dt) { freeifaddrs(ifap); } } else { - error("DISABLED: getifaddrs module"); + collector_error("DISABLED: getifaddrs module"); return 1; } diff --git a/collectors/freebsd.plugin/freebsd_getmntinfo.c b/collectors/freebsd.plugin/freebsd_getmntinfo.c index e8feefc2b..d17cddfc3 100644 --- a/collectors/freebsd.plugin/freebsd_getmntinfo.c +++ b/collectors/freebsd.plugin/freebsd_getmntinfo.c @@ -54,7 +54,7 @@ static void mount_points_cleanup() { struct mount_point *m = mount_points_root, *last = NULL; while(m) { if (unlikely(!m->updated)) { - // info("Removing mount point '%s', linked after '%s'", m->name, last?last->name:"ROOT"); + // collector_info("Removing mount point '%s', linked after '%s'", m->name, last?last->name:"ROOT"); if (mount_points_last_used == m) mount_points_last_used = last; @@ -163,12 +163,12 @@ int do_getmntinfo(int update_every, usec_t dt) { // there is no mount info in sysctl MIBs if (unlikely(!(mntsize = getmntinfo(&mntbuf, MNT_NOWAIT)))) { - error("FREEBSD: getmntinfo() failed"); + collector_error("FREEBSD: getmntinfo() failed"); do_space = 0; - error("DISABLED: disk_space.* charts"); + collector_error("DISABLED: disk_space.* charts"); do_inodes = 0; - error("DISABLED: disk_inodes.* charts"); - error("DISABLED: getmntinfo module"); + collector_error("DISABLED: disk_inodes.* charts"); + collector_error("DISABLED: getmntinfo module"); return 1; } else { int i; @@ -289,7 +289,7 @@ int do_getmntinfo(int update_every, usec_t dt) { } } } else { - error("DISABLED: getmntinfo module"); + collector_error("DISABLED: getmntinfo module"); return 1; } diff --git a/collectors/freebsd.plugin/freebsd_ipfw.c b/collectors/freebsd.plugin/freebsd_ipfw.c index 178eaa36c..dcb771ce9 100644 --- a/collectors/freebsd.plugin/freebsd_ipfw.c +++ b/collectors/freebsd.plugin/freebsd_ipfw.c @@ -6,11 +6,11 @@ #define FREE_MEM_THRESHOLD 10000 // number of unused chunks that trigger memory freeing -#define COMMON_IPFW_ERROR() error("DISABLED: ipfw.packets chart"); \ - error("DISABLED: ipfw.bytes chart"); \ - error("DISABLED: ipfw.dyn_active chart"); \ - error("DISABLED: ipfw.dyn_expired chart"); \ - error("DISABLED: ipfw.mem chart"); +#define COMMON_IPFW_ERROR() collector_error("DISABLED: ipfw.packets chart"); \ + collector_error("DISABLED: ipfw.bytes chart"); \ + collector_error("DISABLED: ipfw.dyn_active chart"); \ + collector_error("DISABLED: ipfw.dyn_expired chart"); \ + collector_error("DISABLED: ipfw.mem chart"); // -------------------------------------------------------------------------------------------------------------------- // ipfw @@ -83,8 +83,8 @@ int do_ipfw(int update_every, usec_t dt) { if (unlikely(ipfw_socket == -1)) ipfw_socket = socket(AF_INET, SOCK_RAW, IPPROTO_RAW); if (unlikely(ipfw_socket == -1)) { - error("FREEBSD: can't get socket for ipfw configuration"); - error("FREEBSD: run netdata as root to get access to ipfw data"); + collector_error("FREEBSD: can't get socket for ipfw configuration"); + collector_error("FREEBSD: run netdata as root to get access to ipfw data"); COMMON_IPFW_ERROR(); return 1; } @@ -100,7 +100,7 @@ int do_ipfw(int update_every, usec_t dt) { error = getsockopt(ipfw_socket, IPPROTO_IP, IP_FW3, op3, optlen); if (error) if (errno != ENOMEM) { - error("FREEBSD: ipfw socket reading error"); + collector_error("FREEBSD: ipfw socket reading error"); COMMON_IPFW_ERROR(); return 1; } @@ -113,7 +113,7 @@ int do_ipfw(int update_every, usec_t dt) { op3->opcode = IP_FW_XGET; error = getsockopt(ipfw_socket, IPPROTO_IP, IP_FW3, op3, optlen); if (error) { - error("FREEBSD: ipfw socket reading error"); + collector_error("FREEBSD: ipfw socket reading error"); COMMON_IPFW_ERROR(); return 1; } @@ -352,7 +352,7 @@ int do_ipfw(int update_every, usec_t dt) { return 0; #else - error("FREEBSD: ipfw charts supported for FreeBSD 11.0 and newer releases only"); + collector_error("FREEBSD: ipfw charts supported for FreeBSD 11.0 and newer releases only"); COMMON_IPFW_ERROR(); return 1; #endif diff --git a/collectors/freebsd.plugin/freebsd_kstat_zfs.c b/collectors/freebsd.plugin/freebsd_kstat_zfs.c index 046a1e693..165efa17c 100644 --- a/collectors/freebsd.plugin/freebsd_kstat_zfs.c +++ b/collectors/freebsd.plugin/freebsd_kstat_zfs.c @@ -238,9 +238,9 @@ int do_kstat_zfs_misc_zio_trim(int update_every, usec_t dt) { GETSYSCTL_SIMPLE("kstat.zfs.misc.zio_trim.success", mib_success, success) || GETSYSCTL_SIMPLE("kstat.zfs.misc.zio_trim.failed", mib_failed, failed) || GETSYSCTL_SIMPLE("kstat.zfs.misc.zio_trim.unsupported", mib_unsupported, unsupported))) { - error("DISABLED: zfs.trim_bytes chart"); - error("DISABLED: zfs.trim_success chart"); - error("DISABLED: kstat.zfs.misc.zio_trim module"); + collector_error("DISABLED: zfs.trim_bytes chart"); + collector_error("DISABLED: zfs.trim_success chart"); + collector_error("DISABLED: kstat.zfs.misc.zio_trim module"); return 1; } else { diff --git a/collectors/freebsd.plugin/freebsd_sysctl.c b/collectors/freebsd.plugin/freebsd_sysctl.c index dd94a1615..7d68bda9b 100644 --- a/collectors/freebsd.plugin/freebsd_sysctl.c +++ b/collectors/freebsd.plugin/freebsd_sysctl.c @@ -96,17 +96,17 @@ int freebsd_plugin_init() { system_pagesize = getpagesize(); if (system_pagesize <= 0) { - error("FREEBSD: can't get system page size"); + collector_error("FREEBSD: can't get system page size"); return 1; } if (unlikely(GETSYSCTL_BY_NAME("kern.smp.cpus", number_of_cpus))) { - error("FREEBSD: can't get number of cpus"); + collector_error("FREEBSD: can't get number of cpus"); return 1; } if (unlikely(!number_of_cpus)) { - error("FREEBSD: wrong number of cpus"); + collector_error("FREEBSD: wrong number of cpus"); return 1; } @@ -126,8 +126,8 @@ int do_vm_loadavg(int update_every, usec_t dt){ struct loadavg sysload; if (unlikely(GETSYSCTL_SIMPLE("vm.loadavg", mib, sysload))) { - error("DISABLED: system.load chart"); - error("DISABLED: vm.loadavg module"); + collector_error("DISABLED: system.load chart"); + collector_error("DISABLED: vm.loadavg module"); return 1; } else { static RRDSET *st = NULL; @@ -185,12 +185,12 @@ int do_vm_vmtotal(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("vm.vmtotal", mib, vmtotal_data))) { do_all_processes = 0; - error("DISABLED: system.active_processes chart"); + collector_error("DISABLED: system.active_processes chart"); do_processes = 0; - error("DISABLED: system.processes chart"); + collector_error("DISABLED: system.processes chart"); do_mem_real = 0; - error("DISABLED: mem.real chart"); - error("DISABLED: vm.vmtotal module"); + collector_error("DISABLED: mem.real chart"); + collector_error("DISABLED: vm.vmtotal module"); return 1; } else { if (likely(do_all_processes)) { @@ -277,7 +277,7 @@ int do_vm_vmtotal(int update_every, usec_t dt) { } } } else { - error("DISABLED: vm.vmtotal module"); + collector_error("DISABLED: vm.vmtotal module"); return 1; } @@ -290,17 +290,17 @@ int do_kern_cp_time(int update_every, usec_t dt) { (void)dt; if (unlikely(CPUSTATES != 5)) { - error("FREEBSD: There are %d CPU states (5 was expected)", CPUSTATES); - error("DISABLED: system.cpu chart"); - error("DISABLED: kern.cp_time module"); + collector_error("FREEBSD: There are %d CPU states (5 was expected)", CPUSTATES); + collector_error("DISABLED: system.cpu chart"); + collector_error("DISABLED: kern.cp_time module"); return 1; } else { static int mib[2] = {0, 0}; long cp_time[CPUSTATES]; if (unlikely(GETSYSCTL_SIMPLE("kern.cp_time", mib, cp_time))) { - error("DISABLED: system.cpu chart"); - error("DISABLED: kern.cp_time module"); + collector_error("DISABLED: system.cpu chart"); + collector_error("DISABLED: kern.cp_time module"); return 1; } else { static RRDSET *st = NULL; @@ -348,9 +348,9 @@ int do_kern_cp_times(int update_every, usec_t dt) { (void)dt; if (unlikely(CPUSTATES != 5)) { - error("FREEBSD: There are %d CPU states (5 was expected)", CPUSTATES); - error("DISABLED: cpu.cpuXX charts"); - error("DISABLED: kern.cp_times module"); + collector_error("FREEBSD: There are %d CPU states (5 was expected)", CPUSTATES); + collector_error("DISABLED: cpu.cpuXX charts"); + collector_error("DISABLED: kern.cp_times module"); return 1; } else { static int mib[2] = {0, 0}; @@ -361,8 +361,8 @@ int do_kern_cp_times(int update_every, usec_t dt) { if(unlikely(number_of_cpus != old_number_of_cpus)) pcpu_cp_time = reallocz(pcpu_cp_time, sizeof(cp_time) * number_of_cpus); if (unlikely(GETSYSCTL_WSIZE("kern.cp_times", mib, pcpu_cp_time, sizeof(cp_time) * number_of_cpus))) { - error("DISABLED: cpu.cpuXX charts"); - error("DISABLED: kern.cp_times module"); + collector_error("DISABLED: cpu.cpuXX charts"); + collector_error("DISABLED: kern.cp_times module"); return 1; } else { int i; @@ -449,8 +449,8 @@ int do_dev_cpu_temperature(int update_every, usec_t dt) { if (unlikely(!(mib[i * 4]))) sprintf(char_mib, "dev.cpu.%d.temperature", i); if (unlikely(getsysctl_simple(char_mib, &mib[i * 4], 4, &pcpu_temperature[i], sizeof(int)))) { - error("DISABLED: cpu.temperature chart"); - error("DISABLED: dev.cpu.temperature module"); + collector_error("DISABLED: cpu.temperature chart"); + collector_error("DISABLED: dev.cpu.temperature module"); return 1; } } @@ -505,8 +505,8 @@ int do_dev_cpu_0_freq(int update_every, usec_t dt) { int cpufreq; if (unlikely(GETSYSCTL_SIMPLE("dev.cpu.0.freq", mib, cpufreq))) { - error("DISABLED: cpu.scaling_cur_freq chart"); - error("DISABLED: dev.cpu.0.freq module"); + collector_error("DISABLED: cpu.scaling_cur_freq chart"); + collector_error("DISABLED: dev.cpu.0.freq module"); return 1; } else { static RRDSET *st = NULL; @@ -547,9 +547,9 @@ int do_hw_intcnt(int update_every, usec_t dt) { size_t intrcnt_size = 0; if (unlikely(GETSYSCTL_SIZE("hw.intrcnt", mib_hw_intrcnt, intrcnt_size))) { - error("DISABLED: system.intr chart"); - error("DISABLED: system.interrupts chart"); - error("DISABLED: hw.intrcnt module"); + collector_error("DISABLED: system.intr chart"); + collector_error("DISABLED: system.interrupts chart"); + collector_error("DISABLED: hw.intrcnt module"); return 1; } else { unsigned long nintr = 0; @@ -560,9 +560,9 @@ int do_hw_intcnt(int update_every, usec_t dt) { if (unlikely(nintr != old_nintr)) intrcnt = reallocz(intrcnt, nintr * sizeof(u_long)); if (unlikely(GETSYSCTL_WSIZE("hw.intrcnt", mib_hw_intrcnt, intrcnt, nintr * sizeof(u_long)))) { - error("DISABLED: system.intr chart"); - error("DISABLED: system.interrupts chart"); - error("DISABLED: hw.intrcnt module"); + collector_error("DISABLED: system.intr chart"); + collector_error("DISABLED: system.interrupts chart"); + collector_error("DISABLED: hw.intrcnt module"); return 1; } else { unsigned long long totalintr = 0; @@ -602,17 +602,17 @@ int do_hw_intcnt(int update_every, usec_t dt) { static char *intrnames = NULL; if (unlikely(GETSYSCTL_SIZE("hw.intrnames", mib_hw_intrnames, size))) { - error("DISABLED: system.intr chart"); - error("DISABLED: system.interrupts chart"); - error("DISABLED: hw.intrcnt module"); + collector_error("DISABLED: system.intr chart"); + collector_error("DISABLED: system.interrupts chart"); + collector_error("DISABLED: hw.intrcnt module"); return 1; } else { if (unlikely(nintr != old_nintr)) intrnames = reallocz(intrnames, size); if (unlikely(GETSYSCTL_WSIZE("hw.intrnames", mib_hw_intrnames, intrnames, size))) { - error("DISABLED: system.intr chart"); - error("DISABLED: system.interrupts chart"); - error("DISABLED: hw.intrcnt module"); + collector_error("DISABLED: system.intr chart"); + collector_error("DISABLED: system.interrupts chart"); + collector_error("DISABLED: hw.intrcnt module"); return 1; } else { static RRDSET *st_interrupts = NULL; @@ -666,8 +666,8 @@ int do_vm_stats_sys_v_intr(int update_every, usec_t dt) { u_int int_number; if (unlikely(GETSYSCTL_SIMPLE("vm.stats.sys.v_intr", mib, int_number))) { - error("DISABLED: system.dev_intr chart"); - error("DISABLED: vm.stats.sys.v_intr module"); + collector_error("DISABLED: system.dev_intr chart"); + collector_error("DISABLED: vm.stats.sys.v_intr module"); return 1; } else { static RRDSET *st = NULL; @@ -707,8 +707,8 @@ int do_vm_stats_sys_v_soft(int update_every, usec_t dt) { u_int soft_intr_number; if (unlikely(GETSYSCTL_SIMPLE("vm.stats.sys.v_soft", mib, soft_intr_number))) { - error("DISABLED: system.dev_intr chart"); - error("DISABLED: vm.stats.sys.v_soft module"); + collector_error("DISABLED: system.dev_intr chart"); + collector_error("DISABLED: vm.stats.sys.v_soft module"); return 1; } else { static RRDSET *st = NULL; @@ -748,8 +748,8 @@ int do_vm_stats_sys_v_swtch(int update_every, usec_t dt) { u_int ctxt_number; if (unlikely(GETSYSCTL_SIMPLE("vm.stats.sys.v_swtch", mib, ctxt_number))) { - error("DISABLED: system.ctxt chart"); - error("DISABLED: vm.stats.sys.v_swtch module"); + collector_error("DISABLED: system.ctxt chart"); + collector_error("DISABLED: vm.stats.sys.v_swtch module"); return 1; } else { static RRDSET *st = NULL; @@ -789,8 +789,8 @@ int do_vm_stats_sys_v_forks(int update_every, usec_t dt) { u_int forks_number; if (unlikely(GETSYSCTL_SIMPLE("vm.stats.vm.v_forks", mib, forks_number))) { - error("DISABLED: system.forks chart"); - error("DISABLED: vm.stats.sys.v_swtch module"); + collector_error("DISABLED: system.forks chart"); + collector_error("DISABLED: vm.stats.sys.v_swtch module"); return 1; } else { @@ -834,8 +834,8 @@ int do_vm_swap_info(int update_every, usec_t dt) { static int mib[3] = {0, 0, 0}; if (unlikely(getsysctl_mib("vm.swap_info", mib, 2))) { - error("DISABLED: system.swap chart"); - error("DISABLED: vm.swap_info module"); + collector_error("DISABLED: system.swap chart"); + collector_error("DISABLED: vm.swap_info module"); return 1; } else { int i; @@ -852,15 +852,15 @@ int do_vm_swap_info(int update_every, usec_t dt) { size = sizeof(xsw); if (unlikely(sysctl(mib, 3, &xsw, &size, NULL, 0) == -1 )) { if (unlikely(errno != ENOENT)) { - error("FREEBSD: sysctl(%s...) failed: %s", "vm.swap_info", strerror(errno)); - error("DISABLED: system.swap chart"); - error("DISABLED: vm.swap_info module"); + collector_error("FREEBSD: sysctl(%s...) failed: %s", "vm.swap_info", strerror(errno)); + collector_error("DISABLED: system.swap chart"); + collector_error("DISABLED: vm.swap_info module"); return 1; } else { if (unlikely(size != sizeof(xsw))) { - error("FREEBSD: sysctl(%s...) expected %lu, got %lu", "vm.swap_info", (unsigned long)sizeof(xsw), (unsigned long)size); - error("DISABLED: system.swap chart"); - error("DISABLED: vm.swap_info module"); + collector_error("FREEBSD: sysctl(%s...) expected %lu, got %lu", "vm.swap_info", (unsigned long)sizeof(xsw), (unsigned long)size); + collector_error("DISABLED: system.swap chart"); + collector_error("DISABLED: vm.swap_info module"); return 1; } else break; } @@ -932,8 +932,8 @@ int do_system_ram(int update_every, usec_t dt) { #endif GETSYSCTL_SIMPLE("vfs.bufspace", mib_vfs_bufspace, vfs_bufspace_count) || GETSYSCTL_SIMPLE("vm.stats.vm.v_free_count", mib_free_count, vmmeter_data.v_free_count))) { - error("DISABLED: system.ram chart"); - error("DISABLED: system.ram module"); + collector_error("DISABLED: system.ram chart"); + collector_error("DISABLED: system.ram module"); return 1; } else { static RRDSET *st = NULL, *st_mem_available = NULL; @@ -1026,8 +1026,8 @@ int do_vm_stats_sys_v_swappgs(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("vm.stats.vm.v_swappgsin", mib_swappgsin, vmmeter_data.v_swappgsin) || GETSYSCTL_SIMPLE("vm.stats.vm.v_swappgsout", mib_swappgsout, vmmeter_data.v_swappgsout))) { - error("DISABLED: system.swapio chart"); - error("DISABLED: vm.stats.vm.v_swappgs module"); + collector_error("DISABLED: system.swapio chart"); + collector_error("DISABLED: vm.stats.vm.v_swappgs module"); return 1; } else { static RRDSET *st = NULL; @@ -1074,8 +1074,8 @@ int do_vm_stats_sys_v_pgfaults(int update_every, usec_t dt) { GETSYSCTL_SIMPLE("vm.stats.vm.v_cow_faults", mib_cow_faults, vmmeter_data.v_cow_faults) || GETSYSCTL_SIMPLE("vm.stats.vm.v_cow_optim", mib_cow_optim, vmmeter_data.v_cow_optim) || GETSYSCTL_SIMPLE("vm.stats.vm.v_intrans", mib_intrans, vmmeter_data.v_intrans))) { - error("DISABLED: mem.pgfaults chart"); - error("DISABLED: vm.stats.vm.v_pgfaults module"); + collector_error("DISABLED: mem.pgfaults chart"); + collector_error("DISABLED: vm.stats.vm.v_pgfaults module"); return 1; } else { static RRDSET *st = NULL; @@ -1131,9 +1131,9 @@ int do_kern_ipc_sem(int update_every, usec_t dt) { } ipc_sem = {0, 0, 0}; if (unlikely(GETSYSCTL_SIMPLE("kern.ipc.semmni", mib_semmni, ipc_sem.semmni))) { - error("DISABLED: system.ipc_semaphores chart"); - error("DISABLED: system.ipc_semaphore_arrays chart"); - error("DISABLED: kern.ipc.sem module"); + collector_error("DISABLED: system.ipc_semaphores chart"); + collector_error("DISABLED: system.ipc_semaphore_arrays chart"); + collector_error("DISABLED: kern.ipc.sem module"); return 1; } else { static struct semid_kernel *ipc_sem_data = NULL; @@ -1145,9 +1145,9 @@ int do_kern_ipc_sem(int update_every, usec_t dt) { old_semmni = ipc_sem.semmni; } if (unlikely(GETSYSCTL_WSIZE("kern.ipc.sema", mib_sema, ipc_sem_data, sizeof(struct semid_kernel) * ipc_sem.semmni))) { - error("DISABLED: system.ipc_semaphores chart"); - error("DISABLED: system.ipc_semaphore_arrays chart"); - error("DISABLED: kern.ipc.sem module"); + collector_error("DISABLED: system.ipc_semaphores chart"); + collector_error("DISABLED: system.ipc_semaphore_arrays chart"); + collector_error("DISABLED: kern.ipc.sem module"); return 1; } else { int i; @@ -1223,9 +1223,9 @@ int do_kern_ipc_shm(int update_every, usec_t dt) { } ipc_shm = {0, 0, 0}; if (unlikely(GETSYSCTL_SIMPLE("kern.ipc.shmmni", mib_shmmni, ipc_shm.shmmni))) { - error("DISABLED: system.ipc_shared_mem_segs chart"); - error("DISABLED: system.ipc_shared_mem_size chart"); - error("DISABLED: kern.ipc.shmmodule"); + collector_error("DISABLED: system.ipc_shared_mem_segs chart"); + collector_error("DISABLED: system.ipc_shared_mem_size chart"); + collector_error("DISABLED: kern.ipc.shmmodule"); return 1; } else { static struct shmid_kernel *ipc_shm_data = NULL; @@ -1238,9 +1238,9 @@ int do_kern_ipc_shm(int update_every, usec_t dt) { } if (unlikely( GETSYSCTL_WSIZE("kern.ipc.shmsegs", mib_shmsegs, ipc_shm_data, sizeof(struct shmid_kernel) * ipc_shm.shmmni))) { - error("DISABLED: system.ipc_shared_mem_segs chart"); - error("DISABLED: system.ipc_shared_mem_size chart"); - error("DISABLED: kern.ipc.shmmodule"); + collector_error("DISABLED: system.ipc_shared_mem_segs chart"); + collector_error("DISABLED: system.ipc_shared_mem_size chart"); + collector_error("DISABLED: kern.ipc.shmmodule"); return 1; } else { unsigned long i; @@ -1318,10 +1318,10 @@ int do_kern_ipc_msq(int update_every, usec_t dt) { } ipc_msq = {0, 0, 0, 0, 0}; if (unlikely(GETSYSCTL_SIMPLE("kern.ipc.msgmni", mib_msgmni, ipc_msq.msgmni))) { - error("DISABLED: system.ipc_msq_queues chart"); - error("DISABLED: system.ipc_msq_messages chart"); - error("DISABLED: system.ipc_msq_size chart"); - error("DISABLED: kern.ipc.msg module"); + collector_error("DISABLED: system.ipc_msq_queues chart"); + collector_error("DISABLED: system.ipc_msq_messages chart"); + collector_error("DISABLED: system.ipc_msq_size chart"); + collector_error("DISABLED: kern.ipc.msg module"); return 1; } else { static struct msqid_kernel *ipc_msq_data = NULL; @@ -1334,10 +1334,10 @@ int do_kern_ipc_msq(int update_every, usec_t dt) { } if (unlikely( GETSYSCTL_WSIZE("kern.ipc.msqids", mib_msqids, ipc_msq_data, sizeof(struct msqid_kernel) * ipc_msq.msgmni))) { - error("DISABLED: system.ipc_msq_queues chart"); - error("DISABLED: system.ipc_msq_messages chart"); - error("DISABLED: system.ipc_msq_size chart"); - error("DISABLED: kern.ipc.msg module"); + collector_error("DISABLED: system.ipc_msq_queues chart"); + collector_error("DISABLED: system.ipc_msq_messages chart"); + collector_error("DISABLED: system.ipc_msq_size chart"); + collector_error("DISABLED: kern.ipc.msg module"); return 1; } else { int i; @@ -1520,11 +1520,11 @@ int do_net_isr(int update_every, usec_t dt) { } if (unlikely(common_error)) { do_netisr = 0; - error("DISABLED: system.softnet_stat chart"); + collector_error("DISABLED: system.softnet_stat chart"); do_netisr_per_core = 0; - error("DISABLED: system.cpuX_softnet_stat chart"); + collector_error("DISABLED: system.cpuX_softnet_stat chart"); common_error = 0; - error("DISABLED: net.isr module"); + collector_error("DISABLED: net.isr module"); return 1; } else { unsigned long i, n; @@ -1554,7 +1554,7 @@ int do_net_isr(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.isr module"); + collector_error("DISABLED: net.isr module"); return 1; } @@ -1662,8 +1662,8 @@ int do_net_inet_tcp_states(int update_every, usec_t dt) { // see http://net-snmp.sourceforge.net/docs/mibs/tcp.html if (unlikely(GETSYSCTL_SIMPLE("net.inet.tcp.states", mib, tcps_states))) { - error("DISABLED: ipv4.tcpsock chart"); - error("DISABLED: net.inet.tcp.states module"); + collector_error("DISABLED: ipv4.tcpsock chart"); + collector_error("DISABLED: net.inet.tcp.states module"); return 1; } else { static RRDSET *st = NULL; @@ -1726,22 +1726,22 @@ int do_net_inet_tcp_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet.tcp.stats", mib, tcpstat))) { do_tcp_packets = 0; - error("DISABLED: ipv4.tcppackets chart"); + collector_error("DISABLED: ipv4.tcppackets chart"); do_tcp_errors = 0; - error("DISABLED: ipv4.tcperrors chart"); + collector_error("DISABLED: ipv4.tcperrors chart"); do_tcp_handshake = 0; - error("DISABLED: ipv4.tcphandshake chart"); + collector_error("DISABLED: ipv4.tcphandshake chart"); do_tcpext_connaborts = 0; - error("DISABLED: ipv4.tcpconnaborts chart"); + collector_error("DISABLED: ipv4.tcpconnaborts chart"); do_tcpext_ofo = 0; - error("DISABLED: ipv4.tcpofo chart"); + collector_error("DISABLED: ipv4.tcpofo chart"); do_tcpext_syncookies = 0; - error("DISABLED: ipv4.tcpsyncookies chart"); + collector_error("DISABLED: ipv4.tcpsyncookies chart"); do_tcpext_listen = 0; - error("DISABLED: ipv4.tcplistenissues chart"); + collector_error("DISABLED: ipv4.tcplistenissues chart"); do_ecn = 0; - error("DISABLED: ipv4.ecnpkts chart"); - error("DISABLED: net.inet.tcp.stats module"); + collector_error("DISABLED: ipv4.ecnpkts chart"); + collector_error("DISABLED: net.inet.tcp.stats module"); return 1; } else { if (likely(do_tcp_packets)) { @@ -2035,7 +2035,7 @@ int do_net_inet_tcp_stats(int update_every, usec_t dt) { } } else { - error("DISABLED: net.inet.tcp.stats module"); + collector_error("DISABLED: net.inet.tcp.stats module"); return 1; } @@ -2060,10 +2060,10 @@ int do_net_inet_udp_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet.udp.stats", mib, udpstat))) { do_udp_packets = 0; - error("DISABLED: ipv4.udppackets chart"); + collector_error("DISABLED: ipv4.udppackets chart"); do_udp_errors = 0; - error("DISABLED: ipv4.udperrors chart"); - error("DISABLED: net.inet.udp.stats module"); + collector_error("DISABLED: ipv4.udperrors chart"); + collector_error("DISABLED: net.inet.udp.stats module"); return 1; } else { if (likely(do_udp_packets)) { @@ -2134,7 +2134,7 @@ int do_net_inet_udp_stats(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.inet.udp.stats module"); + collector_error("DISABLED: net.inet.udp.stats module"); return 1; } @@ -2163,12 +2163,12 @@ int do_net_inet_icmp_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet.icmp.stats", mib, icmpstat))) { do_icmp_packets = 0; - error("DISABLED: ipv4.icmp chart"); + collector_error("DISABLED: ipv4.icmp chart"); do_icmp_errors = 0; - error("DISABLED: ipv4.icmp_errors chart"); + collector_error("DISABLED: ipv4.icmp_errors chart"); do_icmpmsg = 0; - error("DISABLED: ipv4.icmpmsg chart"); - error("DISABLED: net.inet.icmp.stats module"); + collector_error("DISABLED: ipv4.icmpmsg chart"); + collector_error("DISABLED: net.inet.icmp.stats module"); return 1; } else { int i; @@ -2275,7 +2275,7 @@ int do_net_inet_icmp_stats(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.inet.icmp.stats module"); + collector_error("DISABLED: net.inet.icmp.stats module"); return 1; } @@ -2302,14 +2302,14 @@ int do_net_inet_ip_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet.ip.stats", mib, ipstat))) { do_ip_packets = 0; - error("DISABLED: ipv4.packets chart"); + collector_error("DISABLED: ipv4.packets chart"); do_ip_fragsout = 0; - error("DISABLED: ipv4.fragsout chart"); + collector_error("DISABLED: ipv4.fragsout chart"); do_ip_fragsin = 0; - error("DISABLED: ipv4.fragsin chart"); + collector_error("DISABLED: ipv4.fragsin chart"); do_ip_errors = 0; - error("DISABLED: ipv4.errors chart"); - error("DISABLED: net.inet.ip.stats module"); + collector_error("DISABLED: ipv4.errors chart"); + collector_error("DISABLED: net.inet.ip.stats module"); return 1; } else { if (likely(do_ip_packets)) { @@ -2456,7 +2456,7 @@ int do_net_inet_ip_stats(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.inet.ip.stats module"); + collector_error("DISABLED: net.inet.ip.stats module"); return 1; } @@ -2486,14 +2486,14 @@ int do_net_inet6_ip6_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet6.ip6.stats", mib, ip6stat))) { do_ip6_packets = 0; - error("DISABLED: ipv6.packets chart"); + collector_error("DISABLED: ipv6.packets chart"); do_ip6_fragsout = 0; - error("DISABLED: ipv6.fragsout chart"); + collector_error("DISABLED: ipv6.fragsout chart"); do_ip6_fragsin = 0; - error("DISABLED: ipv6.fragsin chart"); + collector_error("DISABLED: ipv6.fragsin chart"); do_ip6_errors = 0; - error("DISABLED: ipv6.errors chart"); - error("DISABLED: net.inet6.ip6.stats module"); + collector_error("DISABLED: ipv6.errors chart"); + collector_error("DISABLED: net.inet6.ip6.stats module"); return 1; } else { if (do_ip6_packets == CONFIG_BOOLEAN_YES || (do_ip6_packets == CONFIG_BOOLEAN_AUTO && @@ -2674,7 +2674,7 @@ int do_net_inet6_ip6_stats(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.inet6.ip6.stats module"); + collector_error("DISABLED: net.inet6.ip6.stats module"); return 1; } @@ -2711,20 +2711,20 @@ int do_net_inet6_icmp6_stats(int update_every, usec_t dt) { if (unlikely(GETSYSCTL_SIMPLE("net.inet6.icmp6.stats", mib, icmp6stat))) { do_icmp6 = 0; - error("DISABLED: ipv6.icmp chart"); + collector_error("DISABLED: ipv6.icmp chart"); do_icmp6_redir = 0; - error("DISABLED: ipv6.icmpredir chart"); + collector_error("DISABLED: ipv6.icmpredir chart"); do_icmp6_errors = 0; - error("DISABLED: ipv6.icmperrors chart"); + collector_error("DISABLED: ipv6.icmperrors chart"); do_icmp6_echos = 0; - error("DISABLED: ipv6.icmpechos chart"); + collector_error("DISABLED: ipv6.icmpechos chart"); do_icmp6_router = 0; - error("DISABLED: ipv6.icmprouter chart"); + collector_error("DISABLED: ipv6.icmprouter chart"); do_icmp6_neighbor = 0; - error("DISABLED: ipv6.icmpneighbor chart"); + collector_error("DISABLED: ipv6.icmpneighbor chart"); do_icmp6_types = 0; - error("DISABLED: ipv6.icmptypes chart"); - error("DISABLED: net.inet6.icmp6.stats module"); + collector_error("DISABLED: ipv6.icmptypes chart"); + collector_error("DISABLED: net.inet6.icmp6.stats module"); return 1; } else { int i; @@ -3054,7 +3054,7 @@ int do_net_inet6_icmp6_stats(int update_every, usec_t dt) { } } } else { - error("DISABLED: net.inet6.icmp6.stats module"); + collector_error("DISABLED: net.inet6.icmp6.stats module"); return 1; } diff --git a/collectors/freebsd.plugin/plugin_freebsd.c b/collectors/freebsd.plugin/plugin_freebsd.c index a52ece3f9..e47b224cf 100644 --- a/collectors/freebsd.plugin/plugin_freebsd.c +++ b/collectors/freebsd.plugin/plugin_freebsd.c @@ -78,7 +78,7 @@ static void freebsd_main_cleanup(void *ptr) struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); static_thread->enabled = NETDATA_MAIN_THREAD_EXITED; } diff --git a/collectors/freeipmi.plugin/README.md b/collectors/freeipmi.plugin/README.md index ff13717d9..e33a9d3b7 100644 --- a/collectors/freeipmi.plugin/README.md +++ b/collectors/freeipmi.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "freeipmi.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/freeipmi.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/freeipmi.plugin/README.md" +sidebar_label: "freeipmi.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # freeipmi.plugin diff --git a/collectors/freeipmi.plugin/freeipmi_plugin.c b/collectors/freeipmi.plugin/freeipmi_plugin.c index 351b6e32b..bcc5139f3 100644 --- a/collectors/freeipmi.plugin/freeipmi_plugin.c +++ b/collectors/freeipmi.plugin/freeipmi_plugin.c @@ -775,7 +775,7 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) int rv = -1; if (!(ctx = ipmi_monitoring_ctx_create ())) { - error("ipmi_monitoring_ctx_create()"); + collector_error("ipmi_monitoring_ctx_create()"); goto cleanup; } @@ -784,8 +784,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) if (ipmi_monitoring_ctx_sdr_cache_directory (ctx, sdr_cache_directory) < 0) { - error("ipmi_monitoring_ctx_sdr_cache_directory(): %s\n", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error("ipmi_monitoring_ctx_sdr_cache_directory(): %s\n", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -796,8 +796,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) if (ipmi_monitoring_ctx_sensor_config_file (ctx, sensor_config_file) < 0) { - error( "ipmi_monitoring_ctx_sensor_config_file(): %s\n", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_ctx_sensor_config_file(): %s\n", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -805,8 +805,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) { if (ipmi_monitoring_ctx_sensor_config_file (ctx, NULL) < 0) { - error( "ipmi_monitoring_ctx_sensor_config_file(): %s\n", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_ctx_sensor_config_file(): %s\n", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -851,8 +851,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sensor_readings_by_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_readings_by_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -867,8 +867,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sensor_readings_by_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_readings_by_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -883,8 +883,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sensor_readings_by_sensor_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_readings_by_sensor_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -920,58 +920,57 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) if ((record_id = ipmi_monitoring_sensor_read_record_id (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_number = ipmi_monitoring_sensor_read_sensor_number (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_number(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_number(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_type = ipmi_monitoring_sensor_read_sensor_type (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if (!(sensor_name = ipmi_monitoring_sensor_read_sensor_name (ctx))) { - error( "ipmi_monitoring_sensor_read_sensor_name(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_name(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_state = ipmi_monitoring_sensor_read_sensor_state (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_state(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_state(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_units = ipmi_monitoring_sensor_read_sensor_units (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_units(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_units(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } #ifdef NETDATA_COMMENTED if ((sensor_bitmask_type = ipmi_monitoring_sensor_read_sensor_bitmask_type (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_bitmask_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_bitmask_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_bitmask = ipmi_monitoring_sensor_read_sensor_bitmask (ctx)) < 0) { - error( - "ipmi_monitoring_sensor_read_sensor_bitmask(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error("ipmi_monitoring_sensor_read_sensor_bitmask(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -986,8 +985,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) if ((sensor_reading_type = ipmi_monitoring_sensor_read_sensor_reading_type (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_sensor_reading_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_sensor_reading_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -996,8 +995,8 @@ _ipmimonitoring_sensors (struct ipmi_monitoring_ipmi_config *ipmi_config) #ifdef NETDATA_COMMENTED if ((event_reading_type_code = ipmi_monitoring_sensor_read_event_reading_type_code (ctx)) < 0) { - error( "ipmi_monitoring_sensor_read_event_reading_type_code(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sensor_read_event_reading_type_code(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } #endif // NETDATA_COMMENTED @@ -1131,7 +1130,7 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if (!(ctx = ipmi_monitoring_ctx_create ())) { - error("ipmi_monitoring_ctx_create()"); + collector_error("ipmi_monitoring_ctx_create()"); goto cleanup; } @@ -1140,8 +1139,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if (ipmi_monitoring_ctx_sdr_cache_directory (ctx, sdr_cache_directory) < 0) { - error( "ipmi_monitoring_ctx_sdr_cache_directory(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_ctx_sdr_cache_directory(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1152,8 +1151,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if (ipmi_monitoring_ctx_sel_config_file (ctx, sel_config_file) < 0) { - error( "ipmi_monitoring_ctx_sel_config_file(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_ctx_sel_config_file(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1161,8 +1160,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) { if (ipmi_monitoring_ctx_sel_config_file (ctx, NULL) < 0) { - error( "ipmi_monitoring_ctx_sel_config_file(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_ctx_sel_config_file(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1192,8 +1191,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sel_by_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_by_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1208,8 +1207,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sel_by_sensor_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_by_sensor_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1225,8 +1224,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sel_by_sensor_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_by_sensor_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1241,8 +1240,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) NULL, NULL)) < 0) { - error( "ipmi_monitoring_sel_by_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_by_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } } @@ -1281,29 +1280,29 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if ((record_id = ipmi_monitoring_sel_read_record_id (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_record_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_record_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((record_type = ipmi_monitoring_sel_read_record_type (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_record_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_record_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((record_type_class = ipmi_monitoring_sel_read_record_type_class (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_record_type_class(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_record_type_class(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sel_state = ipmi_monitoring_sel_read_sel_state (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_sel_state(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_sel_state(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1334,8 +1333,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if (ipmi_monitoring_sel_read_timestamp (ctx, ×tamp) < 0) { - error( "ipmi_monitoring_sel_read_timestamp(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_timestamp(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1363,36 +1362,36 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if (!(sensor_name = ipmi_monitoring_sel_read_sensor_name (ctx))) { - error( "ipmi_monitoring_sel_read_sensor_name(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_sensor_name(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_type = ipmi_monitoring_sel_read_sensor_type (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_sensor_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_sensor_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((sensor_number = ipmi_monitoring_sel_read_sensor_number (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_sensor_number(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_sensor_number(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((event_direction = ipmi_monitoring_sel_read_event_direction (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_event_direction(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_direction(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((event_type_code = ipmi_monitoring_sel_read_event_type_code (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_event_type_code(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_type_code(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1401,29 +1400,29 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) &event_data2, &event_data3) < 0) { - error( "ipmi_monitoring_sel_read_event_data(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_data(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((event_offset_type = ipmi_monitoring_sel_read_event_offset_type (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_event_offset_type(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_offset_type(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if ((event_offset = ipmi_monitoring_sel_read_event_offset (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_event_offset(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_offset(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } if (!(event_offset_string = ipmi_monitoring_sel_read_event_offset_string (ctx))) { - error( "ipmi_monitoring_sel_read_event_offset_string(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_event_offset_string(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1464,8 +1463,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) { if ((manufacturer_id = ipmi_monitoring_sel_read_manufacturer_id (ctx)) < 0) { - error( "ipmi_monitoring_sel_read_manufacturer_id(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_manufacturer_id(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1474,8 +1473,8 @@ _ipmimonitoring_sel (struct ipmi_monitoring_ipmi_config *ipmi_config) if ((oem_data_len = ipmi_monitoring_sel_read_oem_data (ctx, oem_data, 1024)) < 0) { - error( "ipmi_monitoring_sel_read_oem_data(): %s", - ipmi_monitoring_ctx_errormsg (ctx)); + collector_error( "ipmi_monitoring_sel_read_oem_data(): %s", + ipmi_monitoring_ctx_errormsg (ctx)); goto cleanup; } @@ -1596,6 +1595,7 @@ int host_is_local(const char *host) } int main (int argc, char **argv) { + stderror = stderr; clocks_init(); // ------------------------------------------------------------------------ @@ -1779,7 +1779,7 @@ int main (int argc, char **argv) { continue; } - error("freeipmi.plugin: ignoring parameter '%s'", argv[i]); + collector_error("freeipmi.plugin: ignoring parameter '%s'", argv[i]); } errno = 0; @@ -1788,7 +1788,7 @@ int main (int argc, char **argv) { netdata_update_every = freq; else if(freq) - error("update frequency %d seconds is too small for IPMI. Using %d.", freq, netdata_update_every); + collector_error("update frequency %d seconds is too small for IPMI. Using %d.", freq, netdata_update_every); // ------------------------------------------------------------------------ @@ -1813,7 +1813,7 @@ int main (int argc, char **argv) { if(debug) fprintf(stderr, "freeipmi.plugin: IPMI minimum update frequency was calculated to %d seconds.\n", freq); if(freq > netdata_update_every) { - info("enforcing minimum data collection frequency, calculated to %d seconds.", freq); + collector_info("enforcing minimum data collection frequency, calculated to %d seconds.", freq); netdata_update_every = freq; } diff --git a/collectors/idlejitter.plugin/README.md b/collectors/idlejitter.plugin/README.md index 5a92d5317..1a3d80257 100644 --- a/collectors/idlejitter.plugin/README.md +++ b/collectors/idlejitter.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "idlejitter.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/idlejitter.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/idlejitter.plugin/README.md" +sidebar_label: "idlejitter.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/QoS" --> # idlejitter.plugin diff --git a/collectors/idlejitter.plugin/plugin_idlejitter.c b/collectors/idlejitter.plugin/plugin_idlejitter.c index b6339cc0f..d90548869 100644 --- a/collectors/idlejitter.plugin/plugin_idlejitter.c +++ b/collectors/idlejitter.plugin/plugin_idlejitter.c @@ -10,7 +10,7 @@ static void cpuidlejitter_main_cleanup(void *ptr) { struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); static_thread->enabled = NETDATA_MAIN_THREAD_EXITED; } @@ -48,7 +48,7 @@ void *cpuidlejitter_main(void *ptr) { usec_t update_every_ut = localhost->rrd_update_every * USEC_PER_SEC; struct timeval before, after; - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { int iterations = 0; usec_t error_total = 0, error_min = 0, diff --git a/collectors/ioping.plugin/README.md b/collectors/ioping.plugin/README.md index c4c3695c5..1ab9238f4 100644 --- a/collectors/ioping.plugin/README.md +++ b/collectors/ioping.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "ioping.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/ioping.plugin/README.md +title: "Monitor latency for directories/files/devices (ioping.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/ioping.plugin/README.md" +sidebar_label: "Latency monitoring (ioping.plugin)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/QoS" --> # ioping.plugin diff --git a/collectors/macos.plugin/README.md b/collectors/macos.plugin/README.md index 92bbf1eb1..3a3e8a1a2 100644 --- a/collectors/macos.plugin/README.md +++ b/collectors/macos.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "macos.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/macos.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/macos.plugin/README.md" +sidebar_label: "macos.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # macos.plugin diff --git a/collectors/macos.plugin/macos_fw.c b/collectors/macos.plugin/macos_fw.c index 07f7d773d..ca06f428e 100644 --- a/collectors/macos.plugin/macos_fw.c +++ b/collectors/macos.plugin/macos_fw.c @@ -84,14 +84,14 @@ int do_macos_iokit(int update_every, usec_t dt) { /* Get ports and services for drive statistics. */ if (unlikely(IOMainPort(bootstrap_port, &main_port))) { - error("MACOS: IOMasterPort() failed"); + collector_error("MACOS: IOMasterPort() failed"); do_io = 0; - error("DISABLED: system.io"); + collector_error("DISABLED: system.io"); /* Get the list of all drive objects. */ } else if (unlikely(IOServiceGetMatchingServices(main_port, IOServiceMatching("IOBlockStorageDriver"), &drive_list))) { - error("MACOS: IOServiceGetMatchingServices() failed"); + collector_error("MACOS: IOServiceGetMatchingServices() failed"); do_io = 0; - error("DISABLED: system.io"); + collector_error("DISABLED: system.io"); } else { while ((drive = IOIteratorNext(drive_list)) != 0) { properties = 0; @@ -126,9 +126,9 @@ int do_macos_iokit(int update_every, usec_t dt) { /* Obtain the properties for this drive object. */ if (unlikely(IORegistryEntryCreateCFProperties(drive, (CFMutableDictionaryRef *)&properties, kCFAllocatorDefault, 0))) { IOObjectRelease(drive); - error("MACOS: IORegistryEntryCreateCFProperties() failed"); + collector_error("MACOS: IORegistryEntryCreateCFProperties() failed"); do_io = 0; - error("DISABLED: system.io"); + collector_error("DISABLED: system.io"); break; } else if (likely(properties)) { /* Obtain the statistics from the drive properties. */ @@ -413,11 +413,11 @@ int do_macos_iokit(int update_every, usec_t dt) { if (likely(do_space || do_inodes)) { // there is no mount info in sysctl MIBs if (unlikely(!(mntsize = getmntinfo(&mntbuf, MNT_NOWAIT)))) { - error("MACOS: getmntinfo() failed"); + collector_error("MACOS: getmntinfo() failed"); do_space = 0; - error("DISABLED: disk_space.X"); + collector_error("DISABLED: disk_space.X"); do_inodes = 0; - error("DISABLED: disk_inodes.X"); + collector_error("DISABLED: disk_inodes.X"); } else { for (i = 0; i < mntsize; i++) { if (mntbuf[i].f_flags == MNT_RDONLY || @@ -500,9 +500,9 @@ int do_macos_iokit(int update_every, usec_t dt) { if (likely(do_bandwidth)) { if (unlikely(getifaddrs(&ifap))) { - error("MACOS: getifaddrs()"); + collector_error("MACOS: getifaddrs()"); do_bandwidth = 0; - error("DISABLED: system.ipv4"); + collector_error("DISABLED: system.ipv4"); } else { for (ifa = ifap; ifa; ifa = ifa->ifa_next) { if (ifa->ifa_addr->sa_family != AF_LINK) diff --git a/collectors/macos.plugin/macos_mach_smi.c b/collectors/macos.plugin/macos_mach_smi.c index 53b2607b4..f21a56af2 100644 --- a/collectors/macos.plugin/macos_mach_smi.c +++ b/collectors/macos.plugin/macos_mach_smi.c @@ -41,16 +41,16 @@ int do_macos_mach_smi(int update_every, usec_t dt) { if (likely(do_cpu)) { if (unlikely(HOST_CPU_LOAD_INFO_COUNT != 4)) { - error("MACOS: There are %d CPU states (4 was expected)", HOST_CPU_LOAD_INFO_COUNT); + collector_error("MACOS: There are %d CPU states (4 was expected)", HOST_CPU_LOAD_INFO_COUNT); do_cpu = 0; - error("DISABLED: system.cpu"); + collector_error("DISABLED: system.cpu"); } else { count = HOST_CPU_LOAD_INFO_COUNT; kr = host_statistics(host, HOST_CPU_LOAD_INFO, (host_info_t)cp_time, &count); if (unlikely(kr != KERN_SUCCESS)) { - error("MACOS: host_statistics() failed: %s", mach_error_string(kr)); + collector_error("MACOS: host_statistics() failed: %s", mach_error_string(kr)); do_cpu = 0; - error("DISABLED: system.cpu"); + collector_error("DISABLED: system.cpu"); } else { st = rrdset_find_active_bytype_localhost("system", "cpu"); @@ -95,13 +95,13 @@ int do_macos_mach_smi(int update_every, usec_t dt) { kr = host_statistics(host, HOST_VM_INFO, (host_info_t)&vm_statistics, &count); #endif if (unlikely(kr != KERN_SUCCESS)) { - error("MACOS: host_statistics64() failed: %s", mach_error_string(kr)); + collector_error("MACOS: host_statistics64() failed: %s", mach_error_string(kr)); do_ram = 0; - error("DISABLED: system.ram"); + collector_error("DISABLED: system.ram"); do_swapio = 0; - error("DISABLED: system.swapio"); + collector_error("DISABLED: system.swapio"); do_pgfaults = 0; - error("DISABLED: mem.pgfaults"); + collector_error("DISABLED: mem.pgfaults"); } else { if (likely(do_ram)) { st = rrdset_find_active_localhost("system.ram"); diff --git a/collectors/macos.plugin/macos_sysctl.c b/collectors/macos.plugin/macos_sysctl.c index 1f04f6e41..42f01d85a 100644 --- a/collectors/macos.plugin/macos_sysctl.c +++ b/collectors/macos.plugin/macos_sysctl.c @@ -222,7 +222,7 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_loadavg)) { if (unlikely(GETSYSCTL_BY_NAME("vm.loadavg", sysload))) { do_loadavg = 0; - error("DISABLED: system.load"); + collector_error("DISABLED: system.load"); } else { st = rrdset_find_active_bytype_localhost("system", "load"); @@ -260,7 +260,7 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_swap)) { if (unlikely(GETSYSCTL_BY_NAME("vm.swapusage", swap_usage))) { do_swap = 0; - error("DISABLED: system.swap"); + collector_error("DISABLED: system.swap"); } else { st = rrdset_find_active_localhost("system.swap"); if (unlikely(!st)) { @@ -298,15 +298,15 @@ int do_macos_sysctl(int update_every, usec_t dt) { mib[4] = NET_RT_IFLIST2; mib[5] = 0; if (unlikely(sysctl(mib, 6, NULL, &size, NULL, 0))) { - error("MACOS: sysctl(%s...) failed: %s", "net interfaces", strerror(errno)); + collector_error("MACOS: sysctl(%s...) failed: %s", "net interfaces", strerror(errno)); do_bandwidth = 0; - error("DISABLED: system.ipv4"); + collector_error("DISABLED: system.ipv4"); } else { ifstatdata = reallocz(ifstatdata, size); if (unlikely(sysctl(mib, 6, ifstatdata, &size, NULL, 0) < 0)) { - error("MACOS: sysctl(%s...) failed: %s", "net interfaces", strerror(errno)); + collector_error("MACOS: sysctl(%s...) failed: %s", "net interfaces", strerror(errno)); do_bandwidth = 0; - error("DISABLED: system.ipv4"); + collector_error("DISABLED: system.ipv4"); } else { lim = ifstatdata + size; iftot.ift_ibytes = iftot.ift_obytes = 0; @@ -353,19 +353,19 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_tcp_packets || do_tcp_errors || do_tcp_handshake || do_tcpext_connaborts || do_tcpext_ofo || do_tcpext_syscookies || do_ecn)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet.tcp.stats", tcpstat))){ do_tcp_packets = 0; - error("DISABLED: ipv4.tcppackets"); + collector_error("DISABLED: ipv4.tcppackets"); do_tcp_errors = 0; - error("DISABLED: ipv4.tcperrors"); + collector_error("DISABLED: ipv4.tcperrors"); do_tcp_handshake = 0; - error("DISABLED: ipv4.tcphandshake"); + collector_error("DISABLED: ipv4.tcphandshake"); do_tcpext_connaborts = 0; - error("DISABLED: ipv4.tcpconnaborts"); + collector_error("DISABLED: ipv4.tcpconnaborts"); do_tcpext_ofo = 0; - error("DISABLED: ipv4.tcpofo"); + collector_error("DISABLED: ipv4.tcpofo"); do_tcpext_syscookies = 0; - error("DISABLED: ipv4.tcpsyncookies"); + collector_error("DISABLED: ipv4.tcpsyncookies"); do_ecn = 0; - error("DISABLED: ipv4.ecnpkts"); + collector_error("DISABLED: ipv4.ecnpkts"); } else { if (likely(do_tcp_packets)) { st = rrdset_find_active_localhost("ipv4.tcppackets"); @@ -597,9 +597,9 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_udp_packets || do_udp_errors)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet.udp.stats", udpstat))) { do_udp_packets = 0; - error("DISABLED: ipv4.udppackets"); + collector_error("DISABLED: ipv4.udppackets"); do_udp_errors = 0; - error("DISABLED: ipv4.udperrors"); + collector_error("DISABLED: ipv4.udperrors"); } else { if (likely(do_udp_packets)) { st = rrdset_find_active_localhost("ipv4.udppackets"); @@ -673,10 +673,10 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_icmp_packets || do_icmpmsg)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet.icmp.stats", icmpstat))) { do_icmp_packets = 0; - error("DISABLED: ipv4.icmp"); - error("DISABLED: ipv4.icmp_errors"); + collector_error("DISABLED: ipv4.icmp"); + collector_error("DISABLED: ipv4.icmp_errors"); do_icmpmsg = 0; - error("DISABLED: ipv4.icmpmsg"); + collector_error("DISABLED: ipv4.icmpmsg"); } else { for (i = 0; i <= ICMP_MAXTYPE; i++) { icmp_total.msgs_in += icmpstat.icps_inhist[i]; @@ -777,13 +777,13 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_ip_packets || do_ip_fragsout || do_ip_fragsin || do_ip_errors)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet.ip.stats", ipstat))) { do_ip_packets = 0; - error("DISABLED: ipv4.packets"); + collector_error("DISABLED: ipv4.packets"); do_ip_fragsout = 0; - error("DISABLED: ipv4.fragsout"); + collector_error("DISABLED: ipv4.fragsout"); do_ip_fragsin = 0; - error("DISABLED: ipv4.fragsin"); + collector_error("DISABLED: ipv4.fragsin"); do_ip_errors = 0; - error("DISABLED: ipv4.errors"); + collector_error("DISABLED: ipv4.errors"); } else { if (likely(do_ip_packets)) { st = rrdset_find_active_localhost("ipv4.packets"); @@ -919,13 +919,13 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_ip6_packets || do_ip6_fragsout || do_ip6_fragsin || do_ip6_errors)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet6.ip6.stats", ip6stat))) { do_ip6_packets = 0; - error("DISABLED: ipv6.packets"); + collector_error("DISABLED: ipv6.packets"); do_ip6_fragsout = 0; - error("DISABLED: ipv6.fragsout"); + collector_error("DISABLED: ipv6.fragsout"); do_ip6_fragsin = 0; - error("DISABLED: ipv6.fragsin"); + collector_error("DISABLED: ipv6.fragsin"); do_ip6_errors = 0; - error("DISABLED: ipv6.errors"); + collector_error("DISABLED: ipv6.errors"); } else { if (do_ip6_packets == CONFIG_BOOLEAN_YES || (do_ip6_packets == CONFIG_BOOLEAN_AUTO && (ip6stat.ip6s_localout || @@ -1096,7 +1096,7 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_icmp6 || do_icmp6_redir || do_icmp6_errors || do_icmp6_echos || do_icmp6_router || do_icmp6_neighbor || do_icmp6_types)) { if (unlikely(GETSYSCTL_BY_NAME("net.inet6.icmp6.stats", icmp6stat))) { do_icmp6 = 0; - error("DISABLED: ipv6.icmp"); + collector_error("DISABLED: ipv6.icmp"); } else { for (i = 0; i <= ICMP6_MAXTYPE; i++) { icmp6_total.msgs_in += icmp6stat.icp6s_inhist[i]; @@ -1392,7 +1392,7 @@ int do_macos_sysctl(int update_every, usec_t dt) { if (likely(do_uptime)) { if (unlikely(GETSYSCTL_BY_NAME("kern.boottime", boot_time))) { do_uptime = 0; - error("DISABLED: system.uptime"); + collector_error("DISABLED: system.uptime"); } else { clock_gettime(CLOCK_REALTIME, &cur_time); st = rrdset_find_active_localhost("system.uptime"); diff --git a/collectors/macos.plugin/plugin_macos.c b/collectors/macos.plugin/plugin_macos.c index 10472bdb8..f3b860510 100644 --- a/collectors/macos.plugin/plugin_macos.c +++ b/collectors/macos.plugin/plugin_macos.c @@ -32,7 +32,7 @@ static void macos_main_cleanup(void *ptr) struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); static_thread->enabled = NETDATA_MAIN_THREAD_EXITED; } diff --git a/collectors/nfacct.plugin/README.md b/collectors/nfacct.plugin/README.md index bacc8b70e..f57625c82 100644 --- a/collectors/nfacct.plugin/README.md +++ b/collectors/nfacct.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "nfacct.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/nfacct.plugin/README.md +title: "Monitor Netfilter statistics (nfacct.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/nfacct.plugin/README.md" +sidebar_label: "Netfilter statistics (nfacct.plugin)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # nfacct.plugin diff --git a/collectors/nfacct.plugin/plugin_nfacct.c b/collectors/nfacct.plugin/plugin_nfacct.c index eeadb3ccc..430ceab52 100644 --- a/collectors/nfacct.plugin/plugin_nfacct.c +++ b/collectors/nfacct.plugin/plugin_nfacct.c @@ -92,14 +92,14 @@ static int nfstat_init(int update_every) { nfstat_root.mnl = mnl_socket_open(NETLINK_NETFILTER); if(!nfstat_root.mnl) { - error("NFSTAT: mnl_socket_open() failed"); + collector_error("NFSTAT: mnl_socket_open() failed"); return 1; } nfstat_root.seq = (unsigned int)now_realtime_sec() - 1; if(mnl_socket_bind(nfstat_root.mnl, 0, MNL_SOCKET_AUTOPID) < 0) { - error("NFSTAT: mnl_socket_bind() failed"); + collector_error("NFSTAT: mnl_socket_bind() failed"); return 1; } nfstat_root.portid = mnl_socket_get_portid(nfstat_root.mnl); @@ -132,7 +132,7 @@ static int nfct_stats_attr_cb(const struct nlattr *attr, void *data) { return MNL_CB_OK; if (mnl_attr_validate(attr, MNL_TYPE_U32) < 0) { - error("NFSTAT: mnl_attr_validate() failed"); + collector_error("NFSTAT: mnl_attr_validate() failed"); return MNL_CB_ERROR; } @@ -173,7 +173,7 @@ static int nfstat_collect_conntrack() { // send the request if(mnl_socket_sendto(nfstat_root.mnl, nfstat_root.nlh, nfstat_root.nlh->nlmsg_len) < 0) { - error("NFSTAT: mnl_socket_sendto() failed"); + collector_error("NFSTAT: mnl_socket_sendto() failed"); return 1; } @@ -193,7 +193,7 @@ static int nfstat_collect_conntrack() { // verify we run without issues if (ret == -1) { - error("NFSTAT: error communicating with kernel. This plugin can only work when netdata runs as root."); + collector_error("NFSTAT: error communicating with kernel. This plugin can only work when netdata runs as root."); return 1; } @@ -209,7 +209,7 @@ static int nfexp_stats_attr_cb(const struct nlattr *attr, void *data) return MNL_CB_OK; if (mnl_attr_validate(attr, MNL_TYPE_U32) < 0) { - error("NFSTAT EXP: mnl_attr_validate() failed"); + collector_error("NFSTAT EXP: mnl_attr_validate() failed"); return MNL_CB_ERROR; } @@ -245,7 +245,7 @@ static int nfstat_collect_conntrack_expectations() { // send the request if(mnl_socket_sendto(nfstat_root.mnl, nfstat_root.nlh, nfstat_root.nlh->nlmsg_len) < 0) { - error("NFSTAT: mnl_socket_sendto() failed"); + collector_error("NFSTAT: mnl_socket_sendto() failed"); return 1; } @@ -265,7 +265,7 @@ static int nfstat_collect_conntrack_expectations() { // verify we run without issues if (ret == -1) { - error("NFSTAT: error communicating with kernel. This plugin can only work when netdata runs as root."); + collector_error("NFSTAT: error communicating with kernel. This plugin can only work when netdata runs as root."); return 1; } @@ -561,7 +561,7 @@ static int nfacct_init(int update_every) { nfacct_root.nfacct_buffer = nfacct_alloc(); if(!nfacct_root.nfacct_buffer) { - error("nfacct.plugin: nfacct_alloc() failed."); + collector_error("nfacct.plugin: nfacct_alloc() failed."); return 0; } @@ -569,12 +569,12 @@ static int nfacct_init(int update_every) { nfacct_root.mnl = mnl_socket_open(NETLINK_NETFILTER); if(!nfacct_root.mnl) { - error("nfacct.plugin: mnl_socket_open() failed"); + collector_error("nfacct.plugin: mnl_socket_open() failed"); return 1; } if(mnl_socket_bind(nfacct_root.mnl, 0, MNL_SOCKET_AUTOPID) < 0) { - error("nfacct.plugin: mnl_socket_bind() failed"); + collector_error("nfacct.plugin: mnl_socket_bind() failed"); return 1; } nfacct_root.portid = mnl_socket_get_portid(nfacct_root.mnl); @@ -586,7 +586,7 @@ static int nfacct_callback(const struct nlmsghdr *nlh, void *data) { (void)data; if(nfacct_nlmsg_parse_payload(nlh, nfacct_root.nfacct_buffer) < 0) { - error("NFACCT: nfacct_nlmsg_parse_payload() failed."); + collector_error("NFACCT: nfacct_nlmsg_parse_payload() failed."); return MNL_CB_OK; } @@ -612,13 +612,13 @@ static int nfacct_collect() { nfacct_root.seq++; nfacct_root.nlh = nfacct_nlmsg_build_hdr(nfacct_root.buf, NFNL_MSG_ACCT_GET, NLM_F_DUMP, (uint32_t)nfacct_root.seq); if(!nfacct_root.nlh) { - error("NFACCT: nfacct_nlmsg_build_hdr() failed"); + collector_error("NFACCT: nfacct_nlmsg_build_hdr() failed"); return 1; } // send the request if(mnl_socket_sendto(nfacct_root.mnl, nfacct_root.nlh, nfacct_root.nlh->nlmsg_len) < 0) { - error("NFACCT: mnl_socket_sendto() failed"); + collector_error("NFACCT: mnl_socket_sendto() failed"); return 1; } @@ -638,7 +638,7 @@ static int nfacct_collect() { // verify we run without issues if (ret == -1) { - error("NFACCT: error communicating with kernel. This plugin can only work when netdata runs as root."); + collector_error("NFACCT: error communicating with kernel. This plugin can only work when netdata runs as root."); return 1; } @@ -740,11 +740,12 @@ void nfacct_signals() for (i = 0; signals[i]; i++) { if(sigaction(signals[i], &sa, NULL) == -1) - error("Cannot add the handler to signal %d", signals[i]); + collector_error("Cannot add the handler to signal %d", signals[i]); } } int main(int argc, char **argv) { + stderror = stderr; clocks_init(); // ------------------------------------------------------------------------ @@ -813,7 +814,7 @@ int main(int argc, char **argv) { exit(1); } - error("nfacct.plugin: ignoring parameter '%s'", argv[i]); + collector_error("nfacct.plugin: ignoring parameter '%s'", argv[i]); } nfacct_signals(); @@ -823,7 +824,7 @@ int main(int argc, char **argv) { if(freq >= netdata_update_every) netdata_update_every = freq; else if(freq) - error("update frequency %d seconds is too small for NFACCT. Using %d.", freq, netdata_update_every); + collector_error("update frequency %d seconds is too small for NFACCT. Using %d.", freq, netdata_update_every); if (debug) fprintf(stderr, "nfacct.plugin: calling nfacct_init()\n"); @@ -882,5 +883,5 @@ int main(int argc, char **argv) { if(now_monotonic_sec() - started_t > 14400) break; } - info("NFACCT process exiting"); + collector_info("NFACCT process exiting"); } diff --git a/collectors/perf.plugin/README.md b/collectors/perf.plugin/README.md index a7a87aca2..9e114363d 100644 --- a/collectors/perf.plugin/README.md +++ b/collectors/perf.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "perf.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/perf.plugin/README.md +title: "Monitor CPU performance statistics (perf.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/perf.plugin/README.md" +sidebar_label: "CPU performance statistics (perf.plugin)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # perf.plugin diff --git a/collectors/perf.plugin/perf_plugin.c b/collectors/perf.plugin/perf_plugin.c index b2f7d2e17..68c0f917d 100644 --- a/collectors/perf.plugin/perf_plugin.c +++ b/collectors/perf.plugin/perf_plugin.c @@ -294,15 +294,15 @@ static int perf_init() { if(unlikely(fd < 0)) { switch errno { case EACCES: - error("Cannot access to the PMU: Permission denied"); + collector_error("Cannot access to the PMU: Permission denied"); break; case EBUSY: - error("Another event already has exclusive access to the PMU"); + collector_error("Another event already has exclusive access to the PMU"); break; default: - error("Cannot open perf event"); + collector_error("Cannot open perf event"); } - error("Disabling event %u", current_event->id); + collector_error("Disabling event %u", current_event->id); current_event->disabled = 1; } @@ -346,7 +346,7 @@ static void reenable_events() { if(ioctl(current_fd, PERF_EVENT_IOC_DISABLE, PERF_IOC_FLAG_GROUP) == -1 || ioctl(current_fd, PERF_EVENT_IOC_ENABLE, PERF_IOC_FLAG_GROUP) == -1) { - error("Cannot reenable event group"); + collector_error("Cannot reenable event group"); } } } @@ -388,7 +388,7 @@ static int perf_collect() { current_event->updated = 1; } else { - error("Cannot update value for event %u", current_event->id); + collector_error("Cannot update value for event %u", current_event->id); return 1; } } @@ -1272,17 +1272,18 @@ void parse_command_line(int argc, char **argv) { exit(1); } - error("ignoring parameter '%s'", argv[i]); + collector_error("ignoring parameter '%s'", argv[i]); } if(!plugin_enabled){ - info("no charts enabled - nothing to do."); + collector_info("no charts enabled - nothing to do."); printf("DISABLE\n"); exit(1); } } int main(int argc, char **argv) { + stderror = stderr; clocks_init(); // ------------------------------------------------------------------------ @@ -1304,7 +1305,7 @@ int main(int argc, char **argv) { if(freq >= update_every) update_every = freq; else if(freq) - error("update frequency %d seconds is too small for PERF. Using %d.", freq, update_every); + collector_error("update frequency %d seconds is too small for PERF. Using %d.", freq, update_every); if(unlikely(debug)) fprintf(stderr, "perf.plugin: calling perf_init()\n"); int perf = !perf_init(); @@ -1348,6 +1349,6 @@ int main(int argc, char **argv) { if(now_monotonic_sec() - started_t > 14400) break; } - info("process exiting"); + collector_info("process exiting"); perf_free(); } diff --git a/collectors/plugins.d/README.md b/collectors/plugins.d/README.md index 2ecf233f7..8ad1d3a65 100644 --- a/collectors/plugins.d/README.md +++ b/collectors/plugins.d/README.md @@ -1,6 +1,10 @@ <!-- title: "External plugins overview" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/plugins.d/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/plugins.d/README.md" +sidebar_label: "External plugins overview" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "Developers" --> # External plugins overview @@ -12,17 +16,18 @@ from external processes, thus allowing Netdata to use **external plugins**. |plugin|language|O/S|description| |:----:|:------:|:-:|:----------| -|[apps.plugin](/collectors/apps.plugin/README.md)|`C`|linux, freebsd|monitors the whole process tree on Linux and FreeBSD and breaks down system resource usage by **process**, **user** and **user group**.| -|[charts.d.plugin](/collectors/charts.d.plugin/README.md)|`BASH`|all|a **plugin orchestrator** for data collection modules written in `BASH` v4+.| -|[cups.plugin](/collectors/cups.plugin/README.md)|`C`|all|monitors **CUPS**| -|[fping.plugin](/collectors/fping.plugin/README.md)|`C`|all|measures network latency, jitter and packet loss between the monitored node and any number of remote network end points.| -|[ioping.plugin](/collectors/ioping.plugin/README.md)|`C`|all|measures disk latency.| -|[freeipmi.plugin](/collectors/freeipmi.plugin/README.md)|`C`|linux|collects metrics from enterprise hardware sensors, on Linux servers.| -|[nfacct.plugin](/collectors/nfacct.plugin/README.md)|`C`|linux|collects netfilter firewall, connection tracker and accounting metrics using `libmnl` and `libnetfilter_acct`.| -|[xenstat.plugin](/collectors/xenstat.plugin/README.md)|`C`|linux|collects XenServer and XCP-ng metrics using `lxenstat`.| -|[perf.plugin](/collectors/perf.plugin/README.md)|`C`|linux|collects CPU performance metrics using performance monitoring units (PMU).| -|[python.d.plugin](/collectors/python.d.plugin/README.md)|`python`|all|a **plugin orchestrator** for data collection modules written in `python` v2 or v3 (both are supported).| -|[slabinfo.plugin](/collectors/slabinfo.plugin/README.md)|`C`|linux|collects kernel internal cache objects (SLAB) metrics.| +|[apps.plugin](https://github.com/netdata/netdata/blob/master/collectors/apps.plugin/README.md)|`C`|linux, freebsd|monitors the whole process tree on Linux and FreeBSD and breaks down system resource usage by **process**, **user** and **user group**.| +|[charts.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md)|`BASH`|all|a **plugin orchestrator** for data collection modules written in `BASH` v4+.| +|[cups.plugin](https://github.com/netdata/netdata/blob/master/collectors/cups.plugin/README.md)|`C`|all|monitors **CUPS**| +|[ebpf.plugin](https://github.com/netdata/netdata/blob/master/collectors/ebpf.plugin/README.md)|`C`|linux|monitors different metrics on environments using kernel internal functions.| +|[go.d.plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md)|`GO`|all|collects metrics from the system, applications, or third-party APIs.| +|[ioping.plugin](https://github.com/netdata/netdata/blob/master/collectors/ioping.plugin/README.md)|`C`|all|measures disk latency.| +|[freeipmi.plugin](https://github.com/netdata/netdata/blob/master/collectors/freeipmi.plugin/README.md)|`C`|linux|collects metrics from enterprise hardware sensors, on Linux servers.| +|[nfacct.plugin](https://github.com/netdata/netdata/blob/master/collectors/nfacct.plugin/README.md)|`C`|linux|collects netfilter firewall, connection tracker and accounting metrics using `libmnl` and `libnetfilter_acct`.| +|[xenstat.plugin](https://github.com/netdata/netdata/blob/master/collectors/xenstat.plugin/README.md)|`C`|linux|collects XenServer and XCP-ng metrics using `lxenstat`.| +|[perf.plugin](https://github.com/netdata/netdata/blob/master/collectors/perf.plugin/README.md)|`C`|linux|collects CPU performance metrics using performance monitoring units (PMU).| +|[python.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md)|`python`|all|a **plugin orchestrator** for data collection modules written in `python` v2 or v3 (both are supported).| +|[slabinfo.plugin](https://github.com/netdata/netdata/blob/master/collectors/slabinfo.plugin/README.md)|`C`|linux|collects kernel internal cache objects (SLAB) metrics.| Plugin orchestrators may also be described as **modular plugins**. They are modular since they accept custom made modules to be included. Writing modules for these plugins is easier than accessing the native Netdata API directly. You will find modules already available for each orchestrator under the directory of the particular modular plugin (e.g. under python.d.plugin for the python orchestrator). Each of these modular plugins has each own methods for defining modules. Please check the examples and their documentation. @@ -71,7 +76,6 @@ Example: # check for new plugins every = 60 # charts.d = yes - # fping = yes # ioping = yes # python.d = yes ``` @@ -504,12 +508,12 @@ or do not output the line at all. ## Modular Plugins 1. **python**, use `python.d.plugin`, there are many examples in the [python.d - directory](/collectors/python.d.plugin/README.md) + directory](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md) python is ideal for Netdata plugins. It is a simple, yet powerful way to collect data, it has a very small memory footprint, although it is not the most CPU efficient way to do it. 2. **BASH**, use `charts.d.plugin`, there are many examples in the [charts.d - directory](/collectors/charts.d.plugin/README.md) + directory](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md) BASH is the simplest scripting language for collecting values. It is the less efficient though in terms of CPU resources. You can use it to collect data quickly, but extensive use of it might use a lot of system resources. diff --git a/collectors/plugins.d/plugins_d.c b/collectors/plugins.d/plugins_d.c index 79abc7070..7608f3afc 100644 --- a/collectors/plugins.d/plugins_d.c +++ b/collectors/plugins.d/plugins_d.c @@ -21,23 +21,54 @@ inline size_t pluginsd_initialize_plugin_directories() return quoted_strings_splitter(plugins_dir_list, plugin_directories, PLUGINSD_MAX_DIRECTORIES, config_isspace, NULL, NULL, 0); } +static inline void plugin_set_disabled(struct plugind *cd) { + netdata_spinlock_lock(&cd->unsafe.spinlock); + cd->unsafe.enabled = false; + netdata_spinlock_unlock(&cd->unsafe.spinlock); +} + +bool plugin_is_enabled(struct plugind *cd) { + netdata_spinlock_lock(&cd->unsafe.spinlock); + bool ret = cd->unsafe.enabled; + netdata_spinlock_unlock(&cd->unsafe.spinlock); + return ret; +} + +static inline void plugin_set_running(struct plugind *cd) { + netdata_spinlock_lock(&cd->unsafe.spinlock); + cd->unsafe.running = true; + netdata_spinlock_unlock(&cd->unsafe.spinlock); +} + +static inline bool plugin_is_running(struct plugind *cd) { + netdata_spinlock_lock(&cd->unsafe.spinlock); + bool ret = cd->unsafe.running; + netdata_spinlock_unlock(&cd->unsafe.spinlock); + return ret; +} + static void pluginsd_worker_thread_cleanup(void *arg) { struct plugind *cd = (struct plugind *)arg; - if (cd->enabled && !cd->obsolete) { - cd->obsolete = 1; + netdata_spinlock_lock(&cd->unsafe.spinlock); + + cd->unsafe.running = false; + cd->unsafe.thread = 0; + pid_t pid = cd->unsafe.pid; + cd->unsafe.pid = 0; + + netdata_spinlock_unlock(&cd->unsafe.spinlock); + + if (pid) { info("data collection thread exiting"); - if (cd->pid) { - siginfo_t info; - info("killing child process pid %d", cd->pid); - if (killpid(cd->pid) != -1) { - info("waiting for child process pid %d to exit...", cd->pid); - waitid(P_PID, (id_t)cd->pid, &info, WEXITED); - } - cd->pid = 0; + siginfo_t info; + info("killing child process pid %d", pid); + if (killpid(pid) != -1) { + info("waiting for child process pid %d to exit...", pid); + waitid(P_PID, (id_t)pid, &info, WEXITED); } } } @@ -53,8 +84,8 @@ static void pluginsd_worker_thread_handle_success(struct plugind *cd) if (likely(cd->serial_failures <= SERIAL_FAILURES_THRESHOLD)) { info( "'%s' (pid %d) does not generate useful output but it reports success (exits with 0). %s.", - cd->fullfilename, cd->pid, - cd->enabled ? "Waiting a bit before starting it again." : "Will not start it again - it is now disabled."); + cd->fullfilename, cd->unsafe.pid, + plugin_is_enabled(cd) ? "Waiting a bit before starting it again." : "Will not start it again - it is now disabled."); sleep((unsigned int)(cd->update_every * 10)); return; } @@ -63,35 +94,33 @@ static void pluginsd_worker_thread_handle_success(struct plugind *cd) error( "'%s' (pid %d) does not generate useful output, although it reports success (exits with 0)." "We have tried to collect something %zu times - unsuccessfully. Disabling it.", - cd->fullfilename, cd->pid, cd->serial_failures); - cd->enabled = 0; + cd->fullfilename, cd->unsafe.pid, cd->serial_failures); + plugin_set_disabled(cd); return; } - - return; } static void pluginsd_worker_thread_handle_error(struct plugind *cd, int worker_ret_code) { if (worker_ret_code == -1) { - info("'%s' (pid %d) was killed with SIGTERM. Disabling it.", cd->fullfilename, cd->pid); - cd->enabled = 0; + info("'%s' (pid %d) was killed with SIGTERM. Disabling it.", cd->fullfilename, cd->unsafe.pid); + plugin_set_disabled(cd); return; } if (!cd->successful_collections) { error( "'%s' (pid %d) exited with error code %d and haven't collected any data. Disabling it.", cd->fullfilename, - cd->pid, worker_ret_code); - cd->enabled = 0; + cd->unsafe.pid, worker_ret_code); + plugin_set_disabled(cd); return; } if (cd->serial_failures <= SERIAL_FAILURES_THRESHOLD) { error( "'%s' (pid %d) exited with error code %d, but has given useful output in the past (%zu times). %s", - cd->fullfilename, cd->pid, worker_ret_code, cd->successful_collections, - cd->enabled ? "Waiting a bit before starting it again." : "Will not start it again - it is disabled."); + cd->fullfilename, cd->unsafe.pid, worker_ret_code, cd->successful_collections, + plugin_is_enabled(cd) ? "Waiting a bit before starting it again." : "Will not start it again - it is disabled."); sleep((unsigned int)(cd->update_every * 10)); return; } @@ -100,48 +129,47 @@ static void pluginsd_worker_thread_handle_error(struct plugind *cd, int worker_r error( "'%s' (pid %d) exited with error code %d, but has given useful output in the past (%zu times)." "We tried to restart it %zu times, but it failed to generate data. Disabling it.", - cd->fullfilename, cd->pid, worker_ret_code, cd->successful_collections, cd->serial_failures); - cd->enabled = 0; + cd->fullfilename, cd->unsafe.pid, worker_ret_code, cd->successful_collections, cd->serial_failures); + plugin_set_disabled(cd); return; } - - return; } + #undef SERIAL_FAILURES_THRESHOLD -void *pluginsd_worker_thread(void *arg) +static void *pluginsd_worker_thread(void *arg) { worker_register("PLUGINSD"); netdata_thread_cleanup_push(pluginsd_worker_thread_cleanup, arg); struct plugind *cd = (struct plugind *)arg; + plugin_set_running(cd); - cd->obsolete = 0; size_t count = 0; - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { FILE *fp_child_input = NULL; - FILE *fp_child_output = netdata_popen(cd->cmd, &cd->pid, &fp_child_input); + FILE *fp_child_output = netdata_popen(cd->cmd, &cd->unsafe.pid, &fp_child_input); if (unlikely(!fp_child_input || !fp_child_output)) { error("Cannot popen(\"%s\", \"r\").", cd->cmd); break; } - info("connected to '%s' running on pid %d", cd->fullfilename, cd->pid); + info("connected to '%s' running on pid %d", cd->fullfilename, cd->unsafe.pid); count = pluginsd_process(localhost, cd, fp_child_input, fp_child_output, 0); - error("'%s' (pid %d) disconnected after %zu successful data collections (ENDs).", cd->fullfilename, cd->pid, count); - killpid(cd->pid); + error("'%s' (pid %d) disconnected after %zu successful data collections (ENDs).", cd->fullfilename, cd->unsafe.pid, count); + killpid(cd->unsafe.pid); - int worker_ret_code = netdata_pclose(fp_child_input, fp_child_output, cd->pid); + int worker_ret_code = netdata_pclose(fp_child_input, fp_child_output, cd->unsafe.pid); if (likely(worker_ret_code == 0)) pluginsd_worker_thread_handle_success(cd); else pluginsd_worker_thread_handle_error(cd, worker_ret_code); - cd->pid = 0; - if (unlikely(!cd->enabled)) + cd->unsafe.pid = 0; + if (unlikely(!plugin_is_enabled(cd))) break; } worker_unregister(); @@ -158,10 +186,12 @@ static void pluginsd_main_cleanup(void *data) struct plugind *cd; for (cd = pluginsd_root; cd; cd = cd->next) { - if (cd->enabled && !cd->obsolete) { + netdata_spinlock_lock(&cd->unsafe.spinlock); + if (cd->unsafe.enabled && cd->unsafe.running && cd->unsafe.thread != 0) { info("stopping plugin thread: %s", cd->id); - netdata_thread_cancel(cd->thread); + netdata_thread_cancel(cd->unsafe.thread); } + netdata_spinlock_unlock(&cd->unsafe.spinlock); } info("cleanup completed."); @@ -186,12 +216,12 @@ void *pluginsd_main(void *ptr) // so that we don't log broken directories on each loop int directory_errors[PLUGINSD_MAX_DIRECTORIES] = { 0 }; - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { int idx; const char *directory_name; for (idx = 0; idx < PLUGINSD_MAX_DIRECTORIES && (directory_name = plugin_directories[idx]); idx++) { - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; errno = 0; @@ -206,7 +236,7 @@ void *pluginsd_main(void *ptr) struct dirent *file = NULL; while (likely((file = readdir(dir)))) { - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; debug(D_PLUGINSD, "examining file '%s'", file->d_name); @@ -237,7 +267,7 @@ void *pluginsd_main(void *ptr) if (unlikely(strcmp(cd->filename, file->d_name) == 0)) break; - if (likely(cd && !cd->obsolete)) { + if (likely(cd && plugin_is_running(cd))) { debug(D_PLUGINSD, "plugin '%s' is already running", cd->filename); continue; } @@ -252,7 +282,9 @@ void *pluginsd_main(void *ptr) strncpyz(cd->filename, file->d_name, FILENAME_MAX); snprintfz(cd->fullfilename, FILENAME_MAX, "%s/%s", directory_name, cd->filename); - cd->enabled = enabled; + cd->unsafe.enabled = enabled; + cd->unsafe.running = false; + cd->update_every = (int)config_get_number(cd->id, "update every", localhost->rrd_update_every); cd->started_t = now_realtime_sec(); @@ -266,15 +298,16 @@ void *pluginsd_main(void *ptr) cd->next = pluginsd_root; pluginsd_root = cd; - // it is not currently running - cd->obsolete = 1; - - if (cd->enabled) { + if (plugin_is_enabled(cd)) { char tag[NETDATA_THREAD_TAG_MAX + 1]; - snprintfz(tag, NETDATA_THREAD_TAG_MAX, "PLUGINSD[%s]", pluginname); + snprintfz(tag, NETDATA_THREAD_TAG_MAX, "PD[%s]", pluginname); + // spawn a new thread for it - netdata_thread_create( - &cd->thread, tag, NETDATA_THREAD_OPTION_DEFAULT, pluginsd_worker_thread, cd); + netdata_thread_create(&cd->unsafe.thread, + tag, + NETDATA_THREAD_OPTION_DEFAULT, + pluginsd_worker_thread, + cd); } } } diff --git a/collectors/plugins.d/plugins_d.h b/collectors/plugins.d/plugins_d.h index a8acf038a..35af9fe58 100644 --- a/collectors/plugins.d/plugins_d.h +++ b/collectors/plugins.d/plugins_d.h @@ -50,9 +50,6 @@ struct plugind { char fullfilename[FILENAME_MAX+1]; // with path char cmd[PLUGINSD_CMD_MAX+1]; // the command that it executes - volatile pid_t pid; - netdata_thread_t thread; - size_t successful_collections; // the number of times we have seen // values collected from this plugin @@ -60,8 +57,14 @@ struct plugind { // without collecting values int update_every; // the plugin default data collection frequency - volatile sig_atomic_t obsolete; // do not touch this structure after setting this to 1 - volatile sig_atomic_t enabled; // if this is enabled or not + + struct { + SPINLOCK spinlock; + bool running; // do not touch this structure after setting this to 1 + bool enabled; // if this is enabled or not + netdata_thread_t thread; + pid_t pid; + } unsafe; time_t started_t; uint32_t capabilities; // follows the same principles as streaming capabilities diff --git a/collectors/plugins.d/pluginsd_parser.c b/collectors/plugins.d/pluginsd_parser.c index 5501c12fa..2c0f2cbc6 100644 --- a/collectors/plugins.d/pluginsd_parser.c +++ b/collectors/plugins.d/pluginsd_parser.c @@ -148,8 +148,11 @@ PARSER_RC pluginsd_begin(char **words, size_t num_words, void *user) ((PARSER_USER_OBJECT *)user)->st = st; usec_t microseconds = 0; - if (microseconds_txt && *microseconds_txt) - microseconds = str2ull(microseconds_txt); + if (microseconds_txt && *microseconds_txt) { + long long t = str2ll(microseconds_txt, NULL); + if(t >= 0) + microseconds = t; + } #ifdef NETDATA_LOG_REPLICATION_REQUESTS if(st->replay.log_next_data_collection) { @@ -326,7 +329,7 @@ PARSER_RC pluginsd_chart_definition_end(char **words, size_t num_words, void *us { const char *first_entry_txt = get_word(words, num_words, 1); const char *last_entry_txt = get_word(words, num_words, 2); - const char *world_time_txt = get_word(words, num_words, 3); + const char *wall_clock_time_txt = get_word(words, num_words, 3); RRDHOST *host = pluginsd_require_host_from_parent(user, PLUGINSD_KEYWORD_CHART_DEFINITION_END); if(!host) return PLUGINSD_DISABLE_PLUGIN(user); @@ -336,12 +339,7 @@ PARSER_RC pluginsd_chart_definition_end(char **words, size_t num_words, void *us time_t first_entry_child = (first_entry_txt && *first_entry_txt) ? (time_t)str2ul(first_entry_txt) : 0; time_t last_entry_child = (last_entry_txt && *last_entry_txt) ? (time_t)str2ul(last_entry_txt) : 0; - time_t child_world_time = (world_time_txt && *world_time_txt) ? (time_t)str2ul(world_time_txt) : now_realtime_sec(); - - if((first_entry_child != 0 || last_entry_child != 0) && (first_entry_child == 0 || last_entry_child == 0)) - error("PLUGINSD REPLAY ERROR: 'host:%s/chart:%s' got a " PLUGINSD_KEYWORD_CHART_DEFINITION_END " with malformed timings (first time %ld, last time %ld, world time %ld).", - rrdhost_hostname(host), rrdset_id(st), - first_entry_child, last_entry_child, child_world_time); + time_t child_wall_clock_time = (wall_clock_time_txt && *wall_clock_time_txt) ? (time_t)str2ul(wall_clock_time_txt) : now_realtime_sec(); bool ok = true; if(!rrdset_flag_check(st, RRDSET_FLAG_RECEIVER_REPLICATION_IN_PROGRESS)) { @@ -358,7 +356,7 @@ PARSER_RC pluginsd_chart_definition_end(char **words, size_t num_words, void *us PARSER *parser = ((PARSER_USER_OBJECT *)user)->parser; ok = replicate_chart_request(send_to_plugin, parser, host, st, - first_entry_child, last_entry_child, child_world_time, + first_entry_child, last_entry_child, child_wall_clock_time, 0, 0); } #ifdef NETDATA_LOG_REPLICATION_REQUESTS @@ -441,19 +439,20 @@ PARSER_RC pluginsd_dimension(char **words, size_t num_words, void *user) } else rrddim_isnot_obsolete(st, rd); + bool should_update_dimension = false; + if (likely(unhide_dimension)) { rrddim_option_clear(rd, RRDDIM_OPTION_HIDDEN); - if (rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN)) { - rrddim_flag_clear(rd, RRDDIM_FLAG_META_HIDDEN); - metaqueue_dimension_update_flags(rd); - } + should_update_dimension = rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN); } else { rrddim_option_set(rd, RRDDIM_OPTION_HIDDEN); - if (!rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN)) { - rrddim_flag_set(rd, RRDDIM_FLAG_META_HIDDEN); - metaqueue_dimension_update_flags(rd); - } + should_update_dimension = !rrddim_flag_check(rd, RRDDIM_FLAG_META_HIDDEN); + } + + if (should_update_dimension) { + rrddim_flag_set(rd, RRDDIM_FLAG_METADATA_UPDATE); + rrdhost_flag_set(rd->rrdset->rrdhost, RRDHOST_FLAG_METADATA_UPDATE); } return PARSER_RC_OK; @@ -529,7 +528,7 @@ static void inflight_functions_delete_callback(const DICTIONARY_ITEM *item __may } void inflight_functions_init(PARSER *parser) { - parser->inflight.functions = dictionary_create(DICT_OPTION_DONT_OVERWRITE_VALUE); + parser->inflight.functions = dictionary_create_advanced(DICT_OPTION_DONT_OVERWRITE_VALUE, &dictionary_stats_category_functions, 0); dictionary_register_insert_callback(parser->inflight.functions, inflight_functions_insert_callback, parser); dictionary_register_delete_callback(parser->inflight.functions, inflight_functions_delete_callback, parser); dictionary_register_conflict_callback(parser->inflight.functions, inflight_functions_conflict_callback, parser); @@ -883,7 +882,7 @@ PARSER_RC pluginsd_overwrite(char **words __maybe_unused, size_t num_words __may host->rrdlabels = rrdlabels_create(); rrdlabels_migrate_to_these(host->rrdlabels, (DICTIONARY *) (((PARSER_USER_OBJECT *)user)->new_host_labels)); - metaqueue_store_host_labels(host->machine_guid); + rrdhost_flag_set(host, RRDHOST_FLAG_METADATA_LABELS | RRDHOST_FLAG_METADATA_UPDATE); rrdlabels_destroy(((PARSER_USER_OBJECT *)user)->new_host_labels); ((PARSER_USER_OBJECT *)user)->new_host_labels = NULL; @@ -991,7 +990,7 @@ PARSER_RC pluginsd_replay_rrdset_begin(char **words, size_t num_words, void *use if(start_time && end_time && start_time < wall_clock_time + tolerance && end_time < wall_clock_time + tolerance && start_time < end_time) { if (unlikely(end_time - start_time != st->update_every)) - rrdset_set_update_every(st, end_time - start_time); + rrdset_set_update_every_s(st, end_time - start_time); st->last_collected_time.tv_sec = end_time; st->last_collected_time.tv_usec = 0; @@ -1124,6 +1123,9 @@ PARSER_RC pluginsd_replay_set(char **words, size_t num_words, void *user) PARSER_RC pluginsd_replay_rrddim_collection_state(char **words, size_t num_words, void *user) { + if(((PARSER_USER_OBJECT *) user)->replay.rset_enabled == false) + return PARSER_RC_OK; + char *dimension = get_word(words, num_words, 1); char *last_collected_ut_str = get_word(words, num_words, 2); char *last_collected_value_str = get_word(words, num_words, 3); @@ -1156,6 +1158,9 @@ PARSER_RC pluginsd_replay_rrddim_collection_state(char **words, size_t num_words PARSER_RC pluginsd_replay_rrdset_collection_state(char **words, size_t num_words, void *user) { + if(((PARSER_USER_OBJECT *) user)->replay.rset_enabled == false) + return PARSER_RC_OK; + char *last_collected_ut_str = get_word(words, num_words, 1); char *last_updated_ut_str = get_word(words, num_words, 2); @@ -1238,7 +1243,8 @@ PARSER_RC pluginsd_replay_end(char **words, size_t num_words, void *user) time_t started = st->rrdhost->receiver->replication_first_time_t; time_t current = ((PARSER_USER_OBJECT *) user)->replay.end_time; - worker_set_metric(WORKER_RECEIVER_JOB_REPLICATION_COMPLETION, + if(started && current > started) + worker_set_metric(WORKER_RECEIVER_JOB_REPLICATION_COMPLETION, (NETDATA_DOUBLE)(current - started) * 100.0 / (NETDATA_DOUBLE)(now - started)); } @@ -1251,6 +1257,7 @@ PARSER_RC pluginsd_replay_end(char **words, size_t num_words, void *user) st->counter++; st->counter_done++; + store_metric_collection_completed(); #ifdef NETDATA_LOG_REPLICATION_REQUESTS st->replay.start_streaming = false; @@ -1262,7 +1269,7 @@ PARSER_RC pluginsd_replay_end(char **words, size_t num_words, void *user) if (start_streaming) { if (st->update_every != update_every_child) - rrdset_set_update_every(st, update_every_child); + rrdset_set_update_every_s(st, update_every_child); if(rrdset_flag_check(st, RRDSET_FLAG_RECEIVER_REPLICATION_IN_PROGRESS)) { rrdset_flag_set(st, RRDSET_FLAG_RECEIVER_REPLICATION_FINISHED); @@ -1298,10 +1305,10 @@ static void pluginsd_process_thread_cleanup(void *ptr) { inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugin_input, FILE *fp_plugin_output, int trust_durations) { - int enabled = cd->enabled; + int enabled = cd->unsafe.enabled; if (!fp_plugin_input || !fp_plugin_output || !enabled) { - cd->enabled = 0; + cd->unsafe.enabled = 0; return 0; } @@ -1321,7 +1328,7 @@ inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugi clearerr(fp_plugin_output); PARSER_USER_OBJECT user = { - .enabled = cd->enabled, + .enabled = cd->unsafe.enabled, .host = host, .cd = cd, .trust_durations = trust_durations @@ -1339,14 +1346,14 @@ inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugi user.parser = parser; while (likely(!parser_next(parser))) { - if (unlikely(netdata_exit || parser_action(parser, NULL))) + if (unlikely(!service_running(SERVICE_COLLECTORS) || parser_action(parser, NULL))) break; } // free parser with the pop function netdata_thread_cleanup_pop(1); - cd->enabled = user.enabled; + cd->unsafe.enabled = user.enabled; size_t count = user.count; if (likely(count)) { diff --git a/collectors/proc.plugin/README.md b/collectors/proc.plugin/README.md index 32e2112af..f03550604 100644 --- a/collectors/proc.plugin/README.md +++ b/collectors/proc.plugin/README.md @@ -1,6 +1,10 @@ <!-- -title: "proc.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/proc.plugin/README.md +title: "OS provided metrics (proc.plugin)" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/proc.plugin/README.md" +sidebar_label: "OS provided metrics (proc.plugin)" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # proc.plugin @@ -400,7 +404,7 @@ You can set the following values for each configuration option: There are several alarms defined in `health.d/net.conf`. -The tricky ones are `inbound packets dropped` and `inbound packets dropped ratio`. They have quite a strict policy so that they warn users about possible issues. These alarms can be annoying for some network configurations. It is especially true for some bonding configurations if an interface is a child or a bonding interface itself. If it is expected to have a certain number of drops on an interface for a certain network configuration, a separate alarm with different triggering thresholds can be created or the existing one can be disabled for this specific interface. It can be done with the help of the [families](/health/REFERENCE.md#alarm-line-families) line in the alarm configuration. For example, if you want to disable the `inbound packets dropped` alarm for `eth0`, set `families: !eth0 *` in the alarm definition for `template: inbound_packets_dropped`. +The tricky ones are `inbound packets dropped` and `inbound packets dropped ratio`. They have quite a strict policy so that they warn users about possible issues. These alarms can be annoying for some network configurations. It is especially true for some bonding configurations if an interface is a child or a bonding interface itself. If it is expected to have a certain number of drops on an interface for a certain network configuration, a separate alarm with different triggering thresholds can be created or the existing one can be disabled for this specific interface. It can be done with the help of the [families](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alarm-line-families) line in the alarm configuration. For example, if you want to disable the `inbound packets dropped` alarm for `eth0`, set `families: !eth0 *` in the alarm definition for `template: inbound_packets_dropped`. #### configuration diff --git a/collectors/proc.plugin/ipc.c b/collectors/proc.plugin/ipc.c index 7d3d2ecbb..adfc15be5 100644 --- a/collectors/proc.plugin/ipc.c +++ b/collectors/proc.plugin/ipc.c @@ -82,7 +82,7 @@ static inline int ipc_sem_get_limits(struct ipc_limits *lim) { ff = procfile_open(filename, NULL, PROCFILE_FLAG_DEFAULT); if(unlikely(!ff)) { if(unlikely(!error_shown)) { - error("IPC: Cannot open file '%s'.", filename); + collector_error("IPC: Cannot open file '%s'.", filename); error_shown = 1; } goto ipc; @@ -92,7 +92,7 @@ static inline int ipc_sem_get_limits(struct ipc_limits *lim) { ff = procfile_readall(ff); if(unlikely(!ff)) { if(unlikely(!error_shown)) { - error("IPC: Cannot read file '%s'.", filename); + collector_error("IPC: Cannot read file '%s'.", filename); error_shown = 1; } goto ipc; @@ -108,7 +108,7 @@ static inline int ipc_sem_get_limits(struct ipc_limits *lim) { } else { if(unlikely(!error_shown)) { - error("IPC: Invalid content in file '%s'.", filename); + collector_error("IPC: Invalid content in file '%s'.", filename); error_shown = 1; } goto ipc; @@ -122,7 +122,7 @@ ipc: union semun arg = {.array = (ushort *) &seminfo}; if(unlikely(semctl(0, 0, IPC_INFO, arg) < 0)) { - error("IPC: Failed to read '%s' and request IPC_INFO with semctl().", filename); + collector_error("IPC: Failed to read '%s' and request IPC_INFO with semctl().", filename); goto error; } @@ -166,7 +166,7 @@ static inline int ipc_sem_get_status(struct ipc_status *st) { /* kernel not configured for semaphores */ static int error_shown = 0; if(unlikely(!error_shown)) { - error("IPC: kernel is not configured for semaphores"); + collector_error("IPC: kernel is not configured for semaphores"); error_shown = 1; } st->semusz = 0; @@ -195,7 +195,7 @@ int ipc_msq_get_info(char *msg_filename, struct message_queue **message_queue_ro size_t words = 0; if(unlikely(lines < 2)) { - error("Cannot read %s. Expected 2 or more lines, read %zu.", procfile_filename(ff), lines); + collector_error("Cannot read %s. Expected 2 or more lines, read %zu.", procfile_filename(ff), lines); return 1; } @@ -205,7 +205,7 @@ int ipc_msq_get_info(char *msg_filename, struct message_queue **message_queue_ro words = procfile_linewords(ff, l); if(unlikely(words < 2)) continue; if(unlikely(words < 14)) { - error("Cannot read %s line. Expected 14 params, read %zu.", procfile_filename(ff), words); + collector_error("Cannot read %s line. Expected 14 params, read %zu.", procfile_filename(ff), words); continue; } @@ -250,7 +250,7 @@ int ipc_shm_get_info(char *shm_filename, struct shm_stats *shm) { size_t words = 0; if(unlikely(lines < 2)) { - error("Cannot read %s. Expected 2 or more lines, read %zu.", procfile_filename(ff), lines); + collector_error("Cannot read %s. Expected 2 or more lines, read %zu.", procfile_filename(ff), lines); return 1; } @@ -263,7 +263,7 @@ int ipc_shm_get_info(char *shm_filename, struct shm_stats *shm) { words = procfile_linewords(ff, l); if(unlikely(words < 2)) continue; if(unlikely(words < 16)) { - error("Cannot read %s line. Expected 16 params, read %zu.", procfile_filename(ff), words); + collector_error("Cannot read %s line. Expected 16 params, read %zu.", procfile_filename(ff), words); continue; } @@ -306,11 +306,11 @@ int do_ipc(int update_every, usec_t dt) { // make sure it works if(ipc_sem_get_limits(&limits) == -1) { - error("unable to fetch semaphore limits"); + collector_error("unable to fetch semaphore limits"); do_sem = CONFIG_BOOLEAN_NO; } else if(ipc_sem_get_status(&status) == -1) { - error("unable to fetch semaphore statistics"); + collector_error("unable to fetch semaphore statistics"); do_sem = CONFIG_BOOLEAN_NO; } else { @@ -362,7 +362,7 @@ int do_ipc(int update_every, usec_t dt) { } if(unlikely(do_sem == CONFIG_BOOLEAN_NO && do_msg == CONFIG_BOOLEAN_NO)) { - error("ipc module disabled"); + collector_error("ipc module disabled"); return 1; } } @@ -370,7 +370,7 @@ int do_ipc(int update_every, usec_t dt) { if(likely(do_sem != CONFIG_BOOLEAN_NO)) { if(unlikely(read_limits_next < 0)) { if(unlikely(ipc_sem_get_limits(&limits) == -1)) { - error("Unable to fetch semaphore limits."); + collector_error("Unable to fetch semaphore limits."); } else { if(semaphores_max) rrdvar_custom_host_variable_set(localhost, semaphores_max, limits.semmns); @@ -386,7 +386,7 @@ int do_ipc(int update_every, usec_t dt) { read_limits_next--; if(unlikely(ipc_sem_get_status(&status) == -1)) { - error("Unable to get semaphore statistics"); + collector_error("Unable to get semaphore statistics"); return 0; } @@ -478,8 +478,8 @@ int do_ipc(int update_every, usec_t dt) { long long dimensions_num = rrdset_number_of_dimensions(st_msq_messages); if(unlikely(dimensions_num > dimensions_limit)) { - info("Message queue statistics has been disabled"); - info("There are %lld dimensions in memory but limit was set to %lld", dimensions_num, dimensions_limit); + collector_info("Message queue statistics has been disabled"); + collector_info("There are %lld dimensions in memory but limit was set to %lld", dimensions_num, dimensions_limit); rrdset_is_obsolete(st_msq_messages); rrdset_is_obsolete(st_msq_bytes); st_msq_messages = NULL; @@ -487,11 +487,11 @@ int do_ipc(int update_every, usec_t dt) { do_msg = CONFIG_BOOLEAN_NO; } else if(unlikely(!message_queue_root)) { - info("Making chart %s (%s) obsolete since it does not have any dimensions", rrdset_name(st_msq_messages), rrdset_id(st_msq_messages)); + collector_info("Making chart %s (%s) obsolete since it does not have any dimensions", rrdset_name(st_msq_messages), rrdset_id(st_msq_messages)); rrdset_is_obsolete(st_msq_messages); st_msq_messages = NULL; - info("Making chart %s (%s) obsolete since it does not have any dimensions", rrdset_name(st_msq_bytes), rrdset_id(st_msq_bytes)); + collector_info("Making chart %s (%s) obsolete since it does not have any dimensions", rrdset_name(st_msq_bytes), rrdset_id(st_msq_bytes)); rrdset_is_obsolete(st_msq_bytes); st_msq_bytes = NULL; } diff --git a/collectors/proc.plugin/plugin_proc.c b/collectors/proc.plugin/plugin_proc.c index 1b24df45f..1f52713ce 100644 --- a/collectors/proc.plugin/plugin_proc.c +++ b/collectors/proc.plugin/plugin_proc.c @@ -86,7 +86,7 @@ static void proc_main_cleanup(void *ptr) struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); if (netdev_thread) { netdata_thread_join(*netdev_thread, NULL); @@ -98,6 +98,41 @@ static void proc_main_cleanup(void *ptr) worker_unregister(); } +bool inside_lxc_container = false; + +static bool is_lxcfs_proc_mounted() { + procfile *ff = NULL; + + if (unlikely(!ff)) { + char filename[FILENAME_MAX + 1]; + snprintfz(filename, FILENAME_MAX, "/proc/self/mounts"); + ff = procfile_open(filename, " \t", PROCFILE_FLAG_DEFAULT); + if (unlikely(!ff)) + return false; + } + + ff = procfile_readall(ff); + if (unlikely(!ff)) + return false; + + unsigned long l, lines = procfile_lines(ff); + + for (l = 0; l < lines; l++) { + size_t words = procfile_linewords(ff, l); + if (words < 2) { + continue; + } + if (!strcmp(procfile_lineword(ff, l, 0), "lxcfs") && !strncmp(procfile_lineword(ff, l, 1), "/proc", 5)) { + procfile_close(ff); + return true; + } + } + + procfile_close(ff); + + return false; +} + void *proc_main(void *ptr) { worker_register("PROC"); @@ -128,15 +163,17 @@ void *proc_main(void *ptr) heartbeat_t hb; heartbeat_init(&hb); - while (!netdata_exit) { + inside_lxc_container = is_lxcfs_proc_mounted(); + + while (service_running(SERVICE_COLLECTORS)) { worker_is_idle(); usec_t hb_dt = heartbeat_next(&hb, step); - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; for (i = 0; proc_modules[i].name; i++) { - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; struct proc_module *pm = &proc_modules[i]; diff --git a/collectors/proc.plugin/plugin_proc.h b/collectors/proc.plugin/plugin_proc.h index d67ccd6e5..e8746ba36 100644 --- a/collectors/proc.plugin/plugin_proc.h +++ b/collectors/proc.plugin/plugin_proc.h @@ -8,7 +8,7 @@ #define PLUGIN_PROC_CONFIG_NAME "proc" #define PLUGIN_PROC_NAME PLUGIN_PROC_CONFIG_NAME ".plugin" -#define THREAD_NETDEV_NAME "PLUGIN[proc netdev]" +#define THREAD_NETDEV_NAME "P[proc netdev]" void *netdev_main(void *ptr); int do_proc_net_wireless(int update_every, usec_t dt); @@ -48,6 +48,7 @@ int get_numa_node_count(void); // metrics that need to be shared among data collectors extern unsigned long long zfs_arcstats_shrinkable_cache_size_bytes; +extern bool inside_lxc_container; // netdev renames void netdev_rename_device_add( diff --git a/collectors/proc.plugin/proc_diskstats.c b/collectors/proc.plugin/proc_diskstats.c index 28d0e7584..b487f2910 100644 --- a/collectors/proc.plugin/proc_diskstats.c +++ b/collectors/proc.plugin/proc_diskstats.c @@ -214,7 +214,7 @@ static unsigned long long int bcache_read_number_with_units(const char *filename else if(*end == 'T') return (unsigned long long int)(value * 1024.0 * 1024.0 * 1024.0 * 1024.0); else if(unknown_units_error > 0) { - error("bcache file '%s' provides value '%s' with unknown units '%s'", filename, buffer, end); + collector_error("bcache file '%s' provides value '%s' with unknown units '%s'", filename, buffer, end); unknown_units_error--; } } @@ -269,7 +269,7 @@ void bcache_read_priority_stats(struct disk *d, const char *family, int update_e for(l = 0; l < lines ;l++) { size_t words = procfile_linewords(ff, l); if(unlikely(words < 2)) { - if(unlikely(words)) error("Cannot read '%s' line %zu. Expected 2 params, read %zu.", d->bcache_filename_priority_stats, l, words); + if(unlikely(words)) collector_error("Cannot read '%s' line %zu. Expected 2 params, read %zu.", d->bcache_filename_priority_stats, l, words); continue; } @@ -344,7 +344,7 @@ static inline int is_major_enabled(int major) { } static inline int get_disk_name_from_path(const char *path, char *result, size_t result_size, unsigned long major, unsigned long minor, char *disk, char *prefix, int depth) { - //info("DEVICE-MAPPER ('%s', %lu:%lu): examining directory '%s' (allowed depth %d).", disk, major, minor, path, depth); + //collector_info("DEVICE-MAPPER ('%s', %lu:%lu): examining directory '%s' (allowed depth %d).", disk, major, minor, path, depth); int found = 0, preferred = 0; @@ -352,7 +352,7 @@ static inline int get_disk_name_from_path(const char *path, char *result, size_t DIR *dir = opendir(path); if (!dir) { - error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot open directory '%s'.", disk, major, minor, path); + collector_error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot open directory '%s'.", disk, major, minor, path); goto failed; } @@ -363,7 +363,7 @@ static inline int get_disk_name_from_path(const char *path, char *result, size_t continue; if(depth <= 0) { - error("DEVICE-MAPPER ('%s', %lu:%lu): Depth limit reached for path '%s/%s'. Ignoring path.", disk, major, minor, path, de->d_name); + collector_error("DEVICE-MAPPER ('%s', %lu:%lu): Depth limit reached for path '%s/%s'. Ignoring path.", disk, major, minor, path, de->d_name); break; } else { @@ -393,7 +393,7 @@ static inline int get_disk_name_from_path(const char *path, char *result, size_t snprintfz(filename, FILENAME_MAX, "%s/%s", path, de->d_name); ssize_t len = readlink(filename, result, result_size - 1); if(len <= 0) { - error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot read link '%s'.", disk, major, minor, filename); + collector_error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot read link '%s'.", disk, major, minor, filename); continue; } @@ -409,21 +409,21 @@ static inline int get_disk_name_from_path(const char *path, char *result, size_t struct stat sb; if(stat(filename, &sb) == -1) { - error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot stat() file '%s'.", disk, major, minor, filename); + collector_error("DEVICE-MAPPER ('%s', %lu:%lu): Cannot stat() file '%s'.", disk, major, minor, filename); continue; } if((sb.st_mode & S_IFMT) != S_IFBLK) { - //info("DEVICE-MAPPER ('%s', %lu:%lu): file '%s' is not a block device.", disk, major, minor, filename); + //collector_info("DEVICE-MAPPER ('%s', %lu:%lu): file '%s' is not a block device.", disk, major, minor, filename); continue; } if(major(sb.st_rdev) != major || minor(sb.st_rdev) != minor || strcmp(basename(filename), disk)) { - //info("DEVICE-MAPPER ('%s', %lu:%lu): filename '%s' does not match %lu:%lu.", disk, major, minor, filename, (unsigned long)major(sb.st_rdev), (unsigned long)minor(sb.st_rdev)); + //collector_info("DEVICE-MAPPER ('%s', %lu:%lu): filename '%s' does not match %lu:%lu.", disk, major, minor, filename, (unsigned long)major(sb.st_rdev), (unsigned long)minor(sb.st_rdev)); continue; } - //info("DEVICE-MAPPER ('%s', %lu:%lu): filename '%s' matches.", disk, major, minor, filename); + //collector_info("DEVICE-MAPPER ('%s', %lu:%lu): filename '%s' matches.", disk, major, minor, filename); snprintfz(result, result_size - 1, "%s%s%s", (prefix)?prefix:"", (prefix)?"_":"", de->d_name); @@ -672,7 +672,7 @@ static struct disk *get_disk(unsigned long major, unsigned long minor, char *dis break; } if (unlikely(closedir(dirp) == -1)) - error("Unable to close dir %s", buffer); + collector_error("Unable to close dir %s", buffer); } } } @@ -721,15 +721,15 @@ static struct disk *get_disk(unsigned long major, unsigned long minor, char *dis if(likely(tmp)) { d->sector_size = str2i(tmp); if(unlikely(d->sector_size <= 0)) { - error("Invalid sector size %d for device %s in %s. Assuming 512.", d->sector_size, d->device, buffer); + collector_error("Invalid sector size %d for device %s in %s. Assuming 512.", d->sector_size, d->device, buffer); d->sector_size = 512; } } - else error("Cannot read data for sector size for device %s from %s. Assuming 512.", d->device, buffer); + else collector_error("Cannot read data for sector size for device %s from %s. Assuming 512.", d->device, buffer); fclose(fpss); } - else error("Cannot read sector size for device %s from %s. Assuming 512.", d->device, buffer); + else collector_error("Cannot read sector size for device %s from %s. Assuming 512.", d->device, buffer); } */ @@ -748,103 +748,103 @@ static struct disk *get_disk(unsigned long major, unsigned long minor, char *dis if(access(buffer2, R_OK) == 0) d->bcache_filename_cache_congested = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/readahead", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_readaheads = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/cache/cache0/priority_stats", buffer); // only one cache is supported by bcache if(access(buffer2, R_OK) == 0) d->bcache_filename_priority_stats = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/cache/internal/cache_read_races", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_cache_read_races = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/cache/cache0/io_errors", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_cache_io_errors = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/dirty_data", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_dirty_data = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/writeback_rate", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_writeback_rate = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/cache/cache_available_percent", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_cache_available_percent = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_hits", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_hits = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_five_minute/cache_hit_ratio", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_five_minute_cache_hit_ratio = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_hour/cache_hit_ratio", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_hour_cache_hit_ratio = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_day/cache_hit_ratio", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_day_cache_hit_ratio = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_hit_ratio", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_hit_ratio = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_misses", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_misses = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_bypass_hits", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_bypass_hits = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_bypass_misses", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_bypass_misses = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); snprintfz(buffer2, FILENAME_MAX, "%s/stats_total/cache_miss_collisions", buffer); if(access(buffer2, R_OK) == 0) d->bcache_filename_stats_total_cache_miss_collisions = strdupz(buffer2); else - error("bcache file '%s' cannot be read.", buffer2); + collector_error("bcache file '%s' cannot be read.", buffer2); } get_disk_config(d); diff --git a/collectors/proc.plugin/proc_interrupts.c b/collectors/proc.plugin/proc_interrupts.c index f87684758..04d8c73ad 100644 --- a/collectors/proc.plugin/proc_interrupts.c +++ b/collectors/proc.plugin/proc_interrupts.c @@ -78,7 +78,7 @@ int do_proc_interrupts(int update_every, usec_t dt) { size_t words = procfile_linewords(ff, 0); if(unlikely(!lines)) { - error("Cannot read /proc/interrupts, zero lines reported."); + collector_error("Cannot read /proc/interrupts, zero lines reported."); return 1; } @@ -93,7 +93,7 @@ int do_proc_interrupts(int update_every, usec_t dt) { } if(unlikely(!cpus)) { - error("PLUGIN: PROC_INTERRUPTS: Cannot find the number of CPUs in /proc/interrupts"); + collector_error("PLUGIN: PROC_INTERRUPTS: Cannot find the number of CPUs in /proc/interrupts"); return 1; } diff --git a/collectors/proc.plugin/proc_loadavg.c b/collectors/proc.plugin/proc_loadavg.c index d928c8617..e833f69d2 100644 --- a/collectors/proc.plugin/proc_loadavg.c +++ b/collectors/proc.plugin/proc_loadavg.c @@ -32,11 +32,11 @@ int do_proc_loadavg(int update_every, usec_t dt) { } if(unlikely(procfile_lines(ff) < 1)) { - error("/proc/loadavg has no lines."); + collector_error("/proc/loadavg has no lines."); return 1; } if(unlikely(procfile_linewords(ff, 0) < 6)) { - error("/proc/loadavg has less than 6 words in it."); + collector_error("/proc/loadavg has less than 6 words in it."); return 1; } diff --git a/collectors/proc.plugin/proc_mdstat.c b/collectors/proc.plugin/proc_mdstat.c index 63e0c68eb..d6e87fd2d 100644 --- a/collectors/proc.plugin/proc_mdstat.c +++ b/collectors/proc.plugin/proc_mdstat.c @@ -135,7 +135,7 @@ int do_proc_mdstat(int update_every, usec_t dt) size_t words = 0; if (unlikely(lines < 2)) { - error("Cannot read /proc/mdstat. Expected 2 or more lines, read %zu.", lines); + collector_error("Cannot read /proc/mdstat. Expected 2 or more lines, read %zu.", lines); return 1; } @@ -212,7 +212,7 @@ int do_proc_mdstat(int update_every, usec_t dt) s = procfile_lineword(ff, l, words - 2); if (unlikely(s[0] != '[')) { - error("Cannot read /proc/mdstat raid health status. Unexpected format: missing opening bracket."); + collector_error("Cannot read /proc/mdstat raid health status. Unexpected format: missing opening bracket."); continue; } str_total = ++s; @@ -227,7 +227,7 @@ int do_proc_mdstat(int update_every, usec_t dt) s++; } if (unlikely(str_total[0] == '\0' || !str_inuse || str_inuse[0] == '\0')) { - error("Cannot read /proc/mdstat raid health status. Unexpected format."); + collector_error("Cannot read /proc/mdstat raid health status. Unexpected format."); continue; } @@ -260,7 +260,7 @@ int do_proc_mdstat(int update_every, usec_t dt) continue; if (unlikely(words < 7)) { - error("Cannot read /proc/mdstat line. Expected 7 params, read %zu.", words); + collector_error("Cannot read /proc/mdstat line. Expected 7 params, read %zu.", words); continue; } @@ -326,9 +326,9 @@ int do_proc_mdstat(int update_every, usec_t dt) raid->mismatch_cnt_filename = strdupz(filename); } if (unlikely(read_single_number_file(raid->mismatch_cnt_filename, &raid->mismatch_cnt))) { - error("Cannot read file '%s'", raid->mismatch_cnt_filename); + collector_error("Cannot read file '%s'", raid->mismatch_cnt_filename); do_mismatch = CONFIG_BOOLEAN_NO; - error("Monitoring for mismatch count has been disabled"); + collector_error("Monitoring for mismatch count has been disabled"); break; } } diff --git a/collectors/proc.plugin/proc_meminfo.c b/collectors/proc.plugin/proc_meminfo.c index 2f390c653..6988c70e0 100644 --- a/collectors/proc.plugin/proc_meminfo.c +++ b/collectors/proc.plugin/proc_meminfo.c @@ -158,9 +158,11 @@ int do_proc_meminfo(int update_every, usec_t dt) { unsigned long long MemCached = Cached + SReclaimable - Shmem; unsigned long long MemUsed = MemTotal - MemFree - MemCached - Buffers; // The Linux kernel doesn't report ZFS ARC usage as cache memory (the ARC is included in the total used system memory) - MemCached += (zfs_arcstats_shrinkable_cache_size_bytes / 1024); - MemUsed -= (zfs_arcstats_shrinkable_cache_size_bytes / 1024); - MemAvailable += (zfs_arcstats_shrinkable_cache_size_bytes / 1024); + if (!inside_lxc_container) { + MemCached += (zfs_arcstats_shrinkable_cache_size_bytes / 1024); + MemUsed -= (zfs_arcstats_shrinkable_cache_size_bytes / 1024); + MemAvailable += (zfs_arcstats_shrinkable_cache_size_bytes / 1024); + } if(do_ram) { { diff --git a/collectors/proc.plugin/proc_net_dev.c b/collectors/proc.plugin/proc_net_dev.c index e124f631f..3ec8783bd 100644 --- a/collectors/proc.plugin/proc_net_dev.c +++ b/collectors/proc.plugin/proc_net_dev.c @@ -391,7 +391,7 @@ void netdev_rename_device_add( r->processed = 0; netdev_rename_root = r; netdev_pending_renames++; - info("CGROUP: registered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); + collector_info("CGROUP: registered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); } else { if(strcmp(r->container_device, container_device) != 0 || strcmp(r->container_name, container_name) != 0) { @@ -405,7 +405,7 @@ void netdev_rename_device_add( r->processed = 0; netdev_pending_renames++; - info("CGROUP: altered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); + collector_info("CGROUP: altered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); } } @@ -429,7 +429,7 @@ void netdev_rename_device_del(const char *host_device) { if(!r->processed) netdev_pending_renames--; - info("CGROUP: unregistered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); + collector_info("CGROUP: unregistered network interface rename for '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); freez((void *) r->host_device); freez((void *) r->container_name); @@ -445,7 +445,7 @@ void netdev_rename_device_del(const char *host_device) { } static inline void netdev_rename_cgroup(struct netdev *d, struct netdev_rename *r) { - info("CGROUP: renaming network interface '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); + collector_info("CGROUP: renaming network interface '%s' as '%s' under '%s'", r->host_device, r->container_device, r->container_name); netdev_charts_release(d); netdev_free_chart_strings(d); @@ -516,8 +516,7 @@ static inline void netdev_rename_cgroup(struct netdev *d, struct netdev_rename * snprintfz(buffer, RRD_ID_LENGTH_MAX, "%scgroup.net_mtu", r->ctx_prefix); d->chart_ctx_net_mtu = strdupz(buffer); - snprintfz(buffer, RRD_ID_LENGTH_MAX, "net %s", r->container_device); - d->chart_family = strdupz(buffer); + d->chart_family = strdupz("net"); rrdlabels_copy(d->chart_labels, r->chart_labels); @@ -561,7 +560,7 @@ static void netdev_cleanup() { struct netdev *d = netdev_root, *last = NULL; while(d) { if(unlikely(!d->updated)) { - // info("Removing network device '%s', linked after '%s'", d->name, last?last->name:"ROOT"); + // collector_info("Removing network device '%s', linked after '%s'", d->name, last?last->name:"ROOT"); if(netdev_last_used == d) netdev_last_used = last; @@ -875,7 +874,7 @@ int do_proc_net_dev(int update_every, usec_t dt) { now_monotonic_sec() - d->carrier_file_lost_time > READ_RETRY_PERIOD)) { if (read_single_number_file(d->filename_carrier, &d->carrier)) { if (d->carrier_file_exists) - error( + collector_error( "Cannot refresh interface %s carrier state by reading '%s'. Next update is in %d seconds.", d->name, d->filename_carrier, @@ -897,7 +896,7 @@ int do_proc_net_dev(int update_every, usec_t dt) { if (read_file(d->filename_duplex, buffer, STATE_LENGTH_MAX)) { if (d->duplex_file_exists) - error("Cannot refresh interface %s duplex state by reading '%s'.", d->name, d->filename_duplex); + collector_error("Cannot refresh interface %s duplex state by reading '%s'.", d->name, d->filename_duplex); d->duplex_file_exists = 0; d->duplex_file_lost_time = now_monotonic_sec(); d->duplex = NETDEV_DUPLEX_UNKNOWN; @@ -920,7 +919,7 @@ int do_proc_net_dev(int update_every, usec_t dt) { char buffer[STATE_LENGTH_MAX + 1], *trimmed_buffer; if (read_file(d->filename_operstate, buffer, STATE_LENGTH_MAX)) { - error( + collector_error( "Cannot refresh %s operstate by reading '%s'. Will not update its status anymore.", d->name, d->filename_operstate); freez(d->filename_operstate); @@ -933,14 +932,14 @@ int do_proc_net_dev(int update_every, usec_t dt) { if (d->do_mtu != CONFIG_BOOLEAN_NO && d->filename_mtu) { if (read_single_number_file(d->filename_mtu, &d->mtu)) { - error( + collector_error( "Cannot refresh mtu for interface %s by reading '%s'. Stop updating it.", d->name, d->filename_mtu); freez(d->filename_mtu); d->filename_mtu = NULL; } } - //info("PROC_NET_DEV: %s speed %zu, bytes %zu/%zu, packets %zu/%zu/%zu, errors %zu/%zu, drops %zu/%zu, fifo %zu/%zu, compressed %zu/%zu, rframe %zu, tcollisions %zu, tcarrier %zu" + //collector_info("PROC_NET_DEV: %s speed %zu, bytes %zu/%zu, packets %zu/%zu/%zu, errors %zu/%zu, drops %zu/%zu, fifo %zu/%zu, compressed %zu/%zu, rframe %zu, tcollisions %zu, tcarrier %zu" // , d->name, d->speed // , d->rbytes, d->tbytes // , d->rpackets, d->tpackets, d->rmulticast @@ -997,7 +996,7 @@ int do_proc_net_dev(int update_every, usec_t dt) { d->chart_var_speed = rrdsetvar_custom_chart_variable_add_and_acquire(d->st_bandwidth, "nic_speed_max"); if(!d->chart_var_speed) { - error( + collector_error( "Cannot create interface %s chart variable 'nic_speed_max'. Will not update its speed anymore.", d->name); freez(d->filename_speed); @@ -1017,7 +1016,7 @@ int do_proc_net_dev(int update_every, usec_t dt) { if(ret) { if (d->speed_file_exists) - error("Cannot refresh interface %s speed by reading '%s'.", d->name, d->filename_speed); + collector_error("Cannot refresh interface %s speed by reading '%s'.", d->name, d->filename_speed); d->speed_file_exists = 0; d->speed_file_lost_time = now_monotonic_sec(); } @@ -1489,7 +1488,7 @@ static void netdev_main_cleanup(void *ptr) { UNUSED(ptr); - info("cleaning up..."); + collector_info("cleaning up..."); worker_unregister(); } @@ -1505,11 +1504,11 @@ void *netdev_main(void *ptr) heartbeat_t hb; heartbeat_init(&hb); - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { worker_is_idle(); usec_t hb_dt = heartbeat_next(&hb, step); - if (unlikely(netdata_exit)) + if (unlikely(!service_running(SERVICE_COLLECTORS))) break; worker_is_busy(0); diff --git a/collectors/proc.plugin/proc_net_netstat.c b/collectors/proc.plugin/proc_net_netstat.c index f7635e3d0..ce3068c0e 100644 --- a/collectors/proc.plugin/proc_net_netstat.c +++ b/collectors/proc.plugin/proc_net_netstat.c @@ -97,7 +97,7 @@ static void parse_line_pair(procfile *ff_netstat, ARL_BASE *base, size_t header_ size_t w; if(unlikely(vwords > hwords)) { - error("File /proc/net/netstat on header line %zu has %zu words, but on value line %zu has %zu words.", header_line, hwords, values_line, vwords); + collector_error("File /proc/net/netstat on header line %zu has %zu words, but on value line %zu has %zu words.", header_line, hwords, values_line, vwords); vwords = hwords; } @@ -107,16 +107,12 @@ static void parse_line_pair(procfile *ff_netstat, ARL_BASE *base, size_t header_ } } -int do_proc_net_netstat(int update_every, usec_t dt) { - (void)dt; - - static int do_bandwidth = -1, do_inerrors = -1, do_mcast = -1, do_bcast = -1, do_mcast_p = -1, do_bcast_p = -1, do_ecn = -1, \ - do_tcpext_reorder = -1, do_tcpext_syscookies = -1, do_tcpext_ofo = -1, do_tcpext_connaborts = -1, do_tcpext_memory = -1, - do_tcpext_syn_queue = -1, do_tcpext_accept_queue = -1; +static void do_proc_net_snmp6(int update_every) { + static bool do_snmp6 = true; - static int do_ip_packets = -1, do_ip_fragsout = -1, do_ip_fragsin = -1, do_ip_errors = -1, - do_tcp_sockets = -1, do_tcp_packets = -1, do_tcp_errors = -1, do_tcp_handshake = -1, do_tcp_opens = -1, - do_udp_packets = -1, do_udp_errors = -1, do_icmp_packets = -1, do_icmpmsg = -1, do_udplite_packets = -1; + if (!do_snmp6) { + return; + } static int do_ip6_packets = -1, do_ip6_fragsout = -1, do_ip6_fragsin = -1, do_ip6_errors = -1, do_ip6_udplite_packets = -1, do_ip6_udplite_errors = -1, do_ip6_udp_packets = -1, do_ip6_udp_errors = -1, @@ -125,104 +121,10 @@ int do_proc_net_netstat(int update_every, usec_t dt) { do_ip6_icmp_router = -1, do_ip6_icmp_neighbor = -1, do_ip6_icmp_mldv2 = -1, do_ip6_icmp_types = -1, do_ip6_ect = -1; - static uint32_t hash_ipext = 0, hash_tcpext = 0; - static uint32_t hash_ip = 0, hash_icmp = 0, hash_tcp = 0, hash_udp = 0, hash_icmpmsg = 0, hash_udplite = 0; - - static procfile *ff_netstat = NULL; - static procfile *ff_snmp = NULL; static procfile *ff_snmp6 = NULL; - static ARL_BASE *arl_tcpext = NULL; - static ARL_BASE *arl_ipext = NULL; - - static ARL_BASE *arl_ip = NULL; - static ARL_BASE *arl_icmp = NULL; - static ARL_BASE *arl_icmpmsg = NULL; - static ARL_BASE *arl_tcp = NULL; - static ARL_BASE *arl_udp = NULL; - static ARL_BASE *arl_udplite = NULL; - static ARL_BASE *arl_ipv6 = NULL; - static const RRDVAR_ACQUIRED *tcp_max_connections_var = NULL; - - // -------------------------------------------------------------------- - // IP - - // IP bandwidth - static unsigned long long ipext_InOctets = 0; - static unsigned long long ipext_OutOctets = 0; - - // IP input errors - static unsigned long long ipext_InNoRoutes = 0; - static unsigned long long ipext_InTruncatedPkts = 0; - static unsigned long long ipext_InCsumErrors = 0; - - // IP multicast bandwidth - static unsigned long long ipext_InMcastOctets = 0; - static unsigned long long ipext_OutMcastOctets = 0; - - // IP multicast packets - static unsigned long long ipext_InMcastPkts = 0; - static unsigned long long ipext_OutMcastPkts = 0; - - // IP broadcast bandwidth - static unsigned long long ipext_InBcastOctets = 0; - static unsigned long long ipext_OutBcastOctets = 0; - - // IP broadcast packets - static unsigned long long ipext_InBcastPkts = 0; - static unsigned long long ipext_OutBcastPkts = 0; - - // IP ECN - static unsigned long long ipext_InNoECTPkts = 0; - static unsigned long long ipext_InECT1Pkts = 0; - static unsigned long long ipext_InECT0Pkts = 0; - static unsigned long long ipext_InCEPkts = 0; - - // -------------------------------------------------------------------- - // IP TCP - - // IP TCP Reordering - static unsigned long long tcpext_TCPRenoReorder = 0; - static unsigned long long tcpext_TCPFACKReorder = 0; - static unsigned long long tcpext_TCPSACKReorder = 0; - static unsigned long long tcpext_TCPTSReorder = 0; - - // IP TCP SYN Cookies - static unsigned long long tcpext_SyncookiesSent = 0; - static unsigned long long tcpext_SyncookiesRecv = 0; - static unsigned long long tcpext_SyncookiesFailed = 0; - - // IP TCP Out Of Order Queue - // http://www.spinics.net/lists/netdev/msg204696.html - static unsigned long long tcpext_TCPOFOQueue = 0; // Number of packets queued in OFO queue - static unsigned long long tcpext_TCPOFODrop = 0; // Number of packets meant to be queued in OFO but dropped because socket rcvbuf limit hit. - static unsigned long long tcpext_TCPOFOMerge = 0; // Number of packets in OFO that were merged with other packets. - static unsigned long long tcpext_OfoPruned = 0; // packets dropped from out-of-order queue because of socket buffer overrun - - // IP TCP connection resets - // https://github.com/ecki/net-tools/blob/bd8bceaed2311651710331a7f8990c3e31be9840/statistics.c - static unsigned long long tcpext_TCPAbortOnData = 0; // connections reset due to unexpected data - static unsigned long long tcpext_TCPAbortOnClose = 0; // connections reset due to early user close - static unsigned long long tcpext_TCPAbortOnMemory = 0; // connections aborted due to memory pressure - static unsigned long long tcpext_TCPAbortOnTimeout = 0; // connections aborted due to timeout - static unsigned long long tcpext_TCPAbortOnLinger = 0; // connections aborted after user close in linger timeout - static unsigned long long tcpext_TCPAbortFailed = 0; // times unable to send RST due to no memory - - // https://perfchron.com/2015/12/26/investigating-linux-network-issues-with-netstat-and-nstat/ - static unsigned long long tcpext_ListenOverflows = 0; // times the listen queue of a socket overflowed - static unsigned long long tcpext_ListenDrops = 0; // SYNs to LISTEN sockets ignored - - // IP TCP memory pressures - static unsigned long long tcpext_TCPMemoryPressures = 0; - - static unsigned long long tcpext_TCPReqQFullDrop = 0; - static unsigned long long tcpext_TCPReqQFullDoCookies = 0; - - static unsigned long long tcpext_TCPSynRetrans = 0; - - // IPv6 static unsigned long long Ip6InReceives = 0ULL; static unsigned long long Ip6InHdrErrors = 0ULL; static unsigned long long Ip6InTooBigErrors = 0ULL; @@ -316,6 +218,1207 @@ int do_proc_net_netstat(int update_every, usec_t dt) { static unsigned long long UdpLite6SndbufErrors = 0ULL; static unsigned long long UdpLite6InCsumErrors = 0ULL; + // prepare for /proc/net/snmp6 parsing + + if(unlikely(!arl_ipv6)) { + do_ip6_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 packets", CONFIG_BOOLEAN_AUTO); + do_ip6_fragsout = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 fragments sent", CONFIG_BOOLEAN_AUTO); + do_ip6_fragsin = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 fragments assembly", CONFIG_BOOLEAN_AUTO); + do_ip6_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 errors", CONFIG_BOOLEAN_AUTO); + do_ip6_udp_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDP packets", CONFIG_BOOLEAN_AUTO); + do_ip6_udp_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDP errors", CONFIG_BOOLEAN_AUTO); + do_ip6_udplite_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDPlite packets", CONFIG_BOOLEAN_AUTO); + do_ip6_udplite_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDPlite errors", CONFIG_BOOLEAN_AUTO); + do_ip6_bandwidth = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "bandwidth", CONFIG_BOOLEAN_AUTO); + do_ip6_mcast = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "multicast bandwidth", CONFIG_BOOLEAN_AUTO); + do_ip6_bcast = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "broadcast bandwidth", CONFIG_BOOLEAN_AUTO); + do_ip6_mcast_p = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "multicast packets", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_redir = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp redirects", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp errors", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_echos = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp echos", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_groupmemb = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp group membership", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_router = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp router", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_neighbor = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp neighbor", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_mldv2 = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp mldv2", CONFIG_BOOLEAN_AUTO); + do_ip6_icmp_types = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp types", CONFIG_BOOLEAN_AUTO); + do_ip6_ect = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ect", CONFIG_BOOLEAN_AUTO); + + arl_ipv6 = arl_create("snmp6", NULL, 60); + arl_expect(arl_ipv6, "Ip6InReceives", &Ip6InReceives); + arl_expect(arl_ipv6, "Ip6InHdrErrors", &Ip6InHdrErrors); + arl_expect(arl_ipv6, "Ip6InTooBigErrors", &Ip6InTooBigErrors); + arl_expect(arl_ipv6, "Ip6InNoRoutes", &Ip6InNoRoutes); + arl_expect(arl_ipv6, "Ip6InAddrErrors", &Ip6InAddrErrors); + arl_expect(arl_ipv6, "Ip6InUnknownProtos", &Ip6InUnknownProtos); + arl_expect(arl_ipv6, "Ip6InTruncatedPkts", &Ip6InTruncatedPkts); + arl_expect(arl_ipv6, "Ip6InDiscards", &Ip6InDiscards); + arl_expect(arl_ipv6, "Ip6InDelivers", &Ip6InDelivers); + arl_expect(arl_ipv6, "Ip6OutForwDatagrams", &Ip6OutForwDatagrams); + arl_expect(arl_ipv6, "Ip6OutRequests", &Ip6OutRequests); + arl_expect(arl_ipv6, "Ip6OutDiscards", &Ip6OutDiscards); + arl_expect(arl_ipv6, "Ip6OutNoRoutes", &Ip6OutNoRoutes); + arl_expect(arl_ipv6, "Ip6ReasmTimeout", &Ip6ReasmTimeout); + arl_expect(arl_ipv6, "Ip6ReasmReqds", &Ip6ReasmReqds); + arl_expect(arl_ipv6, "Ip6ReasmOKs", &Ip6ReasmOKs); + arl_expect(arl_ipv6, "Ip6ReasmFails", &Ip6ReasmFails); + arl_expect(arl_ipv6, "Ip6FragOKs", &Ip6FragOKs); + arl_expect(arl_ipv6, "Ip6FragFails", &Ip6FragFails); + arl_expect(arl_ipv6, "Ip6FragCreates", &Ip6FragCreates); + arl_expect(arl_ipv6, "Ip6InMcastPkts", &Ip6InMcastPkts); + arl_expect(arl_ipv6, "Ip6OutMcastPkts", &Ip6OutMcastPkts); + arl_expect(arl_ipv6, "Ip6InOctets", &Ip6InOctets); + arl_expect(arl_ipv6, "Ip6OutOctets", &Ip6OutOctets); + arl_expect(arl_ipv6, "Ip6InMcastOctets", &Ip6InMcastOctets); + arl_expect(arl_ipv6, "Ip6OutMcastOctets", &Ip6OutMcastOctets); + arl_expect(arl_ipv6, "Ip6InBcastOctets", &Ip6InBcastOctets); + arl_expect(arl_ipv6, "Ip6OutBcastOctets", &Ip6OutBcastOctets); + arl_expect(arl_ipv6, "Ip6InNoECTPkts", &Ip6InNoECTPkts); + arl_expect(arl_ipv6, "Ip6InECT1Pkts", &Ip6InECT1Pkts); + arl_expect(arl_ipv6, "Ip6InECT0Pkts", &Ip6InECT0Pkts); + arl_expect(arl_ipv6, "Ip6InCEPkts", &Ip6InCEPkts); + arl_expect(arl_ipv6, "Icmp6InMsgs", &Icmp6InMsgs); + arl_expect(arl_ipv6, "Icmp6InErrors", &Icmp6InErrors); + arl_expect(arl_ipv6, "Icmp6OutMsgs", &Icmp6OutMsgs); + arl_expect(arl_ipv6, "Icmp6OutErrors", &Icmp6OutErrors); + arl_expect(arl_ipv6, "Icmp6InCsumErrors", &Icmp6InCsumErrors); + arl_expect(arl_ipv6, "Icmp6InDestUnreachs", &Icmp6InDestUnreachs); + arl_expect(arl_ipv6, "Icmp6InPktTooBigs", &Icmp6InPktTooBigs); + arl_expect(arl_ipv6, "Icmp6InTimeExcds", &Icmp6InTimeExcds); + arl_expect(arl_ipv6, "Icmp6InParmProblems", &Icmp6InParmProblems); + arl_expect(arl_ipv6, "Icmp6InEchos", &Icmp6InEchos); + arl_expect(arl_ipv6, "Icmp6InEchoReplies", &Icmp6InEchoReplies); + arl_expect(arl_ipv6, "Icmp6InGroupMembQueries", &Icmp6InGroupMembQueries); + arl_expect(arl_ipv6, "Icmp6InGroupMembResponses", &Icmp6InGroupMembResponses); + arl_expect(arl_ipv6, "Icmp6InGroupMembReductions", &Icmp6InGroupMembReductions); + arl_expect(arl_ipv6, "Icmp6InRouterSolicits", &Icmp6InRouterSolicits); + arl_expect(arl_ipv6, "Icmp6InRouterAdvertisements", &Icmp6InRouterAdvertisements); + arl_expect(arl_ipv6, "Icmp6InNeighborSolicits", &Icmp6InNeighborSolicits); + arl_expect(arl_ipv6, "Icmp6InNeighborAdvertisements", &Icmp6InNeighborAdvertisements); + arl_expect(arl_ipv6, "Icmp6InRedirects", &Icmp6InRedirects); + arl_expect(arl_ipv6, "Icmp6InMLDv2Reports", &Icmp6InMLDv2Reports); + arl_expect(arl_ipv6, "Icmp6OutDestUnreachs", &Icmp6OutDestUnreachs); + arl_expect(arl_ipv6, "Icmp6OutPktTooBigs", &Icmp6OutPktTooBigs); + arl_expect(arl_ipv6, "Icmp6OutTimeExcds", &Icmp6OutTimeExcds); + arl_expect(arl_ipv6, "Icmp6OutParmProblems", &Icmp6OutParmProblems); + arl_expect(arl_ipv6, "Icmp6OutEchos", &Icmp6OutEchos); + arl_expect(arl_ipv6, "Icmp6OutEchoReplies", &Icmp6OutEchoReplies); + arl_expect(arl_ipv6, "Icmp6OutGroupMembQueries", &Icmp6OutGroupMembQueries); + arl_expect(arl_ipv6, "Icmp6OutGroupMembResponses", &Icmp6OutGroupMembResponses); + arl_expect(arl_ipv6, "Icmp6OutGroupMembReductions", &Icmp6OutGroupMembReductions); + arl_expect(arl_ipv6, "Icmp6OutRouterSolicits", &Icmp6OutRouterSolicits); + arl_expect(arl_ipv6, "Icmp6OutRouterAdvertisements", &Icmp6OutRouterAdvertisements); + arl_expect(arl_ipv6, "Icmp6OutNeighborSolicits", &Icmp6OutNeighborSolicits); + arl_expect(arl_ipv6, "Icmp6OutNeighborAdvertisements", &Icmp6OutNeighborAdvertisements); + arl_expect(arl_ipv6, "Icmp6OutRedirects", &Icmp6OutRedirects); + arl_expect(arl_ipv6, "Icmp6OutMLDv2Reports", &Icmp6OutMLDv2Reports); + arl_expect(arl_ipv6, "Icmp6InType1", &Icmp6InType1); + arl_expect(arl_ipv6, "Icmp6InType128", &Icmp6InType128); + arl_expect(arl_ipv6, "Icmp6InType129", &Icmp6InType129); + arl_expect(arl_ipv6, "Icmp6InType136", &Icmp6InType136); + arl_expect(arl_ipv6, "Icmp6OutType1", &Icmp6OutType1); + arl_expect(arl_ipv6, "Icmp6OutType128", &Icmp6OutType128); + arl_expect(arl_ipv6, "Icmp6OutType129", &Icmp6OutType129); + arl_expect(arl_ipv6, "Icmp6OutType133", &Icmp6OutType133); + arl_expect(arl_ipv6, "Icmp6OutType135", &Icmp6OutType135); + arl_expect(arl_ipv6, "Icmp6OutType143", &Icmp6OutType143); + arl_expect(arl_ipv6, "Udp6InDatagrams", &Udp6InDatagrams); + arl_expect(arl_ipv6, "Udp6NoPorts", &Udp6NoPorts); + arl_expect(arl_ipv6, "Udp6InErrors", &Udp6InErrors); + arl_expect(arl_ipv6, "Udp6OutDatagrams", &Udp6OutDatagrams); + arl_expect(arl_ipv6, "Udp6RcvbufErrors", &Udp6RcvbufErrors); + arl_expect(arl_ipv6, "Udp6SndbufErrors", &Udp6SndbufErrors); + arl_expect(arl_ipv6, "Udp6InCsumErrors", &Udp6InCsumErrors); + arl_expect(arl_ipv6, "Udp6IgnoredMulti", &Udp6IgnoredMulti); + arl_expect(arl_ipv6, "UdpLite6InDatagrams", &UdpLite6InDatagrams); + arl_expect(arl_ipv6, "UdpLite6NoPorts", &UdpLite6NoPorts); + arl_expect(arl_ipv6, "UdpLite6InErrors", &UdpLite6InErrors); + arl_expect(arl_ipv6, "UdpLite6OutDatagrams", &UdpLite6OutDatagrams); + arl_expect(arl_ipv6, "UdpLite6RcvbufErrors", &UdpLite6RcvbufErrors); + arl_expect(arl_ipv6, "UdpLite6SndbufErrors", &UdpLite6SndbufErrors); + arl_expect(arl_ipv6, "UdpLite6InCsumErrors", &UdpLite6InCsumErrors); + } + + // parse /proc/net/snmp + + if (unlikely(!ff_snmp6)) { + char filename[FILENAME_MAX + 1]; + snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/proc/net/snmp6"); + ff_snmp6 = procfile_open( + config_get("plugin:proc:/proc/net/snmp6", "filename to monitor", filename), " \t:", PROCFILE_FLAG_DEFAULT); + if (unlikely(!ff_snmp6)) { + do_snmp6 = false; + return; + } + } + + ff_snmp6 = procfile_readall(ff_snmp6); + if (unlikely(!ff_snmp6)) + return; + + size_t lines, l; + + lines = procfile_lines(ff_snmp6); + + arl_begin(arl_ipv6); + + for (l = 0; l < lines; l++) { + size_t words = procfile_linewords(ff_snmp6, l); + if (unlikely(words < 2)) { + if (unlikely(words)) { + collector_error("Cannot read /proc/net/snmp6 line %zu. Expected 2 params, read %zu.", l, words); + continue; + } + } + + if (unlikely(arl_check(arl_ipv6, procfile_lineword(ff_snmp6, l, 0), procfile_lineword(ff_snmp6, l, 1)))) + break; + } + + if(do_ip6_bandwidth == CONFIG_BOOLEAN_YES || (do_ip6_bandwidth == CONFIG_BOOLEAN_AUTO && + (Ip6InOctets || + Ip6OutOctets || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_bandwidth = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_received = NULL, + *rd_sent = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + "system" + , "ipv6" + , NULL + , "network" + , NULL + , "IPv6 Bandwidth" + , "kilobits/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_SYSTEM_IPV6 + , update_every + , RRDSET_TYPE_AREA + ); + + rd_received = rrddim_add(st, "InOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + rd_sent = rrddim_add(st, "OutOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_received, Ip6InOctets); + rrddim_set_by_pointer(st, rd_sent, Ip6OutOctets); + rrdset_done(st); + } + + if(do_ip6_packets == CONFIG_BOOLEAN_YES || (do_ip6_packets == CONFIG_BOOLEAN_AUTO && + (Ip6InReceives || + Ip6OutRequests || + Ip6InDelivers || + Ip6OutForwDatagrams || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_packets = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_received = NULL, + *rd_sent = NULL, + *rd_forwarded = NULL, + *rd_delivers = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "packets" + , NULL + , "packets" + , NULL + , "IPv6 Packets" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_PACKETS + , update_every + , RRDSET_TYPE_LINE + ); + + rd_received = rrddim_add(st, "InReceives", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_sent = rrddim_add(st, "OutRequests", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_forwarded = rrddim_add(st, "OutForwDatagrams", "forwarded", -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_delivers = rrddim_add(st, "InDelivers", "delivers", 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_received, Ip6InReceives); + rrddim_set_by_pointer(st, rd_sent, Ip6OutRequests); + rrddim_set_by_pointer(st, rd_forwarded, Ip6OutForwDatagrams); + rrddim_set_by_pointer(st, rd_delivers, Ip6InDelivers); + rrdset_done(st); + } + + if(do_ip6_fragsout == CONFIG_BOOLEAN_YES || (do_ip6_fragsout == CONFIG_BOOLEAN_AUTO && + (Ip6FragOKs || + Ip6FragFails || + Ip6FragCreates || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_fragsout = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_ok = NULL, + *rd_failed = NULL, + *rd_all = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "fragsout" + , NULL + , "fragments6" + , NULL + , "IPv6 Fragments Sent" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_FRAGSOUT + , update_every + , RRDSET_TYPE_LINE + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_ok = rrddim_add(st, "FragOKs", "ok", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_failed = rrddim_add(st, "FragFails", "failed", -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_all = rrddim_add(st, "FragCreates", "all", 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_ok, Ip6FragOKs); + rrddim_set_by_pointer(st, rd_failed, Ip6FragFails); + rrddim_set_by_pointer(st, rd_all, Ip6FragCreates); + rrdset_done(st); + } + + if(do_ip6_fragsin == CONFIG_BOOLEAN_YES || (do_ip6_fragsin == CONFIG_BOOLEAN_AUTO && + (Ip6ReasmOKs || + Ip6ReasmFails || + Ip6ReasmTimeout || + Ip6ReasmReqds || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_fragsin = CONFIG_BOOLEAN_YES; + + static RRDSET *st = NULL; + static RRDDIM *rd_ok = NULL, + *rd_failed = NULL, + *rd_timeout = NULL, + *rd_all = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "fragsin" + , NULL + , "fragments6" + , NULL + , "IPv6 Fragments Reassembly" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_FRAGSIN + , update_every + , RRDSET_TYPE_LINE); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_ok = rrddim_add(st, "ReasmOKs", "ok", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_failed = rrddim_add(st, "ReasmFails", "failed", -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_timeout = rrddim_add(st, "ReasmTimeout", "timeout", -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_all = rrddim_add(st, "ReasmReqds", "all", 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_ok, Ip6ReasmOKs); + rrddim_set_by_pointer(st, rd_failed, Ip6ReasmFails); + rrddim_set_by_pointer(st, rd_timeout, Ip6ReasmTimeout); + rrddim_set_by_pointer(st, rd_all, Ip6ReasmReqds); + rrdset_done(st); + } + + if(do_ip6_errors == CONFIG_BOOLEAN_YES || (do_ip6_errors == CONFIG_BOOLEAN_AUTO && + (Ip6InDiscards || + Ip6OutDiscards || + Ip6InHdrErrors || + Ip6InAddrErrors || + Ip6InUnknownProtos || + Ip6InTooBigErrors || + Ip6InTruncatedPkts || + Ip6InNoRoutes || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_errors = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InDiscards = NULL, + *rd_OutDiscards = NULL, + *rd_InHdrErrors = NULL, + *rd_InAddrErrors = NULL, + *rd_InUnknownProtos = NULL, + *rd_InTooBigErrors = NULL, + *rd_InTruncatedPkts = NULL, + *rd_InNoRoutes = NULL, + *rd_OutNoRoutes = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "errors" + , NULL + , "errors" + , NULL + , "IPv6 Errors" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ERRORS + , update_every + , RRDSET_TYPE_LINE + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_InDiscards = rrddim_add(st, "InDiscards", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutDiscards = rrddim_add(st, "OutDiscards", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InHdrErrors = rrddim_add(st, "InHdrErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InAddrErrors = rrddim_add(st, "InAddrErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InUnknownProtos = rrddim_add(st, "InUnknownProtos", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InTooBigErrors = rrddim_add(st, "InTooBigErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InTruncatedPkts = rrddim_add(st, "InTruncatedPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InNoRoutes = rrddim_add(st, "InNoRoutes", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutNoRoutes = rrddim_add(st, "OutNoRoutes", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InDiscards, Ip6InDiscards); + rrddim_set_by_pointer(st, rd_OutDiscards, Ip6OutDiscards); + rrddim_set_by_pointer(st, rd_InHdrErrors, Ip6InHdrErrors); + rrddim_set_by_pointer(st, rd_InAddrErrors, Ip6InAddrErrors); + rrddim_set_by_pointer(st, rd_InUnknownProtos, Ip6InUnknownProtos); + rrddim_set_by_pointer(st, rd_InTooBigErrors, Ip6InTooBigErrors); + rrddim_set_by_pointer(st, rd_InTruncatedPkts, Ip6InTruncatedPkts); + rrddim_set_by_pointer(st, rd_InNoRoutes, Ip6InNoRoutes); + rrddim_set_by_pointer(st, rd_OutNoRoutes, Ip6OutNoRoutes); + rrdset_done(st); + } + + if(do_ip6_udp_packets == CONFIG_BOOLEAN_YES || (do_ip6_udp_packets == CONFIG_BOOLEAN_AUTO && + (Udp6InDatagrams || + Udp6OutDatagrams || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + static RRDSET *st = NULL; + static RRDDIM *rd_received = NULL, + *rd_sent = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "udppackets" + , NULL + , "udp6" + , NULL + , "IPv6 UDP Packets" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_UDP_PACKETS + , update_every + , RRDSET_TYPE_LINE + ); + + rd_received = rrddim_add(st, "InDatagrams", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_sent = rrddim_add(st, "OutDatagrams", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_received, Udp6InDatagrams); + rrddim_set_by_pointer(st, rd_sent, Udp6OutDatagrams); + rrdset_done(st); + } + + if(do_ip6_udp_errors == CONFIG_BOOLEAN_YES || (do_ip6_udp_errors == CONFIG_BOOLEAN_AUTO && + (Udp6InErrors || + Udp6NoPorts || + Udp6RcvbufErrors || + Udp6SndbufErrors || + Udp6InCsumErrors || + Udp6IgnoredMulti || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_udp_errors = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_RcvbufErrors = NULL, + *rd_SndbufErrors = NULL, + *rd_InErrors = NULL, + *rd_NoPorts = NULL, + *rd_InCsumErrors = NULL, + *rd_IgnoredMulti = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "udperrors" + , NULL + , "udp6" + , NULL + , "IPv6 UDP Errors" + , "events/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_UDP_ERRORS + , update_every + , RRDSET_TYPE_LINE + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_RcvbufErrors = rrddim_add(st, "RcvbufErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_SndbufErrors = rrddim_add(st, "SndbufErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_NoPorts = rrddim_add(st, "NoPorts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_IgnoredMulti = rrddim_add(st, "IgnoredMulti", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_RcvbufErrors, Udp6RcvbufErrors); + rrddim_set_by_pointer(st, rd_SndbufErrors, Udp6SndbufErrors); + rrddim_set_by_pointer(st, rd_InErrors, Udp6InErrors); + rrddim_set_by_pointer(st, rd_NoPorts, Udp6NoPorts); + rrddim_set_by_pointer(st, rd_InCsumErrors, Udp6InCsumErrors); + rrddim_set_by_pointer(st, rd_IgnoredMulti, Udp6IgnoredMulti); + rrdset_done(st); + } + + if(do_ip6_udplite_packets == CONFIG_BOOLEAN_YES || (do_ip6_udplite_packets == CONFIG_BOOLEAN_AUTO && + (UdpLite6InDatagrams || + UdpLite6OutDatagrams || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + static RRDSET *st = NULL; + static RRDDIM *rd_received = NULL, + *rd_sent = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "udplitepackets" + , NULL + , "udplite6" + , NULL + , "IPv6 UDPlite Packets" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_UDPLITE_PACKETS + , update_every + , RRDSET_TYPE_LINE + ); + + rd_received = rrddim_add(st, "InDatagrams", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_sent = rrddim_add(st, "OutDatagrams", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_received, UdpLite6InDatagrams); + rrddim_set_by_pointer(st, rd_sent, UdpLite6OutDatagrams); + rrdset_done(st); + } + + if(do_ip6_udplite_errors == CONFIG_BOOLEAN_YES || (do_ip6_udplite_errors == CONFIG_BOOLEAN_AUTO && + (UdpLite6InErrors || + UdpLite6NoPorts || + UdpLite6RcvbufErrors || + UdpLite6SndbufErrors || + Udp6InCsumErrors || + UdpLite6InCsumErrors || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_udplite_errors = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_RcvbufErrors = NULL, + *rd_SndbufErrors = NULL, + *rd_InErrors = NULL, + *rd_NoPorts = NULL, + *rd_InCsumErrors = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "udpliteerrors" + , NULL + , "udplite6" + , NULL + , "IPv6 UDP Lite Errors" + , "events/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_UDPLITE_ERRORS + , update_every + , RRDSET_TYPE_LINE + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_RcvbufErrors = rrddim_add(st, "RcvbufErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_SndbufErrors = rrddim_add(st, "SndbufErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_NoPorts = rrddim_add(st, "NoPorts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InErrors, UdpLite6InErrors); + rrddim_set_by_pointer(st, rd_NoPorts, UdpLite6NoPorts); + rrddim_set_by_pointer(st, rd_RcvbufErrors, UdpLite6RcvbufErrors); + rrddim_set_by_pointer(st, rd_SndbufErrors, UdpLite6SndbufErrors); + rrddim_set_by_pointer(st, rd_InCsumErrors, UdpLite6InCsumErrors); + rrdset_done(st); + } + + if(do_ip6_mcast == CONFIG_BOOLEAN_YES || (do_ip6_mcast == CONFIG_BOOLEAN_AUTO && + (Ip6OutMcastOctets || + Ip6InMcastOctets || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_mcast = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_Ip6InMcastOctets = NULL, + *rd_Ip6OutMcastOctets = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "mcast" + , NULL + , "multicast6" + , NULL + , "IPv6 Multicast Bandwidth" + , "kilobits/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_MCAST + , update_every + , RRDSET_TYPE_AREA + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_Ip6InMcastOctets = rrddim_add(st, "InMcastOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + rd_Ip6OutMcastOctets = rrddim_add(st, "OutMcastOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_Ip6InMcastOctets, Ip6InMcastOctets); + rrddim_set_by_pointer(st, rd_Ip6OutMcastOctets, Ip6OutMcastOctets); + rrdset_done(st); + } + + if(do_ip6_bcast == CONFIG_BOOLEAN_YES || (do_ip6_bcast == CONFIG_BOOLEAN_AUTO && + (Ip6OutBcastOctets || + Ip6InBcastOctets || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_bcast = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_Ip6InBcastOctets = NULL, + *rd_Ip6OutBcastOctets = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "bcast" + , NULL + , "broadcast6" + , NULL + , "IPv6 Broadcast Bandwidth" + , "kilobits/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_BCAST + , update_every + , RRDSET_TYPE_AREA + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_Ip6InBcastOctets = rrddim_add(st, "InBcastOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + rd_Ip6OutBcastOctets = rrddim_add(st, "OutBcastOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_Ip6InBcastOctets, Ip6InBcastOctets); + rrddim_set_by_pointer(st, rd_Ip6OutBcastOctets, Ip6OutBcastOctets); + rrdset_done(st); + } + + if(do_ip6_mcast_p == CONFIG_BOOLEAN_YES || (do_ip6_mcast_p == CONFIG_BOOLEAN_AUTO && + (Ip6OutMcastPkts || + Ip6InMcastPkts || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_mcast_p = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_Ip6InMcastPkts = NULL, + *rd_Ip6OutMcastPkts = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "mcastpkts" + , NULL + , "multicast6" + , NULL + , "IPv6 Multicast Packets" + , "packets/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_MCAST_PACKETS + , update_every + , RRDSET_TYPE_LINE + ); + rrdset_flag_set(st, RRDSET_FLAG_DETAIL); + + rd_Ip6InMcastPkts = rrddim_add(st, "InMcastPkts", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_Ip6OutMcastPkts = rrddim_add(st, "OutMcastPkts", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_Ip6InMcastPkts, Ip6InMcastPkts); + rrddim_set_by_pointer(st, rd_Ip6OutMcastPkts, Ip6OutMcastPkts); + rrdset_done(st); + } + + if(do_ip6_icmp == CONFIG_BOOLEAN_YES || (do_ip6_icmp == CONFIG_BOOLEAN_AUTO && + (Icmp6InMsgs || + Icmp6OutMsgs || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_Icmp6InMsgs = NULL, + *rd_Icmp6OutMsgs = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmp" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Messages" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP + , update_every + , RRDSET_TYPE_LINE + ); + + rd_Icmp6InMsgs = rrddim_add(st, "InMsgs", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_Icmp6OutMsgs = rrddim_add(st, "OutMsgs", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_Icmp6InMsgs, Icmp6InMsgs); + rrddim_set_by_pointer(st, rd_Icmp6OutMsgs, Icmp6OutMsgs); + rrdset_done(st); + } + + if(do_ip6_icmp_redir == CONFIG_BOOLEAN_YES || (do_ip6_icmp_redir == CONFIG_BOOLEAN_AUTO && + (Icmp6InRedirects || + Icmp6OutRedirects || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_redir = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_Icmp6InRedirects = NULL, + *rd_Icmp6OutRedirects = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmpredir" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Redirects" + , "redirects/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_REDIR + , update_every + , RRDSET_TYPE_LINE + ); + + rd_Icmp6InRedirects = rrddim_add(st, "InRedirects", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_Icmp6OutRedirects = rrddim_add(st, "OutRedirects", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_Icmp6InRedirects, Icmp6InRedirects); + rrddim_set_by_pointer(st, rd_Icmp6OutRedirects, Icmp6OutRedirects); + rrdset_done(st); + } + + if(do_ip6_icmp_errors == CONFIG_BOOLEAN_YES || (do_ip6_icmp_errors == CONFIG_BOOLEAN_AUTO && + (Icmp6InErrors || + Icmp6OutErrors || + Icmp6InCsumErrors || + Icmp6InDestUnreachs || + Icmp6InPktTooBigs || + Icmp6InTimeExcds || + Icmp6InParmProblems || + Icmp6OutDestUnreachs || + Icmp6OutPktTooBigs || + Icmp6OutTimeExcds || + Icmp6OutParmProblems || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_errors = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InErrors = NULL, + *rd_OutErrors = NULL, + *rd_InCsumErrors = NULL, + *rd_InDestUnreachs = NULL, + *rd_InPktTooBigs = NULL, + *rd_InTimeExcds = NULL, + *rd_InParmProblems = NULL, + *rd_OutDestUnreachs = NULL, + *rd_OutPktTooBigs = NULL, + *rd_OutTimeExcds = NULL, + *rd_OutParmProblems = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmperrors" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Errors" + , "errors/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_ERRORS + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutErrors = rrddim_add(st, "OutErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InDestUnreachs = rrddim_add(st, "InDestUnreachs", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InPktTooBigs = rrddim_add(st, "InPktTooBigs", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InTimeExcds = rrddim_add(st, "InTimeExcds", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InParmProblems = rrddim_add(st, "InParmProblems", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutDestUnreachs = rrddim_add(st, "OutDestUnreachs", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutPktTooBigs = rrddim_add(st, "OutPktTooBigs", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutTimeExcds = rrddim_add(st, "OutTimeExcds", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutParmProblems = rrddim_add(st, "OutParmProblems", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InErrors, Icmp6InErrors); + rrddim_set_by_pointer(st, rd_OutErrors, Icmp6OutErrors); + rrddim_set_by_pointer(st, rd_InCsumErrors, Icmp6InCsumErrors); + rrddim_set_by_pointer(st, rd_InDestUnreachs, Icmp6InDestUnreachs); + rrddim_set_by_pointer(st, rd_InPktTooBigs, Icmp6InPktTooBigs); + rrddim_set_by_pointer(st, rd_InTimeExcds, Icmp6InTimeExcds); + rrddim_set_by_pointer(st, rd_InParmProblems, Icmp6InParmProblems); + rrddim_set_by_pointer(st, rd_OutDestUnreachs, Icmp6OutDestUnreachs); + rrddim_set_by_pointer(st, rd_OutPktTooBigs, Icmp6OutPktTooBigs); + rrddim_set_by_pointer(st, rd_OutTimeExcds, Icmp6OutTimeExcds); + rrddim_set_by_pointer(st, rd_OutParmProblems, Icmp6OutParmProblems); + rrdset_done(st); + } + + if(do_ip6_icmp_echos == CONFIG_BOOLEAN_YES || (do_ip6_icmp_echos == CONFIG_BOOLEAN_AUTO && + (Icmp6InEchos || + Icmp6OutEchos || + Icmp6InEchoReplies || + Icmp6OutEchoReplies || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_echos = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InEchos = NULL, + *rd_OutEchos = NULL, + *rd_InEchoReplies = NULL, + *rd_OutEchoReplies = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmpechos" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Echo" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_ECHOS + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InEchos = rrddim_add(st, "InEchos", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutEchos = rrddim_add(st, "OutEchos", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InEchoReplies = rrddim_add(st, "InEchoReplies", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutEchoReplies = rrddim_add(st, "OutEchoReplies", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InEchos, Icmp6InEchos); + rrddim_set_by_pointer(st, rd_OutEchos, Icmp6OutEchos); + rrddim_set_by_pointer(st, rd_InEchoReplies, Icmp6InEchoReplies); + rrddim_set_by_pointer(st, rd_OutEchoReplies, Icmp6OutEchoReplies); + rrdset_done(st); + } + + if(do_ip6_icmp_groupmemb == CONFIG_BOOLEAN_YES || (do_ip6_icmp_groupmemb == CONFIG_BOOLEAN_AUTO && + (Icmp6InGroupMembQueries || + Icmp6OutGroupMembQueries || + Icmp6InGroupMembResponses || + Icmp6OutGroupMembResponses || + Icmp6InGroupMembReductions || + Icmp6OutGroupMembReductions || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_groupmemb = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InQueries = NULL, + *rd_OutQueries = NULL, + *rd_InResponses = NULL, + *rd_OutResponses = NULL, + *rd_InReductions = NULL, + *rd_OutReductions = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "groupmemb" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Group Membership" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_GROUPMEMB + , update_every + , RRDSET_TYPE_LINE); + + rd_InQueries = rrddim_add(st, "InQueries", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutQueries = rrddim_add(st, "OutQueries", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InResponses = rrddim_add(st, "InResponses", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutResponses = rrddim_add(st, "OutResponses", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InReductions = rrddim_add(st, "InReductions", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutReductions = rrddim_add(st, "OutReductions", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InQueries, Icmp6InGroupMembQueries); + rrddim_set_by_pointer(st, rd_OutQueries, Icmp6OutGroupMembQueries); + rrddim_set_by_pointer(st, rd_InResponses, Icmp6InGroupMembResponses); + rrddim_set_by_pointer(st, rd_OutResponses, Icmp6OutGroupMembResponses); + rrddim_set_by_pointer(st, rd_InReductions, Icmp6InGroupMembReductions); + rrddim_set_by_pointer(st, rd_OutReductions, Icmp6OutGroupMembReductions); + rrdset_done(st); + } + + if(do_ip6_icmp_router == CONFIG_BOOLEAN_YES || (do_ip6_icmp_router == CONFIG_BOOLEAN_AUTO && + (Icmp6InRouterSolicits || + Icmp6OutRouterSolicits || + Icmp6InRouterAdvertisements || + Icmp6OutRouterAdvertisements || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_router = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InSolicits = NULL, + *rd_OutSolicits = NULL, + *rd_InAdvertisements = NULL, + *rd_OutAdvertisements = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmprouter" + , NULL + , "icmp6" + , NULL + , "IPv6 Router Messages" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_ROUTER + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InSolicits = rrddim_add(st, "InSolicits", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutSolicits = rrddim_add(st, "OutSolicits", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InAdvertisements = rrddim_add(st, "InAdvertisements", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutAdvertisements = rrddim_add(st, "OutAdvertisements", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InSolicits, Icmp6InRouterSolicits); + rrddim_set_by_pointer(st, rd_OutSolicits, Icmp6OutRouterSolicits); + rrddim_set_by_pointer(st, rd_InAdvertisements, Icmp6InRouterAdvertisements); + rrddim_set_by_pointer(st, rd_OutAdvertisements, Icmp6OutRouterAdvertisements); + rrdset_done(st); + } + + if(do_ip6_icmp_neighbor == CONFIG_BOOLEAN_YES || (do_ip6_icmp_neighbor == CONFIG_BOOLEAN_AUTO && + (Icmp6InNeighborSolicits || + Icmp6OutNeighborSolicits || + Icmp6InNeighborAdvertisements || + Icmp6OutNeighborAdvertisements || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_neighbor = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InSolicits = NULL, + *rd_OutSolicits = NULL, + *rd_InAdvertisements = NULL, + *rd_OutAdvertisements = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmpneighbor" + , NULL + , "icmp6" + , NULL + , "IPv6 Neighbor Messages" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_NEIGHBOR + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InSolicits = rrddim_add(st, "InSolicits", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutSolicits = rrddim_add(st, "OutSolicits", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InAdvertisements = rrddim_add(st, "InAdvertisements", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutAdvertisements = rrddim_add(st, "OutAdvertisements", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InSolicits, Icmp6InNeighborSolicits); + rrddim_set_by_pointer(st, rd_OutSolicits, Icmp6OutNeighborSolicits); + rrddim_set_by_pointer(st, rd_InAdvertisements, Icmp6InNeighborAdvertisements); + rrddim_set_by_pointer(st, rd_OutAdvertisements, Icmp6OutNeighborAdvertisements); + rrdset_done(st); + } + + if(do_ip6_icmp_mldv2 == CONFIG_BOOLEAN_YES || (do_ip6_icmp_mldv2 == CONFIG_BOOLEAN_AUTO && + (Icmp6InMLDv2Reports || + Icmp6OutMLDv2Reports || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_mldv2 = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InMLDv2Reports = NULL, + *rd_OutMLDv2Reports = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmpmldv2" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP MLDv2 Reports" + , "reports/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_LDV2 + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InMLDv2Reports = rrddim_add(st, "InMLDv2Reports", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutMLDv2Reports = rrddim_add(st, "OutMLDv2Reports", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InMLDv2Reports, Icmp6InMLDv2Reports); + rrddim_set_by_pointer(st, rd_OutMLDv2Reports, Icmp6OutMLDv2Reports); + rrdset_done(st); + } + + if(do_ip6_icmp_types == CONFIG_BOOLEAN_YES || (do_ip6_icmp_types == CONFIG_BOOLEAN_AUTO && + (Icmp6InType1 || + Icmp6InType128 || + Icmp6InType129 || + Icmp6InType136 || + Icmp6OutType1 || + Icmp6OutType128 || + Icmp6OutType129 || + Icmp6OutType133 || + Icmp6OutType135 || + Icmp6OutType143 || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_icmp_types = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InType1 = NULL, + *rd_InType128 = NULL, + *rd_InType129 = NULL, + *rd_InType136 = NULL, + *rd_OutType1 = NULL, + *rd_OutType128 = NULL, + *rd_OutType129 = NULL, + *rd_OutType133 = NULL, + *rd_OutType135 = NULL, + *rd_OutType143 = NULL; + + if(unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6 + , "icmptypes" + , NULL + , "icmp6" + , NULL + , "IPv6 ICMP Types" + , "messages/s" + , PLUGIN_PROC_NAME + , PLUGIN_PROC_MODULE_NETSTAT_NAME + , NETDATA_CHART_PRIO_IPV6_ICMP_TYPES + , update_every + , RRDSET_TYPE_LINE + ); + + rd_InType1 = rrddim_add(st, "InType1", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InType128 = rrddim_add(st, "InType128", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InType129 = rrddim_add(st, "InType129", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InType136 = rrddim_add(st, "InType136", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType1 = rrddim_add(st, "OutType1", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType128 = rrddim_add(st, "OutType128", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType129 = rrddim_add(st, "OutType129", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType133 = rrddim_add(st, "OutType133", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType135 = rrddim_add(st, "OutType135", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_OutType143 = rrddim_add(st, "OutType143", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InType1, Icmp6InType1); + rrddim_set_by_pointer(st, rd_InType128, Icmp6InType128); + rrddim_set_by_pointer(st, rd_InType129, Icmp6InType129); + rrddim_set_by_pointer(st, rd_InType136, Icmp6InType136); + rrddim_set_by_pointer(st, rd_OutType1, Icmp6OutType1); + rrddim_set_by_pointer(st, rd_OutType128, Icmp6OutType128); + rrddim_set_by_pointer(st, rd_OutType129, Icmp6OutType129); + rrddim_set_by_pointer(st, rd_OutType133, Icmp6OutType133); + rrddim_set_by_pointer(st, rd_OutType135, Icmp6OutType135); + rrddim_set_by_pointer(st, rd_OutType143, Icmp6OutType143); + rrdset_done(st); + } + + if (do_ip6_ect == CONFIG_BOOLEAN_YES || + (do_ip6_ect == CONFIG_BOOLEAN_AUTO && (Ip6InNoECTPkts || Ip6InECT1Pkts || Ip6InECT0Pkts || Ip6InCEPkts || + netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { + do_ip6_ect = CONFIG_BOOLEAN_YES; + static RRDSET *st = NULL; + static RRDDIM *rd_InNoECTPkts = NULL, *rd_InECT1Pkts = NULL, *rd_InECT0Pkts = NULL, *rd_InCEPkts = NULL; + + if (unlikely(!st)) { + st = rrdset_create_localhost( + RRD_TYPE_NET_SNMP6, + "ect", + NULL, + "packets", + NULL, + "IPv6 ECT Packets", + "packets/s", + PLUGIN_PROC_NAME, + PLUGIN_PROC_MODULE_NETSTAT_NAME, + NETDATA_CHART_PRIO_IPV6_ECT, + update_every, + RRDSET_TYPE_LINE); + + rd_InNoECTPkts = rrddim_add(st, "InNoECTPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InECT1Pkts = rrddim_add(st, "InECT1Pkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InECT0Pkts = rrddim_add(st, "InECT0Pkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + rd_InCEPkts = rrddim_add(st, "InCEPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); + } + + rrddim_set_by_pointer(st, rd_InNoECTPkts, Ip6InNoECTPkts); + rrddim_set_by_pointer(st, rd_InECT1Pkts, Ip6InECT1Pkts); + rrddim_set_by_pointer(st, rd_InECT0Pkts, Ip6InECT0Pkts); + rrddim_set_by_pointer(st, rd_InCEPkts, Ip6InCEPkts); + rrdset_done(st); + } +} + +int do_proc_net_netstat(int update_every, usec_t dt) { + (void)dt; + + static int do_bandwidth = -1, do_inerrors = -1, do_mcast = -1, do_bcast = -1, do_mcast_p = -1, do_bcast_p = -1, do_ecn = -1, \ + do_tcpext_reorder = -1, do_tcpext_syscookies = -1, do_tcpext_ofo = -1, do_tcpext_connaborts = -1, do_tcpext_memory = -1, + do_tcpext_syn_queue = -1, do_tcpext_accept_queue = -1; + + static int do_ip_packets = -1, do_ip_fragsout = -1, do_ip_fragsin = -1, do_ip_errors = -1, + do_tcp_sockets = -1, do_tcp_packets = -1, do_tcp_errors = -1, do_tcp_handshake = -1, do_tcp_opens = -1, + do_udp_packets = -1, do_udp_errors = -1, do_icmp_packets = -1, do_icmpmsg = -1, do_udplite_packets = -1; + + static uint32_t hash_ipext = 0, hash_tcpext = 0; + static uint32_t hash_ip = 0, hash_icmp = 0, hash_tcp = 0, hash_udp = 0, hash_icmpmsg = 0, hash_udplite = 0; + + static procfile *ff_netstat = NULL; + static procfile *ff_snmp = NULL; + + static ARL_BASE *arl_tcpext = NULL; + static ARL_BASE *arl_ipext = NULL; + + static ARL_BASE *arl_ip = NULL; + static ARL_BASE *arl_icmp = NULL; + static ARL_BASE *arl_icmpmsg = NULL; + static ARL_BASE *arl_tcp = NULL; + static ARL_BASE *arl_udp = NULL; + static ARL_BASE *arl_udplite = NULL; + + static const RRDVAR_ACQUIRED *tcp_max_connections_var = NULL; + + // -------------------------------------------------------------------- + // IP + + // IP bandwidth + static unsigned long long ipext_InOctets = 0; + static unsigned long long ipext_OutOctets = 0; + + // IP input errors + static unsigned long long ipext_InNoRoutes = 0; + static unsigned long long ipext_InTruncatedPkts = 0; + static unsigned long long ipext_InCsumErrors = 0; + + // IP multicast bandwidth + static unsigned long long ipext_InMcastOctets = 0; + static unsigned long long ipext_OutMcastOctets = 0; + + // IP multicast packets + static unsigned long long ipext_InMcastPkts = 0; + static unsigned long long ipext_OutMcastPkts = 0; + + // IP broadcast bandwidth + static unsigned long long ipext_InBcastOctets = 0; + static unsigned long long ipext_OutBcastOctets = 0; + + // IP broadcast packets + static unsigned long long ipext_InBcastPkts = 0; + static unsigned long long ipext_OutBcastPkts = 0; + + // IP ECN + static unsigned long long ipext_InNoECTPkts = 0; + static unsigned long long ipext_InECT1Pkts = 0; + static unsigned long long ipext_InECT0Pkts = 0; + static unsigned long long ipext_InCEPkts = 0; + + // -------------------------------------------------------------------- + // IP TCP + + // IP TCP Reordering + static unsigned long long tcpext_TCPRenoReorder = 0; + static unsigned long long tcpext_TCPFACKReorder = 0; + static unsigned long long tcpext_TCPSACKReorder = 0; + static unsigned long long tcpext_TCPTSReorder = 0; + + // IP TCP SYN Cookies + static unsigned long long tcpext_SyncookiesSent = 0; + static unsigned long long tcpext_SyncookiesRecv = 0; + static unsigned long long tcpext_SyncookiesFailed = 0; + + // IP TCP Out Of Order Queue + // http://www.spinics.net/lists/netdev/msg204696.html + static unsigned long long tcpext_TCPOFOQueue = 0; // Number of packets queued in OFO queue + static unsigned long long tcpext_TCPOFODrop = 0; // Number of packets meant to be queued in OFO but dropped because socket rcvbuf limit hit. + static unsigned long long tcpext_TCPOFOMerge = 0; // Number of packets in OFO that were merged with other packets. + static unsigned long long tcpext_OfoPruned = 0; // packets dropped from out-of-order queue because of socket buffer overrun + + // IP TCP connection resets + // https://github.com/ecki/net-tools/blob/bd8bceaed2311651710331a7f8990c3e31be9840/statistics.c + static unsigned long long tcpext_TCPAbortOnData = 0; // connections reset due to unexpected data + static unsigned long long tcpext_TCPAbortOnClose = 0; // connections reset due to early user close + static unsigned long long tcpext_TCPAbortOnMemory = 0; // connections aborted due to memory pressure + static unsigned long long tcpext_TCPAbortOnTimeout = 0; // connections aborted due to timeout + static unsigned long long tcpext_TCPAbortOnLinger = 0; // connections aborted after user close in linger timeout + static unsigned long long tcpext_TCPAbortFailed = 0; // times unable to send RST due to no memory + + // https://perfchron.com/2015/12/26/investigating-linux-network-issues-with-netstat-and-nstat/ + static unsigned long long tcpext_ListenOverflows = 0; // times the listen queue of a socket overflowed + static unsigned long long tcpext_ListenDrops = 0; // SYNs to LISTEN sockets ignored + + // IP TCP memory pressures + static unsigned long long tcpext_TCPMemoryPressures = 0; + + static unsigned long long tcpext_TCPReqQFullDrop = 0; + static unsigned long long tcpext_TCPReqQFullDoCookies = 0; + + static unsigned long long tcpext_TCPSynRetrans = 0; + // prepare for /proc/net/netstat parsing if(unlikely(!arl_ipext)) { @@ -547,127 +1650,6 @@ int do_proc_net_netstat(int update_every, usec_t dt) { tcp_max_connections_var = rrdvar_custom_host_variable_add_and_acquire(localhost, "tcp_max_connections"); } - // prepare for /proc/net/snmp6 parsing - - if(unlikely(!arl_ipv6)) { - do_ip6_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 packets", CONFIG_BOOLEAN_AUTO); - do_ip6_fragsout = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 fragments sent", CONFIG_BOOLEAN_AUTO); - do_ip6_fragsin = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 fragments assembly", CONFIG_BOOLEAN_AUTO); - do_ip6_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 errors", CONFIG_BOOLEAN_AUTO); - do_ip6_udp_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDP packets", CONFIG_BOOLEAN_AUTO); - do_ip6_udp_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDP errors", CONFIG_BOOLEAN_AUTO); - do_ip6_udplite_packets = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDPlite packets", CONFIG_BOOLEAN_AUTO); - do_ip6_udplite_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ipv6 UDPlite errors", CONFIG_BOOLEAN_AUTO); - do_ip6_bandwidth = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "bandwidth", CONFIG_BOOLEAN_AUTO); - do_ip6_mcast = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "multicast bandwidth", CONFIG_BOOLEAN_AUTO); - do_ip6_bcast = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "broadcast bandwidth", CONFIG_BOOLEAN_AUTO); - do_ip6_mcast_p = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "multicast packets", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_redir = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp redirects", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_errors = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp errors", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_echos = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp echos", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_groupmemb = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp group membership", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_router = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp router", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_neighbor = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp neighbor", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_mldv2 = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp mldv2", CONFIG_BOOLEAN_AUTO); - do_ip6_icmp_types = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "icmp types", CONFIG_BOOLEAN_AUTO); - do_ip6_ect = config_get_boolean_ondemand("plugin:proc:/proc/net/snmp6", "ect", CONFIG_BOOLEAN_AUTO); - - arl_ipv6 = arl_create("snmp6", NULL, 60); - arl_expect(arl_ipv6, "Ip6InReceives", &Ip6InReceives); - arl_expect(arl_ipv6, "Ip6InHdrErrors", &Ip6InHdrErrors); - arl_expect(arl_ipv6, "Ip6InTooBigErrors", &Ip6InTooBigErrors); - arl_expect(arl_ipv6, "Ip6InNoRoutes", &Ip6InNoRoutes); - arl_expect(arl_ipv6, "Ip6InAddrErrors", &Ip6InAddrErrors); - arl_expect(arl_ipv6, "Ip6InUnknownProtos", &Ip6InUnknownProtos); - arl_expect(arl_ipv6, "Ip6InTruncatedPkts", &Ip6InTruncatedPkts); - arl_expect(arl_ipv6, "Ip6InDiscards", &Ip6InDiscards); - arl_expect(arl_ipv6, "Ip6InDelivers", &Ip6InDelivers); - arl_expect(arl_ipv6, "Ip6OutForwDatagrams", &Ip6OutForwDatagrams); - arl_expect(arl_ipv6, "Ip6OutRequests", &Ip6OutRequests); - arl_expect(arl_ipv6, "Ip6OutDiscards", &Ip6OutDiscards); - arl_expect(arl_ipv6, "Ip6OutNoRoutes", &Ip6OutNoRoutes); - arl_expect(arl_ipv6, "Ip6ReasmTimeout", &Ip6ReasmTimeout); - arl_expect(arl_ipv6, "Ip6ReasmReqds", &Ip6ReasmReqds); - arl_expect(arl_ipv6, "Ip6ReasmOKs", &Ip6ReasmOKs); - arl_expect(arl_ipv6, "Ip6ReasmFails", &Ip6ReasmFails); - arl_expect(arl_ipv6, "Ip6FragOKs", &Ip6FragOKs); - arl_expect(arl_ipv6, "Ip6FragFails", &Ip6FragFails); - arl_expect(arl_ipv6, "Ip6FragCreates", &Ip6FragCreates); - arl_expect(arl_ipv6, "Ip6InMcastPkts", &Ip6InMcastPkts); - arl_expect(arl_ipv6, "Ip6OutMcastPkts", &Ip6OutMcastPkts); - arl_expect(arl_ipv6, "Ip6InOctets", &Ip6InOctets); - arl_expect(arl_ipv6, "Ip6OutOctets", &Ip6OutOctets); - arl_expect(arl_ipv6, "Ip6InMcastOctets", &Ip6InMcastOctets); - arl_expect(arl_ipv6, "Ip6OutMcastOctets", &Ip6OutMcastOctets); - arl_expect(arl_ipv6, "Ip6InBcastOctets", &Ip6InBcastOctets); - arl_expect(arl_ipv6, "Ip6OutBcastOctets", &Ip6OutBcastOctets); - arl_expect(arl_ipv6, "Ip6InNoECTPkts", &Ip6InNoECTPkts); - arl_expect(arl_ipv6, "Ip6InECT1Pkts", &Ip6InECT1Pkts); - arl_expect(arl_ipv6, "Ip6InECT0Pkts", &Ip6InECT0Pkts); - arl_expect(arl_ipv6, "Ip6InCEPkts", &Ip6InCEPkts); - arl_expect(arl_ipv6, "Icmp6InMsgs", &Icmp6InMsgs); - arl_expect(arl_ipv6, "Icmp6InErrors", &Icmp6InErrors); - arl_expect(arl_ipv6, "Icmp6OutMsgs", &Icmp6OutMsgs); - arl_expect(arl_ipv6, "Icmp6OutErrors", &Icmp6OutErrors); - arl_expect(arl_ipv6, "Icmp6InCsumErrors", &Icmp6InCsumErrors); - arl_expect(arl_ipv6, "Icmp6InDestUnreachs", &Icmp6InDestUnreachs); - arl_expect(arl_ipv6, "Icmp6InPktTooBigs", &Icmp6InPktTooBigs); - arl_expect(arl_ipv6, "Icmp6InTimeExcds", &Icmp6InTimeExcds); - arl_expect(arl_ipv6, "Icmp6InParmProblems", &Icmp6InParmProblems); - arl_expect(arl_ipv6, "Icmp6InEchos", &Icmp6InEchos); - arl_expect(arl_ipv6, "Icmp6InEchoReplies", &Icmp6InEchoReplies); - arl_expect(arl_ipv6, "Icmp6InGroupMembQueries", &Icmp6InGroupMembQueries); - arl_expect(arl_ipv6, "Icmp6InGroupMembResponses", &Icmp6InGroupMembResponses); - arl_expect(arl_ipv6, "Icmp6InGroupMembReductions", &Icmp6InGroupMembReductions); - arl_expect(arl_ipv6, "Icmp6InRouterSolicits", &Icmp6InRouterSolicits); - arl_expect(arl_ipv6, "Icmp6InRouterAdvertisements", &Icmp6InRouterAdvertisements); - arl_expect(arl_ipv6, "Icmp6InNeighborSolicits", &Icmp6InNeighborSolicits); - arl_expect(arl_ipv6, "Icmp6InNeighborAdvertisements", &Icmp6InNeighborAdvertisements); - arl_expect(arl_ipv6, "Icmp6InRedirects", &Icmp6InRedirects); - arl_expect(arl_ipv6, "Icmp6InMLDv2Reports", &Icmp6InMLDv2Reports); - arl_expect(arl_ipv6, "Icmp6OutDestUnreachs", &Icmp6OutDestUnreachs); - arl_expect(arl_ipv6, "Icmp6OutPktTooBigs", &Icmp6OutPktTooBigs); - arl_expect(arl_ipv6, "Icmp6OutTimeExcds", &Icmp6OutTimeExcds); - arl_expect(arl_ipv6, "Icmp6OutParmProblems", &Icmp6OutParmProblems); - arl_expect(arl_ipv6, "Icmp6OutEchos", &Icmp6OutEchos); - arl_expect(arl_ipv6, "Icmp6OutEchoReplies", &Icmp6OutEchoReplies); - arl_expect(arl_ipv6, "Icmp6OutGroupMembQueries", &Icmp6OutGroupMembQueries); - arl_expect(arl_ipv6, "Icmp6OutGroupMembResponses", &Icmp6OutGroupMembResponses); - arl_expect(arl_ipv6, "Icmp6OutGroupMembReductions", &Icmp6OutGroupMembReductions); - arl_expect(arl_ipv6, "Icmp6OutRouterSolicits", &Icmp6OutRouterSolicits); - arl_expect(arl_ipv6, "Icmp6OutRouterAdvertisements", &Icmp6OutRouterAdvertisements); - arl_expect(arl_ipv6, "Icmp6OutNeighborSolicits", &Icmp6OutNeighborSolicits); - arl_expect(arl_ipv6, "Icmp6OutNeighborAdvertisements", &Icmp6OutNeighborAdvertisements); - arl_expect(arl_ipv6, "Icmp6OutRedirects", &Icmp6OutRedirects); - arl_expect(arl_ipv6, "Icmp6OutMLDv2Reports", &Icmp6OutMLDv2Reports); - arl_expect(arl_ipv6, "Icmp6InType1", &Icmp6InType1); - arl_expect(arl_ipv6, "Icmp6InType128", &Icmp6InType128); - arl_expect(arl_ipv6, "Icmp6InType129", &Icmp6InType129); - arl_expect(arl_ipv6, "Icmp6InType136", &Icmp6InType136); - arl_expect(arl_ipv6, "Icmp6OutType1", &Icmp6OutType1); - arl_expect(arl_ipv6, "Icmp6OutType128", &Icmp6OutType128); - arl_expect(arl_ipv6, "Icmp6OutType129", &Icmp6OutType129); - arl_expect(arl_ipv6, "Icmp6OutType133", &Icmp6OutType133); - arl_expect(arl_ipv6, "Icmp6OutType135", &Icmp6OutType135); - arl_expect(arl_ipv6, "Icmp6OutType143", &Icmp6OutType143); - arl_expect(arl_ipv6, "Udp6InDatagrams", &Udp6InDatagrams); - arl_expect(arl_ipv6, "Udp6NoPorts", &Udp6NoPorts); - arl_expect(arl_ipv6, "Udp6InErrors", &Udp6InErrors); - arl_expect(arl_ipv6, "Udp6OutDatagrams", &Udp6OutDatagrams); - arl_expect(arl_ipv6, "Udp6RcvbufErrors", &Udp6RcvbufErrors); - arl_expect(arl_ipv6, "Udp6SndbufErrors", &Udp6SndbufErrors); - arl_expect(arl_ipv6, "Udp6InCsumErrors", &Udp6InCsumErrors); - arl_expect(arl_ipv6, "Udp6IgnoredMulti", &Udp6IgnoredMulti); - arl_expect(arl_ipv6, "UdpLite6InDatagrams", &UdpLite6InDatagrams); - arl_expect(arl_ipv6, "UdpLite6NoPorts", &UdpLite6NoPorts); - arl_expect(arl_ipv6, "UdpLite6InErrors", &UdpLite6InErrors); - arl_expect(arl_ipv6, "UdpLite6OutDatagrams", &UdpLite6OutDatagrams); - arl_expect(arl_ipv6, "UdpLite6RcvbufErrors", &UdpLite6RcvbufErrors); - arl_expect(arl_ipv6, "UdpLite6SndbufErrors", &UdpLite6SndbufErrors); - arl_expect(arl_ipv6, "UdpLite6InCsumErrors", &UdpLite6InCsumErrors); - } - size_t lines, l, words; // parse /proc/net/netstat @@ -696,7 +1678,7 @@ int do_proc_net_netstat(int update_every, usec_t dt) { words = procfile_linewords(ff_netstat, l); if(unlikely(words < 2)) { - error("Cannot read /proc/net/netstat IpExt line. Expected 2+ params, read %zu.", words); + collector_error("Cannot read /proc/net/netstat IpExt line. Expected 2+ params, read %zu.", words); continue; } @@ -708,7 +1690,7 @@ int do_proc_net_netstat(int update_every, usec_t dt) { words = procfile_linewords(ff_netstat, l); if(unlikely(words < 2)) { - error("Cannot read /proc/net/netstat TcpExt line. Expected 2+ params, read %zu.", words); + collector_error("Cannot read /proc/net/netstat TcpExt line. Expected 2+ params, read %zu.", words); continue; } @@ -739,13 +1721,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "Ip") != 0) { - error("Cannot read Ip line from /proc/net/snmp."); + collector_error("Cannot read Ip line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 3) { - error("Cannot read /proc/net/snmp Ip line. Expected 3+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp Ip line. Expected 3+ params, read %zu.", words); continue; } @@ -759,13 +1741,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "Icmp") != 0) { - error("Cannot read Icmp line from /proc/net/snmp."); + collector_error("Cannot read Icmp line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 3) { - error("Cannot read /proc/net/snmp Icmp line. Expected 3+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp Icmp line. Expected 3+ params, read %zu.", words); continue; } @@ -779,13 +1761,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "IcmpMsg") != 0) { - error("Cannot read IcmpMsg line from /proc/net/snmp."); + collector_error("Cannot read IcmpMsg line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 2) { - error("Cannot read /proc/net/snmp IcmpMsg line. Expected 2+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp IcmpMsg line. Expected 2+ params, read %zu.", words); continue; } @@ -799,13 +1781,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "Tcp") != 0) { - error("Cannot read Tcp line from /proc/net/snmp."); + collector_error("Cannot read Tcp line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 3) { - error("Cannot read /proc/net/snmp Tcp line. Expected 3+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp Tcp line. Expected 3+ params, read %zu.", words); continue; } @@ -819,13 +1801,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "Udp") != 0) { - error("Cannot read Udp line from /proc/net/snmp."); + collector_error("Cannot read Udp line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 3) { - error("Cannot read /proc/net/snmp Udp line. Expected 3+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp Udp line. Expected 3+ params, read %zu.", words); continue; } @@ -839,13 +1821,13 @@ int do_proc_net_netstat(int update_every, usec_t dt) { size_t h = l++; if(strcmp(procfile_lineword(ff_snmp, l, 0), "UdpLite") != 0) { - error("Cannot read UdpLite line from /proc/net/snmp."); + collector_error("Cannot read UdpLite line from /proc/net/snmp."); break; } words = procfile_linewords(ff_snmp, l); if(words < 3) { - error("Cannot read /proc/net/snmp UdpLite line. Expected 3+ params, read %zu.", words); + collector_error("Cannot read /proc/net/snmp UdpLite line. Expected 3+ params, read %zu.", words); continue; } @@ -857,36 +1839,6 @@ int do_proc_net_netstat(int update_every, usec_t dt) { } } - // parse /proc/net/snmp - - if(unlikely(!ff_snmp6)) { - char filename[FILENAME_MAX + 1]; - snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/proc/net/snmp6"); - ff_snmp6 = procfile_open(config_get("plugin:proc:/proc/net/snmp6", "filename to monitor", filename), " \t:", PROCFILE_FLAG_DEFAULT); - if(unlikely(!ff_snmp6)) - return 1; - } - - ff_snmp6 = procfile_readall(ff_snmp6); - if(unlikely(!ff_snmp6)) - return 0; // we return 0, so that we will retry to open it next time - - lines = procfile_lines(ff_snmp6); - - arl_begin(arl_ipv6); - - for(l = 0; l < lines ;l++) { - size_t words = procfile_linewords(ff_snmp6, l); - if(unlikely(words < 2)) { - if(unlikely(words)) error("Cannot read /proc/net/snmp6 line %zu. Expected 2 params, read %zu.", l, words); - continue; - } - - if(unlikely(arl_check(arl_ipv6, - procfile_lineword(ff_snmp6, l, 0), - procfile_lineword(ff_snmp6, l, 1)))) break; - } - // netstat IpExt charts if(do_bandwidth == CONFIG_BOOLEAN_YES || (do_bandwidth == CONFIG_BOOLEAN_AUTO && @@ -2157,954 +3109,7 @@ int do_proc_net_netstat(int update_every, usec_t dt) { } } - // snmp6 charts - - if(do_ip6_bandwidth == CONFIG_BOOLEAN_YES || (do_ip6_bandwidth == CONFIG_BOOLEAN_AUTO && - (Ip6InOctets || - Ip6OutOctets || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_bandwidth = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_received = NULL, - *rd_sent = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - "system" - , "ipv6" - , NULL - , "network" - , NULL - , "IPv6 Bandwidth" - , "kilobits/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_SYSTEM_IPV6 - , update_every - , RRDSET_TYPE_AREA - ); - - rd_received = rrddim_add(st, "InOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - rd_sent = rrddim_add(st, "OutOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_received, Ip6InOctets); - rrddim_set_by_pointer(st, rd_sent, Ip6OutOctets); - rrdset_done(st); - } - - if(do_ip6_packets == CONFIG_BOOLEAN_YES || (do_ip6_packets == CONFIG_BOOLEAN_AUTO && - (Ip6InReceives || - Ip6OutRequests || - Ip6InDelivers || - Ip6OutForwDatagrams || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_packets = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_received = NULL, - *rd_sent = NULL, - *rd_forwarded = NULL, - *rd_delivers = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "packets" - , NULL - , "packets" - , NULL - , "IPv6 Packets" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_PACKETS - , update_every - , RRDSET_TYPE_LINE - ); - - rd_received = rrddim_add(st, "InReceives", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_sent = rrddim_add(st, "OutRequests", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_forwarded = rrddim_add(st, "OutForwDatagrams", "forwarded", -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_delivers = rrddim_add(st, "InDelivers", "delivers", 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_received, Ip6InReceives); - rrddim_set_by_pointer(st, rd_sent, Ip6OutRequests); - rrddim_set_by_pointer(st, rd_forwarded, Ip6OutForwDatagrams); - rrddim_set_by_pointer(st, rd_delivers, Ip6InDelivers); - rrdset_done(st); - } - - if(do_ip6_fragsout == CONFIG_BOOLEAN_YES || (do_ip6_fragsout == CONFIG_BOOLEAN_AUTO && - (Ip6FragOKs || - Ip6FragFails || - Ip6FragCreates || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_fragsout = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_ok = NULL, - *rd_failed = NULL, - *rd_all = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "fragsout" - , NULL - , "fragments6" - , NULL - , "IPv6 Fragments Sent" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_FRAGSOUT - , update_every - , RRDSET_TYPE_LINE - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_ok = rrddim_add(st, "FragOKs", "ok", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_failed = rrddim_add(st, "FragFails", "failed", -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_all = rrddim_add(st, "FragCreates", "all", 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_ok, Ip6FragOKs); - rrddim_set_by_pointer(st, rd_failed, Ip6FragFails); - rrddim_set_by_pointer(st, rd_all, Ip6FragCreates); - rrdset_done(st); - } - - if(do_ip6_fragsin == CONFIG_BOOLEAN_YES || (do_ip6_fragsin == CONFIG_BOOLEAN_AUTO && - (Ip6ReasmOKs || - Ip6ReasmFails || - Ip6ReasmTimeout || - Ip6ReasmReqds || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_fragsin = CONFIG_BOOLEAN_YES; - - static RRDSET *st = NULL; - static RRDDIM *rd_ok = NULL, - *rd_failed = NULL, - *rd_timeout = NULL, - *rd_all = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "fragsin" - , NULL - , "fragments6" - , NULL - , "IPv6 Fragments Reassembly" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_FRAGSIN - , update_every - , RRDSET_TYPE_LINE); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_ok = rrddim_add(st, "ReasmOKs", "ok", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_failed = rrddim_add(st, "ReasmFails", "failed", -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_timeout = rrddim_add(st, "ReasmTimeout", "timeout", -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_all = rrddim_add(st, "ReasmReqds", "all", 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_ok, Ip6ReasmOKs); - rrddim_set_by_pointer(st, rd_failed, Ip6ReasmFails); - rrddim_set_by_pointer(st, rd_timeout, Ip6ReasmTimeout); - rrddim_set_by_pointer(st, rd_all, Ip6ReasmReqds); - rrdset_done(st); - } - - if(do_ip6_errors == CONFIG_BOOLEAN_YES || (do_ip6_errors == CONFIG_BOOLEAN_AUTO && - (Ip6InDiscards || - Ip6OutDiscards || - Ip6InHdrErrors || - Ip6InAddrErrors || - Ip6InUnknownProtos || - Ip6InTooBigErrors || - Ip6InTruncatedPkts || - Ip6InNoRoutes || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_errors = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InDiscards = NULL, - *rd_OutDiscards = NULL, - *rd_InHdrErrors = NULL, - *rd_InAddrErrors = NULL, - *rd_InUnknownProtos = NULL, - *rd_InTooBigErrors = NULL, - *rd_InTruncatedPkts = NULL, - *rd_InNoRoutes = NULL, - *rd_OutNoRoutes = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "errors" - , NULL - , "errors" - , NULL - , "IPv6 Errors" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ERRORS - , update_every - , RRDSET_TYPE_LINE - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_InDiscards = rrddim_add(st, "InDiscards", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutDiscards = rrddim_add(st, "OutDiscards", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InHdrErrors = rrddim_add(st, "InHdrErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InAddrErrors = rrddim_add(st, "InAddrErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InUnknownProtos = rrddim_add(st, "InUnknownProtos", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InTooBigErrors = rrddim_add(st, "InTooBigErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InTruncatedPkts = rrddim_add(st, "InTruncatedPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InNoRoutes = rrddim_add(st, "InNoRoutes", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutNoRoutes = rrddim_add(st, "OutNoRoutes", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InDiscards, Ip6InDiscards); - rrddim_set_by_pointer(st, rd_OutDiscards, Ip6OutDiscards); - rrddim_set_by_pointer(st, rd_InHdrErrors, Ip6InHdrErrors); - rrddim_set_by_pointer(st, rd_InAddrErrors, Ip6InAddrErrors); - rrddim_set_by_pointer(st, rd_InUnknownProtos, Ip6InUnknownProtos); - rrddim_set_by_pointer(st, rd_InTooBigErrors, Ip6InTooBigErrors); - rrddim_set_by_pointer(st, rd_InTruncatedPkts, Ip6InTruncatedPkts); - rrddim_set_by_pointer(st, rd_InNoRoutes, Ip6InNoRoutes); - rrddim_set_by_pointer(st, rd_OutNoRoutes, Ip6OutNoRoutes); - rrdset_done(st); - } - - if(do_ip6_udp_packets == CONFIG_BOOLEAN_YES || (do_ip6_udp_packets == CONFIG_BOOLEAN_AUTO && - (Udp6InDatagrams || - Udp6OutDatagrams || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_udp_packets = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_received = NULL, - *rd_sent = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "udppackets" - , NULL - , "udp6" - , NULL - , "IPv6 UDP Packets" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_UDP_PACKETS - , update_every - , RRDSET_TYPE_LINE - ); - - rd_received = rrddim_add(st, "InDatagrams", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_sent = rrddim_add(st, "OutDatagrams", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_received, Udp6InDatagrams); - rrddim_set_by_pointer(st, rd_sent, Udp6OutDatagrams); - rrdset_done(st); - } - - if(do_ip6_udp_errors == CONFIG_BOOLEAN_YES || (do_ip6_udp_errors == CONFIG_BOOLEAN_AUTO && - (Udp6InErrors || - Udp6NoPorts || - Udp6RcvbufErrors || - Udp6SndbufErrors || - Udp6InCsumErrors || - Udp6IgnoredMulti || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_udp_errors = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_RcvbufErrors = NULL, - *rd_SndbufErrors = NULL, - *rd_InErrors = NULL, - *rd_NoPorts = NULL, - *rd_InCsumErrors = NULL, - *rd_IgnoredMulti = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "udperrors" - , NULL - , "udp6" - , NULL - , "IPv6 UDP Errors" - , "events/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_UDP_ERRORS - , update_every - , RRDSET_TYPE_LINE - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_RcvbufErrors = rrddim_add(st, "RcvbufErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_SndbufErrors = rrddim_add(st, "SndbufErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_NoPorts = rrddim_add(st, "NoPorts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_IgnoredMulti = rrddim_add(st, "IgnoredMulti", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_RcvbufErrors, Udp6RcvbufErrors); - rrddim_set_by_pointer(st, rd_SndbufErrors, Udp6SndbufErrors); - rrddim_set_by_pointer(st, rd_InErrors, Udp6InErrors); - rrddim_set_by_pointer(st, rd_NoPorts, Udp6NoPorts); - rrddim_set_by_pointer(st, rd_InCsumErrors, Udp6InCsumErrors); - rrddim_set_by_pointer(st, rd_IgnoredMulti, Udp6IgnoredMulti); - rrdset_done(st); - } - - if(do_ip6_udplite_packets == CONFIG_BOOLEAN_YES || (do_ip6_udplite_packets == CONFIG_BOOLEAN_AUTO && - (UdpLite6InDatagrams || - UdpLite6OutDatagrams || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_udplite_packets = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_received = NULL, - *rd_sent = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "udplitepackets" - , NULL - , "udplite6" - , NULL - , "IPv6 UDPlite Packets" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_UDPLITE_PACKETS - , update_every - , RRDSET_TYPE_LINE - ); - - rd_received = rrddim_add(st, "InDatagrams", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_sent = rrddim_add(st, "OutDatagrams", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_received, UdpLite6InDatagrams); - rrddim_set_by_pointer(st, rd_sent, UdpLite6OutDatagrams); - rrdset_done(st); - } - - if(do_ip6_udplite_errors == CONFIG_BOOLEAN_YES || (do_ip6_udplite_errors == CONFIG_BOOLEAN_AUTO && - (UdpLite6InErrors || - UdpLite6NoPorts || - UdpLite6RcvbufErrors || - UdpLite6SndbufErrors || - Udp6InCsumErrors || - UdpLite6InCsumErrors || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_udplite_errors = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_RcvbufErrors = NULL, - *rd_SndbufErrors = NULL, - *rd_InErrors = NULL, - *rd_NoPorts = NULL, - *rd_InCsumErrors = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "udpliteerrors" - , NULL - , "udplite6" - , NULL - , "IPv6 UDP Lite Errors" - , "events/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_UDPLITE_ERRORS - , update_every - , RRDSET_TYPE_LINE - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_RcvbufErrors = rrddim_add(st, "RcvbufErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_SndbufErrors = rrddim_add(st, "SndbufErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_NoPorts = rrddim_add(st, "NoPorts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InErrors, UdpLite6InErrors); - rrddim_set_by_pointer(st, rd_NoPorts, UdpLite6NoPorts); - rrddim_set_by_pointer(st, rd_RcvbufErrors, UdpLite6RcvbufErrors); - rrddim_set_by_pointer(st, rd_SndbufErrors, UdpLite6SndbufErrors); - rrddim_set_by_pointer(st, rd_InCsumErrors, UdpLite6InCsumErrors); - rrdset_done(st); - } - - if(do_ip6_mcast == CONFIG_BOOLEAN_YES || (do_ip6_mcast == CONFIG_BOOLEAN_AUTO && - (Ip6OutMcastOctets || - Ip6InMcastOctets || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_mcast = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_Ip6InMcastOctets = NULL, - *rd_Ip6OutMcastOctets = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "mcast" - , NULL - , "multicast6" - , NULL - , "IPv6 Multicast Bandwidth" - , "kilobits/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_MCAST - , update_every - , RRDSET_TYPE_AREA - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_Ip6InMcastOctets = rrddim_add(st, "InMcastOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - rd_Ip6OutMcastOctets = rrddim_add(st, "OutMcastOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_Ip6InMcastOctets, Ip6InMcastOctets); - rrddim_set_by_pointer(st, rd_Ip6OutMcastOctets, Ip6OutMcastOctets); - rrdset_done(st); - } - - if(do_ip6_bcast == CONFIG_BOOLEAN_YES || (do_ip6_bcast == CONFIG_BOOLEAN_AUTO && - (Ip6OutBcastOctets || - Ip6InBcastOctets || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_bcast = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_Ip6InBcastOctets = NULL, - *rd_Ip6OutBcastOctets = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "bcast" - , NULL - , "broadcast6" - , NULL - , "IPv6 Broadcast Bandwidth" - , "kilobits/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_BCAST - , update_every - , RRDSET_TYPE_AREA - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_Ip6InBcastOctets = rrddim_add(st, "InBcastOctets", "received", 8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - rd_Ip6OutBcastOctets = rrddim_add(st, "OutBcastOctets", "sent", -8, BITS_IN_A_KILOBIT, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_Ip6InBcastOctets, Ip6InBcastOctets); - rrddim_set_by_pointer(st, rd_Ip6OutBcastOctets, Ip6OutBcastOctets); - rrdset_done(st); - } - - if(do_ip6_mcast_p == CONFIG_BOOLEAN_YES || (do_ip6_mcast_p == CONFIG_BOOLEAN_AUTO && - (Ip6OutMcastPkts || - Ip6InMcastPkts || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_mcast_p = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_Ip6InMcastPkts = NULL, - *rd_Ip6OutMcastPkts = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "mcastpkts" - , NULL - , "multicast6" - , NULL - , "IPv6 Multicast Packets" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_MCAST_PACKETS - , update_every - , RRDSET_TYPE_LINE - ); - rrdset_flag_set(st, RRDSET_FLAG_DETAIL); - - rd_Ip6InMcastPkts = rrddim_add(st, "InMcastPkts", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_Ip6OutMcastPkts = rrddim_add(st, "OutMcastPkts", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_Ip6InMcastPkts, Ip6InMcastPkts); - rrddim_set_by_pointer(st, rd_Ip6OutMcastPkts, Ip6OutMcastPkts); - rrdset_done(st); - } - - if(do_ip6_icmp == CONFIG_BOOLEAN_YES || (do_ip6_icmp == CONFIG_BOOLEAN_AUTO && - (Icmp6InMsgs || - Icmp6OutMsgs || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_Icmp6InMsgs = NULL, - *rd_Icmp6OutMsgs = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmp" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Messages" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP - , update_every - , RRDSET_TYPE_LINE - ); - - rd_Icmp6InMsgs = rrddim_add(st, "InMsgs", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_Icmp6OutMsgs = rrddim_add(st, "OutMsgs", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_Icmp6InMsgs, Icmp6InMsgs); - rrddim_set_by_pointer(st, rd_Icmp6OutMsgs, Icmp6OutMsgs); - rrdset_done(st); - } - - if(do_ip6_icmp_redir == CONFIG_BOOLEAN_YES || (do_ip6_icmp_redir == CONFIG_BOOLEAN_AUTO && - (Icmp6InRedirects || - Icmp6OutRedirects || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_redir = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_Icmp6InRedirects = NULL, - *rd_Icmp6OutRedirects = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmpredir" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Redirects" - , "redirects/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_REDIR - , update_every - , RRDSET_TYPE_LINE - ); - - rd_Icmp6InRedirects = rrddim_add(st, "InRedirects", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_Icmp6OutRedirects = rrddim_add(st, "OutRedirects", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_Icmp6InRedirects, Icmp6InRedirects); - rrddim_set_by_pointer(st, rd_Icmp6OutRedirects, Icmp6OutRedirects); - rrdset_done(st); - } - - if(do_ip6_icmp_errors == CONFIG_BOOLEAN_YES || (do_ip6_icmp_errors == CONFIG_BOOLEAN_AUTO && - (Icmp6InErrors || - Icmp6OutErrors || - Icmp6InCsumErrors || - Icmp6InDestUnreachs || - Icmp6InPktTooBigs || - Icmp6InTimeExcds || - Icmp6InParmProblems || - Icmp6OutDestUnreachs || - Icmp6OutPktTooBigs || - Icmp6OutTimeExcds || - Icmp6OutParmProblems || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_errors = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InErrors = NULL, - *rd_OutErrors = NULL, - *rd_InCsumErrors = NULL, - *rd_InDestUnreachs = NULL, - *rd_InPktTooBigs = NULL, - *rd_InTimeExcds = NULL, - *rd_InParmProblems = NULL, - *rd_OutDestUnreachs = NULL, - *rd_OutPktTooBigs = NULL, - *rd_OutTimeExcds = NULL, - *rd_OutParmProblems = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmperrors" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Errors" - , "errors/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_ERRORS - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InErrors = rrddim_add(st, "InErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutErrors = rrddim_add(st, "OutErrors", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InCsumErrors = rrddim_add(st, "InCsumErrors", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InDestUnreachs = rrddim_add(st, "InDestUnreachs", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InPktTooBigs = rrddim_add(st, "InPktTooBigs", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InTimeExcds = rrddim_add(st, "InTimeExcds", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InParmProblems = rrddim_add(st, "InParmProblems", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutDestUnreachs = rrddim_add(st, "OutDestUnreachs", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutPktTooBigs = rrddim_add(st, "OutPktTooBigs", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutTimeExcds = rrddim_add(st, "OutTimeExcds", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutParmProblems = rrddim_add(st, "OutParmProblems", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InErrors, Icmp6InErrors); - rrddim_set_by_pointer(st, rd_OutErrors, Icmp6OutErrors); - rrddim_set_by_pointer(st, rd_InCsumErrors, Icmp6InCsumErrors); - rrddim_set_by_pointer(st, rd_InDestUnreachs, Icmp6InDestUnreachs); - rrddim_set_by_pointer(st, rd_InPktTooBigs, Icmp6InPktTooBigs); - rrddim_set_by_pointer(st, rd_InTimeExcds, Icmp6InTimeExcds); - rrddim_set_by_pointer(st, rd_InParmProblems, Icmp6InParmProblems); - rrddim_set_by_pointer(st, rd_OutDestUnreachs, Icmp6OutDestUnreachs); - rrddim_set_by_pointer(st, rd_OutPktTooBigs, Icmp6OutPktTooBigs); - rrddim_set_by_pointer(st, rd_OutTimeExcds, Icmp6OutTimeExcds); - rrddim_set_by_pointer(st, rd_OutParmProblems, Icmp6OutParmProblems); - rrdset_done(st); - } - - if(do_ip6_icmp_echos == CONFIG_BOOLEAN_YES || (do_ip6_icmp_echos == CONFIG_BOOLEAN_AUTO && - (Icmp6InEchos || - Icmp6OutEchos || - Icmp6InEchoReplies || - Icmp6OutEchoReplies || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_echos = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InEchos = NULL, - *rd_OutEchos = NULL, - *rd_InEchoReplies = NULL, - *rd_OutEchoReplies = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmpechos" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Echo" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_ECHOS - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InEchos = rrddim_add(st, "InEchos", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutEchos = rrddim_add(st, "OutEchos", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InEchoReplies = rrddim_add(st, "InEchoReplies", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutEchoReplies = rrddim_add(st, "OutEchoReplies", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InEchos, Icmp6InEchos); - rrddim_set_by_pointer(st, rd_OutEchos, Icmp6OutEchos); - rrddim_set_by_pointer(st, rd_InEchoReplies, Icmp6InEchoReplies); - rrddim_set_by_pointer(st, rd_OutEchoReplies, Icmp6OutEchoReplies); - rrdset_done(st); - } - - if(do_ip6_icmp_groupmemb == CONFIG_BOOLEAN_YES || (do_ip6_icmp_groupmemb == CONFIG_BOOLEAN_AUTO && - (Icmp6InGroupMembQueries || - Icmp6OutGroupMembQueries || - Icmp6InGroupMembResponses || - Icmp6OutGroupMembResponses || - Icmp6InGroupMembReductions || - Icmp6OutGroupMembReductions || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_groupmemb = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InQueries = NULL, - *rd_OutQueries = NULL, - *rd_InResponses = NULL, - *rd_OutResponses = NULL, - *rd_InReductions = NULL, - *rd_OutReductions = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "groupmemb" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Group Membership" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_GROUPMEMB - , update_every - , RRDSET_TYPE_LINE); - - rd_InQueries = rrddim_add(st, "InQueries", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutQueries = rrddim_add(st, "OutQueries", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InResponses = rrddim_add(st, "InResponses", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutResponses = rrddim_add(st, "OutResponses", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InReductions = rrddim_add(st, "InReductions", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutReductions = rrddim_add(st, "OutReductions", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InQueries, Icmp6InGroupMembQueries); - rrddim_set_by_pointer(st, rd_OutQueries, Icmp6OutGroupMembQueries); - rrddim_set_by_pointer(st, rd_InResponses, Icmp6InGroupMembResponses); - rrddim_set_by_pointer(st, rd_OutResponses, Icmp6OutGroupMembResponses); - rrddim_set_by_pointer(st, rd_InReductions, Icmp6InGroupMembReductions); - rrddim_set_by_pointer(st, rd_OutReductions, Icmp6OutGroupMembReductions); - rrdset_done(st); - } - - if(do_ip6_icmp_router == CONFIG_BOOLEAN_YES || (do_ip6_icmp_router == CONFIG_BOOLEAN_AUTO && - (Icmp6InRouterSolicits || - Icmp6OutRouterSolicits || - Icmp6InRouterAdvertisements || - Icmp6OutRouterAdvertisements || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_router = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InSolicits = NULL, - *rd_OutSolicits = NULL, - *rd_InAdvertisements = NULL, - *rd_OutAdvertisements = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmprouter" - , NULL - , "icmp6" - , NULL - , "IPv6 Router Messages" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_ROUTER - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InSolicits = rrddim_add(st, "InSolicits", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutSolicits = rrddim_add(st, "OutSolicits", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InAdvertisements = rrddim_add(st, "InAdvertisements", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutAdvertisements = rrddim_add(st, "OutAdvertisements", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InSolicits, Icmp6InRouterSolicits); - rrddim_set_by_pointer(st, rd_OutSolicits, Icmp6OutRouterSolicits); - rrddim_set_by_pointer(st, rd_InAdvertisements, Icmp6InRouterAdvertisements); - rrddim_set_by_pointer(st, rd_OutAdvertisements, Icmp6OutRouterAdvertisements); - rrdset_done(st); - } - - if(do_ip6_icmp_neighbor == CONFIG_BOOLEAN_YES || (do_ip6_icmp_neighbor == CONFIG_BOOLEAN_AUTO && - (Icmp6InNeighborSolicits || - Icmp6OutNeighborSolicits || - Icmp6InNeighborAdvertisements || - Icmp6OutNeighborAdvertisements || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_neighbor = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InSolicits = NULL, - *rd_OutSolicits = NULL, - *rd_InAdvertisements = NULL, - *rd_OutAdvertisements = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmpneighbor" - , NULL - , "icmp6" - , NULL - , "IPv6 Neighbor Messages" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_NEIGHBOR - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InSolicits = rrddim_add(st, "InSolicits", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutSolicits = rrddim_add(st, "OutSolicits", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InAdvertisements = rrddim_add(st, "InAdvertisements", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutAdvertisements = rrddim_add(st, "OutAdvertisements", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InSolicits, Icmp6InNeighborSolicits); - rrddim_set_by_pointer(st, rd_OutSolicits, Icmp6OutNeighborSolicits); - rrddim_set_by_pointer(st, rd_InAdvertisements, Icmp6InNeighborAdvertisements); - rrddim_set_by_pointer(st, rd_OutAdvertisements, Icmp6OutNeighborAdvertisements); - rrdset_done(st); - } - - if(do_ip6_icmp_mldv2 == CONFIG_BOOLEAN_YES || (do_ip6_icmp_mldv2 == CONFIG_BOOLEAN_AUTO && - (Icmp6InMLDv2Reports || - Icmp6OutMLDv2Reports || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_mldv2 = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InMLDv2Reports = NULL, - *rd_OutMLDv2Reports = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmpmldv2" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP MLDv2 Reports" - , "reports/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_LDV2 - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InMLDv2Reports = rrddim_add(st, "InMLDv2Reports", "received", 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutMLDv2Reports = rrddim_add(st, "OutMLDv2Reports", "sent", -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InMLDv2Reports, Icmp6InMLDv2Reports); - rrddim_set_by_pointer(st, rd_OutMLDv2Reports, Icmp6OutMLDv2Reports); - rrdset_done(st); - } - - if(do_ip6_icmp_types == CONFIG_BOOLEAN_YES || (do_ip6_icmp_types == CONFIG_BOOLEAN_AUTO && - (Icmp6InType1 || - Icmp6InType128 || - Icmp6InType129 || - Icmp6InType136 || - Icmp6OutType1 || - Icmp6OutType128 || - Icmp6OutType129 || - Icmp6OutType133 || - Icmp6OutType135 || - Icmp6OutType143 || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_icmp_types = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InType1 = NULL, - *rd_InType128 = NULL, - *rd_InType129 = NULL, - *rd_InType136 = NULL, - *rd_OutType1 = NULL, - *rd_OutType128 = NULL, - *rd_OutType129 = NULL, - *rd_OutType133 = NULL, - *rd_OutType135 = NULL, - *rd_OutType143 = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "icmptypes" - , NULL - , "icmp6" - , NULL - , "IPv6 ICMP Types" - , "messages/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ICMP_TYPES - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InType1 = rrddim_add(st, "InType1", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InType128 = rrddim_add(st, "InType128", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InType129 = rrddim_add(st, "InType129", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InType136 = rrddim_add(st, "InType136", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType1 = rrddim_add(st, "OutType1", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType128 = rrddim_add(st, "OutType128", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType129 = rrddim_add(st, "OutType129", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType133 = rrddim_add(st, "OutType133", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType135 = rrddim_add(st, "OutType135", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_OutType143 = rrddim_add(st, "OutType143", NULL, -1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InType1, Icmp6InType1); - rrddim_set_by_pointer(st, rd_InType128, Icmp6InType128); - rrddim_set_by_pointer(st, rd_InType129, Icmp6InType129); - rrddim_set_by_pointer(st, rd_InType136, Icmp6InType136); - rrddim_set_by_pointer(st, rd_OutType1, Icmp6OutType1); - rrddim_set_by_pointer(st, rd_OutType128, Icmp6OutType128); - rrddim_set_by_pointer(st, rd_OutType129, Icmp6OutType129); - rrddim_set_by_pointer(st, rd_OutType133, Icmp6OutType133); - rrddim_set_by_pointer(st, rd_OutType135, Icmp6OutType135); - rrddim_set_by_pointer(st, rd_OutType143, Icmp6OutType143); - rrdset_done(st); - } - - if(do_ip6_ect == CONFIG_BOOLEAN_YES || (do_ip6_ect == CONFIG_BOOLEAN_AUTO && - (Ip6InNoECTPkts || - Ip6InECT1Pkts || - Ip6InECT0Pkts || - Ip6InCEPkts || - netdata_zero_metrics_enabled == CONFIG_BOOLEAN_YES))) { - do_ip6_ect = CONFIG_BOOLEAN_YES; - static RRDSET *st = NULL; - static RRDDIM *rd_InNoECTPkts = NULL, - *rd_InECT1Pkts = NULL, - *rd_InECT0Pkts = NULL, - *rd_InCEPkts = NULL; - - if(unlikely(!st)) { - st = rrdset_create_localhost( - RRD_TYPE_NET_SNMP6 - , "ect" - , NULL - , "packets" - , NULL - , "IPv6 ECT Packets" - , "packets/s" - , PLUGIN_PROC_NAME - , PLUGIN_PROC_MODULE_NETSTAT_NAME - , NETDATA_CHART_PRIO_IPV6_ECT - , update_every - , RRDSET_TYPE_LINE - ); - - rd_InNoECTPkts = rrddim_add(st, "InNoECTPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InECT1Pkts = rrddim_add(st, "InECT1Pkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InECT0Pkts = rrddim_add(st, "InECT0Pkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - rd_InCEPkts = rrddim_add(st, "InCEPkts", NULL, 1, 1, RRD_ALGORITHM_INCREMENTAL); - } - - rrddim_set_by_pointer(st, rd_InNoECTPkts, Ip6InNoECTPkts); - rrddim_set_by_pointer(st, rd_InECT1Pkts, Ip6InECT1Pkts); - rrddim_set_by_pointer(st, rd_InECT0Pkts, Ip6InECT0Pkts); - rrddim_set_by_pointer(st, rd_InCEPkts, Ip6InCEPkts); - rrdset_done(st); - } - + do_proc_net_snmp6(update_every); + return 0; } diff --git a/collectors/proc.plugin/proc_net_rpc_nfs.c b/collectors/proc.plugin/proc_net_rpc_nfs.c index b1ff4e05a..0ab9d28b5 100644 --- a/collectors/proc.plugin/proc_net_rpc_nfs.c +++ b/collectors/proc.plugin/proc_net_rpc_nfs.c @@ -183,7 +183,7 @@ int do_proc_net_rpc_nfs(int update_every, usec_t dt) { if(do_net == 1 && strcmp(type, "net") == 0) { if(words < 5) { - error("%s line of /proc/net/rpc/nfs has %zu words, expected %d", type, words, 5); + collector_error("%s line of /proc/net/rpc/nfs has %zu words, expected %d", type, words, 5); continue; } @@ -198,7 +198,7 @@ int do_proc_net_rpc_nfs(int update_every, usec_t dt) { } else if(do_rpc == 1 && strcmp(type, "rpc") == 0) { if(words < 4) { - error("%s line of /proc/net/rpc/nfs has %zu words, expected %d", type, words, 6); + collector_error("%s line of /proc/net/rpc/nfs has %zu words, expected %d", type, words, 6); continue; } @@ -224,7 +224,7 @@ int do_proc_net_rpc_nfs(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc2_warning) { - error("Disabling /proc/net/rpc/nfs v2 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_error("Disabling /proc/net/rpc/nfs v2 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc2_warning = 1; } do_proc2 = 0; @@ -245,7 +245,7 @@ int do_proc_net_rpc_nfs(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc3_warning) { - info("Disabling /proc/net/rpc/nfs v3 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_info("Disabling /proc/net/rpc/nfs v3 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc3_warning = 1; } do_proc3 = 0; @@ -266,7 +266,7 @@ int do_proc_net_rpc_nfs(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc4_warning) { - info("Disabling /proc/net/rpc/nfs v4 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_info("Disabling /proc/net/rpc/nfs v4 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc4_warning = 1; } do_proc4 = 0; diff --git a/collectors/proc.plugin/proc_net_rpc_nfsd.c b/collectors/proc.plugin/proc_net_rpc_nfsd.c index bd1da8889..faa6b5c46 100644 --- a/collectors/proc.plugin/proc_net_rpc_nfsd.c +++ b/collectors/proc.plugin/proc_net_rpc_nfsd.c @@ -282,7 +282,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { if(do_rc == 1 && strcmp(type, "rc") == 0) { if(unlikely(words < 4)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 4); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 4); continue; } @@ -296,7 +296,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { } else if(do_fh == 1 && strcmp(type, "fh") == 0) { if(unlikely(words < 6)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 6); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 6); continue; } @@ -309,7 +309,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { } else if(do_io == 1 && strcmp(type, "io") == 0) { if(unlikely(words < 3)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 3); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 3); continue; } @@ -322,7 +322,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { } else if(do_th == 1 && strcmp(type, "th") == 0) { if(unlikely(words < 13)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 13); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 13); continue; } @@ -335,7 +335,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { } else if(do_net == 1 && strcmp(type, "net") == 0) { if(unlikely(words < 5)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 5); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 5); continue; } @@ -350,7 +350,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { } else if(do_rpc == 1 && strcmp(type, "rpc") == 0) { if(unlikely(words < 6)) { - error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 6); + collector_error("%s line of /proc/net/rpc/nfsd has %zu words, expected %d", type, words, 6); continue; } @@ -377,7 +377,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc2_warning) { - error("Disabling /proc/net/rpc/nfsd v2 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_error("Disabling /proc/net/rpc/nfsd v2 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc2_warning = 1; } do_proc2 = 0; @@ -398,7 +398,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc3_warning) { - info("Disabling /proc/net/rpc/nfsd v3 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_info("Disabling /proc/net/rpc/nfsd v3 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc3_warning = 1; } do_proc3 = 0; @@ -419,7 +419,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc4_warning) { - info("Disabling /proc/net/rpc/nfsd v4 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_info("Disabling /proc/net/rpc/nfsd v4 procedure calls chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc4_warning = 1; } do_proc4 = 0; @@ -440,7 +440,7 @@ int do_proc_net_rpc_nfsd(int update_every, usec_t dt) { if(sum == 0ULL) { if(!proc4ops_warning) { - info("Disabling /proc/net/rpc/nfsd v4 operations chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); + collector_info("Disabling /proc/net/rpc/nfsd v4 operations chart. It seems unused on this machine. It will be enabled automatically when found with data in it."); proc4ops_warning = 1; } do_proc4ops = 0; diff --git a/collectors/proc.plugin/proc_net_sctp_snmp.c b/collectors/proc.plugin/proc_net_sctp_snmp.c index 292449a73..e67143e69 100644 --- a/collectors/proc.plugin/proc_net_sctp_snmp.c +++ b/collectors/proc.plugin/proc_net_sctp_snmp.c @@ -113,7 +113,7 @@ int do_proc_net_sctp_snmp(int update_every, usec_t dt) { for(l = 0; l < lines ;l++) { size_t words = procfile_linewords(ff, l); if(unlikely(words < 2)) { - if(unlikely(words)) error("Cannot read /proc/net/sctp/snmp line %zu. Expected 2 params, read %zu.", l, words); + if(unlikely(words)) collector_error("Cannot read /proc/net/sctp/snmp line %zu. Expected 2 params, read %zu.", l, words); continue; } diff --git a/collectors/proc.plugin/proc_net_softnet_stat.c b/collectors/proc.plugin/proc_net_softnet_stat.c index 65239246a..dfd372b2a 100644 --- a/collectors/proc.plugin/proc_net_softnet_stat.c +++ b/collectors/proc.plugin/proc_net_softnet_stat.c @@ -40,7 +40,7 @@ int do_proc_net_softnet_stat(int update_every, usec_t dt) { size_t words = procfile_linewords(ff, 0), w; if(unlikely(!lines || !words)) { - error("Cannot read /proc/net/softnet_stat, %zu lines and %zu columns reported.", lines, words); + collector_error("Cannot read /proc/net/softnet_stat, %zu lines and %zu columns reported.", lines, words); return 1; } diff --git a/collectors/proc.plugin/proc_net_stat_conntrack.c b/collectors/proc.plugin/proc_net_stat_conntrack.c index f9dbdf47c..e8fbdbb66 100644 --- a/collectors/proc.plugin/proc_net_stat_conntrack.c +++ b/collectors/proc.plugin/proc_net_stat_conntrack.c @@ -69,7 +69,7 @@ int do_proc_net_stat_conntrack(int update_every, usec_t dt) { for(l = 1; l < lines ;l++) { size_t words = procfile_linewords(ff, l); if(unlikely(words < 17)) { - if(unlikely(words)) error("Cannot read /proc/net/stat/nf_conntrack line. Expected 17 params, read %zu.", words); + if(unlikely(words)) collector_error("Cannot read /proc/net/stat/nf_conntrack line. Expected 17 params, read %zu.", words); continue; } diff --git a/collectors/proc.plugin/proc_net_stat_synproxy.c b/collectors/proc.plugin/proc_net_stat_synproxy.c index 0a74b3575..e23a0ab7b 100644 --- a/collectors/proc.plugin/proc_net_stat_synproxy.c +++ b/collectors/proc.plugin/proc_net_stat_synproxy.c @@ -34,7 +34,7 @@ int do_proc_net_stat_synproxy(int update_every, usec_t dt) { // make sure we have 3 lines size_t lines = procfile_lines(ff), l; if(unlikely(lines < 2)) { - error("/proc/net/stat/synproxy has %zu lines, expected no less than 2. Disabling it.", lines); + collector_error("/proc/net/stat/synproxy has %zu lines, expected no less than 2. Disabling it.", lines); return 1; } diff --git a/collectors/proc.plugin/proc_pagetypeinfo.c b/collectors/proc.plugin/proc_pagetypeinfo.c index dc006aa59..e12c5bff8 100644 --- a/collectors/proc.plugin/proc_pagetypeinfo.c +++ b/collectors/proc.plugin/proc_pagetypeinfo.c @@ -112,7 +112,7 @@ int do_proc_pagetypeinfo(int update_every, usec_t dt) { ff_lines = procfile_lines(ff); if(unlikely(!ff_lines)) { - error("PLUGIN: PROC_PAGETYPEINFO: Cannot read %s, zero lines reported.", ff_path); + collector_error("PLUGIN: PROC_PAGETYPEINFO: Cannot read %s, zero lines reported.", ff_path); return 1; } @@ -135,21 +135,21 @@ int do_proc_pagetypeinfo(int update_every, usec_t dt) { pagelines_cnt++; } if (pagelines_cnt == 0) { - error("PLUGIN: PROC_PAGETYPEINFO: Unable to parse any valid line in %s", ff_path); + collector_error("PLUGIN: PROC_PAGETYPEINFO: Unable to parse any valid line in %s", ff_path); return 1; } // 4th line is the "Free pages count per migrate type at order". Just subtract these 8 words. pageorders_cnt = procfile_linewords(ff, 3); if (pageorders_cnt < 9) { - error("PLUGIN: PROC_PAGETYPEINFO: Unable to parse Line 4 of %s", ff_path); + collector_error("PLUGIN: PROC_PAGETYPEINFO: Unable to parse Line 4 of %s", ff_path); return 1; } pageorders_cnt -= 9; if (pageorders_cnt > MAX_PAGETYPE_ORDER) { - error("PLUGIN: PROC_PAGETYPEINFO: pageorder found (%lu) is higher than max %d", + collector_error("PLUGIN: PROC_PAGETYPEINFO: pageorder found (%lu) is higher than max %d", (long unsigned int) pageorders_cnt, MAX_PAGETYPE_ORDER); return 1; } @@ -158,7 +158,7 @@ int do_proc_pagetypeinfo(int update_every, usec_t dt) { if (!pagelines) { pagelines = callocz(pagelines_cnt, sizeof(struct pageline)); if (!pagelines) { - error("PLUGIN: PROC_PAGETYPEINFO: Cannot allocate %lu pagelines of %lu B", + collector_error("PLUGIN: PROC_PAGETYPEINFO: Cannot allocate %lu pagelines of %lu B", (long unsigned int) pagelines_cnt, (long unsigned int) sizeof(struct pageline)); return 1; } @@ -291,8 +291,8 @@ int do_proc_pagetypeinfo(int update_every, usec_t dt) { size_t words = procfile_linewords(ff, l); if (words != 7+pageorders_cnt) { - error("PLUGIN: PROC_PAGETYPEINFO: Unable to read line %lu, %lu words found instead of %lu", - l+1, (long unsigned int) words, (long unsigned int) 7+pageorders_cnt); + collector_error("PLUGIN: PROC_PAGETYPEINFO: Unable to read line %lu, %lu words found instead of %lu", + l+1, (long unsigned int) words, (long unsigned int) 7+pageorders_cnt); break; } diff --git a/collectors/proc.plugin/proc_pressure.c b/collectors/proc.plugin/proc_pressure.c index 6649aa630..80b08d9ad 100644 --- a/collectors/proc.plugin/proc_pressure.c +++ b/collectors/proc.plugin/proc_pressure.c @@ -171,7 +171,7 @@ int do_proc_pressure(int update_every, usec_t dt) { ff = procfile_open(filename, " =", PROCFILE_FLAG_DEFAULT); if (unlikely(!ff)) { - error("Cannot read pressure information from %s.", filename); + collector_error("Cannot read pressure information from %s.", filename); fail_count++; continue; } @@ -186,7 +186,7 @@ int do_proc_pressure(int update_every, usec_t dt) { size_t lines = procfile_lines(ff); if (unlikely(lines < 1)) { - error("%s has no lines.", procfile_filename(ff)); + collector_error("%s has no lines.", procfile_filename(ff)); fail_count++; continue; } diff --git a/collectors/proc.plugin/proc_self_mountinfo.c b/collectors/proc.plugin/proc_self_mountinfo.c index 9310f2ffc..0483749c3 100644 --- a/collectors/proc.plugin/proc_self_mountinfo.c +++ b/collectors/proc.plugin/proc_self_mountinfo.c @@ -227,8 +227,9 @@ struct mountinfo *mountinfo_read(int do_statvfs) { struct mountinfo *root = NULL, *last = NULL, *mi = NULL; // create a dictionary to track uniqueness - DICTIONARY *dict = dictionary_create( - DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_NAME_LINK_DONT_CLONE); + DICTIONARY *dict = dictionary_create_advanced( + DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_NAME_LINK_DONT_CLONE, + &dictionary_stats_category_collectors, 0); unsigned long l, lines = procfile_lines(ff); for(l = 0; l < lines ;l++) { @@ -252,7 +253,7 @@ struct mountinfo *mountinfo_read(int do_statvfs) { for(minor = major; *minor && *minor != ':' ;minor++) ; if(unlikely(!*minor)) { - error("Cannot parse major:minor on '%s' at line %lu of '%s'", major, l + 1, filename); + collector_error("Cannot parse major:minor on '%s' at line %lu of '%s'", major, l + 1, filename); freez(mi); continue; } @@ -442,7 +443,7 @@ struct mountinfo *mountinfo_read(int do_statvfs) { #ifdef NETDATA_INTERNAL_CHECKS if(unlikely(!mi)) { - error("Mount point '%s' not found in /proc/self/mountinfo", mnt->mnt_dir); + collector_error("Mount point '%s' not found in /proc/self/mountinfo", mnt->mnt_dir); } #endif } diff --git a/collectors/proc.plugin/proc_softirqs.c b/collectors/proc.plugin/proc_softirqs.c index 4c4df7668..0d5d8ef9c 100644 --- a/collectors/proc.plugin/proc_softirqs.c +++ b/collectors/proc.plugin/proc_softirqs.c @@ -75,7 +75,7 @@ int do_proc_softirqs(int update_every, usec_t dt) { size_t words = procfile_linewords(ff, 0); if(unlikely(!lines)) { - error("Cannot read /proc/softirqs, zero lines reported."); + collector_error("Cannot read /proc/softirqs, zero lines reported."); return 1; } @@ -90,7 +90,7 @@ int do_proc_softirqs(int update_every, usec_t dt) { } if(unlikely(!cpus)) { - error("PLUGIN: PROC_SOFTIRQS: Cannot find the number of CPUs in /proc/softirqs"); + collector_error("PLUGIN: PROC_SOFTIRQS: Cannot find the number of CPUs in /proc/softirqs"); return 1; } diff --git a/collectors/proc.plugin/proc_spl_kstat_zfs.c b/collectors/proc.plugin/proc_spl_kstat_zfs.c index 8938d6431..0db9970c3 100644 --- a/collectors/proc.plugin/proc_spl_kstat_zfs.c +++ b/collectors/proc.plugin/proc_spl_kstat_zfs.c @@ -140,7 +140,7 @@ int do_proc_spl_kstat_zfs_arcstats(int update_every, usec_t dt) { if(likely(!do_zfs_stats)) { DIR *dir = opendir(dirname); if(unlikely(!dir)) { - error("Cannot read directory '%s'", dirname); + collector_error("Cannot read directory '%s'", dirname); return 1; } @@ -177,7 +177,7 @@ int do_proc_spl_kstat_zfs_arcstats(int update_every, usec_t dt) { for(l = 0; l < lines ;l++) { size_t words = procfile_linewords(ff, l); if(unlikely(words < 3)) { - if(unlikely(words)) error("Cannot read " ZFS_PROC_ARCSTATS " line %zu. Expected 3 params, read %zu.", l, words); + if(unlikely(words)) collector_error("Cannot read " ZFS_PROC_ARCSTATS " line %zu. Expected 3 params, read %zu.", l, words); continue; } @@ -285,6 +285,8 @@ int update_zfs_pool_state_chart(const DICTIONARY_ITEM *item, void *pool_p, void pool->rd_offline = rrddim_add(pool->st, "offline", NULL, 1, 1, RRD_ALGORITHM_ABSOLUTE); pool->rd_removed = rrddim_add(pool->st, "removed", NULL, 1, 1, RRD_ALGORITHM_ABSOLUTE); pool->rd_unavail = rrddim_add(pool->st, "unavail", NULL, 1, 1, RRD_ALGORITHM_ABSOLUTE); + + rrdlabels_add(pool->st->rrdlabels, "pool", name, RRDLABEL_SRC_AUTO); } rrddim_set_by_pointer(pool->st, pool->rd_online, pool->online); @@ -320,7 +322,7 @@ int do_proc_spl_kstat_zfs_pool_state(int update_every, usec_t dt) snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/proc/spl/kstat/zfs"); dirname = config_get("plugin:proc:" ZFS_PROC_POOLS, "directory to monitor", filename); - zfs_pools = dictionary_create(DICT_OPTION_SINGLE_THREADED); + zfs_pools = dictionary_create_advanced(DICT_OPTION_SINGLE_THREADED, &dictionary_stats_category_collectors, 0); do_zfs_pool_state = 1; } @@ -328,7 +330,7 @@ int do_proc_spl_kstat_zfs_pool_state(int update_every, usec_t dt) if (likely(do_zfs_pool_state)) { DIR *dir = opendir(dirname); if (unlikely(!dir)) { - error("Cannot read directory '%s'", dirname); + collector_error("Cannot read directory '%s'", dirname); return 1; } @@ -392,7 +394,7 @@ int do_proc_spl_kstat_zfs_pool_state(int update_every, usec_t dt) char *c = strchr(state, '\n'); if (c) *c = '\0'; - error("ZFS POOLS: Undefined state %s for zpool %s, disabling the chart", state, de->d_name); + collector_error("ZFS POOLS: Undefined state %s for zpool %s, disabling the chart", state, de->d_name); } } } @@ -402,7 +404,7 @@ int do_proc_spl_kstat_zfs_pool_state(int update_every, usec_t dt) } if (do_zfs_pool_state && pool_found && !state_file_found) { - info("ZFS POOLS: State files not found. Disabling the module."); + collector_info("ZFS POOLS: State files not found. Disabling the module."); do_zfs_pool_state = 0; } diff --git a/collectors/proc.plugin/proc_stat.c b/collectors/proc.plugin/proc_stat.c index 33fe93234..2ca7c42e1 100644 --- a/collectors/proc.plugin/proc_stat.c +++ b/collectors/proc.plugin/proc_stat.c @@ -69,7 +69,7 @@ static int read_per_core_files(struct cpu_chart *all_cpu_charts, size_t len, siz if(unlikely(f->fd == -1)) { f->fd = open(f->filename, O_RDONLY); if (unlikely(f->fd == -1)) { - error("Cannot open file '%s'", f->filename); + collector_error("Cannot open file '%s'", f->filename); continue; } } @@ -78,7 +78,7 @@ static int read_per_core_files(struct cpu_chart *all_cpu_charts, size_t len, siz if(unlikely(ret < 0)) { // cannot read that file - error("Cannot read file '%s'", f->filename); + collector_error("Cannot read file '%s'", f->filename); close(f->fd); f->fd = -1; continue; @@ -94,7 +94,7 @@ static int read_per_core_files(struct cpu_chart *all_cpu_charts, size_t len, siz f->fd = -1; } else if(lseek(f->fd, 0, SEEK_SET) == -1) { - error("Cannot seek in file '%s'", f->filename); + collector_error("Cannot seek in file '%s'", f->filename); close(f->fd); f->fd = -1; } @@ -133,14 +133,14 @@ static int read_per_core_time_in_state_files(struct cpu_chart *all_cpu_charts, s tsf->ff = procfile_open(tsf->filename, " \t:", PROCFILE_FLAG_DEFAULT); if(unlikely(!tsf->ff)) { - error("Cannot open file '%s'", tsf->filename); + collector_error("Cannot open file '%s'", tsf->filename); continue; } } tsf->ff = procfile_readall(tsf->ff); if(unlikely(!tsf->ff)) { - error("Cannot read file '%s'", tsf->filename); + collector_error("Cannot read file '%s'", tsf->filename); procfile_close(tsf->ff); tsf->ff = NULL; continue; @@ -179,7 +179,7 @@ static int read_per_core_time_in_state_files(struct cpu_chart *all_cpu_charts, s words = procfile_linewords(tsf->ff, l); if(unlikely(words < 2)) { - error("Cannot read time_in_state line. Expected 2 params, read %zu.", words); + collector_error("Cannot read time_in_state line. Expected 2 params, read %zu.", words); continue; } frequency = str2ull(procfile_lineword(tsf->ff, l, 0)); @@ -273,11 +273,11 @@ static void* wake_cpu_thread(void* core) { thread = pthread_self(); if(unlikely(pthread_setaffinity_np(thread, sizeof(cpu_set_t), &cpu_set))) { if(unlikely(errors < 8)) { - error("Cannot set CPU affinity for core %d", *(int*)core); + collector_error("Cannot set CPU affinity for core %d", *(int*)core); errors++; } else if(unlikely(errors < 9)) { - error("CPU affinity errors are disabled"); + collector_error("CPU affinity errors are disabled"); errors++; } } @@ -312,14 +312,14 @@ static int read_schedstat(char *schedstat_filename, struct per_core_cpuidle_char if(likely(row_key[0] == 'c' && row_key[1] == 'p' && row_key[2] == 'u')) { words = procfile_linewords(ff, l); if(unlikely(words < 10)) { - error("Cannot read /proc/schedstat cpu line. Expected 9 params, read %zu.", words); + collector_error("Cannot read /proc/schedstat cpu line. Expected 9 params, read %zu.", words); return 1; } cores_found++; size_t core = str2ul(&row_key[3]); if(unlikely(core >= cores_found)) { - error("Core %zu found but no more than %zu cores were expected.", core, cores_found); + collector_error("Core %zu found but no more than %zu cores were expected.", core, cores_found); return 1; } @@ -343,7 +343,7 @@ static int read_one_state(char *buf, const char *filename, int *fd) { if(unlikely(ret <= 0)) { // cannot read that file - error("Cannot read file '%s'", filename); + collector_error("Cannot read file '%s'", filename); close(*fd); *fd = -1; return 0; @@ -359,7 +359,7 @@ static int read_one_state(char *buf, const char *filename, int *fd) { *fd = -1; } else if(lseek(*fd, 0, SEEK_SET) == -1) { - error("Cannot seek in file '%s'", filename); + collector_error("Cannot seek in file '%s'", filename); close(*fd); *fd = -1; } @@ -413,14 +413,14 @@ static int read_cpuidle_states(char *cpuidle_name_filename , char *cpuidle_time_ int fd = open(filename, O_RDONLY, 0666); if(unlikely(fd == -1)) { - error("Cannot open file '%s'", filename); + collector_error("Cannot open file '%s'", filename); cc->rescan_cpu_states = 1; return 1; } ssize_t r = read(fd, name_buf, 50); if(unlikely(r < 1)) { - error("Cannot read file '%s'", filename); + collector_error("Cannot read file '%s'", filename); close(fd); cc->rescan_cpu_states = 1; return 1; @@ -445,7 +445,7 @@ static int read_cpuidle_states(char *cpuidle_name_filename , char *cpuidle_time_ if(unlikely(cs->time_fd == -1)) { cs->time_fd = open(cs->time_filename, O_RDONLY); if (unlikely(cs->time_fd == -1)) { - error("Cannot open file '%s'", cs->time_filename); + collector_error("Cannot open file '%s'", cs->time_filename); cc->rescan_cpu_states = 1; return 1; } @@ -483,7 +483,7 @@ int do_proc_stat(int update_every, usec_t dt) { *time_in_state_filename = NULL, *schedstat_filename = NULL, *cpuidle_name_filename = NULL, *cpuidle_time_filename = NULL; static const RRDVAR_ACQUIRED *cpus_var = NULL; static int accurate_freq_avail = 0, accurate_freq_is_used = 0; - size_t cores_found = (size_t)processors; + size_t cores_found = (size_t)get_system_cpus(); if(unlikely(do_cpu == -1)) { do_cpu = config_get_boolean("plugin:proc:/proc/stat", "cpu utilization", CONFIG_BOOLEAN_YES); @@ -494,7 +494,7 @@ int do_proc_stat(int update_every, usec_t dt) { do_processes = config_get_boolean("plugin:proc:/proc/stat", "processes running", CONFIG_BOOLEAN_YES); // give sane defaults based on the number of processors - if(unlikely(processors > 50)) { + if(unlikely(get_system_cpus() > 50)) { // the system has too many processors keep_per_core_fds_open = CONFIG_BOOLEAN_NO; do_core_throttle_count = CONFIG_BOOLEAN_NO; @@ -510,7 +510,7 @@ int do_proc_stat(int update_every, usec_t dt) { do_cpu_freq = CONFIG_BOOLEAN_YES; do_cpuidle = CONFIG_BOOLEAN_YES; } - if(unlikely(processors > 24)) { + if(unlikely(get_system_cpus() > 24)) { // the system has too many processors keep_cpuidle_fds_open = CONFIG_BOOLEAN_NO; } @@ -585,7 +585,7 @@ int do_proc_stat(int update_every, usec_t dt) { if(likely(row_key[0] == 'c' && row_key[1] == 'p' && row_key[2] == 'u')) { words = procfile_linewords(ff, l); if(unlikely(words < 9)) { - error("Cannot read /proc/stat cpu line. Expected 9 params, read %zu.", words); + collector_error("Cannot read /proc/stat cpu line. Expected 9 params, read %zu.", words); continue; } @@ -712,6 +712,12 @@ int do_proc_stat(int update_every, usec_t dt) { cpu_chart->rd_idle = rrddim_add(cpu_chart->st, "idle", NULL, multiplier, divisor, RRD_ALGORITHM_PCENT_OVER_DIFF_TOTAL); rrddim_hide(cpu_chart->st, "idle"); + if (core > 0) { + char cpu_core[50 + 1]; + snprintfz(cpu_core, 50, "cpu%lu", core - 1); + rrdlabels_add(cpu_chart->st->rrdlabels, "cpu", cpu_core, RRDLABEL_SRC_AUTO); + } + if(unlikely(core == 0 && cpus_var == NULL)) cpus_var = rrdvar_custom_host_variable_add_and_acquire(localhost, "active_processors"); } @@ -930,7 +936,7 @@ int do_proc_stat(int update_every, usec_t dt) { if(r > 0 && !accurate_freq_is_used) { accurate_freq_is_used = 1; snprintfz(filename, FILENAME_MAX, time_in_state_filename, "cpu*"); - info("cpufreq is using %s", filename); + collector_info("cpufreq is using %s", filename); } } if (r < 1) { @@ -938,7 +944,7 @@ int do_proc_stat(int update_every, usec_t dt) { if(accurate_freq_is_used) { accurate_freq_is_used = 0; snprintfz(filename, FILENAME_MAX, scaling_cur_freq_filename, "cpu*"); - info("cpufreq fell back to %s", filename); + collector_info("cpufreq fell back to %s", filename); } } @@ -993,13 +999,13 @@ int do_proc_stat(int update_every, usec_t dt) { } } else - error("Cannot read current process affinity"); + collector_error("Cannot read current process affinity"); // These threads are very ephemeral and don't need to have a specific name if(unlikely(pthread_create(&thread, NULL, wake_cpu_thread, (void *)&core))) - error("Cannot create wake_cpu_thread"); + collector_error("Cannot create wake_cpu_thread"); else if(unlikely(pthread_join(thread, NULL))) - error("Cannot join wake_cpu_thread"); + collector_error("Cannot join wake_cpu_thread"); cpu_states_updated = 1; } } diff --git a/collectors/proc.plugin/proc_vmstat.c b/collectors/proc.plugin/proc_vmstat.c index b8defc455..638d1690c 100644 --- a/collectors/proc.plugin/proc_vmstat.c +++ b/collectors/proc.plugin/proc_vmstat.c @@ -100,7 +100,7 @@ int do_proc_vmstat(int update_every, usec_t dt) { for(l = 0; l < lines ;l++) { size_t words = procfile_linewords(ff, l); if(unlikely(words < 2)) { - if(unlikely(words)) error("Cannot read /proc/vmstat line %zu. Expected 2 params, read %zu.", l, words); + if(unlikely(words)) collector_error("Cannot read /proc/vmstat line %zu. Expected 2 params, read %zu.", l, words); continue; } diff --git a/collectors/proc.plugin/sys_block_zram.c b/collectors/proc.plugin/sys_block_zram.c index 6bae54243..1be725b10 100644 --- a/collectors/proc.plugin/sys_block_zram.c +++ b/collectors/proc.plugin/sys_block_zram.c @@ -144,17 +144,17 @@ static int init_devices(DICTIONARY *devices, unsigned int zram_id, int update_ev snprintfz(filename, FILENAME_MAX, "/dev/%s", de->d_name); if (unlikely(stat(filename, &st) != 0)) { - error("ZRAM : Unable to stat %s: %s", filename, strerror(errno)); + collector_error("ZRAM : Unable to stat %s: %s", filename, strerror(errno)); continue; } if (major(st.st_rdev) == zram_id) { - info("ZRAM : Found device %s", filename); + collector_info("ZRAM : Found device %s", filename); snprintfz(filename, FILENAME_MAX, "/sys/block/%s/mm_stat", de->d_name); ff = procfile_open(filename, " \t:", PROCFILE_FLAG_DEFAULT); if (ff == NULL) { - error("ZRAM : Failed to open %s: %s", filename, strerror(errno)); + collector_error("ZRAM : Failed to open %s: %s", filename, strerror(errno)); continue; } device.file = ff; @@ -170,7 +170,7 @@ static int init_devices(DICTIONARY *devices, unsigned int zram_id, int update_ev static void free_device(DICTIONARY *dict, const char *name) { ZRAM_DEVICE *d = (ZRAM_DEVICE*)dictionary_get(dict, name); - info("ZRAM : Disabling monitoring of device %s", name); + collector_info("ZRAM : Disabling monitoring of device %s", name); rrdset_obsolete_and_pointer_null(d->st_usage); rrdset_obsolete_and_pointer_null(d->st_savings); rrdset_obsolete_and_pointer_null(d->st_alloc_efficiency); @@ -252,7 +252,7 @@ int do_sys_block_zram(int update_every, usec_t dt) { ff = procfile_open("/proc/devices", " \t:", PROCFILE_FLAG_DEFAULT); if (ff == NULL) { - error("Cannot read /proc/devices"); + collector_error("Cannot read /proc/devices"); return 1; } ff = procfile_readall(ff); @@ -267,7 +267,7 @@ int do_sys_block_zram(int update_every, usec_t dt) { } procfile_close(ff); - devices = dictionary_create(DICT_OPTION_SINGLE_THREADED); + devices = dictionary_create_advanced(DICT_OPTION_SINGLE_THREADED, &dictionary_stats_category_collectors, 0); device_count = init_devices(devices, (unsigned int)zram_id, update_every); } diff --git a/collectors/proc.plugin/sys_class_infiniband.c b/collectors/proc.plugin/sys_class_infiniband.c index fca0cb8a2..5f5e53239 100644 --- a/collectors/proc.plugin/sys_class_infiniband.c +++ b/collectors/proc.plugin/sys_class_infiniband.c @@ -200,7 +200,7 @@ static struct ibport { #define GEN_DO_HWCOUNTER_READ(NAME, GRP, DESC, DIR, PORT, HW, ...) \ if (HW->file_##NAME) { \ if (read_single_number_file(HW->file_##NAME, (unsigned long long *)&HW->NAME)) { \ - error("cannot read iface '%s' hwcounter '" #HW "'", PORT->name); \ + collector_error("cannot read iface '%s' hwcounter '" #HW "'", PORT->name); \ HW->file_##NAME = NULL; \ } \ } @@ -469,7 +469,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt) snprintfz(buffer, FILENAME_MAX, "%s/%s/%s", ports_dirname, port_dent->d_name, "rate"); char buffer_rate[65]; if (read_file(buffer, buffer_rate, 64)) { - error("Unable to read '%s'", buffer); + collector_error("Unable to read '%s'", buffer); p->width = 1; } else { char *buffer_width = strstr(buffer_rate, "("); @@ -480,12 +480,11 @@ int do_sys_class_infiniband(int update_every, usec_t dt) } if (!p->discovered) - info( - "Infiniband card %s port %s at speed %" PRIu64 " width %" PRIu64 "", - dev_dent->d_name, - port_dent->d_name, - p->speed, - p->width); + collector_info("Infiniband card %s port %s at speed %" PRIu64 " width %" PRIu64 "", + dev_dent->d_name, + port_dent->d_name, + p->speed, + p->width); p->discovered = 1; } @@ -511,7 +510,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt) #define GEN_DO_COUNTER_READ(NAME, GRP, DESC, DIR, PORT, ...) \ if (PORT->file_##NAME) { \ if (read_single_number_file(PORT->file_##NAME, (unsigned long long *)&PORT->NAME)) { \ - error("cannot read iface '%s' counter '" #NAME "'", PORT->name); \ + collector_error("cannot read iface '%s' counter '" #NAME "'", PORT->name); \ PORT->file_##NAME = NULL; \ } \ } @@ -650,7 +649,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt) // Unknown vendor, should not happen else { - error( + collector_error( "Unmanaged vendor for '%s', do_hwerrors should have been set to no. Please report this bug", port->name); port->do_hwerrors = CONFIG_BOOLEAN_NO; @@ -686,7 +685,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt) // Unknown vendor, should not happen else { - error( + collector_error( "Unmanaged vendor for '%s', do_hwpackets should have been set to no. Please report this bug", port->name); port->do_hwpackets = CONFIG_BOOLEAN_NO; diff --git a/collectors/proc.plugin/sys_class_power_supply.c b/collectors/proc.plugin/sys_class_power_supply.c index dde421503..ec36a295f 100644 --- a/collectors/proc.plugin/sys_class_power_supply.c +++ b/collectors/proc.plugin/sys_class_power_supply.c @@ -137,7 +137,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { DIR *dir = opendir(dirname); if(unlikely(!dir)) { - error("Cannot read directory '%s'", dirname); + collector_error("Cannot read directory '%s'", dirname); return 1; } @@ -247,7 +247,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { if(unlikely(ps->capacity->fd == -1)) { ps->capacity->fd = open(ps->capacity->filename, O_RDONLY, 0666); if(unlikely(ps->capacity->fd == -1)) { - error("Cannot open file '%s'", ps->capacity->filename); + collector_error("Cannot open file '%s'", ps->capacity->filename); power_supply_free(ps); ps = NULL; } @@ -257,7 +257,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { { ssize_t r = read(ps->capacity->fd, buffer, 30); if(unlikely(r < 1)) { - error("Cannot read file '%s'", ps->capacity->filename); + collector_error("Cannot read file '%s'", ps->capacity->filename); power_supply_free(ps); ps = NULL; } @@ -270,7 +270,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { ps->capacity->fd = -1; } else if(unlikely(lseek(ps->capacity->fd, 0, SEEK_SET) == -1)) { - error("Cannot seek in file '%s'", ps->capacity->filename); + collector_error("Cannot seek in file '%s'", ps->capacity->filename); close(ps->capacity->fd); ps->capacity->fd = -1; } @@ -292,7 +292,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { if(unlikely(pd->fd == -1)) { pd->fd = open(pd->filename, O_RDONLY, 0666); if(unlikely(pd->fd == -1)) { - error("Cannot open file '%s'", pd->filename); + collector_error("Cannot open file '%s'", pd->filename); read_error = 1; power_supply_free(ps); break; @@ -301,7 +301,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { ssize_t r = read(pd->fd, buffer, 30); if(unlikely(r < 1)) { - error("Cannot read file '%s'", pd->filename); + collector_error("Cannot read file '%s'", pd->filename); read_error = 1; power_supply_free(ps); break; @@ -314,7 +314,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) { pd->fd = -1; } else if(unlikely(lseek(pd->fd, 0, SEEK_SET) == -1)) { - error("Cannot seek in file '%s'", pd->filename); + collector_error("Cannot seek in file '%s'", pd->filename); close(pd->fd); pd->fd = -1; } diff --git a/collectors/proc.plugin/sys_devices_system_edac_mc.c b/collectors/proc.plugin/sys_devices_system_edac_mc.c index 13d209781..fe8250963 100644 --- a/collectors/proc.plugin/sys_devices_system_edac_mc.c +++ b/collectors/proc.plugin/sys_devices_system_edac_mc.c @@ -30,7 +30,7 @@ static void find_all_mc() { DIR *dir = opendir(dirname); if(unlikely(!dir)) { - error("Cannot read ECC memory errors directory '%s'", dirname); + collector_error("Cannot read ECC memory errors directory '%s'", dirname); return; } diff --git a/collectors/proc.plugin/sys_devices_system_node.c b/collectors/proc.plugin/sys_devices_system_node.c index 90aafd56a..068d739db 100644 --- a/collectors/proc.plugin/sys_devices_system_node.c +++ b/collectors/proc.plugin/sys_devices_system_node.c @@ -19,7 +19,7 @@ static int find_all_nodes() { DIR *dir = opendir(dirname); if(!dir) { - error("Cannot read NUMA node directory '%s'", dirname); + collector_error("Cannot read NUMA node directory '%s'", dirname); return 0; } @@ -134,7 +134,7 @@ int do_proc_sys_devices_system_node(int update_every, usec_t dt) { if(unlikely(words < 2)) { if(unlikely(words)) - error("Cannot read %s numastat line %zu. Expected 2 params, read %zu.", m->name, l, words); + collector_error("Cannot read %s numastat line %zu. Expected 2 params, read %zu.", m->name, l, words); continue; } diff --git a/collectors/proc.plugin/sys_fs_btrfs.c b/collectors/proc.plugin/sys_fs_btrfs.c index 3b9841fec..6abfd7852 100644 --- a/collectors/proc.plugin/sys_fs_btrfs.c +++ b/collectors/proc.plugin/sys_fs_btrfs.c @@ -90,7 +90,7 @@ static inline void btrfs_free_disk(BTRFS_DISK *d) { } static inline void btrfs_free_node(BTRFS_NODE *node) { - // info("BTRFS: destroying '%s'", node->id); + // collector_info("BTRFS: destroying '%s'", node->id); if(node->st_allocation_disks) rrdset_is_obsolete(node->st_allocation_disks); @@ -136,7 +136,7 @@ static inline int find_btrfs_disks(BTRFS_NODE *node, const char *path) { DIR *dir = opendir(path); if (!dir) { if(!node->logged_error) { - error("BTRFS: Cannot open directory '%s'.", path); + collector_error("BTRFS: Cannot open directory '%s'.", path); node->logged_error = 1; } return 1; @@ -149,7 +149,7 @@ static inline int find_btrfs_disks(BTRFS_NODE *node, const char *path) { || !strcmp(de->d_name, ".") || !strcmp(de->d_name, "..") ) { - // info("BTRFS: ignoring '%s'", de->d_name); + // collector_info("BTRFS: ignoring '%s'", de->d_name); continue; } @@ -200,13 +200,13 @@ static inline int find_btrfs_disks(BTRFS_NODE *node, const char *path) { // update the values if(read_single_number_file(d->size_filename, &d->size) != 0) { - error("BTRFS: failed to read '%s'", d->size_filename); + collector_error("BTRFS: failed to read '%s'", d->size_filename); d->exists = 0; continue; } if(read_single_number_file(d->hw_sector_size_filename, &d->hw_sector_size) != 0) { - error("BTRFS: failed to read '%s'", d->hw_sector_size_filename); + collector_error("BTRFS: failed to read '%s'", d->hw_sector_size_filename); d->exists = 0; continue; } @@ -257,7 +257,7 @@ static inline int find_all_btrfs_pools(const char *path) { DIR *dir = opendir(path); if (!dir) { if(!logged_error) { - error("BTRFS: Cannot open directory '%s'.", path); + collector_error("BTRFS: Cannot open directory '%s'.", path); logged_error = 1; } return 1; @@ -271,7 +271,7 @@ static inline int find_all_btrfs_pools(const char *path) { || !strcmp(de->d_name, "..") || !strcmp(de->d_name, "features") ) { - // info("BTRFS: ignoring '%s'", de->d_name); + // collector_info("BTRFS: ignoring '%s'", de->d_name); continue; } @@ -285,7 +285,7 @@ static inline int find_all_btrfs_pools(const char *path) { // did we find it? if(node) { - // info("BTRFS: already exists '%s'", de->d_name); + // collector_info("BTRFS: already exists '%s'", de->d_name); node->exists = 1; // update the disk sizes @@ -295,7 +295,7 @@ static inline int find_all_btrfs_pools(const char *path) { continue; } - // info("BTRFS: adding '%s'", de->d_name); + // collector_info("BTRFS: adding '%s'", de->d_name); // not found, create it node = callocz(sizeof(BTRFS_NODE), 1); @@ -309,7 +309,7 @@ static inline int find_all_btrfs_pools(const char *path) { snprintfz(filename, FILENAME_MAX, "%s/%s/label", path, de->d_name); if(read_file(filename, label, FILENAME_MAX) != 0) { - error("BTRFS: failed to read '%s'", filename); + collector_error("BTRFS: failed to read '%s'", filename); btrfs_free_node(node); continue; } @@ -326,21 +326,21 @@ static inline int find_all_btrfs_pools(const char *path) { //snprintfz(filename, FILENAME_MAX, "%s/%s/sectorsize", path, de->d_name); //if(read_single_number_file(filename, &node->sectorsize) != 0) { - // error("BTRFS: failed to read '%s'", filename); + // collector_error("BTRFS: failed to read '%s'", filename); // btrfs_free_node(node); // continue; //} //snprintfz(filename, FILENAME_MAX, "%s/%s/nodesize", path, de->d_name); //if(read_single_number_file(filename, &node->nodesize) != 0) { - // error("BTRFS: failed to read '%s'", filename); + // collector_error("BTRFS: failed to read '%s'", filename); // btrfs_free_node(node); // continue; //} //snprintfz(filename, FILENAME_MAX, "%s/%s/quota_override", path, de->d_name); //if(read_single_number_file(filename, &node->quota_override) != 0) { - // error("BTRFS: failed to read '%s'", filename); + // collector_error("BTRFS: failed to read '%s'", filename); // btrfs_free_node(node); // continue; //} @@ -351,7 +351,7 @@ static inline int find_all_btrfs_pools(const char *path) { #define init_btrfs_allocation_field(FIELD) {\ snprintfz(filename, FILENAME_MAX, "%s/%s/allocation/" #FIELD, path, de->d_name); \ if(read_single_number_file(filename, &node->allocation_ ## FIELD) != 0) {\ - error("BTRFS: failed to read '%s'", filename);\ + collector_error("BTRFS: failed to read '%s'", filename);\ btrfs_free_node(node);\ continue;\ }\ @@ -362,7 +362,7 @@ static inline int find_all_btrfs_pools(const char *path) { #define init_btrfs_allocation_section_field(SECTION, FIELD) {\ snprintfz(filename, FILENAME_MAX, "%s/%s/allocation/" #SECTION "/" #FIELD, path, de->d_name); \ if(read_single_number_file(filename, &node->allocation_ ## SECTION ## _ ## FIELD) != 0) {\ - error("BTRFS: failed to read '%s'", filename);\ + collector_error("BTRFS: failed to read '%s'", filename);\ btrfs_free_node(node);\ continue;\ }\ @@ -411,7 +411,7 @@ static inline int find_all_btrfs_pools(const char *path) { // -------------------------------------------------------------------- // link it - // info("BTRFS: linking '%s'", node->id); + // collector_info("BTRFS: linking '%s'", node->id); node->next = nodes; nodes = node; } @@ -505,7 +505,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) { || collect_btrfs_allocation_section_field(metadata, disk_used) != 0 || collect_btrfs_allocation_section_field(system, disk_total) != 0 || collect_btrfs_allocation_section_field(system, disk_used) != 0) { - error("BTRFS: failed to collect physical disks allocation for '%s'", node->id); + collector_error("BTRFS: failed to collect physical disks allocation for '%s'", node->id); // make it refresh btrfs at the next iteration refresh_delta = refresh_every; continue; @@ -515,7 +515,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) { if(do_allocation_data != CONFIG_BOOLEAN_NO) { if (collect_btrfs_allocation_section_field(data, total_bytes) != 0 || collect_btrfs_allocation_section_field(data, bytes_used) != 0) { - error("BTRFS: failed to collect allocation/data for '%s'", node->id); + collector_error("BTRFS: failed to collect allocation/data for '%s'", node->id); // make it refresh btrfs at the next iteration refresh_delta = refresh_every; continue; @@ -527,7 +527,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) { || collect_btrfs_allocation_section_field(metadata, bytes_used) != 0 || collect_btrfs_allocation_field(global_rsv_size) != 0 ) { - error("BTRFS: failed to collect allocation/metadata for '%s'", node->id); + collector_error("BTRFS: failed to collect allocation/metadata for '%s'", node->id); // make it refresh btrfs at the next iteration refresh_delta = refresh_every; continue; @@ -537,7 +537,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) { if(do_allocation_system != CONFIG_BOOLEAN_NO) { if (collect_btrfs_allocation_section_field(system, total_bytes) != 0 || collect_btrfs_allocation_section_field(system, bytes_used) != 0) { - error("BTRFS: failed to collect allocation/system for '%s'", node->id); + collector_error("BTRFS: failed to collect allocation/system for '%s'", node->id); // make it refresh btrfs at the next iteration refresh_delta = refresh_every; continue; diff --git a/collectors/python.d.plugin/Makefile.am b/collectors/python.d.plugin/Makefile.am index 1bbbf8ca0..6ea7b21b5 100644 --- a/collectors/python.d.plugin/Makefile.am +++ b/collectors/python.d.plugin/Makefile.am @@ -48,7 +48,6 @@ include bind_rndc/Makefile.inc include boinc/Makefile.inc include ceph/Makefile.inc include changefinder/Makefile.inc -include dockerd/Makefile.inc include dovecot/Makefile.inc include example/Makefile.inc include exim/Makefile.inc @@ -61,10 +60,8 @@ include hpssa/Makefile.inc include icecast/Makefile.inc include ipfs/Makefile.inc include litespeed/Makefile.inc -include logind/Makefile.inc include megacli/Makefile.inc include memcached/Makefile.inc -include mongodb/Makefile.inc include monit/Makefile.inc include nvidia_smi/Makefile.inc include nsd/Makefile.inc @@ -83,7 +80,6 @@ include samba/Makefile.inc include sensors/Makefile.inc include smartd_log/Makefile.inc include spigotmc/Makefile.inc -include springboot/Makefile.inc include squid/Makefile.inc include tomcat/Makefile.inc include tor/Makefile.inc diff --git a/collectors/python.d.plugin/README.md b/collectors/python.d.plugin/README.md index 2f5ebfcb1..b6d658fae 100644 --- a/collectors/python.d.plugin/README.md +++ b/collectors/python.d.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "python.d.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/README.md" +sidebar_label: "python.d.plugin" +learn_status: "Published" +learn_topic_type: "Tasks" +learn_rel_path: "Developers/Collectors" --> # python.d.plugin @@ -86,7 +90,7 @@ plugin](https://raw.githubusercontent.com/netdata/netdata/master/collectors/pyth Netdata (as opposed to having to install Netdata from source again with your new changes) to can copy over the relevant file to where Netdata expects it and then either `sudo systemctl restart netdata` to have it be picked up and used by Netdata or you can just run the updated collector in debug mode by following a process like below (this assumes you have -[installed Netdata from a GitHub fork](https://learn.netdata.cloud/docs/agent/packaging/installer/methods/manual) you +[installed Netdata from a GitHub fork](https://github.com/netdata/netdata/blob/master/packaging/installer/methods/manual.md) you have made to do your development on). ```bash @@ -125,7 +129,7 @@ CHART = { ]} ``` -All names are better explained in the [External Plugins](/collectors/plugins.d/README.md) section. +All names are better explained in the [External Plugins](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md) section. Parameters like `priority` and `update_every` are handled by `python.d.plugin`. ### `Service` class @@ -227,7 +231,7 @@ For additional security it uses python `subprocess.Popen` (without `shell=True` _Examples: `apache`, `nginx`, `tomcat`_ -_Multiple Endpoints (urls) Examples: [`rabbitmq`](/collectors/python.d.plugin/rabbitmq/README.md) (simpler). +_Multiple Endpoints (urls) Examples: [`rabbitmq`](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/rabbitmq/README.md) (simpler). _Variables from config file_: `url`, `user`, `pass`. diff --git a/collectors/python.d.plugin/adaptec_raid/README.md b/collectors/python.d.plugin/adaptec_raid/README.md index da5d13b16..90ef8fa3c 100644 --- a/collectors/python.d.plugin/adaptec_raid/README.md +++ b/collectors/python.d.plugin/adaptec_raid/README.md @@ -1,7 +1,10 @@ <!-- title: "Adaptec RAID controller monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/adaptec_raid/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/adaptec_raid/README.md" sidebar_label: "Adaptec RAID" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Hardware" --> # Adaptec RAID controller monitoring with Netdata @@ -52,7 +55,7 @@ systemctl restart netdata.service ## Enable the collector The `adaptec_raid` collector is disabled by default. To enable it, use `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. ```bash @@ -61,12 +64,12 @@ sudo ./edit-config python.d.conf ``` Change the value of the `adaptec_raid` setting to `yes`. Save the file and restart the Netdata Agent with `sudo -systemctl restart netdata`, or the [appropriate method](/docs/configure/start-stop-restart.md) for your system. +systemctl restart netdata`, or the [appropriate method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system. ## Configuration Edit the `python.d/adaptec_raid.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/alarms/README.md b/collectors/python.d.plugin/alarms/README.md index 8dc666f5b..4804bd0d7 100644 --- a/collectors/python.d.plugin/alarms/README.md +++ b/collectors/python.d.plugin/alarms/README.md @@ -1,6 +1,9 @@ <!-- title: "Alarms" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/alarms/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/alarms/README.md" +sidebar_label: "alarms" +learn_status: "Unpublished" +learn_topic_type: "References" --> # Alarms - graphing Netdata alarm states over time @@ -23,7 +26,7 @@ Below is an example of the chart produced when running `stress-ng --all 2` for a ## Configuration -Enable the collector and [restart Netdata](/docs/configure/start-stop-restart.md). +Enable the collector and [restart Netdata](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md). ```bash cd /etc/netdata/ @@ -33,7 +36,7 @@ sudo systemctl restart netdata ``` If needed, edit the `python.d/alarms.conf` configuration file using `edit-config` from the your agent's [config -directory](/docs/configure/nodes.md), which is usually at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is usually at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/am2320/README.md b/collectors/python.d.plugin/am2320/README.md index 3503d7c17..070e8eb38 100644 --- a/collectors/python.d.plugin/am2320/README.md +++ b/collectors/python.d.plugin/am2320/README.md @@ -1,7 +1,10 @@ <!-- title: "AM2320 sensor monitoring with netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/am2320/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/am2320/README.md" sidebar_label: "AM2320" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # AM2320 sensor monitoring with netdata @@ -21,7 +24,7 @@ It produces the following charts: ## Configuration Edit the `python.d/am2320.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/anomalies/README.md b/collectors/python.d.plugin/anomalies/README.md index aaf39ab92..7c59275f9 100644 --- a/collectors/python.d.plugin/anomalies/README.md +++ b/collectors/python.d.plugin/anomalies/README.md @@ -1,13 +1,17 @@ <!-- title: "Anomaly detection with Netdata" description: "Use ML-driven anomaly detection to narrow your focus to only affected metrics and services/processes on your node to shorten root cause analysis." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/anomalies/README.md -sidebar_url: Anomalies +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/anomalies/README.md" +sidebar_url: "Anomalies" +sidebar_label: "anomalies" +learn_status: "Unpublished" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Misc" --> # Anomaly detection with Netdata -**Note**: Check out the [Netdata Anomaly Advisor](https://learn.netdata.cloud/docs/cloud/insights/anomaly-advisor) for a more native anomaly detection experience within Netdata. +**Note**: Check out the [Netdata Anomaly Advisor](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/anomaly-advisor.mdx) for a more native anomaly detection experience within Netdata. This collector uses the Python [PyOD](https://pyod.readthedocs.io/en/latest/index.html) library to perform unsupervised [anomaly detection](https://en.wikipedia.org/wiki/Anomaly_detection) on your Netdata charts and/or dimensions. @@ -70,7 +74,7 @@ The configuration for the anomalies collector defines how it will behave on your _**Note**: If you are unsure about any of the below configuration options then it's best to just ignore all this and leave the `anomalies.conf` file alone to begin with. Then you can return to it later if you would like to tune things a bit more once the collector is running for a while and you have a feeling for its performance on your node._ Edit the `python.d/anomalies.conf` configuration file using `edit-config` from the your agent's [config -directory](/docs/configure/nodes.md), which is usually at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is usually at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -216,7 +220,7 @@ If you would like to go deeper on what exactly the anomalies collector is doing ## Notes -- Python 3 is required as the [`netdata-pandas`](https://github.com/netdata/netdata-pandas) package uses Python async libraries ([asks](https://pypi.org/project/asks/) and [trio](https://pypi.org/project/trio/)) to make asynchronous calls to the [Netdata REST API](https://learn.netdata.cloud/docs/agent/web/api) to get the required data for each chart. +- Python 3 is required as the [`netdata-pandas`](https://github.com/netdata/netdata-pandas) package uses Python async libraries ([asks](https://pypi.org/project/asks/) and [trio](https://pypi.org/project/trio/)) to make asynchronous calls to the [Netdata REST API](https://github.com/netdata/netdata/blob/master/web/api/README.md) to get the required data for each chart. - Python 3 is also required for the underlying ML libraries of [numba](https://pypi.org/project/numba/), [scikit-learn](https://pypi.org/project/scikit-learn/), and [PyOD](https://pypi.org/project/pyod/). - It may take a few hours or so (depending on your choice of `train_secs_n`) for the collector to 'settle' into it's typical behaviour in terms of the trained models and probabilities you will see in the normal running of your node. - As this collector does most of the work in Python itself, with [PyOD](https://pyod.readthedocs.io/en/latest/) leveraging [numba](https://numba.pydata.org/) under the hood, you may want to try it out first on a test or development system to get a sense of its performance characteristics on a node similar to where you would like to use it. @@ -231,7 +235,7 @@ If you would like to go deeper on what exactly the anomalies collector is doing - If you activate this collector on a fresh node, it might take a little while to build up enough data to calculate a realistic and useful model. - Some models like `iforest` can be comparatively expensive (on same n1-standard-2 system above ~2s runtime during predict, ~40s training time, ~50% cpu on both train and predict) so if you would like to use it you might be advised to set a relatively high `update_every` maybe 10, 15 or 30 in `anomalies.conf`. - Setting a higher `train_every_n` and `update_every` is an easy way to devote less resources on the node to anomaly detection. Specifying less charts and a lower `train_n_secs` will also help reduce resources at the expense of covering less charts and maybe a more noisy model if you set `train_n_secs` to be too small for how your node tends to behave. -- If you would like to enable this on a Rasberry Pi, then check out [this guide](https://learn.netdata.cloud/guides/monitor/raspberry-pi-anomaly-detection) which will guide you through first installing LLVM. +- If you would like to enable this on a Raspberry Pi, then check out [this guide](https://github.com/netdata/netdata/blob/master/docs/guides/monitor/raspberry-pi-anomaly-detection.md) which will guide you through first installing LLVM. ## Useful links and further reading diff --git a/collectors/python.d.plugin/beanstalk/README.md b/collectors/python.d.plugin/beanstalk/README.md index 3b632597e..7e7f30de9 100644 --- a/collectors/python.d.plugin/beanstalk/README.md +++ b/collectors/python.d.plugin/beanstalk/README.md @@ -1,7 +1,10 @@ <!-- title: "Beanstalk monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/beanstalk/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/beanstalk/README.md" sidebar_label: "Beanstalk" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Message brokers" --> # Beanstalk monitoring with Netdata @@ -112,7 +115,7 @@ Provides server and tube-level statistics. ## Configuration Edit the `python.d/beanstalk.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/bind_rndc/README.md b/collectors/python.d.plugin/bind_rndc/README.md index 2d747f81b..e87001884 100644 --- a/collectors/python.d.plugin/bind_rndc/README.md +++ b/collectors/python.d.plugin/bind_rndc/README.md @@ -1,7 +1,10 @@ <!-- title: "ISC Bind monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/bind_rndc/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/bind_rndc/README.md" sidebar_label: "ISC Bind" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # ISC Bind monitoring with Netdata @@ -58,7 +61,7 @@ It produces: ## Configuration Edit the `python.d/bind_rndc.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/boinc/README.md b/collectors/python.d.plugin/boinc/README.md index 4da2d52bf..149d37ca1 100644 --- a/collectors/python.d.plugin/boinc/README.md +++ b/collectors/python.d.plugin/boinc/README.md @@ -1,7 +1,10 @@ <!-- title: "BOINC monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/boinc/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/boinc/README.md" sidebar_label: "BOINC" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Distributed computing" --> # BOINC monitoring with Netdata @@ -13,7 +16,7 @@ It provides charts tracking the total number of tasks and active tasks, as well ## Configuration Edit the `python.d/boinc.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/ceph/README.md b/collectors/python.d.plugin/ceph/README.md index b75ba6d4f..e7d0f51e2 100644 --- a/collectors/python.d.plugin/ceph/README.md +++ b/collectors/python.d.plugin/ceph/README.md @@ -1,7 +1,10 @@ <!-- title: "CEPH monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ceph/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ceph/README.md" sidebar_label: "CEPH" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Storage" --> # CEPH monitoring with Netdata @@ -28,7 +31,7 @@ Monitors the ceph cluster usage and consumption data of a server, and produces: ## Configuration Edit the `python.d/ceph.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/changefinder/README.md b/collectors/python.d.plugin/changefinder/README.md index 7ec3a2539..326a69dd5 100644 --- a/collectors/python.d.plugin/changefinder/README.md +++ b/collectors/python.d.plugin/changefinder/README.md @@ -1,7 +1,11 @@ <!-- title: "Online change point detection with Netdata" description: "Use ML-driven change point detection to narrow your focus and shorten root cause analysis." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/changefinder/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/changefinder/README.md" +sidebar_label: "changefinder" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/QoS" --> # Online changepoint detection with Netdata @@ -93,7 +97,7 @@ leave the `changefinder.conf` file alone to begin with. Then you can return to i a bit more once the collector is running for a while and you have a feeling for its performance on your node._ Edit the `python.d/changefinder.conf` configuration file using `edit-config` from the your -agent's [config directory](/docs/configure/nodes.md), which is usually at `/etc/netdata`. +agent's [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is usually at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/dockerd/Makefile.inc b/collectors/python.d.plugin/dockerd/Makefile.inc deleted file mode 100644 index b100bc6a1..000000000 --- a/collectors/python.d.plugin/dockerd/Makefile.inc +++ /dev/null @@ -1,13 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-or-later - -# THIS IS NOT A COMPLETE Makefile -# IT IS INCLUDED BY ITS PARENT'S Makefile.am -# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT - -# install these files -dist_python_DATA += dockerd/dockerd.chart.py -dist_pythonconfig_DATA += dockerd/dockerd.conf - -# do not install these files, but include them in the distribution -dist_noinst_DATA += dockerd/README.md dockerd/Makefile.inc - diff --git a/collectors/python.d.plugin/dockerd/README.md b/collectors/python.d.plugin/dockerd/README.md deleted file mode 100644 index 6470a7c0b..000000000 --- a/collectors/python.d.plugin/dockerd/README.md +++ /dev/null @@ -1,46 +0,0 @@ -<!-- -title: "Docker Engine monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/dockerd/README.md -sidebar_label: "Docker Engine" ---> - -# Docker Engine monitoring with Netdata - -Collects docker container health metrics. - -**Requirement:** - -- `docker` package, required version 3.2.0+ - -Following charts are drawn: - -1. **running containers** - - - count - -2. **healthy containers** - - - count - -3. **unhealthy containers** - - - count - -## Configuration - -Edit the `python.d/dockerd.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/dockerd.conf -``` - -```yaml - update_every : 1 - priority : 60000 -``` - ---- - - diff --git a/collectors/python.d.plugin/dockerd/dockerd.chart.py b/collectors/python.d.plugin/dockerd/dockerd.chart.py deleted file mode 100644 index bd9640bbf..000000000 --- a/collectors/python.d.plugin/dockerd/dockerd.chart.py +++ /dev/null @@ -1,86 +0,0 @@ -# -*- coding: utf-8 -*- -# Description: docker netdata python.d module -# Author: Kévin Darcel (@tuxity) - -try: - import docker - - HAS_DOCKER = True -except ImportError: - HAS_DOCKER = False - -from distutils.version import StrictVersion - -from bases.FrameworkServices.SimpleService import SimpleService - -# charts order (can be overridden if you want less charts, or different order) -ORDER = [ - 'running_containers', - 'healthy_containers', - 'unhealthy_containers' -] - -CHARTS = { - 'running_containers': { - 'options': [None, 'Number of running containers', 'containers', 'running containers', - 'docker.running_containers', 'line'], - 'lines': [ - ['running_containers', 'running'] - ] - }, - 'healthy_containers': { - 'options': [None, 'Number of healthy containers', 'containers', 'healthy containers', - 'docker.healthy_containers', 'line'], - 'lines': [ - ['healthy_containers', 'healthy'] - ] - }, - 'unhealthy_containers': { - 'options': [None, 'Number of unhealthy containers', 'containers', 'unhealthy containers', - 'docker.unhealthy_containers', 'line'], - 'lines': [ - ['unhealthy_containers', 'unhealthy'] - ] - } -} - -MIN_REQUIRED_VERSION = '3.2.0' - - -class Service(SimpleService): - def __init__(self, configuration=None, name=None): - SimpleService.__init__(self, configuration=configuration, name=name) - self.order = ORDER - self.definitions = CHARTS - self.client = None - - def check(self): - if not HAS_DOCKER: - self.error("'docker' package is needed to use dockerd module") - return False - - if StrictVersion(docker.__version__) < StrictVersion(MIN_REQUIRED_VERSION): - self.error("installed 'docker' package version {0}, minimum required version {1}, please upgrade".format( - docker.__version__, - MIN_REQUIRED_VERSION, - )) - return False - - self.client = docker.DockerClient(base_url=self.configuration.get('url', 'unix://var/run/docker.sock')) - - try: - self.client.ping() - except docker.errors.APIError as error: - self.error(error) - return False - - return True - - def get_data(self): - data = dict() - - data['running_containers'] = len(self.client.containers.list(sparse=True)) - data['healthy_containers'] = len(self.client.containers.list(filters={'health': 'healthy'}, sparse=True)) - data['unhealthy_containers'] = len(self.client.containers.list(filters={'health': 'unhealthy'}, sparse=True)) - - return data or None diff --git a/collectors/python.d.plugin/dockerd/dockerd.conf b/collectors/python.d.plugin/dockerd/dockerd.conf deleted file mode 100644 index 96c8ee0d8..000000000 --- a/collectors/python.d.plugin/dockerd/dockerd.conf +++ /dev/null @@ -1,77 +0,0 @@ -# netdata python.d.plugin configuration for dockerd health data API -# -# This file is in YaML format. Generally the format is: -# -# name: value -# -# There are 2 sections: -# - global variables -# - one or more JOBS -# -# JOBS allow you to collect values from multiple sources. -# Each source will have its own set of charts. -# -# JOB parameters have to be indented (using spaces only, example below). - -# ---------------------------------------------------------------------- -# Global Variables -# These variables set the defaults for all JOBs, however each JOB -# may define its own, overriding the defaults. - -# update_every sets the default data collection frequency. -# If unset, the python.d.plugin default is used. -# update_every: 1 - -# priority controls the order of charts at the netdata dashboard. -# Lower numbers move the charts towards the top of the page. -# If unset, the default for python.d.plugin is used. -# priority: 60000 - -# penalty indicates whether to apply penalty to update_every in case of failures. -# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes. -# penalty: yes - -# autodetection_retry sets the job re-check interval in seconds. -# The job is not deleted if check fails. -# Attempts to start the job are made once every autodetection_retry. -# This feature is disabled by default. -# autodetection_retry: 0 - -# ---------------------------------------------------------------------- -# JOBS (data collection sources) -# -# The default JOBS share the same *name*. JOBS with the same name -# are mutually exclusive. Only one of them will be allowed running at -# any time. This allows autodetection to try several alternatives and -# pick the one that works. -# -# Any number of jobs is supported. -# -# All python.d.plugin JOBS (for all its modules) support a set of -# predefined parameters. These are: -# -# job_name: -# name: myname # the JOB's name as it will appear at the -# # dashboard (by default is the job_name) -# # JOBs sharing a name are mutually exclusive -# update_every: 1 # the JOB's data collection frequency -# priority: 60000 # the JOB's order on the dashboard -# penalty: yes # the JOB's penalty -# autodetection_retry: 0 # the JOB's re-check interval in seconds -# -# Additionally to the above, dockerd plugin also supports the following: -# -# url: '<scheme>://<host>:<port>/<health_page_api>' -# # http://localhost:8080/health -# -# if the URL is password protected, the following are supported: -# -# user: 'username' -# pass: 'password' -# -# ---------------------------------------------------------------------- -# AUTO-DETECTION JOBS -# only one of them will run (they have the same name) -# -local: - url: 'unix://var/run/docker.sock' diff --git a/collectors/python.d.plugin/dovecot/README.md b/collectors/python.d.plugin/dovecot/README.md index e6bbf0d74..358f1ba81 100644 --- a/collectors/python.d.plugin/dovecot/README.md +++ b/collectors/python.d.plugin/dovecot/README.md @@ -1,7 +1,10 @@ <!-- title: "Dovecot monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/dovecot/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/dovecot/README.md" sidebar_label: "Dovecot" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Dovecot monitoring with Netdata @@ -78,7 +81,7 @@ Module gives information with following charts: ## Configuration Edit the `python.d/dovecot.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/example/README.md b/collectors/python.d.plugin/example/README.md index 0b80aa9ea..7e6d2b913 100644 --- a/collectors/python.d.plugin/example/README.md +++ b/collectors/python.d.plugin/example/README.md @@ -1,6 +1,10 @@ <!-- -title: "Example" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/example/README.md +title: "Example module in Python" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/example/README.md" +sidebar_label: "Example module in Python" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Mock Collectors" --> # Example @@ -9,6 +13,6 @@ You can add custom data collectors using Python. Netdata provides an [example python data collection module](https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin/example). -If you want to write your own collector, read our [writing a new Python module](/collectors/python.d.plugin/README.md#how-to-write-a-new-module) tutorial. +If you want to write your own collector, read our [writing a new Python module](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md#how-to-write-a-new-module) tutorial. diff --git a/collectors/python.d.plugin/exim/README.md b/collectors/python.d.plugin/exim/README.md index 92b2d7a5b..a9c66c057 100644 --- a/collectors/python.d.plugin/exim/README.md +++ b/collectors/python.d.plugin/exim/README.md @@ -1,7 +1,10 @@ <!-- title: "Exim monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/exim/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/exim/README.md" sidebar_label: "Exim" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Exim monitoring with Netdata diff --git a/collectors/python.d.plugin/fail2ban/README.md b/collectors/python.d.plugin/fail2ban/README.md index be09e1857..6b2c6bba1 100644 --- a/collectors/python.d.plugin/fail2ban/README.md +++ b/collectors/python.d.plugin/fail2ban/README.md @@ -1,7 +1,10 @@ <!-- title: "Fail2ban monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/fail2ban/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/fail2ban/README.md" sidebar_label: "Fail2ban" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apps" --> # Fail2ban monitoring with Netdata @@ -58,7 +61,7 @@ To persist the changes after rotating the log file, add `create 640 root netdata ## Configuration Edit the `python.d/fail2ban.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/gearman/README.md b/collectors/python.d.plugin/gearman/README.md index 34ea584ab..9ac53cb8e 100644 --- a/collectors/python.d.plugin/gearman/README.md +++ b/collectors/python.d.plugin/gearman/README.md @@ -1,7 +1,10 @@ <!-- title: "Gearman monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/gearman/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/gearman/README.md" sidebar_label: "Gearman" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Distributed computing" --> # Gearman monitoring with Netdata @@ -27,7 +30,7 @@ It produces: ## Configuration Edit the `python.d/gearman.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/go_expvar/README.md b/collectors/python.d.plugin/go_expvar/README.md index feb150dd9..ff786e7c4 100644 --- a/collectors/python.d.plugin/go_expvar/README.md +++ b/collectors/python.d.plugin/go_expvar/README.md @@ -1,7 +1,10 @@ <!-- title: "Go applications monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/go_expvar/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/go_expvar/README.md" sidebar_label: "Go applications" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Application Performance Monitoring" --> # Go applications monitoring with Netdata @@ -209,8 +212,8 @@ See [this issue](https://github.com/netdata/netdata/pull/1902#issuecomment-28449 Please see these two links to the official Netdata documentation for more information about the values: -- [External plugins - charts](/collectors/plugins.d/README.md#chart) -- [Chart variables](/collectors/python.d.plugin/README.md#global-variables-order-and-chart) +- [External plugins - charts](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md#chart) +- [Chart variables](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md#global-variables-order-and-chart) **Line definitions** @@ -233,7 +236,7 @@ hidden: False ``` Please see the following link for more information about the options and their default values: -[External plugins - dimensions](/collectors/plugins.d/README.md#dimension) +[External plugins - dimensions](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md#dimension) Apart from top-level expvars, this plugin can also parse expvars stored in a multi-level map; All dicts in the resulting JSON document are then flattened to one level. @@ -255,7 +258,7 @@ the first defined key wins and all subsequent keys with the same name are ignore ## Enable the collector The `go_expvar` collector is disabled by default. To enable it, use `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -268,7 +271,7 @@ restart netdata`, or the appropriate method for your system, to finish enabling ## Configuration Edit the `python.d/go_expvar.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/haproxy/README.md b/collectors/python.d.plugin/haproxy/README.md index f16e7258e..1aa1a214a 100644 --- a/collectors/python.d.plugin/haproxy/README.md +++ b/collectors/python.d.plugin/haproxy/README.md @@ -1,7 +1,10 @@ <!-- title: "HAProxy monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/haproxy/README.md -sidebar_label: "HAProxy" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/haproxy/README.md" +sidebar_label: "haproxy-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # HAProxy monitoring with Netdata @@ -39,7 +42,7 @@ It produces: ## Configuration Edit the `python.d/haproxy.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/hddtemp/README.md b/collectors/python.d.plugin/hddtemp/README.md index d8aba62d2..6a253b5bf 100644 --- a/collectors/python.d.plugin/hddtemp/README.md +++ b/collectors/python.d.plugin/hddtemp/README.md @@ -1,7 +1,10 @@ <!-- title: "Hard drive temperature monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/hddtemp/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/hddtemp/README.md" sidebar_label: "Hard drive temperature" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Hardware" --> # Hard drive temperature monitoring with Netdata @@ -16,7 +19,7 @@ It produces one chart **Temperature** with dynamic number of dimensions (one per ## Configuration Edit the `python.d/hddtemp.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/hpssa/README.md b/collectors/python.d.plugin/hpssa/README.md index c1d218279..72dc78032 100644 --- a/collectors/python.d.plugin/hpssa/README.md +++ b/collectors/python.d.plugin/hpssa/README.md @@ -1,7 +1,10 @@ <!-- title: "HP Smart Storage Arrays monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/hpssa/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/hpssa/README.md" sidebar_label: "HP Smart Storage Arrays" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Storage" --> # HP Smart Storage Arrays monitoring with Netdata @@ -51,7 +54,7 @@ systemctl restart netdata.service ## Enable the collector The `hpssa` collector is disabled by default. To enable it, use `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. ```bash @@ -60,12 +63,12 @@ sudo ./edit-config python.d.conf ``` Change the value of the `hpssa` setting to `yes`. Save the file and restart the Netdata Agent with `sudo systemctl -restart netdata`, or the [appropriate method](/docs/configure/start-stop-restart.md) for your system. +restart netdata`, or the [appropriate method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system. ## Configuration Edit the `python.d/hpssa.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -79,5 +82,5 @@ ssacli_path: /usr/sbin/ssacli ``` Save the file and restart the Netdata Agent with `sudo systemctl restart netdata`, or the [appropriate -method](/docs/configure/start-stop-restart.md) for your system. +method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system. diff --git a/collectors/python.d.plugin/icecast/README.md b/collectors/python.d.plugin/icecast/README.md index c122f76a3..6fca34ba6 100644 --- a/collectors/python.d.plugin/icecast/README.md +++ b/collectors/python.d.plugin/icecast/README.md @@ -1,7 +1,10 @@ <!-- title: "Icecast monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/icecast/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/icecast/README.md" sidebar_label: "Icecast" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # Icecast monitoring with Netdata @@ -21,7 +24,7 @@ It produces the following charts: ## Configuration Edit the `python.d/icecast.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/ipfs/README.md b/collectors/python.d.plugin/ipfs/README.md index 3a7c43632..8f5e53b10 100644 --- a/collectors/python.d.plugin/ipfs/README.md +++ b/collectors/python.d.plugin/ipfs/README.md @@ -1,7 +1,10 @@ <!-- title: "IPFS monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ipfs/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ipfs/README.md" sidebar_label: "IPFS" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Storage" --> # IPFS monitoring with Netdata @@ -20,7 +23,7 @@ It produces the following charts: ## Configuration Edit the `python.d/ipfs.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/litespeed/README.md b/collectors/python.d.plugin/litespeed/README.md index b58b23d7e..b9bad4635 100644 --- a/collectors/python.d.plugin/litespeed/README.md +++ b/collectors/python.d.plugin/litespeed/README.md @@ -1,7 +1,10 @@ <!-- title: "LiteSpeed monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/litespeed/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/litespeed/README.md" sidebar_label: "LiteSpeed" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Application Performance Monitoring" --> # LiteSpeed monitoring with Netdata @@ -53,7 +56,7 @@ It produces: ## Configuration Edit the `python.d/litespeed.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/logind/Makefile.inc b/collectors/python.d.plugin/logind/Makefile.inc deleted file mode 100644 index adadab120..000000000 --- a/collectors/python.d.plugin/logind/Makefile.inc +++ /dev/null @@ -1,13 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-or-later - -# THIS IS NOT A COMPLETE Makefile -# IT IS INCLUDED BY ITS PARENT'S Makefile.am -# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT - -# install these files -dist_python_DATA += logind/logind.chart.py -dist_pythonconfig_DATA += logind/logind.conf - -# do not install these files, but include them in the distribution -dist_noinst_DATA += logind/README.md logind/Makefile.inc - diff --git a/collectors/python.d.plugin/logind/README.md b/collectors/python.d.plugin/logind/README.md deleted file mode 100644 index 442d388d0..000000000 --- a/collectors/python.d.plugin/logind/README.md +++ /dev/null @@ -1,86 +0,0 @@ -<!-- -title: "systemd-logind monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/logind/README.md -sidebar_label: "systemd-logind" ---> - -# Systemd-Logind monitoring with Netdata - -Monitors active sessions, users, and seats tracked by `systemd-logind` or `elogind`. - -It provides the following charts: - -1. **Sessions** Tracks the total number of sessions. - - - Graphical: Local graphical sessions (running X11, or Wayland, or something else). - - Console: Local console sessions. - - Remote: Remote sessions. - -2. **Users** Tracks total number of unique user logins of each type. - - - Graphical - - Console - - Remote - -3. **Seats** Total number of seats in use. - - - Seats - -## Enable the collector - -The `logind` collector is disabled by default. To enable it, use `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d.conf -``` - -Change the value of the `logind` setting to `yes`. Save the file and restart the Netdata Agent with `sudo systemctl -restart netdata`, or the appropriate method for your system, to finish enabling the `logind` collector. - -## Configuration - -This module needs no configuration. Just make sure the `netdata` user -can run the `loginctl` command and get a session list without having to -specify a path. - -This will work with any command that can output data in the _exact_ -same format as `loginctl list-sessions --no-legend`. If you have some -other command you want to use that outputs data in this format, you can -specify it using the `command` key like so: - -```yaml -command: '/path/to/other/command' -``` - -Edit the `python.d/logind.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/logind.conf -``` - -## Notes - -- This module's ability to track logins is dependent on what PAM services - are configured to register sessions with logind. In particular, for - most systems, it will only track TTY logins, local desktop logins, - and logins through remote shell connections. - -- The users chart counts _usernames_ not UID's. This is potentially - important in configurations where multiple users have the same UID. - -- The users chart counts any given user name up to once for _each_ type - of login. So if the same user has a graphical and a console login on a - system, they will show up once in the graphical count, and once in the - console count. - -- Because the data collection process is rather expensive, this plugin - is currently disabled by default, and needs to be explicitly enabled in - `/etc/netdata/python.d.conf` before it will run. - ---- - - diff --git a/collectors/python.d.plugin/logind/logind.chart.py b/collectors/python.d.plugin/logind/logind.chart.py deleted file mode 100644 index 708668649..000000000 --- a/collectors/python.d.plugin/logind/logind.chart.py +++ /dev/null @@ -1,85 +0,0 @@ -# -*- coding: utf-8 -*- -# Description: logind netdata python.d module -# Author: Austin S. Hemmelgarn (Ferroin) -# SPDX-License-Identifier: GPL-3.0-or-later - -from bases.FrameworkServices.ExecutableService import ExecutableService - -priority = 59999 -disabled_by_default = True - -LOGINCTL_COMMAND = 'loginctl list-sessions --no-legend' - -ORDER = [ - 'sessions', - 'users', - 'seats', -] - -CHARTS = { - 'sessions': { - 'options': [None, 'Logind Sessions', 'sessions', 'sessions', 'logind.sessions', 'stacked'], - 'lines': [ - ['sessions_graphical', 'Graphical', 'absolute', 1, 1], - ['sessions_console', 'Console', 'absolute', 1, 1], - ['sessions_remote', 'Remote', 'absolute', 1, 1] - ] - }, - 'users': { - 'options': [None, 'Logind Users', 'users', 'users', 'logind.users', 'stacked'], - 'lines': [ - ['users_graphical', 'Graphical', 'absolute', 1, 1], - ['users_console', 'Console', 'absolute', 1, 1], - ['users_remote', 'Remote', 'absolute', 1, 1] - ] - }, - 'seats': { - 'options': [None, 'Logind Seats', 'seats', 'seats', 'logind.seats', 'line'], - 'lines': [ - ['seats', 'Active Seats', 'absolute', 1, 1] - ] - } -} - - -class Service(ExecutableService): - def __init__(self, configuration=None, name=None): - ExecutableService.__init__(self, configuration=configuration, name=name) - self.order = ORDER - self.definitions = CHARTS - self.command = LOGINCTL_COMMAND - - def _get_data(self): - ret = { - 'sessions_graphical': 0, - 'sessions_console': 0, - 'sessions_remote': 0, - } - users = { - 'graphical': list(), - 'console': list(), - 'remote': list() - } - seats = list() - data = self._get_raw_data() - - for item in data: - fields = item.split() - if len(fields) == 3: - users['remote'].append(fields[2]) - ret['sessions_remote'] += 1 - elif len(fields) == 4: - users['graphical'].append(fields[2]) - ret['sessions_graphical'] += 1 - seats.append(fields[3]) - elif len(fields) == 5: - users['console'].append(fields[2]) - ret['sessions_console'] += 1 - seats.append(fields[3]) - - ret['users_graphical'] = len(set(users['graphical'])) - ret['users_console'] = len(set(users['console'])) - ret['users_remote'] = len(set(users['remote'])) - ret['seats'] = len(set(seats)) - - return ret diff --git a/collectors/python.d.plugin/logind/logind.conf b/collectors/python.d.plugin/logind/logind.conf deleted file mode 100644 index 01a859d21..000000000 --- a/collectors/python.d.plugin/logind/logind.conf +++ /dev/null @@ -1,60 +0,0 @@ -# netdata python.d.plugin configuration for logind -# -# This file is in YaML format. Generally the format is: -# -# name: value -# -# There are 2 sections: -# - global variables -# - one or more JOBS -# -# JOBS allow you to collect values from multiple sources. -# Each source will have its own set of charts. -# -# JOB parameters have to be indented (using spaces only, example below). - -# ---------------------------------------------------------------------- -# Global Variables -# These variables set the defaults for all JOBs, however each JOB -# may define its own, overriding the defaults. - -# update_every sets the default data collection frequency. -# If unset, the python.d.plugin default is used. -# update_every: 1 - -# priority controls the order of charts at the netdata dashboard. -# Lower numbers move the charts towards the top of the page. -# If unset, the default for python.d.plugin is used. -# priority: 60000 - -# penalty indicates whether to apply penalty to update_every in case of failures. -# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes. -# penalty: yes - -# autodetection_retry sets the job re-check interval in seconds. -# The job is not deleted if check fails. -# Attempts to start the job are made once every autodetection_retry. -# This feature is disabled by default. -# autodetection_retry: 0 - -# ---------------------------------------------------------------------- -# JOBS (data collection sources) -# -# The default JOBS share the same *name*. JOBS with the same name -# are mutually exclusive. Only one of them will be allowed running at -# any time. This allows autodetection to try several alternatives and -# pick the one that works. -# -# Any number of jobs is supported. -# -# All python.d.plugin JOBS (for all its modules) support a set of -# predefined parameters. These are: -# -# job_name: -# name: myname # the JOB's name as it will appear at the -# # dashboard (by default is the job_name) -# # JOBs sharing a name are mutually exclusive -# update_every: 1 # the JOB's data collection frequency -# priority: 60000 # the JOB's order on the dashboard -# penalty: yes # the JOB's penalty -# autodetection_retry: 0 # the JOB's re-check interval in seconds diff --git a/collectors/python.d.plugin/megacli/README.md b/collectors/python.d.plugin/megacli/README.md index 3c99c3de8..3900de381 100644 --- a/collectors/python.d.plugin/megacli/README.md +++ b/collectors/python.d.plugin/megacli/README.md @@ -1,7 +1,10 @@ <!-- title: "MegaRAID controller monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/megacli/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/megacli/README.md" sidebar_label: "MegaRAID controllers" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # MegaRAID controller monitoring with Netdata @@ -53,7 +56,7 @@ systemctl restart netdata.service ## Enable the collector The `megacli` collector is disabled by default. To enable it, use `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. ```bash @@ -67,7 +70,7 @@ with `sudo systemctl restart netdata`, or the appropriate method for your system ## Configuration Edit the `python.d/megacli.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -81,6 +84,6 @@ do_battery: yes ``` Save the file and restart the Netdata Agent with `sudo systemctl restart netdata`, or the [appropriate -method](/docs/configure/start-stop-restart.md) for your system. +method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system. diff --git a/collectors/python.d.plugin/memcached/README.md b/collectors/python.d.plugin/memcached/README.md index 19139ee92..4158ab19c 100644 --- a/collectors/python.d.plugin/memcached/README.md +++ b/collectors/python.d.plugin/memcached/README.md @@ -1,7 +1,10 @@ <!-- title: "Memcached monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/memcached/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/memcached/README.md" sidebar_label: "Memcached" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Databases" --> # Memcached monitoring with Netdata @@ -76,7 +79,7 @@ Collects memory-caching system performance metrics. It reads server response to ## Configuration Edit the `python.d/memcached.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/mongodb/Makefile.inc b/collectors/python.d.plugin/mongodb/Makefile.inc deleted file mode 100644 index 784945aa6..000000000 --- a/collectors/python.d.plugin/mongodb/Makefile.inc +++ /dev/null @@ -1,13 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-or-later - -# THIS IS NOT A COMPLETE Makefile -# IT IS INCLUDED BY ITS PARENT'S Makefile.am -# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT - -# install these files -dist_python_DATA += mongodb/mongodb.chart.py -dist_pythonconfig_DATA += mongodb/mongodb.conf - -# do not install these files, but include them in the distribution -dist_noinst_DATA += mongodb/README.md mongodb/Makefile.inc - diff --git a/collectors/python.d.plugin/mongodb/README.md b/collectors/python.d.plugin/mongodb/README.md deleted file mode 100644 index b6dd9c5f4..000000000 --- a/collectors/python.d.plugin/mongodb/README.md +++ /dev/null @@ -1,210 +0,0 @@ -<!-- -title: "MongoDB monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/mongodb/README.md -sidebar_label: "MongoDB" ---> - -# MongoDB monitoring with Netdata - -Monitors performance and health metrics of MongoDB. - -## Requirements - -- `python-pymongo` package v2.4+. - -You need to install it manually. - -Number of charts depends on mongodb version, storage engine and other features (replication): - -1. **Read requests**: - - - query - - getmore (operation the cursor executes to get additional data from query) - -2. **Write requests**: - - - insert - - delete - - update - -3. **Active clients**: - - - readers (number of clients with read operations in progress or queued) - - writers (number of clients with write operations in progress or queued) - -4. **Journal transactions**: - - - commits (count of transactions that have been written to the journal) - -5. **Data written to the journal**: - - - volume (volume of data) - -6. **Background flush** (MMAPv1): - - - average ms (average time taken by flushes to execute) - - last ms (time taken by the last flush) - -7. **Read tickets** (WiredTiger): - - - in use (number of read tickets in use) - - available (number of available read tickets remaining) - -8. **Write tickets** (WiredTiger): - - - in use (number of write tickets in use) - - available (number of available write tickets remaining) - -9. **Cursors**: - -- opened (number of cursors currently opened by MongoDB for clients) -- timedOut (number of cursors that have timed) -- noTimeout (number of open cursors with timeout disabled) - -10. **Connections**: - - - connected (number of clients currently connected to the database server) - - unused (number of unused connections available for new clients) - -11. **Memory usage metrics**: - - - virtual - - resident (amount of memory used by the database process) - - mapped - - non mapped - -12. **Page faults**: - - - page faults (number of times MongoDB had to request from disk) - -13. **Cache metrics** (WiredTiger): - - - percentage of bytes currently in the cache (amount of space taken by cached data) - - percentage of tracked dirty bytes in the cache (amount of space taken by dirty data) - -14. **Pages evicted from cache** (WiredTiger): - - - modified - - unmodified - -15. **Queued requests**: - - - readers (number of read request currently queued) - - writers (number of write request currently queued) - -16. **Errors**: - - - msg (number of message assertions raised) - - warning (number of warning assertions raised) - - regular (number of regular assertions raised) - - user (number of assertions corresponding to errors generated by users) - -17. **Storage metrics** (one chart for every database) - - - dataSize (size of all documents + padding in the database) - - indexSize (size of all indexes in the database) - - storageSize (size of all extents in the database) - -18. **Documents in the database** (one chart for all databases) - -- documents (number of objects in the database among all the collections) - -19. **tcmalloc metrics** - - - central cache free - - current total thread cache - - pageheap free - - pageheap unmapped - - thread cache free - - transfer cache free - - heap size - -20. **Commands total/failed rate** - - - count - - createIndex - - delete - - eval - - findAndModify - - insert - -21. **Locks metrics** (acquireCount metrics - number of times the lock was acquired in the specified mode) - - - Global lock - - Database lock - - Collection lock - - Metadata lock - - oplog lock - -22. **Replica set members state** - - - state - -23. **Oplog window** - - - window (interval of time between the oldest and the latest entries in the oplog) - -24. **Replication lag** - - - member (time when last entry from the oplog was applied for every member) - -25. **Replication set member heartbeat latency** - - - member (time when last heartbeat was received from replica set member) - -## Prerequisite - -Create a read-only user for Netdata in the admin database. - -1. Authenticate as the admin user. - -``` -use admin -db.auth("admin", "<MONGODB_ADMIN_PASSWORD>") -``` - -2. Create a user. - -``` -# MongoDB 2.x. -db.addUser("netdata", "<UNIQUE_PASSWORD>", true) - -# MongoDB 3.x or higher. -db.createUser({ - "user":"netdata", - "pwd": "<UNIQUE_PASSWORD>", - "roles" : [ - {role: 'read', db: 'admin' }, - {role: 'clusterMonitor', db: 'admin'}, - {role: 'read', db: 'local' } - ] -}) -``` - -## Configuration - -Edit the `python.d/mongodb.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/mongodb.conf -``` - -Sample: - -```yaml -local: - name : 'local' - authdb: 'admin' - host : '127.0.0.1' - port : 27017 - user : 'netdata' - pass : 'netdata' -``` - -If no configuration is given, module will attempt to connect to mongodb daemon on `127.0.0.1:27017` address - ---- - - diff --git a/collectors/python.d.plugin/mongodb/mongodb.chart.py b/collectors/python.d.plugin/mongodb/mongodb.chart.py deleted file mode 100644 index 5e8fec834..000000000 --- a/collectors/python.d.plugin/mongodb/mongodb.chart.py +++ /dev/null @@ -1,786 +0,0 @@ -# -*- coding: utf-8 -*- -# Description: mongodb netdata python.d module -# Author: ilyam8 -# SPDX-License-Identifier: GPL-3.0-or-later - -import ssl - -from copy import deepcopy -from datetime import datetime -from sys import exc_info - -try: - from pymongo import MongoClient, ASCENDING, DESCENDING, version_tuple - from pymongo.errors import PyMongoError - - PYMONGO = True -except ImportError: - PYMONGO = False - -from bases.FrameworkServices.SimpleService import SimpleService - -REPL_SET_STATES = [ - ('1', 'primary'), - ('8', 'down'), - ('2', 'secondary'), - ('3', 'recovering'), - ('5', 'startup2'), - ('4', 'fatal'), - ('7', 'arbiter'), - ('6', 'unknown'), - ('9', 'rollback'), - ('10', 'removed'), - ('0', 'startup') -] - - -def multiply_by_100(value): - return value * 100 - - -DEFAULT_METRICS = [ - ('opcounters.delete', None, None), - ('opcounters.update', None, None), - ('opcounters.insert', None, None), - ('opcounters.query', None, None), - ('opcounters.getmore', None, None), - ('globalLock.activeClients.readers', 'activeClients_readers', None), - ('globalLock.activeClients.writers', 'activeClients_writers', None), - ('connections.available', 'connections_available', None), - ('connections.current', 'connections_current', None), - ('mem.mapped', None, None), - ('mem.resident', None, None), - ('mem.virtual', None, None), - ('globalLock.currentQueue.readers', 'currentQueue_readers', None), - ('globalLock.currentQueue.writers', 'currentQueue_writers', None), - ('asserts.msg', None, None), - ('asserts.regular', None, None), - ('asserts.user', None, None), - ('asserts.warning', None, None), - ('extra_info.page_faults', None, None), - ('metrics.record.moves', None, None), - ('backgroundFlushing.average_ms', None, multiply_by_100), - ('backgroundFlushing.last_ms', None, multiply_by_100), - ('backgroundFlushing.flushes', None, multiply_by_100), - ('metrics.cursor.timedOut', None, None), - ('metrics.cursor.open.total', 'cursor_total', None), - ('metrics.cursor.open.noTimeout', None, None), - ('cursors.timedOut', None, None), - ('cursors.totalOpen', 'cursor_total', None) -] - -DUR = [ - ('dur.commits', None, None), - ('dur.journaledMB', None, multiply_by_100) -] - -WIREDTIGER = [ - ('wiredTiger.concurrentTransactions.read.available', 'wiredTigerRead_available', None), - ('wiredTiger.concurrentTransactions.read.out', 'wiredTigerRead_out', None), - ('wiredTiger.concurrentTransactions.write.available', 'wiredTigerWrite_available', None), - ('wiredTiger.concurrentTransactions.write.out', 'wiredTigerWrite_out', None), - ('wiredTiger.cache.bytes currently in the cache', None, None), - ('wiredTiger.cache.tracked dirty bytes in the cache', None, None), - ('wiredTiger.cache.maximum bytes configured', None, None), - ('wiredTiger.cache.unmodified pages evicted', 'unmodified', None), - ('wiredTiger.cache.modified pages evicted', 'modified', None) -] - -TCMALLOC = [ - ('tcmalloc.generic.current_allocated_bytes', None, None), - ('tcmalloc.generic.heap_size', None, None), - ('tcmalloc.tcmalloc.central_cache_free_bytes', None, None), - ('tcmalloc.tcmalloc.current_total_thread_cache_bytes', None, None), - ('tcmalloc.tcmalloc.pageheap_free_bytes', None, None), - ('tcmalloc.tcmalloc.pageheap_unmapped_bytes', None, None), - ('tcmalloc.tcmalloc.thread_cache_free_bytes', None, None), - ('tcmalloc.tcmalloc.transfer_cache_free_bytes', None, None) -] - -COMMANDS = [ - ('metrics.commands.count.total', 'count_total', None), - ('metrics.commands.createIndexes.total', 'createIndexes_total', None), - ('metrics.commands.delete.total', 'delete_total', None), - ('metrics.commands.eval.total', 'eval_total', None), - ('metrics.commands.findAndModify.total', 'findAndModify_total', None), - ('metrics.commands.insert.total', 'insert_total', None), - ('metrics.commands.delete.total', 'delete_total', None), - ('metrics.commands.count.failed', 'count_failed', None), - ('metrics.commands.createIndexes.failed', 'createIndexes_failed', None), - ('metrics.commands.delete.failed', 'delete_failed', None), - ('metrics.commands.eval.failed', 'eval_failed', None), - ('metrics.commands.findAndModify.failed', 'findAndModify_failed', None), - ('metrics.commands.insert.failed', 'insert_failed', None), - ('metrics.commands.delete.failed', 'delete_failed', None) -] - -LOCKS = [ - ('locks.Collection.acquireCount.R', 'Collection_R', None), - ('locks.Collection.acquireCount.r', 'Collection_r', None), - ('locks.Collection.acquireCount.W', 'Collection_W', None), - ('locks.Collection.acquireCount.w', 'Collection_w', None), - ('locks.Database.acquireCount.R', 'Database_R', None), - ('locks.Database.acquireCount.r', 'Database_r', None), - ('locks.Database.acquireCount.W', 'Database_W', None), - ('locks.Database.acquireCount.w', 'Database_w', None), - ('locks.Global.acquireCount.R', 'Global_R', None), - ('locks.Global.acquireCount.r', 'Global_r', None), - ('locks.Global.acquireCount.W', 'Global_W', None), - ('locks.Global.acquireCount.w', 'Global_w', None), - ('locks.Metadata.acquireCount.R', 'Metadata_R', None), - ('locks.Metadata.acquireCount.w', 'Metadata_w', None), - ('locks.oplog.acquireCount.r', 'oplog_r', None), - ('locks.oplog.acquireCount.w', 'oplog_w', None) -] - -DBSTATS = [ - 'dataSize', - 'indexSize', - 'storageSize', - 'objects' -] - -# charts order (can be overridden if you want less charts, or different order) -ORDER = [ - 'read_operations', - 'write_operations', - 'active_clients', - 'journaling_transactions', - 'journaling_volume', - 'background_flush_average', - 'background_flush_last', - 'background_flush_rate', - 'wiredtiger_read', - 'wiredtiger_write', - 'cursors', - 'connections', - 'memory', - 'page_faults', - 'queued_requests', - 'record_moves', - 'wiredtiger_cache', - 'wiredtiger_pages_evicted', - 'asserts', - 'locks_collection', - 'locks_database', - 'locks_global', - 'locks_metadata', - 'locks_oplog', - 'dbstats_objects', - 'tcmalloc_generic', - 'tcmalloc_metrics', - 'command_total_rate', - 'command_failed_rate' -] - -CHARTS = { - 'read_operations': { - 'options': [None, 'Received read requests', 'requests/s', 'throughput metrics', - 'mongodb.read_operations', 'line'], - 'lines': [ - ['query', None, 'incremental'], - ['getmore', None, 'incremental'] - ] - }, - 'write_operations': { - 'options': [None, 'Received write requests', 'requests/s', 'throughput metrics', - 'mongodb.write_operations', 'line'], - 'lines': [ - ['insert', None, 'incremental'], - ['update', None, 'incremental'], - ['delete', None, 'incremental'] - ] - }, - 'active_clients': { - 'options': [None, 'Clients with read or write operations in progress or queued', 'clients', - 'throughput metrics', 'mongodb.active_clients', 'line'], - 'lines': [ - ['activeClients_readers', 'readers', 'absolute'], - ['activeClients_writers', 'writers', 'absolute'] - ] - }, - 'journaling_transactions': { - 'options': [None, 'Transactions that have been written to the journal', 'commits', - 'database performance', 'mongodb.journaling_transactions', 'line'], - 'lines': [ - ['commits', None, 'absolute'] - ] - }, - 'journaling_volume': { - 'options': [None, 'Volume of data written to the journal', 'MiB', 'database performance', - 'mongodb.journaling_volume', 'line'], - 'lines': [ - ['journaledMB', 'volume', 'absolute', 1, 100] - ] - }, - 'background_flush_average': { - 'options': [None, 'Average time taken by flushes to execute', 'milliseconds', 'database performance', - 'mongodb.background_flush_average', 'line'], - 'lines': [ - ['average_ms', 'time', 'absolute', 1, 100] - ] - }, - 'background_flush_last': { - 'options': [None, 'Time taken by the last flush operation to execute', 'milliseconds', 'database performance', - 'mongodb.background_flush_last', 'line'], - 'lines': [ - ['last_ms', 'time', 'absolute', 1, 100] - ] - }, - 'background_flush_rate': { - 'options': [None, 'Flushes rate', 'flushes', 'database performance', 'mongodb.background_flush_rate', 'line'], - 'lines': [ - ['flushes', 'flushes', 'incremental', 1, 1] - ] - }, - 'wiredtiger_read': { - 'options': [None, 'Read tickets in use and remaining', 'tickets', 'database performance', - 'mongodb.wiredtiger_read', 'stacked'], - 'lines': [ - ['wiredTigerRead_available', 'available', 'absolute', 1, 1], - ['wiredTigerRead_out', 'inuse', 'absolute', 1, 1] - ] - }, - 'wiredtiger_write': { - 'options': [None, 'Write tickets in use and remaining', 'tickets', 'database performance', - 'mongodb.wiredtiger_write', 'stacked'], - 'lines': [ - ['wiredTigerWrite_available', 'available', 'absolute', 1, 1], - ['wiredTigerWrite_out', 'inuse', 'absolute', 1, 1] - ] - }, - 'cursors': { - 'options': [None, 'Currently opened cursors, cursors with timeout disabled and timed out cursors', - 'cursors', 'database performance', 'mongodb.cursors', 'stacked'], - 'lines': [ - ['cursor_total', 'opened', 'absolute', 1, 1], - ['noTimeout', None, 'absolute', 1, 1], - ['timedOut', None, 'incremental', 1, 1] - ] - }, - 'connections': { - 'options': [None, 'Currently connected clients and unused connections', 'connections', - 'resource utilization', 'mongodb.connections', 'stacked'], - 'lines': [ - ['connections_available', 'unused', 'absolute', 1, 1], - ['connections_current', 'connected', 'absolute', 1, 1] - ] - }, - 'memory': { - 'options': [None, 'Memory metrics', 'MiB', 'resource utilization', 'mongodb.memory', 'stacked'], - 'lines': [ - ['virtual', None, 'absolute', 1, 1], - ['resident', None, 'absolute', 1, 1], - ['nonmapped', None, 'absolute', 1, 1], - ['mapped', None, 'absolute', 1, 1] - ] - }, - 'page_faults': { - 'options': [None, 'Number of times MongoDB had to fetch data from disk', 'request/s', - 'resource utilization', 'mongodb.page_faults', 'line'], - 'lines': [ - ['page_faults', None, 'incremental', 1, 1] - ] - }, - 'queued_requests': { - 'options': [None, 'Currently queued read and write requests', 'requests', 'resource saturation', - 'mongodb.queued_requests', 'line'], - 'lines': [ - ['currentQueue_readers', 'readers', 'absolute', 1, 1], - ['currentQueue_writers', 'writers', 'absolute', 1, 1] - ] - }, - 'record_moves': { - 'options': [None, 'Number of times documents had to be moved on-disk', 'number', - 'resource saturation', 'mongodb.record_moves', 'line'], - 'lines': [ - ['moves', None, 'incremental', 1, 1] - ] - }, - 'asserts': { - 'options': [ - None, - 'Number of message, warning, regular, corresponding to errors generated by users assertions raised', - 'number', 'errors (asserts)', 'mongodb.asserts', 'line'], - 'lines': [ - ['msg', None, 'incremental', 1, 1], - ['warning', None, 'incremental', 1, 1], - ['regular', None, 'incremental', 1, 1], - ['user', None, 'incremental', 1, 1] - ] - }, - 'wiredtiger_cache': { - 'options': [None, 'The percentage of the wiredTiger cache that is in use and cache with dirty bytes', - 'percentage', 'resource utilization', 'mongodb.wiredtiger_cache', 'stacked'], - 'lines': [ - ['wiredTiger_percent_clean', 'inuse', 'absolute', 1, 1000], - ['wiredTiger_percent_dirty', 'dirty', 'absolute', 1, 1000] - ] - }, - 'wiredtiger_pages_evicted': { - 'options': [None, 'Pages evicted from the cache', - 'pages', 'resource utilization', 'mongodb.wiredtiger_pages_evicted', 'stacked'], - 'lines': [ - ['unmodified', None, 'absolute', 1, 1], - ['modified', None, 'absolute', 1, 1] - ] - }, - 'dbstats_objects': { - 'options': [None, 'Number of documents in the database among all the collections', 'documents', - 'storage size metrics', 'mongodb.dbstats_objects', 'stacked'], - 'lines': [] - }, - 'tcmalloc_generic': { - 'options': [None, 'Tcmalloc generic metrics', 'MiB', 'tcmalloc', 'mongodb.tcmalloc_generic', 'stacked'], - 'lines': [ - ['current_allocated_bytes', 'allocated', 'absolute', 1, 1 << 20], - ['heap_size', 'heap_size', 'absolute', 1, 1 << 20] - ] - }, - 'tcmalloc_metrics': { - 'options': [None, 'Tcmalloc metrics', 'KiB', 'tcmalloc', 'mongodb.tcmalloc_metrics', 'stacked'], - 'lines': [ - ['central_cache_free_bytes', 'central_cache_free', 'absolute', 1, 1024], - ['current_total_thread_cache_bytes', 'current_total_thread_cache', 'absolute', 1, 1024], - ['pageheap_free_bytes', 'pageheap_free', 'absolute', 1, 1024], - ['pageheap_unmapped_bytes', 'pageheap_unmapped', 'absolute', 1, 1024], - ['thread_cache_free_bytes', 'thread_cache_free', 'absolute', 1, 1024], - ['transfer_cache_free_bytes', 'transfer_cache_free', 'absolute', 1, 1024] - ] - }, - 'command_total_rate': { - 'options': [None, 'Commands total rate', 'commands/s', 'commands', 'mongodb.command_total_rate', 'stacked'], - 'lines': [ - ['count_total', 'count', 'incremental', 1, 1], - ['createIndexes_total', 'createIndexes', 'incremental', 1, 1], - ['delete_total', 'delete', 'incremental', 1, 1], - ['eval_total', 'eval', 'incremental', 1, 1], - ['findAndModify_total', 'findAndModify', 'incremental', 1, 1], - ['insert_total', 'insert', 'incremental', 1, 1], - ['update_total', 'update', 'incremental', 1, 1] - ] - }, - 'command_failed_rate': { - 'options': [None, 'Commands failed rate', 'commands/s', 'commands', 'mongodb.command_failed_rate', 'stacked'], - 'lines': [ - ['count_failed', 'count', 'incremental', 1, 1], - ['createIndexes_failed', 'createIndexes', 'incremental', 1, 1], - ['delete_failed', 'delete', 'incremental', 1, 1], - ['eval_failed', 'eval', 'incremental', 1, 1], - ['findAndModify_failed', 'findAndModify', 'incremental', 1, 1], - ['insert_failed', 'insert', 'incremental', 1, 1], - ['update_failed', 'update', 'incremental', 1, 1] - ] - }, - 'locks_collection': { - 'options': [None, 'Collection lock. Number of times the lock was acquired in the specified mode', - 'locks', 'locks metrics', 'mongodb.locks_collection', 'stacked'], - 'lines': [ - ['Collection_R', 'shared', 'incremental'], - ['Collection_W', 'exclusive', 'incremental'], - ['Collection_r', 'intent_shared', 'incremental'], - ['Collection_w', 'intent_exclusive', 'incremental'] - ] - }, - 'locks_database': { - 'options': [None, 'Database lock. Number of times the lock was acquired in the specified mode', - 'locks', 'locks metrics', 'mongodb.locks_database', 'stacked'], - 'lines': [ - ['Database_R', 'shared', 'incremental'], - ['Database_W', 'exclusive', 'incremental'], - ['Database_r', 'intent_shared', 'incremental'], - ['Database_w', 'intent_exclusive', 'incremental'] - ] - }, - 'locks_global': { - 'options': [None, 'Global lock. Number of times the lock was acquired in the specified mode', - 'locks', 'locks metrics', 'mongodb.locks_global', 'stacked'], - 'lines': [ - ['Global_R', 'shared', 'incremental'], - ['Global_W', 'exclusive', 'incremental'], - ['Global_r', 'intent_shared', 'incremental'], - ['Global_w', 'intent_exclusive', 'incremental'] - ] - }, - 'locks_metadata': { - 'options': [None, 'Metadata lock. Number of times the lock was acquired in the specified mode', - 'locks', 'locks metrics', 'mongodb.locks_metadata', 'stacked'], - 'lines': [ - ['Metadata_R', 'shared', 'incremental'], - ['Metadata_w', 'intent_exclusive', 'incremental'] - ] - }, - 'locks_oplog': { - 'options': [None, 'Lock on the oplog. Number of times the lock was acquired in the specified mode', - 'locks', 'locks metrics', 'mongodb.locks_oplog', 'stacked'], - 'lines': [ - ['oplog_r', 'intent_shared', 'incremental'], - ['oplog_w', 'intent_exclusive', 'incremental'] - ] - } -} - -DEFAULT_HOST = '127.0.0.1' -DEFAULT_PORT = 27017 -DEFAULT_TIMEOUT = 100 -DEFAULT_AUTHDB = 'admin' - -CONN_PARAM_HOST = 'host' -CONN_PARAM_PORT = 'port' -CONN_PARAM_SERVER_SELECTION_TIMEOUT_MS = 'serverselectiontimeoutms' -CONN_PARAM_SSL_SSL = 'ssl' -CONN_PARAM_SSL_CERT_REQS = 'ssl_cert_reqs' -CONN_PARAM_SSL_CA_CERTS = 'ssl_ca_certs' -CONN_PARAM_SSL_CRL_FILE = 'ssl_crlfile' -CONN_PARAM_SSL_CERT_FILE = 'ssl_certfile' -CONN_PARAM_SSL_KEY_FILE = 'ssl_keyfile' -CONN_PARAM_SSL_PEM_PASSPHRASE = 'ssl_pem_passphrase' - - -class Service(SimpleService): - def __init__(self, configuration=None, name=None): - SimpleService.__init__(self, configuration=configuration, name=name) - self.order = ORDER[:] - self.definitions = deepcopy(CHARTS) - self.authdb = self.configuration.get('authdb', DEFAULT_AUTHDB) - self.user = self.configuration.get('user') - self.password = self.configuration.get('pass') - self.metrics_to_collect = deepcopy(DEFAULT_METRICS) - self.connection = None - self.do_replica = None - self.databases = list() - - def check(self): - if not PYMONGO: - self.error('Pymongo package v2.4+ is needed to use mongodb.chart.py') - return False - self.connection, server_status, error = self._create_connection() - if error: - self.error(error) - return False - - self.build_metrics_to_collect_(server_status) - - try: - data = self._get_data() - except (LookupError, SyntaxError, AttributeError): - self.error('Type: %s, error: %s' % (str(exc_info()[0]), str(exc_info()[1]))) - return False - if isinstance(data, dict) and data: - self._data_from_check = data - self.create_charts_(server_status) - return True - self.error('_get_data() returned no data or type is not <dict>') - return False - - def build_metrics_to_collect_(self, server_status): - - self.do_replica = 'repl' in server_status - if 'dur' in server_status: - self.metrics_to_collect.extend(DUR) - if 'tcmalloc' in server_status: - self.metrics_to_collect.extend(TCMALLOC) - if 'commands' in server_status['metrics']: - self.metrics_to_collect.extend(COMMANDS) - if 'wiredTiger' in server_status: - self.metrics_to_collect.extend(WIREDTIGER) - has_locks = 'locks' in server_status - if has_locks and 'Collection' in server_status['locks']: - self.metrics_to_collect.extend(LOCKS) - - def create_charts_(self, server_status): - - if 'dur' not in server_status: - self.order.remove('journaling_transactions') - self.order.remove('journaling_volume') - - if 'backgroundFlushing' not in server_status: - self.order.remove('background_flush_average') - self.order.remove('background_flush_last') - self.order.remove('background_flush_rate') - - if 'wiredTiger' not in server_status: - self.order.remove('wiredtiger_write') - self.order.remove('wiredtiger_read') - self.order.remove('wiredtiger_cache') - - if 'tcmalloc' not in server_status: - self.order.remove('tcmalloc_generic') - self.order.remove('tcmalloc_metrics') - - if 'commands' not in server_status['metrics']: - self.order.remove('command_total_rate') - self.order.remove('command_failed_rate') - - has_no_locks = 'locks' not in server_status - if has_no_locks or 'Collection' not in server_status['locks']: - self.order.remove('locks_collection') - self.order.remove('locks_database') - self.order.remove('locks_global') - self.order.remove('locks_metadata') - - if has_no_locks or 'oplog' not in server_status['locks']: - self.order.remove('locks_oplog') - - for dbase in self.databases: - self.order.append('_'.join([dbase, 'dbstats'])) - self.definitions['_'.join([dbase, 'dbstats'])] = { - 'options': [None, '%s: size of all documents, indexes, extents' % dbase, 'KB', - 'storage size metrics', 'mongodb.dbstats', 'line'], - 'lines': [ - ['_'.join([dbase, 'dataSize']), 'documents', 'absolute', 1, 1024], - ['_'.join([dbase, 'indexSize']), 'indexes', 'absolute', 1, 1024], - ['_'.join([dbase, 'storageSize']), 'extents', 'absolute', 1, 1024] - ]} - self.definitions['dbstats_objects']['lines'].append(['_'.join([dbase, 'objects']), dbase, 'absolute']) - - if self.do_replica: - def create_lines(hosts, string): - lines = list() - for host in hosts: - dim_id = '_'.join([host, string]) - lines.append([dim_id, host, 'absolute', 1, 1000]) - return lines - - def create_state_lines(states): - lines = list() - for state, description in states: - dim_id = '_'.join([host, 'state', state]) - lines.append([dim_id, description, 'absolute', 1, 1]) - return lines - - all_hosts = server_status['repl']['hosts'] + server_status['repl'].get('arbiters', list()) - this_host = server_status['repl']['me'] - other_hosts = [host for host in all_hosts if host != this_host] - - if 'local' in self.databases: - self.order.append('oplog_window') - self.definitions['oplog_window'] = { - 'options': [None, 'Interval of time between the oldest and the latest entries in the oplog', - 'seconds', 'replication and oplog', 'mongodb.oplog_window', 'line'], - 'lines': [['timeDiff', 'window', 'absolute', 1, 1000]]} - # Create "heartbeat delay" chart - self.order.append('heartbeat_delay') - self.definitions['heartbeat_delay'] = { - 'options': [ - None, - 'Time when last heartbeat was received from the replica set member (lastHeartbeatRecv)', - 'seconds ago', 'replication and oplog', 'mongodb.replication_heartbeat_delay', 'stacked'], - 'lines': create_lines(other_hosts, 'heartbeat_lag')} - # Create "optimedate delay" chart - self.order.append('optimedate_delay') - self.definitions['optimedate_delay'] = { - 'options': [None, 'Time when last entry from the oplog was applied (optimeDate)', - 'seconds ago', 'replication and oplog', 'mongodb.replication_optimedate_delay', 'stacked'], - 'lines': create_lines(all_hosts, 'optimedate')} - # Create "replica set members state" chart - for host in all_hosts: - chart_name = '_'.join([host, 'state']) - self.order.append(chart_name) - self.definitions[chart_name] = { - 'options': [None, 'Replica set member (%s) current state' % host, 'state', - 'replication and oplog', 'mongodb.replication_state', 'line'], - 'lines': create_state_lines(REPL_SET_STATES)} - - def _get_raw_data(self): - raw_data = dict() - - raw_data.update(self.get_server_status() or dict()) - raw_data.update(self.get_db_stats() or dict()) - raw_data.update(self.get_repl_set_get_status() or dict()) - raw_data.update(self.get_get_replication_info() or dict()) - - return raw_data or None - - def get_server_status(self): - raw_data = dict() - try: - raw_data['serverStatus'] = self.connection.admin.command('serverStatus') - except PyMongoError: - return None - else: - return raw_data - - def get_db_stats(self): - if not self.databases: - return None - - raw_data = dict() - raw_data['dbStats'] = dict() - try: - for dbase in self.databases: - raw_data['dbStats'][dbase] = self.connection[dbase].command('dbStats') - return raw_data - except PyMongoError: - return None - - def get_repl_set_get_status(self): - if not self.do_replica: - return None - - raw_data = dict() - try: - raw_data['replSetGetStatus'] = self.connection.admin.command('replSetGetStatus') - return raw_data - except PyMongoError: - return None - - def get_get_replication_info(self): - if not (self.do_replica and 'local' in self.databases): - return None - - raw_data = dict() - raw_data['getReplicationInfo'] = dict() - try: - raw_data['getReplicationInfo']['ASCENDING'] = self.connection.local.oplog.rs.find().sort( - '$natural', ASCENDING).limit(1)[0] - raw_data['getReplicationInfo']['DESCENDING'] = self.connection.local.oplog.rs.find().sort( - '$natural', DESCENDING).limit(1)[0] - return raw_data - except PyMongoError: - return None - - def _get_data(self): - """ - :return: dict - """ - raw_data = self._get_raw_data() - - if not raw_data: - return None - - data = dict() - serverStatus = raw_data['serverStatus'] - dbStats = raw_data.get('dbStats') - replSetGetStatus = raw_data.get('replSetGetStatus') - getReplicationInfo = raw_data.get('getReplicationInfo') - utc_now = datetime.utcnow() - - # serverStatus - for metric, new_name, func in self.metrics_to_collect: - value = serverStatus - for key in metric.split('.'): - try: - value = value[key] - except KeyError: - break - - if not isinstance(value, dict) and key: - data[new_name or key] = value if not func else func(value) - - if 'mapped' in serverStatus['mem']: - data['nonmapped'] = data['virtual'] - serverStatus['mem'].get('mappedWithJournal', data['mapped']) - - if data.get('maximum bytes configured'): - maximum = data['maximum bytes configured'] - data['wiredTiger_percent_clean'] = int(data['bytes currently in the cache'] * 100 / maximum * 1000) - data['wiredTiger_percent_dirty'] = int(data['tracked dirty bytes in the cache'] * 100 / maximum * 1000) - - # dbStats - if dbStats: - for dbase in dbStats: - for metric in DBSTATS: - key = '_'.join([dbase, metric]) - data[key] = dbStats[dbase][metric] - - # replSetGetStatus - if replSetGetStatus: - other_hosts = list() - members = replSetGetStatus['members'] - unix_epoch = datetime(1970, 1, 1, 0, 0) - - for member in members: - if not member.get('self'): - other_hosts.append(member) - - # Replica set time diff between current time and time when last entry from the oplog was applied - if member.get('optimeDate', unix_epoch) != unix_epoch: - member_optimedate = member['name'] + '_optimedate' - delta = utc_now - member['optimeDate'] - data[member_optimedate] = int(delta_calculation(delta=delta, multiplier=1000)) - - # Replica set members state - member_state = member['name'] + '_state' - for elem in REPL_SET_STATES: - state = elem[0] - data.update({'_'.join([member_state, state]): 0}) - data.update({'_'.join([member_state, str(member['state'])]): member['state']}) - - # Heartbeat lag calculation - for other in other_hosts: - if other['lastHeartbeatRecv'] != unix_epoch: - node = other['name'] + '_heartbeat_lag' - delta = utc_now - other['lastHeartbeatRecv'] - data[node] = int(delta_calculation(delta=delta, multiplier=1000)) - - if getReplicationInfo: - first_event = getReplicationInfo['ASCENDING']['ts'].as_datetime() - last_event = getReplicationInfo['DESCENDING']['ts'].as_datetime() - data['timeDiff'] = int(delta_calculation(delta=last_event - first_event, multiplier=1000)) - - return data - - def build_ssl_connection_params(self): - conf = self.configuration - - def cert_req(v): - if v is None: - return None - if not v: - return ssl.CERT_NONE - return ssl.CERT_REQUIRED - - ssl_params = { - CONN_PARAM_SSL_SSL: conf.get(CONN_PARAM_SSL_SSL), - CONN_PARAM_SSL_CERT_REQS: cert_req(conf.get(CONN_PARAM_SSL_CERT_REQS)), - CONN_PARAM_SSL_CA_CERTS: conf.get(CONN_PARAM_SSL_CA_CERTS), - CONN_PARAM_SSL_CRL_FILE: conf.get(CONN_PARAM_SSL_CRL_FILE), - CONN_PARAM_SSL_CERT_FILE: conf.get(CONN_PARAM_SSL_CERT_FILE), - CONN_PARAM_SSL_KEY_FILE: conf.get(CONN_PARAM_SSL_KEY_FILE), - CONN_PARAM_SSL_PEM_PASSPHRASE: conf.get(CONN_PARAM_SSL_PEM_PASSPHRASE), - } - - ssl_params = dict((k, v) for k, v in ssl_params.items() if v is not None) - - return ssl_params - - def build_connection_params(self): - conf = self.configuration - params = { - CONN_PARAM_HOST: conf.get(CONN_PARAM_HOST, DEFAULT_HOST), - CONN_PARAM_PORT: conf.get(CONN_PARAM_PORT, DEFAULT_PORT), - } - if hasattr(MongoClient, 'server_selection_timeout') or version_tuple[0] >= 4: - params[CONN_PARAM_SERVER_SELECTION_TIMEOUT_MS] = conf.get('timeout', DEFAULT_TIMEOUT) - - params.update(self.build_ssl_connection_params()) - return params - - def _create_connection(self): - params = self.build_connection_params() - self.debug('creating connection, connection params: {0}'.format(sorted(params))) - - try: - connection = MongoClient(**params) - if self.user and self.password: - self.debug('authenticating, user: {0}, password: {1}'.format(self.user, self.password)) - getattr(connection, self.authdb).authenticate(name=self.user, password=self.password) - else: - self.debug('skip authenticating, user and password are not set') - # elif self.user: - # connection.admin.authenticate(name=self.user, mechanism='MONGODB-X509') - server_status = connection.admin.command('serverStatus') - except PyMongoError as error: - return None, None, str(error) - else: - try: - self.databases = connection.database_names() - except PyMongoError as error: - self.info('Can\'t collect databases: %s' % str(error)) - return connection, server_status, None - - -def delta_calculation(delta, multiplier=1): - if hasattr(delta, 'total_seconds'): - return delta.total_seconds() * multiplier - return (delta.microseconds + (delta.seconds + delta.days * 24 * 3600) * 10 ** 6) / 10.0 ** 6 * multiplier diff --git a/collectors/python.d.plugin/mongodb/mongodb.conf b/collectors/python.d.plugin/mongodb/mongodb.conf deleted file mode 100644 index 9f660f594..000000000 --- a/collectors/python.d.plugin/mongodb/mongodb.conf +++ /dev/null @@ -1,102 +0,0 @@ -# netdata python.d.plugin configuration for mongodb -# -# This file is in YaML format. Generally the format is: -# -# name: value -# -# There are 2 sections: -# - global variables -# - one or more JOBS -# -# JOBS allow you to collect values from multiple sources. -# Each source will have its own set of charts. -# -# JOB parameters have to be indented (using spaces only, example below). - -# ---------------------------------------------------------------------- -# Global Variables -# These variables set the defaults for all JOBs, however each JOB -# may define its own, overriding the defaults. - -# update_every sets the default data collection frequency. -# If unset, the python.d.plugin default is used. -# update_every: 1 - -# priority controls the order of charts at the netdata dashboard. -# Lower numbers move the charts towards the top of the page. -# If unset, the default for python.d.plugin is used. -# priority: 60000 - -# penalty indicates whether to apply penalty to update_every in case of failures. -# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes. -# penalty: yes - -# autodetection_retry sets the job re-check interval in seconds. -# The job is not deleted if check fails. -# Attempts to start the job are made once every autodetection_retry. -# This feature is disabled by default. -# autodetection_retry: 0 - -# ---------------------------------------------------------------------- -# JOBS (data collection sources) -# -# The default JOBS share the same *name*. JOBS with the same name -# are mutually exclusive. Only one of them will be allowed running at -# any time. This allows autodetection to try several alternatives and -# pick the one that works. -# -# Any number of jobs is supported. -# -# All python.d.plugin JOBS (for all its modules) support a set of -# predefined parameters. These are: -# -# job_name: -# name: myname # the JOB's name as it will appear at the -# # dashboard (by default is the job_name) -# # JOBs sharing a name are mutually exclusive -# update_every: 1 # the JOB's data collection frequency -# priority: 60000 # the JOB's order on the dashboard -# penalty: yes # the JOB's penalty -# autodetection_retry: 0 # the JOB's re-check interval in seconds -# -# Additionally to the above, mongodb also supports the following: -# -# host: 'IP or HOSTNAME' # type <str> the host to connect to -# port: PORT # type <int> the port to connect to -# -# in all cases, the following can also be set: -# -# authdb: 'dbname' # database to authenticate the user against, -# # defaults to "admin". -# user: 'username' # the mongodb username to use -# pass: 'password' # the mongodb password to use -# -# SSL connection parameters (https://api.mongodb.com/python/current/examples/tls.html): -# -# ssl: yes # connect to the server using TLS -# ssl_cert_reqs: yes # require a certificate from the server when TLS is enabled -# ssl_ca_certs: '/path/to/ca.pem' # use a specific set of CA certificates -# ssl_crlfile: '/path/to/crl.pem' # use a certificate revocation lists -# ssl_certfile: '/path/to/client.pem' # use a client certificate -# ssl_keyfile: '/path/to/key.pem' # use a specific client certificate key -# ssl_pem_passphrase: 'passphrase' # use a passphrase to decrypt encrypted private keys -# - -# ---------------------------------------------------------------------- -# to connect to the mongodb on localhost, without a password: -# ---------------------------------------------------------------------- -# AUTO-DETECTION JOBS -# only one of them will run (they have the same name) - -local: - name : 'local' - host : '127.0.0.1' - port : 27017 - -# authsample: -# name : 'secure' -# host : 'mongodb.example.com' -# port : 27017 -# authdb : 'admin' -# user : 'monitor' -# pass : 'supersecret' diff --git a/collectors/python.d.plugin/monit/README.md b/collectors/python.d.plugin/monit/README.md index 13960256b..816143ebf 100644 --- a/collectors/python.d.plugin/monit/README.md +++ b/collectors/python.d.plugin/monit/README.md @@ -1,34 +1,40 @@ <!-- title: "Monit monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/monit/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/monit/README.md" sidebar_label: "Monit" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Storage" --> # Monit monitoring with Netdata -Monit monitoring module. Data is grabbed from stats XML interface (exists for a long time, but not mentioned in official documentation). Mostly this plugin shows statuses of monit targets, i.e. [statuses of specified checks](https://mmonit.com/monit/documentation/monit.html#Service-checks). +Monit monitoring module. Data is grabbed from stats XML interface (exists for a long time, but not mentioned in official +documentation). Mostly this plugin shows statuses of monit targets, i.e. +[statuses of specified checks](https://mmonit.com/monit/documentation/monit.html#Service-checks). -1. **Filesystems** +1. **Filesystems** - - Filesystems - - Directories - - Files - - Pipes + - Filesystems + - Directories + - Files + - Pipes -2. **Applications** +2. **Applications** - - Processes (+threads/childs) - - Programs + - Processes (+threads/childs) + - Programs -3. **Network** +3. **Network** - - Hosts (+latency) - - Network interfaces + - Hosts (+latency) + - Network interfaces ## Configuration -Edit the `python.d/monit.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Edit the `python.d/monit.conf` configuration file using `edit-config` from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically +at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -39,10 +45,10 @@ Sample: ```yaml local: - name : 'local' - url : 'http://localhost:2812' - user: : admin - pass: : monit + name: 'local' + url: 'http://localhost:2812' + user: : admin + pass: : monit ``` If no configuration is given, module will attempt to connect to monit as `http://localhost:2812`. diff --git a/collectors/python.d.plugin/nsd/README.md b/collectors/python.d.plugin/nsd/README.md index e5183aeb7..f99726c30 100644 --- a/collectors/python.d.plugin/nsd/README.md +++ b/collectors/python.d.plugin/nsd/README.md @@ -1,7 +1,10 @@ <!-- title: "NSD monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/nsd/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/nsd/README.md" sidebar_label: "NSD" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # NSD monitoring with Netdata diff --git a/collectors/python.d.plugin/ntpd/README.md b/collectors/python.d.plugin/ntpd/README.md index 9832707bd..8ae923da5 100644 --- a/collectors/python.d.plugin/ntpd/README.md +++ b/collectors/python.d.plugin/ntpd/README.md @@ -1,90 +1,14 @@ <!-- title: "NTP daemon monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ntpd/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/ntpd/README.md" sidebar_label: "NTP daemon" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Qos" --> # NTP daemon monitoring with Netdata -Monitors the system variables of the local `ntpd` daemon (optional incl. variables of the polled peers) using the NTP Control Message Protocol via UDP socket, similar to `ntpq`, the [standard NTP query program](http://doc.ntp.org/current-stable/ntpq.html). - -## Requirements - -- Version: `NTPv4` -- Local interrogation allowed in `/etc/ntp.conf` (default): - -``` -# Local users may interrogate the ntp server more closely. -restrict 127.0.0.1 -restrict ::1 -``` - -It produces: - -1. system - - - offset - - jitter - - frequency - - delay - - dispersion - - stratum - - tc - - precision - -2. peers - - - offset - - delay - - dispersion - - jitter - - rootdelay - - rootdispersion - - stratum - - hmode - - pmode - - hpoll - - ppoll - - precision - -## Configuration - -Edit the `python.d/ntpd.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/ntpd.conf -``` - -Sample: - -```yaml -update_every: 10 - -host: 'localhost' -port: '123' -show_peers: yes -# hide peers with source address in ranges 127.0.0.0/8 and 192.168.0.0/16 -peer_filter: '(127\..*)|(192\.168\..*)' -# check for new/changed peers every 60 updates -peer_rescan: 60 -``` - -Sample (multiple jobs): - -Note: `ntp.conf` on the host `otherhost` must be configured to allow queries from our local host by including a line like `restrict <IP> nomodify notrap nopeer`. - -```yaml -local: - host: 'localhost' - -otherhost: - host: 'otherhost' -``` - -If no configuration is given, module will attempt to connect to `ntpd` on `::1:123` or `127.0.0.1:123` and show charts for the systemvars. Use `show_peers: yes` to also show the charts for configured peers. Local peers in the range `127.0.0.0/8` are hidden by default, use `peer_filter: ''` to show all peers. - ---- - - +This collector is deprecated. +Use [go.d/ntpd](https://github.com/netdata/go.d.plugin/tree/master/modules/ntpd#ntp-daemon-monitoring-with-netdata) +instead.
\ No newline at end of file diff --git a/collectors/python.d.plugin/ntpd/ntpd.chart.py b/collectors/python.d.plugin/ntpd/ntpd.chart.py index 275d2276c..077124b4f 100644 --- a/collectors/python.d.plugin/ntpd/ntpd.chart.py +++ b/collectors/python.d.plugin/ntpd/ntpd.chart.py @@ -9,6 +9,8 @@ import struct from bases.FrameworkServices.SocketService import SocketService +disabled_by_default = True + # NTP Control Message Protocol constants MODE = 6 HEADER_FORMAT = '!BBHHHHH' diff --git a/collectors/python.d.plugin/nvidia_smi/README.md b/collectors/python.d.plugin/nvidia_smi/README.md index bb4169441..ce5473c26 100644 --- a/collectors/python.d.plugin/nvidia_smi/README.md +++ b/collectors/python.d.plugin/nvidia_smi/README.md @@ -1,14 +1,17 @@ <!-- title: "Nvidia GPU monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/nvidia_smi/README.md -sidebar_label: "Nvidia GPUs" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/nvidia_smi/README.md" +sidebar_label: "nvidia_smi-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # Nvidia GPU monitoring with Netdata Monitors performance metrics (memory usage, fan speed, pcie bandwidth utilization, temperature, etc.) using `nvidia-smi` cli tool. -> **Warning**: this collector does not work when the Netdata Agent is [running in a container](https://learn.netdata.cloud/docs/agent/packaging/docker). +> **Warning**: this collector does not work when the Netdata Agent is [running in a container](https://github.com/netdata/netdata/blob/master/packaging/docker/README.md). ## Requirements and Notes @@ -48,7 +51,7 @@ It produces the following charts: ## Configuration Edit the `python.d/nvidia_smi.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/nvidia_smi/nvidia_smi.chart.py b/collectors/python.d.plugin/nvidia_smi/nvidia_smi.chart.py index 23e90e658..6affae7b8 100644 --- a/collectors/python.d.plugin/nvidia_smi/nvidia_smi.chart.py +++ b/collectors/python.d.plugin/nvidia_smi/nvidia_smi.chart.py @@ -22,6 +22,7 @@ EMPTY_ROW_LIMIT = 500 POLLER_BREAK_ROW = '</nvidia_smi_log>' PCI_BANDWIDTH = 'pci_bandwidth' +PCI_BANDWIDTH_PERCENT = 'pci_bandwidth_percent' FAN_SPEED = 'fan_speed' GPU_UTIL = 'gpu_utilization' MEM_UTIL = 'mem_utilization' @@ -38,6 +39,7 @@ USER_NUM = 'user_num' ORDER = [ PCI_BANDWIDTH, + PCI_BANDWIDTH_PERCENT, FAN_SPEED, GPU_UTIL, MEM_UTIL, @@ -56,7 +58,22 @@ ORDER = [ # https://docs.nvidia.com/gameworks/content/gameworkslibrary/coresdk/nvapi/group__gpupstate.html POWER_STATES = ['P' + str(i) for i in range(0, 16)] - +# PCI Transfer data rate in gigabits per second (Gb/s) per generation +PCI_SPEED = { + "1": 2.5, + "2": 5, + "3": 8, + "4": 16, + "5": 32 +} +# PCI encoding per generation +PCI_ENCODING = { + "1": 2/10, + "2": 2/10, + "3": 2/130, + "4": 2/130, + "5": 2/130 +} def gpu_charts(gpu): fam = gpu.full_name() @@ -68,6 +85,13 @@ def gpu_charts(gpu): ['tx_util', 'tx', 'absolute', 1, -1], ] }, + PCI_BANDWIDTH_PERCENT: { + 'options': [None, 'PCI Express Bandwidth Percent', 'percentage', fam, 'nvidia_smi.pci_bandwidth_percent', 'area'], + 'lines': [ + ['rx_util_percent', 'rx_percent'], + ['tx_util_percent', 'tx_percent'], + ] + }, FAN_SPEED: { 'options': [None, 'Fan Speed', 'percentage', fam, 'nvidia_smi.fan_speed', 'line'], 'lines': [ @@ -327,6 +351,24 @@ class GPU: return 'gpu{0} {1}'.format(self.num, self.name()) @handle_attr_error + def pci_link_gen(self): + return self.root.find('pci').find('pci_gpu_link_info').find('pcie_gen').find('max_link_gen').text + + @handle_attr_error + def pci_link_width(self): + return self.root.find('pci').find('pci_gpu_link_info').find('link_widths').find('max_link_width').text.split('x')[0] + + def pci_bw_max(self): + link_gen = self.pci_link_gen() + link_width = int(self.pci_link_width()) + if link_gen not in PCI_SPEED or link_gen not in PCI_ENCODING or not link_width: + return None + # Maximum PCIe Bandwidth = SPEED * WIDTH * (1 - ENCODING) - 1Gb/s. + # see details https://enterprise-support.nvidia.com/s/article/understanding-pcie-configuration-for-maximum-performance + # return max bandwidth in kilobytes per second (kB/s) + return (PCI_SPEED[link_gen] * link_width * (1- PCI_ENCODING[link_gen]) - 1) * 1000 * 1000 / 8 + + @handle_attr_error def rx_util(self): return self.root.find('pci').find('rx_util').text.split()[0] @@ -439,6 +481,15 @@ class GPU: 'power_draw': self.power_draw(), } + pci_bw_max = self.pci_bw_max() + if not pci_bw_max: + data['rx_util_percent'] = 0 + data['tx_util_percent'] = 0 + else : + data['rx_util_percent'] = str(int(int(self.rx_util())*100/self.pci_bw_max())) + data['tx_util_percent'] = str(int(int(self.tx_util())*100/self.pci_bw_max())) + + for v in POWER_STATES: data['power_state_' + v.lower()] = 0 p_state = self.power_state() diff --git a/collectors/python.d.plugin/openldap/README.md b/collectors/python.d.plugin/openldap/README.md index b0cd1db42..4f29bbb49 100644 --- a/collectors/python.d.plugin/openldap/README.md +++ b/collectors/python.d.plugin/openldap/README.md @@ -1,7 +1,10 @@ <!-- title: "OpenLDAP monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/openldap/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/openldap/README.md" sidebar_label: "OpenLDAP" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # OpenLDAP monitoring with Netdata @@ -56,7 +59,7 @@ Statistics are taken from LDAP monitoring interface. Manual page, slapd-monitor( ## Configuration Edit the `python.d/openldap.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/oracledb/README.md b/collectors/python.d.plugin/oracledb/README.md index 88024f8c5..78f807d61 100644 --- a/collectors/python.d.plugin/oracledb/README.md +++ b/collectors/python.d.plugin/oracledb/README.md @@ -1,7 +1,10 @@ <!-- title: "OracleDB monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/oracledb/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/oracledb/README.md" sidebar_label: "OracleDB" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Databases" --> # OracleDB monitoring with Netdata @@ -71,7 +74,7 @@ GRANT SELECT_CATALOG_ROLE TO netdata; ## Configuration Edit the `python.d/oracledb.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/postfix/README.md b/collectors/python.d.plugin/postfix/README.md index 1a546c614..8d646ad51 100644 --- a/collectors/python.d.plugin/postfix/README.md +++ b/collectors/python.d.plugin/postfix/README.md @@ -1,7 +1,10 @@ <!-- title: "Postfix monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/postfix/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/postfix/README.md" sidebar_label: "Postfix" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Postfix monitoring with Netdata diff --git a/collectors/python.d.plugin/proxysql/README.md b/collectors/python.d.plugin/proxysql/README.md index 8c6a394f1..d6c626b51 100644 --- a/collectors/python.d.plugin/proxysql/README.md +++ b/collectors/python.d.plugin/proxysql/README.md @@ -1,106 +1,14 @@ <!-- title: "ProxySQL monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/proxysql/README.md -sidebar_label: "ProxySQL" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/proxysql/README.md" +sidebar_label: "proxysql-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Databases" --> # ProxySQL monitoring with Netdata -Monitors database backend and frontend performance metrics. - -## Requirements - -- python library [MySQLdb](https://github.com/PyMySQL/mysqlclient-python) (faster) or [PyMySQL](https://github.com/PyMySQL/PyMySQL) (slower) -- `netdata` local user to connect to the ProxySQL server. - -To create the `netdata` user, follow [the documentation](https://github.com/sysown/proxysql/wiki/Users-configuration#creating-a-new-user). - -## Charts - -It produces: - -1. **Connections (frontend)** - - - connected: number of frontend connections currently connected - - aborted: number of frontend connections aborted due to invalid credential or max_connections reached - - non_idle: number of frontend connections that are not currently idle - - created: number of frontend connections created - -2. **Questions (frontend)** - - - questions: total number of queries sent from frontends - - slow_queries: number of queries that ran for longer than the threshold in milliseconds defined in global variable `mysql-long_query_time` - -3. **Overall Bandwidth (backends)** - - - in - - out - -4. **Status (backends)** - - - Backends - - `1=ONLINE`: backend server is fully operational - - `2=SHUNNED`: backend sever is temporarily taken out of use because of either too many connection errors in a time that was too short, or replication lag exceeded the allowed threshold - - `3=OFFLINE_SOFT`: when a server is put into OFFLINE_SOFT mode, new incoming connections aren't accepted anymore, while the existing connections are kept until they became inactive. In other words, connections are kept in use until the current transaction is completed. This allows to gracefully detach a backend - - `4=OFFLINE_HARD`: when a server is put into OFFLINE_HARD mode, the existing connections are dropped, while new incoming connections aren't accepted either. This is equivalent to deleting the server from a hostgroup, or temporarily taking it out of the hostgroup for maintenance work - - `-1`: Unknown status - -5. **Bandwidth (backends)** - - - Backends - - in - - out - -6. **Queries (backends)** - - - Backends - - queries - -7. **Latency (backends)** - - - Backends - - ping time - -8. **Pool connections (backends)** - - - Backends - - Used: The number of connections are currently used by ProxySQL for sending queries to the backend server. - - Free: The number of connections are currently free. - - Established/OK: The number of connections were established successfully. - - Error: The number of connections weren't established successfully. - -9. **Commands** - - - Commands - - Count - - Duration (Total duration for each command) - -10. **Commands Histogram** - - - Commands - - 100us, 500us, ..., 10s, inf: the total number of commands of the given type which executed within the specified time limit and the previous one. - -## Configuration - -Edit the `python.d/proxysql.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/proxysql.conf -``` - -```yaml -tcpipv4: - name : 'local' - user : 'stats' - pass : 'stats' - host : '127.0.0.1' - port : '6032' -``` - -If no configuration is given, module will fail to run. - ---- - - +This collector is deprecated. +Use [go.d/proxysql](https://github.com/netdata/go.d.plugin/tree/master/modules/proxysql#proxysql-monitoring-with-netdata) +instead.
\ No newline at end of file diff --git a/collectors/python.d.plugin/proxysql/proxysql.chart.py b/collectors/python.d.plugin/proxysql/proxysql.chart.py index 982c28ee7..7e06b7bdc 100644 --- a/collectors/python.d.plugin/proxysql/proxysql.chart.py +++ b/collectors/python.d.plugin/proxysql/proxysql.chart.py @@ -6,6 +6,8 @@ from bases.FrameworkServices.MySQLService import MySQLService +disabled_by_default = True + def query(table, *params): return 'SELECT {params} FROM {table}'.format(table=table, params=', '.join(params)) diff --git a/collectors/python.d.plugin/puppet/README.md b/collectors/python.d.plugin/puppet/README.md index 1b06d181b..8b98b8a2d 100644 --- a/collectors/python.d.plugin/puppet/README.md +++ b/collectors/python.d.plugin/puppet/README.md @@ -1,7 +1,10 @@ <!-- title: "Puppet monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/puppet/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/puppet/README.md" sidebar_label: "Puppet" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Provisioning tools" --> # Puppet monitoring with Netdata @@ -33,7 +36,7 @@ Following charts are drawn: ## Configuration Edit the `python.d/puppet.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/python.d.conf b/collectors/python.d.plugin/python.d.conf index 7b43ee205..41385dac6 100644 --- a/collectors/python.d.plugin/python.d.conf +++ b/collectors/python.d.plugin/python.d.conf @@ -34,7 +34,6 @@ gc_interval: 300 # boinc: yes # ceph: yes # changefinder: no -# dockerd: yes # dovecot: yes # this is just an example @@ -51,7 +50,6 @@ hpssa: no # icecast: yes # ipfs: yes # litespeed: yes -logind: no # megacli: yes # memcached: yes # mongodb: yes @@ -73,7 +71,6 @@ logind: no # sensors: yes # smartd_log: yes # spigotmc: yes -# springboot: yes # squid: yes # traefik: yes # tomcat: yes diff --git a/collectors/python.d.plugin/rabbitmq/README.md b/collectors/python.d.plugin/rabbitmq/README.md index 927adcc68..19df65694 100644 --- a/collectors/python.d.plugin/rabbitmq/README.md +++ b/collectors/python.d.plugin/rabbitmq/README.md @@ -1,7 +1,10 @@ <!-- title: "RabbitMQ monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/rabbitmq/README.md -sidebar_label: "RabbitMQ" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/rabbitmq/README.md" +sidebar_label: "rabbitmq-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Message brokers" --> # RabbitMQ monitoring with Netdata @@ -93,7 +96,7 @@ Per Vhost charts: ## Configuration Edit the `python.d/rabbitmq.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/rethinkdbs/README.md b/collectors/python.d.plugin/rethinkdbs/README.md index d3fa3553a..578c1c0b1 100644 --- a/collectors/python.d.plugin/rethinkdbs/README.md +++ b/collectors/python.d.plugin/rethinkdbs/README.md @@ -1,7 +1,10 @@ <!-- title: "RethinkDB monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/rethinkdbs/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/rethinkdbs/README.md" sidebar_label: "RethinkDB" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Databases" --> # RethinkDB monitoring with Netdata @@ -10,27 +13,28 @@ Collects database server and cluster statistics. Following charts are drawn: -1. **Connected Servers** +1. **Connected Servers** - - connected - - missing + - connected + - missing -2. **Active Clients** +2. **Active Clients** - - active + - active -3. **Queries** per second +3. **Queries** per second - - queries + - queries -4. **Documents** per second +4. **Documents** per second - - documents + - documents ## Configuration -Edit the `python.d/rethinkdbs.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Edit the `python.d/rethinkdbs.conf` configuration file using `edit-config` from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically +at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -39,11 +43,11 @@ sudo ./edit-config python.d/rethinkdbs.conf ```yaml localhost: - name : 'local' - host : '127.0.0.1' - port : 28015 - user : "user" - password : "pass" + name: 'local' + host: '127.0.0.1' + port: 28015 + user: "user" + password: "pass" ``` When no configuration file is found, module tries to connect to `127.0.0.1:28015`. diff --git a/collectors/python.d.plugin/retroshare/README.md b/collectors/python.d.plugin/retroshare/README.md index 297df9fca..142b7d5bf 100644 --- a/collectors/python.d.plugin/retroshare/README.md +++ b/collectors/python.d.plugin/retroshare/README.md @@ -1,7 +1,10 @@ <!-- title: "RetroShare monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/retroshare/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/retroshare/README.md" sidebar_label: "RetroShare" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apm" --> # RetroShare monitoring with Netdata @@ -22,7 +25,7 @@ This module produces the following charts: ## Configuration Edit the `python.d/retroshare.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/riakkv/README.md b/collectors/python.d.plugin/riakkv/README.md index fe62c6718..5e533a419 100644 --- a/collectors/python.d.plugin/riakkv/README.md +++ b/collectors/python.d.plugin/riakkv/README.md @@ -1,7 +1,10 @@ <!-- title: "Riak KV monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/riakkv/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/riakkv/README.md" sidebar_label: "Riak KV" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Databases" --> # Riak KV monitoring with Netdata @@ -103,7 +106,7 @@ listed ## Configuration Edit the `python.d/riakkv.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/samba/README.md b/collectors/python.d.plugin/samba/README.md index 767df12de..41ae1c5ba 100644 --- a/collectors/python.d.plugin/samba/README.md +++ b/collectors/python.d.plugin/samba/README.md @@ -1,7 +1,10 @@ <!-- title: "Samba monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/samba/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/samba/README.md" sidebar_label: "Samba" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apps" --> # Samba monitoring with Netdata @@ -95,7 +98,7 @@ systemctl restart netdata.service ## Enable the collector The `samba` collector is disabled by default. To enable it, use `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`, to edit the `python.d.conf` file. ```bash @@ -104,12 +107,12 @@ sudo ./edit-config python.d.conf ``` Change the value of the `samba` setting to `yes`. Save the file and restart the Netdata Agent with `sudo systemctl -restart netdata`, or the [appropriate method](/docs/configure/start-stop-restart.md) for your system. +restart netdata`, or the [appropriate method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system. ## Configuration Edit the `python.d/samba.conf` configuration file using `edit-config` from the -Netdata [config directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/sensors/README.md b/collectors/python.d.plugin/sensors/README.md index e791195d4..f5f435854 100644 --- a/collectors/python.d.plugin/sensors/README.md +++ b/collectors/python.d.plugin/sensors/README.md @@ -1,7 +1,10 @@ <!-- title: "Linux machine sensors monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/sensors/README.md -sidebar_label: "Linux machine sensors" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/sensors/README.md" +sidebar_label: "sensors-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # Linux machine sensors monitoring with Netdata @@ -13,7 +16,7 @@ Charts are created dynamically. ## Configuration Edit the `python.d/sensors.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -26,7 +29,7 @@ There have been reports from users that on certain servers, ACPI ring buffer err We are tracking such cases in issue [#827](https://github.com/netdata/netdata/issues/827). Please join this discussion for help. -When `lm-sensors` doesn't work on your device (e.g. for RPi temperatures), use [the legacy bash collector](https://learn.netdata.cloud/docs/agent/collectors/charts.d.plugin/sensors) +When `lm-sensors` doesn't work on your device (e.g. for RPi temperatures), use [the legacy bash collector](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/sensors/README.md) --- diff --git a/collectors/python.d.plugin/smartd_log/README.md b/collectors/python.d.plugin/smartd_log/README.md index eef34ce43..7c1e845f8 100644 --- a/collectors/python.d.plugin/smartd_log/README.md +++ b/collectors/python.d.plugin/smartd_log/README.md @@ -1,7 +1,10 @@ <!-- title: "Storage devices monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/smartd_log/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/smartd_log/README.md" sidebar_label: "S.M.A.R.T. attributes" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Devices" --> # Storage devices monitoring with Netdata @@ -106,7 +109,7 @@ Otherwise, all the smartd `.csv` files may get written to `/var/lib/smartmontool ## Configuration Edit the `python.d/smartd_log.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/spigotmc/README.md b/collectors/python.d.plugin/spigotmc/README.md index 06483188b..6d8e4b62b 100644 --- a/collectors/python.d.plugin/spigotmc/README.md +++ b/collectors/python.d.plugin/spigotmc/README.md @@ -1,7 +1,10 @@ <!-- title: "SpigotMC monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/spigotmc/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/spigotmc/README.md" sidebar_label: "SpigotMC" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # SpigotMC monitoring with Netdata @@ -18,7 +21,7 @@ the data returned by the `tps` or `list` console commands. ## Configuration Edit the `python.d/spigotmc.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/springboot/Makefile.inc b/collectors/python.d.plugin/springboot/Makefile.inc deleted file mode 100644 index 06775f937..000000000 --- a/collectors/python.d.plugin/springboot/Makefile.inc +++ /dev/null @@ -1,13 +0,0 @@ -# SPDX-License-Identifier: GPL-3.0-or-later - -# THIS IS NOT A COMPLETE Makefile -# IT IS INCLUDED BY ITS PARENT'S Makefile.am -# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT - -# install these files -dist_python_DATA += springboot/springboot.chart.py -dist_pythonconfig_DATA += springboot/springboot.conf - -# do not install these files, but include them in the distribution -dist_noinst_DATA += springboot/README.md springboot/Makefile.inc - diff --git a/collectors/python.d.plugin/springboot/README.md b/collectors/python.d.plugin/springboot/README.md deleted file mode 100644 index cdbc9a900..000000000 --- a/collectors/python.d.plugin/springboot/README.md +++ /dev/null @@ -1,145 +0,0 @@ -<!-- -title: "Java Spring Boot 2 application monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/springboot/README.md -sidebar_label: "Java Spring Boot 2 applications" ---> - -# Java Spring Boot 2 application monitoring with Netdata - -Monitors one or more Java Spring-boot applications depending on configuration. -Netdata can be used to monitor running Java [Spring Boot](https://spring.io/) applications that expose their metrics with the use of the **Spring Boot Actuator** included in Spring Boot library. - -## Configuration - -The Spring Boot Actuator exposes these metrics over HTTP and is very easy to use: - -- add `org.springframework.boot:spring-boot-starter-actuator` to your application dependencies -- set `endpoints.metrics.sensitive=false` in your `application.properties` - -You can create custom Metrics by add and inject a PublicMetrics in your application. -This is a example to add custom metrics: - -```java -package com.example; - -import org.springframework.boot.actuate.endpoint.PublicMetrics; -import org.springframework.boot.actuate.metrics.Metric; -import org.springframework.stereotype.Service; - -import java.lang.management.ManagementFactory; -import java.lang.management.MemoryPoolMXBean; -import java.util.ArrayList; -import java.util.Collection; - -@Service -public class HeapPoolMetrics implements PublicMetrics { - - private static final String PREFIX = "mempool."; - private static final String KEY_EDEN = PREFIX + "eden"; - private static final String KEY_SURVIVOR = PREFIX + "survivor"; - private static final String KEY_TENURED = PREFIX + "tenured"; - - @Override - public Collection<Metric<?>> metrics() { - Collection<Metric<?>> result = new ArrayList<>(4); - for (MemoryPoolMXBean mem : ManagementFactory.getMemoryPoolMXBeans()) { - String poolName = mem.getName(); - String name = null; - if (poolName.indexOf("Eden Space") != -1) { - name = KEY_EDEN; - } else if (poolName.indexOf("Survivor Space") != -1) { - name = KEY_SURVIVOR; - } else if (poolName.indexOf("Tenured Gen") != -1 || poolName.indexOf("Old Gen") != -1) { - name = KEY_TENURED; - } - - if (name != null) { - result.add(newMemoryMetric(name, mem.getUsage().getMax())); - result.add(newMemoryMetric(name + ".init", mem.getUsage().getInit())); - result.add(newMemoryMetric(name + ".committed", mem.getUsage().getCommitted())); - result.add(newMemoryMetric(name + ".used", mem.getUsage().getUsed())); - } - } - return result; - } - - private Metric<Long> newMemoryMetric(String name, long bytes) { - return new Metric<>(name, bytes / 1024); - } -} -``` - -Please refer [Spring Boot Actuator: Production-ready Features](https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html#production-ready) and [81. Actuator - Part IX. ‘How-to’ guides](https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-actuator) for more information. - -## Charts - -1. **Response Codes** in requests/s - - - 1xx - - 2xx - - 3xx - - 4xx - - 5xx - - others - -2. **Threads** - - - daemon - - total - -3. **GC Time** in milliseconds and **GC Operations** in operations/s - - - Copy - - MarkSweep - - ... - -4. **Heap Memory Usage** in KB - - - used - - committed - -## Usage - -Edit the `python.d/springboot.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. - -```bash -cd /etc/netdata # Replace this path with your Netdata config directory, if different -sudo ./edit-config python.d/springboot.conf -``` - -This module defines some common charts, and you can add custom charts by change the configurations. - -The configuration format is like: - -```yaml -<id>: - name: '<name>' - url: '<metrics endpoint>' # ex. http://localhost:8080/metrics - user: '<username>' # optional - pass: '<password>' # optional - defaults: - [<chart-id>]: true|false - extras: - - id: '<chart-id>' - options: - title: '***' - units: '***' - family: '***' - context: 'springboot.***' - charttype: 'stacked' | 'area' | 'line' - lines: - - { dimension: 'myapp_ok', name: 'ok', algorithm: 'absolute', multiplier: 1, divisor: 1} # it shows "myapp.ok" metrics - - { dimension: 'myapp_ng', name: 'ng', algorithm: 'absolute', multiplier: 1, divisor: 1} # it shows "myapp.ng" metrics -``` - -By default, it creates `response_code`, `threads`, `gc_time`, `gc_ope` abd `heap` charts. -You can disable the default charts by set `defaults.<chart-id>: false`. - -The dimension name of extras charts should replace `.` to `_`. - -Please check -[springboot.conf](https://raw.githubusercontent.com/netdata/netdata/master/collectors/python.d.plugin/springboot/springboot.conf) -for more examples. - - diff --git a/collectors/python.d.plugin/springboot/springboot.chart.py b/collectors/python.d.plugin/springboot/springboot.chart.py deleted file mode 100644 index dbe11d6b8..000000000 --- a/collectors/python.d.plugin/springboot/springboot.chart.py +++ /dev/null @@ -1,160 +0,0 @@ -# -*- coding: utf-8 -*- -# Description: tomcat netdata python.d module -# Author: Wing924 -# SPDX-License-Identifier: GPL-3.0-or-later - -import json - -from bases.FrameworkServices.UrlService import UrlService - -DEFAULT_ORDER = [ - 'response_code', - 'threads', - 'gc_time', - 'gc_ope', - 'heap', -] - -DEFAULT_CHARTS = { - 'response_code': { - 'options': [None, "Response Codes", "requests/s", "response", "springboot.response_code", "stacked"], - 'lines': [ - ["resp_other", 'Other', 'incremental'], - ["resp_1xx", '1xx', 'incremental'], - ["resp_2xx", '2xx', 'incremental'], - ["resp_3xx", '3xx', 'incremental'], - ["resp_4xx", '4xx', 'incremental'], - ["resp_5xx", '5xx', 'incremental'], - ] - }, - 'threads': { - 'options': [None, "Threads", "current threads", "threads", "springboot.threads", "area"], - 'lines': [ - ["threads_daemon", 'daemon', 'absolute'], - ["threads", 'total', 'absolute'], - ] - }, - 'gc_time': { - 'options': [None, "GC Time", "milliseconds", "garbage collection", "springboot.gc_time", "stacked"], - 'lines': [ - ["gc_copy_time", 'Copy', 'incremental'], - ["gc_marksweepcompact_time", 'MarkSweepCompact', 'incremental'], - ["gc_parnew_time", 'ParNew', 'incremental'], - ["gc_concurrentmarksweep_time", 'ConcurrentMarkSweep', 'incremental'], - ["gc_ps_scavenge_time", 'PS Scavenge', 'incremental'], - ["gc_ps_marksweep_time", 'PS MarkSweep', 'incremental'], - ["gc_g1_young_generation_time", 'G1 Young Generation', 'incremental'], - ["gc_g1_old_generation_time", 'G1 Old Generation', 'incremental'], - ] - }, - 'gc_ope': { - 'options': [None, "GC Operations", "operations/s", "garbage collection", "springboot.gc_ope", "stacked"], - 'lines': [ - ["gc_copy_count", 'Copy', 'incremental'], - ["gc_marksweepcompact_count", 'MarkSweepCompact', 'incremental'], - ["gc_parnew_count", 'ParNew', 'incremental'], - ["gc_concurrentmarksweep_count", 'ConcurrentMarkSweep', 'incremental'], - ["gc_ps_scavenge_count", 'PS Scavenge', 'incremental'], - ["gc_ps_marksweep_count", 'PS MarkSweep', 'incremental'], - ["gc_g1_young_generation_count", 'G1 Young Generation', 'incremental'], - ["gc_g1_old_generation_count", 'G1 Old Generation', 'incremental'], - ] - }, - 'heap': { - 'options': [None, "Heap Memory Usage", "KiB", "heap memory", "springboot.heap", "area"], - 'lines': [ - ["heap_committed", 'committed', "absolute"], - ["heap_used", 'used', "absolute"], - ] - } -} - - -class ExtraChartError(ValueError): - pass - - -class Service(UrlService): - def __init__(self, configuration=None, name=None): - UrlService.__init__(self, configuration=configuration, name=name) - self.url = self.configuration.get('url', "http://localhost:8080/metrics") - self._setup_charts() - - def _get_data(self): - """ - Format data received from http request - :return: dict - """ - raw_data = self._get_raw_data() - if not raw_data: - return None - - try: - data = json.loads(raw_data) - except ValueError: - self.debug('%s is not a valid JSON page' % self.url) - return None - - result = { - 'resp_1xx': 0, - 'resp_2xx': 0, - 'resp_3xx': 0, - 'resp_4xx': 0, - 'resp_5xx': 0, - 'resp_other': 0, - } - - for key, value in data.iteritems(): - if 'counter.status.' in key: - status_type = key[15:16] + 'xx' - if status_type[0] not in '12345': - status_type = 'other' - result['resp_' + status_type] += value - else: - result[key.replace('.', '_')] = value - - return result or None - - def _setup_charts(self): - self.order = [] - self.definitions = {} - defaults = self.configuration.get('defaults', {}) - - for chart in DEFAULT_ORDER: - if defaults.get(chart, True): - self.order.append(chart) - self.definitions[chart] = DEFAULT_CHARTS[chart] - - for extra in self.configuration.get('extras', []): - self._add_extra_chart(extra) - self.order.append(extra['id']) - - def _add_extra_chart(self, chart): - chart_id = chart.get('id', None) or self.die('id is not defined in extra chart') - options = chart.get('options', None) or self.die('option is not defined in extra chart: %s' % chart_id) - lines = chart.get('lines', None) or self.die('lines is not defined in extra chart: %s' % chart_id) - - title = options.get('title', None) or self.die('title is missing: %s' % chart_id) - units = options.get('units', None) or self.die('units is missing: %s' % chart_id) - family = options.get('family', title) - context = options.get('context', 'springboot.' + title) - charttype = options.get('charttype', 'line') - - result = { - 'options': [None, title, units, family, context, charttype], - 'lines': [], - } - - for line in lines: - dimension = line.get('dimension', None) or self.die('dimension is missing: %s' % chart_id) - name = line.get('name', dimension) - algorithm = line.get('algorithm', 'absolute') - multiplier = line.get('multiplier', 1) - divisor = line.get('divisor', 1) - result['lines'].append([dimension, name, algorithm, multiplier, divisor]) - - self.definitions[chart_id] = result - - @staticmethod - def die(error_message): - raise ExtraChartError(error_message) diff --git a/collectors/python.d.plugin/springboot/springboot.conf b/collectors/python.d.plugin/springboot/springboot.conf deleted file mode 100644 index 0cb369cd8..000000000 --- a/collectors/python.d.plugin/springboot/springboot.conf +++ /dev/null @@ -1,118 +0,0 @@ -# netdata python.d.plugin configuration for springboot -# -# This file is in YaML format. Generally the format is: -# -# name: value -# -# There are 2 sections: -# - global variables -# - one or more JOBS -# -# JOBS allow you to collect values from multiple sources. -# Each source will have its own set of charts. -# -# JOB parameters have to be indented (using spaces only, example below). - -# ---------------------------------------------------------------------- -# Global Variables -# These variables set the defaults for all JOBs, however each JOB -# may define its own, overriding the defaults. - -# update_every sets the default data collection frequency. -# If unset, the python.d.plugin default is used. -# update_every: 1 - -# priority controls the order of charts at the netdata dashboard. -# Lower numbers move the charts towards the top of the page. -# If unset, the default for python.d.plugin is used. -# priority: 60000 - -# penalty indicates whether to apply penalty to update_every in case of failures. -# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes. -# penalty: yes - -# autodetection_retry sets the job re-check interval in seconds. -# The job is not deleted if check fails. -# Attempts to start the job are made once every autodetection_retry. -# This feature is disabled by default. -# autodetection_retry: 0 - -# ---------------------------------------------------------------------- -# JOBS (data collection sources) -# -# Any number of jobs is supported. -# -# All python.d.plugin JOBS (for all its modules) support a set of -# predefined parameters. These are: -# -# job_name: -# name: myname # the JOB's name as it will appear at the -# # dashboard (by default is the job_name) -# # JOBs sharing a name are mutually exclusive -# update_every: 1 # the JOB's data collection frequency -# priority: 60000 # the JOB's order on the dashboard -# penalty: yes # the JOB's penalty -# autodetection_retry: 0 # the JOB's re-check interval in seconds -# -# Additionally to the above, this plugin also supports the following: -# -# url: 'http://127.0.0.1/metrics' # the URL of the spring boot actuator metrics -# -# if the URL is password protected, the following are supported: -# -# user: 'username' -# pass: 'password' -# -# defaults: -# [chart_id]: true | false # enables/disables default charts, defaults true. -# extras: {} # defines extra charts to monitor, please see the example below -# - id: [chart_id] -# options: {} -# lines: [] -# -# If all defaults is disabled and no extra charts are defined, this module will disable itself, as it has no data to -# collect. -# -# Configuration example -# --------------------- -# example: -# name: 'example' -# url: 'http://localhost:8080/metrics' -# defaults: -# response_code: true -# threads: true -# gc_time: true -# gc_ope: true -# heap: false -# extras: -# - id: 'heap' -# options: { title: 'Heap Memory Usage', units: 'KB', family: 'heap memory', context: 'springboot.heap', charttype: 'stacked' } -# lines: -# - { dimension: 'mem_free', name: 'free'} -# - { dimension: 'mempool_eden_used', name: 'eden', algorithm: 'absolute', multiplier: 1, divisor: 1} -# - { dimension: 'mempool_survivor_used', name: 'survivor', algorithm: 'absolute', multiplier: 1, divisor: 1} -# - { dimension: 'mempool_tenured_used', name: 'tenured', algorithm: 'absolute', multiplier: 1, divisor: 1} -# - id: 'heap_eden' -# options: { title: 'Eden Memory Usage', units: 'KB', family: 'heap memory', context: 'springboot.heap_eden', charttype: 'area' } -# lines: -# - { dimension: 'mempool_eden_used', name: 'used'} -# - { dimension: 'mempool_eden_committed', name: 'committed'} -# - id: 'heap_survivor' -# options: { title: 'Survivor Memory Usage', units: 'KB', family: 'heap memory', context: 'springboot.heap_survivor', charttype: 'area' } -# lines: -# - { dimension: 'mempool_survivor_used', name: 'used'} -# - { dimension: 'mempool_survivor_committed', name: 'committed'} -# - id: 'heap_tenured' -# options: { title: 'Tenured Memory Usage', units: 'KB', family: 'heap memory', context: 'springboot.heap_tenured', charttype: 'area' } -# lines: -# - { dimension: 'mempool_tenured_used', name: 'used'} -# - { dimension: 'mempool_tenured_committed', name: 'committed'} - - -local: - name: 'local' - url: 'http://localhost:8080/metrics' - -local_ip: - name: 'local' - url: 'http://127.0.0.1:8080/metrics' diff --git a/collectors/python.d.plugin/squid/README.md b/collectors/python.d.plugin/squid/README.md index c29b69a19..ac6c83714 100644 --- a/collectors/python.d.plugin/squid/README.md +++ b/collectors/python.d.plugin/squid/README.md @@ -1,7 +1,10 @@ <!-- title: "Squid monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/squid/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/squid/README.md" sidebar_label: "Squid" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Squid monitoring with Netdata @@ -35,7 +38,7 @@ It produces following charts: ## Configuration Edit the `python.d/squid.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/tomcat/README.md b/collectors/python.d.plugin/tomcat/README.md index b7525b88a..66ed6d97a 100644 --- a/collectors/python.d.plugin/tomcat/README.md +++ b/collectors/python.d.plugin/tomcat/README.md @@ -1,7 +1,10 @@ <!-- title: "Apache Tomcat monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tomcat/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tomcat/README.md" sidebar_label: "Tomcat" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Apache Tomcat monitoring with Netdata @@ -30,7 +33,7 @@ Charts: ## Configuration Edit the `python.d/tomcat.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/tor/README.md b/collectors/python.d.plugin/tor/README.md index b57d77c08..c66803766 100644 --- a/collectors/python.d.plugin/tor/README.md +++ b/collectors/python.d.plugin/tor/README.md @@ -1,7 +1,10 @@ <!-- title: "Tor monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tor/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tor/README.md" sidebar_label: "Tor" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apps" --> # Tor monitoring with Netdata @@ -23,7 +26,7 @@ It produces only one chart: ## Configuration Edit the `python.d/tor.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/traefik/README.md b/collectors/python.d.plugin/traefik/README.md index 251cdf2e4..cf30a82a4 100644 --- a/collectors/python.d.plugin/traefik/README.md +++ b/collectors/python.d.plugin/traefik/README.md @@ -1,7 +1,10 @@ <!-- title: "Traefik monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/traefik/README.md -sidebar_label: "Traefik" +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/traefik/README.md" +sidebar_label: "traefik-python.d.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Traefik monitoring with Netdata @@ -10,45 +13,46 @@ Uses the `health` API to provide statistics. It produces: -1. **Responses** by statuses +1. **Responses** by statuses - - success (1xx, 2xx, 304) - - error (5xx) - - redirect (3xx except 304) - - bad (4xx) - - other (all other responses) + - success (1xx, 2xx, 304) + - error (5xx) + - redirect (3xx except 304) + - bad (4xx) + - other (all other responses) -2. **Responses** by codes +2. **Responses** by codes - - 2xx (successful) - - 5xx (internal server errors) - - 3xx (redirect) - - 4xx (bad) - - 1xx (informational) - - other (non-standart responses) + - 2xx (successful) + - 5xx (internal server errors) + - 3xx (redirect) + - 4xx (bad) + - 1xx (informational) + - other (non-standart responses) -3. **Detailed Response Codes** requests/s (number of responses for each response code family individually) +3. **Detailed Response Codes** requests/s (number of responses for each response code family individually) -4. **Requests**/s +4. **Requests**/s - - request statistics + - request statistics -5. **Total response time** +5. **Total response time** - - sum of all response time + - sum of all response time -6. **Average response time** +6. **Average response time** -7. **Average response time per iteration** +7. **Average response time per iteration** -8. **Uptime** +8. **Uptime** - - Traefik server uptime + - Traefik server uptime ## Configuration -Edit the `python.d/traefik.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +Edit the `python.d/traefik.conf` configuration file using `edit-config` from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically +at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different @@ -60,11 +64,11 @@ Needs only `url` to server's `health` Here is an example for local server: ```yaml -update_every : 1 -priority : 60000 +update_every: 1 +priority: 60000 local: - url : 'http://localhost:8080/health' + url: 'http://localhost:8080/health' ``` Without configuration, module attempts to connect to `http://localhost:8080/health`. diff --git a/collectors/python.d.plugin/uwsgi/README.md b/collectors/python.d.plugin/uwsgi/README.md index 58db1a41a..dcc2dc38e 100644 --- a/collectors/python.d.plugin/uwsgi/README.md +++ b/collectors/python.d.plugin/uwsgi/README.md @@ -1,7 +1,10 @@ <!-- title: "uWSGI monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/uwsgi/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/uwsgi/README.md" sidebar_label: "uWSGI" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # uWSGI monitoring with Netdata @@ -29,7 +32,7 @@ Following charts are drawn: ## Configuration Edit the `python.d/uwsgi.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/varnish/README.md b/collectors/python.d.plugin/varnish/README.md index 018905f06..ebcc00c51 100644 --- a/collectors/python.d.plugin/varnish/README.md +++ b/collectors/python.d.plugin/varnish/README.md @@ -1,7 +1,10 @@ <!-- title: "Varnish Cache monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/varnish/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/varnish/README.md" sidebar_label: "Varnish Cache" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Webapps" --> # Varnish Cache monitoring with Netdata @@ -45,7 +48,7 @@ For every storage (SMF, SMA, or MSE): ## Configuration Edit the `python.d/varnish.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/w1sensor/README.md b/collectors/python.d.plugin/w1sensor/README.md index b6d2b2d63..12a14a19a 100644 --- a/collectors/python.d.plugin/w1sensor/README.md +++ b/collectors/python.d.plugin/w1sensor/README.md @@ -1,7 +1,10 @@ <!-- title: "1-Wire Sensors monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/w1sensor/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/w1sensor/README.md" sidebar_label: "1-Wire sensors" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Remotes/Devices" --> # 1-Wire Sensors monitoring with Netdata @@ -16,7 +19,7 @@ Charts are created dynamically based on the number of detected sensors. ## Configuration Edit the `python.d/w1sensor.conf` configuration file using `edit-config` from the Netdata [config -directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`. +directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md), which is typically at `/etc/netdata`. ```bash cd /etc/netdata # Replace this path with your Netdata config directory, if different diff --git a/collectors/python.d.plugin/w1sensor/w1sensor.chart.py b/collectors/python.d.plugin/w1sensor/w1sensor.chart.py index c4f847bf0..66797ced3 100644 --- a/collectors/python.d.plugin/w1sensor/w1sensor.chart.py +++ b/collectors/python.d.plugin/w1sensor/w1sensor.chart.py @@ -15,7 +15,7 @@ update_every = 5 W1_DIR = '/sys/bus/w1/devices/' # Lines matching the following regular expression contain a temperature value -RE_TEMP = re.compile(r' t=(\d+)') +RE_TEMP = re.compile(r' t=(-?\d+)') ORDER = [ 'temp', diff --git a/collectors/python.d.plugin/zscores/README.md b/collectors/python.d.plugin/zscores/README.md index 4f84a6c1f..d89aa6a0f 100644 --- a/collectors/python.d.plugin/zscores/README.md +++ b/collectors/python.d.plugin/zscores/README.md @@ -1,14 +1,18 @@ <!-- title: "zscores" description: "Use statistical anomaly detection to narrow your focus and shorten root cause analysis." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/zscores/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/zscores/README.md" +sidebar_label: "zscores" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Uncategorized" --> # Z-Scores - basic anomaly detection for your key metrics and charts Smoothed, rolling [Z-Scores](https://en.wikipedia.org/wiki/Standard_score) for selected metrics or charts. -This collector uses the [Netdata rest api](https://learn.netdata.cloud/docs/agent/web/api) to get the `mean` and `stddev` +This collector uses the [Netdata rest api](https://github.com/netdata/netdata/blob/master/web/api/README.md) to get the `mean` and `stddev` for each dimension on specified charts over a time range (defined by `train_secs` and `offset_secs`). For each dimension it will calculate a Z-Score as `z = (x - mean) / stddev` (clipped at `z_clip`). Scores are then smoothed over time (`z_smooth_n`) and, if `mode: 'per_chart'`, aggregated across dimensions to a smoothed, rolling chart level Z-Score diff --git a/collectors/slabinfo.plugin/README.md b/collectors/slabinfo.plugin/README.md index 2f4904660..320b1fc9f 100644 --- a/collectors/slabinfo.plugin/README.md +++ b/collectors/slabinfo.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "slabinfo.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/slabinfo.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/slabinfo.plugin/README.md" +sidebar_label: "slabinfo.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # slabinfo.plugin diff --git a/collectors/slabinfo.plugin/slabinfo.c b/collectors/slabinfo.plugin/slabinfo.c index 2e47ee229..52b53cd20 100644 --- a/collectors/slabinfo.plugin/slabinfo.c +++ b/collectors/slabinfo.plugin/slabinfo.c @@ -142,14 +142,14 @@ struct slabinfo *read_file_slabinfo() { if(unlikely(!ff)) { ff = procfile_reopen(ff, PLUGIN_SLABINFO_PROCFILE, " ,:" , PROCFILE_FLAG_DEFAULT); if(unlikely(!ff)) { - error("<- Cannot open file '%s", PLUGIN_SLABINFO_PROCFILE); + collector_error("<- Cannot open file '%s", PLUGIN_SLABINFO_PROCFILE); exit(1); } } ff = procfile_readall(ff); if(unlikely(!ff)) { - error("<- Cannot read file '%s'", PLUGIN_SLABINFO_PROCFILE); + collector_error("<- Cannot read file '%s'", PLUGIN_SLABINFO_PROCFILE); exit(0); } @@ -336,6 +336,7 @@ void usage(void) { } int main(int argc, char **argv) { + stderror = stderr; clocks_init(); program_name = argv[0]; @@ -350,7 +351,7 @@ int main(int argc, char **argv) { n = (int) str2l(argv[i]); if (n > 0) { if (n >= UPDATE_EVERY_MAX) { - error("Invalid interval value: %s", argv[i]); + collector_error("Invalid interval value: %s", argv[i]); exit(1); } freq = n; @@ -383,7 +384,7 @@ int main(int argc, char **argv) { if(freq >= update_every) update_every = freq; else if(freq) - error("update frequency %d seconds is too small for slabinfo. Using %d.", freq, update_every); + collector_error("update frequency %d seconds is too small for slabinfo. Using %d.", freq, update_every); // Call the main function. Time drift to be added diff --git a/collectors/statsd.plugin/README.md b/collectors/statsd.plugin/README.md index b46ca28d9..d65476ff4 100644 --- a/collectors/statsd.plugin/README.md +++ b/collectors/statsd.plugin/README.md @@ -1,7 +1,11 @@ <!-- title: "statsd.plugin" description: "The Netdata Agent is a fully-featured StatsD server that collects metrics from any custom application and visualizes them in real-time." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/README.md" +sidebar_label: "statsd.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apm" --> StatsD is a system to collect data from any application. Applications send metrics to it, usually via non-blocking UDP communication, and StatsD servers collect these metrics, perform a few simple calculations on them and push them to backend time-series databases. @@ -25,11 +29,11 @@ On synthetic charts, we can have alarms as with any metric and chart. - [K6 load testing tool](https://k6.io) - **Description:** k6 is a developer-centric, free and open-source load testing tool built for making performance testing a productive and enjoyable experience. - - [Documentation](/collectors/statsd.plugin/k6.md) + - [Documentation](https://github.com/netdata/netdata/blob/master/collectors/statsd.plugin/k6.md) - [Configuration](https://github.com/netdata/netdata/blob/master/collectors/statsd.plugin/k6.conf) - [Asterisk](https://www.asterisk.org/) - **Description:** Asterisk is an Open Source PBX and telephony toolkit. - - [Documentation](/collectors/statsd.plugin/asterisk.md) + - [Documentation](https://github.com/netdata/netdata/blob/master/collectors/statsd.plugin/asterisk.md) - [Configuration](https://github.com/netdata/netdata/blob/master/collectors/statsd.plugin/asterisk.conf) ## Metrics supported by Netdata @@ -202,7 +206,7 @@ Netdata can visualize StatsD collected metrics in 2 ways: ### Private metric charts -Private charts are controlled with `create private charts for metrics matching = *`. This setting accepts a space-separated list of [simple patterns](/libnetdata/simple_pattern/README.md). Netdata will create private charts for all metrics **by default**. +Private charts are controlled with `create private charts for metrics matching = *`. This setting accepts a space-separated list of [simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md). Netdata will create private charts for all metrics **by default**. For example, to render charts for all `myapp.*` metrics, except `myapp.*.badmetric`, use: @@ -210,7 +214,7 @@ For example, to render charts for all `myapp.*` metrics, except `myapp.*.badmetr create private charts for metrics matching = !myapp.*.badmetric myapp.* ``` -You can specify Netdata StatsD to have a different `memory mode` than the rest of the Netdata Agent. You can read more about `memory mode` in the [documentation](/database/README.md). +You can specify Netdata StatsD to have a different `memory mode` than the rest of the Netdata Agent. You can read more about `memory mode` in the [documentation](https://github.com/netdata/netdata/blob/master/database/README.md). The default behavior is to use the same settings as the rest of the Netdata Agent. If you wish to change them, edit the following settings: - `private charts memory mode` @@ -289,7 +293,7 @@ Synthetic charts are organized in - **charts for each application** aka family in Netdata Dashboard. - **StatsD metrics for each chart** /aka charts and context Netdata Dashboard. -> You can read more about how the Netdata Agent organizes information in the relevant [documentation](/web/README.md) +> You can read more about how the Netdata Agent organizes information in the relevant [documentation](https://github.com/netdata/netdata/blob/master/web/README.md) For each application you need to create a `.conf` file in `/etc/netdata/statsd.d`. @@ -326,7 +330,7 @@ Using the above configuration `myapp` should get its own section on the dashboar `[app]` starts a new application definition. The supported settings in this section are: - `name` defines the name of the app. -- `metrics` is a Netdata [simple pattern](/libnetdata/simple_pattern/README.md). This pattern should match all the possible StatsD metrics that will be participating in the application `myapp`. +- `metrics` is a Netdata [simple pattern](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md). This pattern should match all the possible StatsD metrics that will be participating in the application `myapp`. - `private charts = yes|no`, enables or disables private charts for the metrics matched. - `gaps when not collected = yes|no`, enables or disables gaps on the charts of the application in case that no metrics are collected. - `memory mode` sets the memory mode for all charts of the application. The default is the global default for Netdata (not the global default for StatsD private charts). We suggest not to use this (we have commented it out in the example) and let your app use the global default for Netdata, which is our dbengine. @@ -352,7 +356,7 @@ So, the format is this: dimension = [pattern] METRIC NAME TYPE MULTIPLIER DIVIDER OPTIONS ``` -`pattern` is a keyword. When set, `METRIC` is expected to be a Netdata [simple pattern](/libnetdata/simple_pattern/README.md) that will be used to match all the StatsD metrics to be added to the chart. So, `pattern` automatically matches any number of StatsD metrics, all of which will be added as separate chart dimensions. +`pattern` is a keyword. When set, `METRIC` is expected to be a Netdata [simple pattern](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md) that will be used to match all the StatsD metrics to be added to the chart. So, `pattern` automatically matches any number of StatsD metrics, all of which will be added as separate chart dimensions. `TYPE`, `MULTIPLIER`, `DIVIDER` and `OPTIONS` are optional. diff --git a/collectors/statsd.plugin/asterisk.md b/collectors/statsd.plugin/asterisk.md index 94da94ee2..9d7948111 100644 --- a/collectors/statsd.plugin/asterisk.md +++ b/collectors/statsd.plugin/asterisk.md @@ -1,8 +1,10 @@ <!-- title: "Asterisk monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/asterisk.md - +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/asterisk.md" sidebar_label: "Asterisk" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apm/Statsd" --> # Asterisk monitoring with Netdata diff --git a/collectors/statsd.plugin/k6.md b/collectors/statsd.plugin/k6.md index 4f8c70133..7a1e36773 100644 --- a/collectors/statsd.plugin/k6.md +++ b/collectors/statsd.plugin/k6.md @@ -1,8 +1,10 @@ <!-- title: "K6 load test monitoring with Netdata" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/k6.md - +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/statsd.plugin/k6.md" sidebar_label: "K6 Load Testing" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Apm/Statsd" --> # K6 Load Testing monitoring with Netdata diff --git a/collectors/statsd.plugin/statsd.c b/collectors/statsd.plugin/statsd.c index 67d7ed2e2..d15129b9c 100644 --- a/collectors/statsd.plugin/statsd.c +++ b/collectors/statsd.plugin/statsd.c @@ -234,7 +234,8 @@ typedef struct statsd_app { // global statsd data struct collection_thread_status { - int status; + SPINLOCK spinlock; + bool running; size_t max_sockets; netdata_thread_t thread; @@ -433,7 +434,7 @@ static inline NETDATA_DOUBLE statsd_parse_float(const char *v, NETDATA_DOUBLE de char *e = NULL; value = str2ndd(v, &e); if(unlikely(e && *e)) - error("STATSD: excess data '%s' after value '%s'", e, v); + collector_error("STATSD: excess data '%s' after value '%s'", e, v); } else value = def; @@ -455,7 +456,7 @@ static inline long long statsd_parse_int(const char *v, long long def) { char *e = NULL; value = str2ll(v, &e); if(unlikely(e && *e)) - error("STATSD: excess data '%s' after value '%s'", e, v); + collector_error("STATSD: excess data '%s' after value '%s'", e, v); } else value = def; @@ -483,7 +484,7 @@ static inline void statsd_process_gauge(STATSD_METRIC *m, const char *value, con if(!is_metric_useful_for_collection(m)) return; if(unlikely(!value || !*value)) { - error("STATSD: metric '%s' of type gauge, with empty value is ignored.", m->name); + collector_error("STATSD: metric '%s' of type gauge, with empty value is ignored.", m->name); return; } @@ -531,7 +532,7 @@ static inline void statsd_process_histogram_or_timer(STATSD_METRIC *m, const cha if(!is_metric_useful_for_collection(m)) return; if(unlikely(!value || !*value)) { - error("STATSD: metric of type %s, with empty value is ignored.", type); + collector_error("STATSD: metric of type %s, with empty value is ignored.", type); return; } @@ -594,7 +595,7 @@ static inline void statsd_process_set(STATSD_METRIC *m, const char *value) { } if (unlikely(!m->set.dict)) { - m->set.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); + m->set.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); dictionary_register_insert_callback(m->set.dict, dictionary_metric_set_value_insert_callback, m); m->set.unique = 0; } @@ -634,7 +635,7 @@ static inline void statsd_process_dictionary(STATSD_METRIC *m, const char *value statsd_reset_metric(m); if (unlikely(!m->dictionary.dict)) { - m->dictionary.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); + m->dictionary.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); dictionary_register_insert_callback(m->dictionary.dict, dictionary_metric_dict_value_insert_callback, m); m->dictionary.unique = 0; } @@ -873,21 +874,19 @@ struct statsd_tcp { char buffer[]; }; -#ifdef HAVE_RECVMMSG struct statsd_udp { - int *running; + struct collection_thread_status *status; STATSD_SOCKET_DATA_TYPE type; + +#ifdef HAVE_RECVMMSG size_t size; struct iovec *iovecs; struct mmsghdr *msgs; -}; #else -struct statsd_udp { int *running; - STATSD_SOCKET_DATA_TYPE type; char buffer[STATSD_UDP_BUFFER_SIZE]; -}; #endif +}; // new TCP client connected static void *statsd_add_callback(POLLINFO *pi, short int *events, void *data) { @@ -1097,9 +1096,11 @@ static int statsd_snd_callback(POLLINFO *pi, short int *events) { void statsd_collector_thread_cleanup(void *data) { struct statsd_udp *d = data; - *d->running = 0; + netdata_spinlock_lock(&d->status->spinlock); + d->status->running = false; + netdata_spinlock_unlock(&d->status->spinlock); - info("cleaning up..."); + collector_info("cleaning up..."); #ifdef HAVE_RECVMMSG size_t i; @@ -1114,9 +1115,15 @@ void statsd_collector_thread_cleanup(void *data) { worker_unregister(); } +static bool statsd_should_stop(void) { + return !service_running(SERVICE_COLLECTORS); +} + void *statsd_collector_thread(void *ptr) { struct collection_thread_status *status = ptr; - status->status = 1; + netdata_spinlock_lock(&status->spinlock); + status->running = true; + netdata_spinlock_unlock(&status->spinlock); worker_register("STATSD"); worker_register_job_name(WORKER_JOB_TYPE_TCP_CONNECTED, "tcp connect"); @@ -1124,10 +1131,10 @@ void *statsd_collector_thread(void *ptr) { worker_register_job_name(WORKER_JOB_TYPE_RCV_DATA, "receive"); worker_register_job_name(WORKER_JOB_TYPE_SND_DATA, "send"); - info("STATSD collector thread started with taskid %d", gettid()); + collector_info("STATSD collector thread started with taskid %d", gettid()); struct statsd_udp *d = callocz(sizeof(struct statsd_udp), 1); - d->running = &status->status; + d->status = status; netdata_thread_cleanup_push(statsd_collector_thread_cleanup, d); @@ -1152,6 +1159,7 @@ void *statsd_collector_thread(void *ptr) { , statsd_rcv_callback , statsd_snd_callback , NULL + , statsd_should_stop , NULL // No access control pattern , 0 // No dns lookups for access control pattern , (void *)d @@ -1329,7 +1337,7 @@ static int statsd_readfile(const char *filename, STATSD_APP *app, STATSD_APP_CHA else if(app) { if(!strcmp(s, "dictionary")) { if(!app->dict) - app->dict = dictionary_create(DICT_OPTION_SINGLE_THREADED); + app->dict = dictionary_create_advanced(DICT_OPTION_SINGLE_THREADED, &dictionary_stats_category_collectors, 0); dict = app->dict; } @@ -1934,7 +1942,7 @@ static inline void statsd_flush_dictionary(STATSD_METRIC *m) { if(m->dictionary.unique >= statsd.dictionary_max_unique) { if(!(m->options & STATSD_METRIC_OPTION_COLLECTION_FULL_LOGGED)) { m->options |= STATSD_METRIC_OPTION_COLLECTION_FULL_LOGGED; - info( + collector_info( "STATSD dictionary '%s' reach max of %zu items - try increasing 'dictionaries max unique dimensions' in netdata.conf", m->name, m->dictionary.unique); @@ -2307,7 +2315,7 @@ static inline void statsd_flush_index_metrics(STATSD_INDEX *index, void (*flush_ if(unlikely(!(m->options & STATSD_METRIC_OPTION_PRIVATE_CHART_CHECKED))) { if(unlikely(statsd.private_charts >= statsd.max_private_charts_hard)) { debug(D_STATSD, "STATSD: metric '%s' will not be charted, because the hard limit of the maximum number of charts has been reached.", m->name); - info("STATSD: metric '%s' will not be charted, because the hard limit of the maximum number of charts (%zu) has been reached. Increase the number of charts by editing netdata.conf, [statsd] section.", m->name, statsd.max_private_charts_hard); + collector_info("STATSD: metric '%s' will not be charted, because the hard limit of the maximum number of charts (%zu) has been reached. Increase the number of charts by editing netdata.conf, [statsd] section.", m->name, statsd.max_private_charts_hard); m->options &= ~STATSD_METRIC_OPTION_PRIVATE_CHART_ENABLED; } else { @@ -2353,22 +2361,24 @@ static int statsd_listen_sockets_setup(void) { static void statsd_main_cleanup(void *data) { struct netdata_static_thread *static_thread = (struct netdata_static_thread *)data; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); if (statsd.collection_threads_status) { int i; for (i = 0; i < statsd.threads; i++) { - if(statsd.collection_threads_status[i].status) { - info("STATSD: stopping data collection thread %d...", i + 1); + netdata_spinlock_lock(&statsd.collection_threads_status[i].spinlock); + if(statsd.collection_threads_status[i].running) { + collector_info("STATSD: stopping data collection thread %d...", i + 1); netdata_thread_cancel(statsd.collection_threads_status[i].thread); } else { - info("STATSD: data collection thread %d found stopped.", i + 1); + collector_info("STATSD: data collection thread %d found stopped.", i + 1); } + netdata_spinlock_unlock(&statsd.collection_threads_status[i].spinlock); } } - info("STATSD: closing sockets..."); + collector_info("STATSD: closing sockets..."); listen_sockets_close(&statsd.sockets); // destroy the dictionaries @@ -2380,7 +2390,7 @@ static void statsd_main_cleanup(void *data) { dictionary_destroy(statsd.sets.dict); dictionary_destroy(statsd.timers.dict); - info("STATSD: cleanup completed."); + collector_info("STATSD: cleanup completed."); static_thread->enabled = NETDATA_MAIN_THREAD_EXITED; worker_unregister(); @@ -2412,13 +2422,13 @@ void *statsd_main(void *ptr) { netdata_thread_cleanup_push(statsd_main_cleanup, ptr); - statsd.gauges.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.meters.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.counters.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.histograms.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.dictionaries.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.sets.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); - statsd.timers.dict = dictionary_create(STATSD_DICTIONARY_OPTIONS); + statsd.gauges.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.meters.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.counters.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.histograms.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.dictionaries.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.sets.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); + statsd.timers.dict = dictionary_create_advanced(STATSD_DICTIONARY_OPTIONS, &dictionary_stats_category_collectors, 0); dictionary_register_insert_callback(statsd.gauges.dict, dictionary_metric_insert_callback, &statsd.gauges); dictionary_register_insert_callback(statsd.meters.dict, dictionary_metric_insert_callback, &statsd.meters); @@ -2444,7 +2454,7 @@ void *statsd_main(void *ptr) { statsd.update_every = default_rrd_update_every; statsd.update_every = (int)config_get_number(CONFIG_SECTION_STATSD, "update every (flushInterval)", statsd.update_every); if(statsd.update_every < default_rrd_update_every) { - error("STATSD: minimum flush interval %d given, but the minimum is the update every of netdata. Using %d", statsd.update_every, default_rrd_update_every); + collector_error("STATSD: minimum flush interval %d given, but the minimum is the update every of netdata. Using %d", statsd.update_every, default_rrd_update_every); statsd.update_every = default_rrd_update_every; } @@ -2461,7 +2471,7 @@ void *statsd_main(void *ptr) { statsd.histogram_percentile = (double)config_get_float(CONFIG_SECTION_STATSD, "histograms and timers percentile (percentThreshold)", statsd.histogram_percentile); if(isless(statsd.histogram_percentile, 0) || isgreater(statsd.histogram_percentile, 100)) { - error("STATSD: invalid histograms and timers percentile %0.5f given", statsd.histogram_percentile); + collector_error("STATSD: invalid histograms and timers percentile %0.5f given", statsd.histogram_percentile); statsd.histogram_percentile = 95.0; } { @@ -2508,7 +2518,7 @@ void *statsd_main(void *ptr) { #ifdef STATSD_MULTITHREADED statsd.threads = (int)config_get_number(CONFIG_SECTION_STATSD, "threads", processors); if(statsd.threads < 1) { - error("STATSD: Invalid number of threads %d, using %d", statsd.threads, processors); + collector_error("STATSD: Invalid number of threads %d, using %d", statsd.threads, processors); statsd.threads = processors; config_set_number(CONFIG_SECTION_STATSD, "collector threads", statsd.threads); } @@ -2526,7 +2536,7 @@ void *statsd_main(void *ptr) { statsd_listen_sockets_setup(); if(!statsd.sockets.opened) { - error("STATSD: No statsd sockets to listen to. statsd will be disabled."); + collector_error("STATSD: No statsd sockets to listen to. statsd will be disabled."); goto cleanup; } @@ -2536,7 +2546,8 @@ void *statsd_main(void *ptr) { for(i = 0; i < statsd.threads ;i++) { statsd.collection_threads_status[i].max_sockets = max_sockets / statsd.threads; char tag[NETDATA_THREAD_TAG_MAX + 1]; - snprintfz(tag, NETDATA_THREAD_TAG_MAX, "STATSD_COLLECTOR[%d]", i + 1); + snprintfz(tag, NETDATA_THREAD_TAG_MAX, "STATSD_IN[%d]", i + 1); + netdata_spinlock_init(&statsd.collection_threads_status[i].spinlock); netdata_thread_create(&statsd.collection_threads_status[i].thread, tag, NETDATA_THREAD_OPTION_DEFAULT, statsd_collector_thread, &statsd.collection_threads_status[i]); } @@ -2753,7 +2764,7 @@ void *statsd_main(void *ptr) { usec_t step = statsd.update_every * USEC_PER_SEC; heartbeat_t hb; heartbeat_init(&hb); - while(!netdata_exit) { + while(service_running(SERVICE_COLLECTORS)) { worker_is_idle(); heartbeat_next(&hb, step); @@ -2781,7 +2792,7 @@ void *statsd_main(void *ptr) { worker_is_busy(WORKER_STATSD_FLUSH_STATS); statsd_update_all_app_charts(); - if(unlikely(netdata_exit)) + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; if(global_statistics_enabled) { diff --git a/collectors/tc.plugin/README.md b/collectors/tc.plugin/README.md index 32c20f490..bf8655a43 100644 --- a/collectors/tc.plugin/README.md +++ b/collectors/tc.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "tc.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/tc.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/tc.plugin/README.md" +sidebar_label: "tc.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Networking" --> # tc.plugin @@ -67,7 +71,7 @@ QoS is about 2 features: When your system is under a DDoS attack, it will get a lot more bandwidth compared to the one it can handle and probably your applications will crash. Setting a limit on the inbound traffic using QoS, will protect your servers (throttle the requests) and depending on the size of the attack may allow your legitimate users to access the server, while the attack is taking place. - Using QoS together with a [SYNPROXY](/collectors/proc.plugin/README.md) will provide a great degree of protection against most DDoS attacks. Actually when I wrote that article, a few folks tried to DDoS the Netdata demo site to see in real-time the SYNPROXY operation. They did not do it right, but anyway a great deal of requests reached the Netdata server. What saved Netdata was QoS. The Netdata demo server has QoS installed, so the requests were throttled and the server did not even reach the point of resource starvation. Read about it [here](/collectors/proc.plugin/README.md). + Using QoS together with a [SYNPROXY](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md) will provide a great degree of protection against most DDoS attacks. Actually when I wrote that article, a few folks tried to DDoS the Netdata demo site to see in real-time the SYNPROXY operation. They did not do it right, but anyway a great deal of requests reached the Netdata server. What saved Netdata was QoS. The Netdata demo server has QoS installed, so the requests were throttled and the server did not even reach the point of resource starvation. Read about it [here](https://github.com/netdata/netdata/blob/master/collectors/proc.plugin/README.md). On top of all these, QoS is extremely light. You will configure it once, and this is it. It will not bother you again and it will not use any noticeable CPU resources, especially on application and database servers. diff --git a/collectors/tc.plugin/plugin_tc.c b/collectors/tc.plugin/plugin_tc.c index a2e72ee33..a8ceca449 100644 --- a/collectors/tc.plugin/plugin_tc.c +++ b/collectors/tc.plugin/plugin_tc.c @@ -89,7 +89,7 @@ static bool tc_class_conflict_callback(const DICTIONARY_ITEM *item __maybe_unuse struct tc_class *c = old_value; (void)c; struct tc_class *new_c = new_value; (void)new_c; - error("TC: class '%s' is already in device '%s'. Ignoring duplicate.", dictionary_acquired_item_name(item), string2str(d->id)); + collector_error("TC: class '%s' is already in device '%s'. Ignoring duplicate.", dictionary_acquired_item_name(item), string2str(d->id)); tc_class_free_callback(item, new_value, data); @@ -98,7 +98,7 @@ static bool tc_class_conflict_callback(const DICTIONARY_ITEM *item __maybe_unuse static void tc_class_index_init(struct tc_device *d) { if(!d->classes) { - d->classes = dictionary_create(DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_SINGLE_THREADED); + d->classes = dictionary_create_advanced(DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_SINGLE_THREADED, &dictionary_stats_category_collectors, 0); dictionary_register_delete_callback(d->classes, tc_class_free_callback, d); dictionary_register_conflict_callback(d->classes, tc_class_conflict_callback, d); @@ -144,8 +144,9 @@ static void tc_device_free_callback(const DICTIONARY_ITEM *item __maybe_unused, static void tc_device_index_init() { if(!tc_device_root_index) { - tc_device_root_index = dictionary_create( - DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_SINGLE_THREADED | DICT_OPTION_ADD_IN_FRONT); + tc_device_root_index = dictionary_create_advanced( + DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_SINGLE_THREADED | DICT_OPTION_ADD_IN_FRONT, + &dictionary_stats_category_collectors, 0); dictionary_register_insert_callback(tc_device_root_index, tc_device_add_callback, NULL); dictionary_register_delete_callback(tc_device_root_index, tc_device_free_callback, NULL); @@ -276,7 +277,7 @@ static inline void tc_device_commit(struct tc_device *d) { } if(unlikely(updated_classes && updated_qdiscs)) { - error("TC: device '%s' has active both classes (%d) and qdiscs (%d). Will render only qdiscs.", string2str(d->id), updated_classes, updated_qdiscs); + collector_error("TC: device '%s' has active both classes (%d) and qdiscs (%d). Will render only qdiscs.", string2str(d->id), updated_classes, updated_qdiscs); // set all classes to !updated dfe_start_read(d->classes, c) { @@ -352,7 +353,7 @@ static inline void tc_device_commit(struct tc_device *d) { } //if(unlikely(!c->hasparent)) { - // if(root) error("TC: multiple root class/qdisc for device '%s' (old: '%s', new: '%s')", d->id, root->id, c->id); + // if(root) collector_error("TC: multiple root class/qdisc for device '%s' (old: '%s', new: '%s')", d->id, root->id, c->id); // root = c; // debug(D_TC_LOOP, "TC: found root class/qdisc '%s'", root->id); //} @@ -855,14 +856,14 @@ static void tc_main_cleanup(void *ptr) { struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr; static_thread->enabled = NETDATA_MAIN_THREAD_EXITING; - info("cleaning up..."); + collector_info("cleaning up..."); if(tc_child_pid) { - info("TC: killing with SIGTERM tc-qos-helper process %d", tc_child_pid); + collector_info("TC: killing with SIGTERM tc-qos-helper process %d", tc_child_pid); if(killpid(tc_child_pid) != -1) { siginfo_t info; - info("TC: waiting for tc plugin child process pid %d to exit...", tc_child_pid); + collector_info("TC: waiting for tc plugin child process pid %d to exit...", tc_child_pid); waitid(P_PID, (id_t) tc_child_pid, &info, WEXITED); } @@ -929,7 +930,7 @@ void *tc_main(void *ptr) { snprintfz(command, TC_LINE_MAX, "%s/tc-qos-helper.sh", netdata_configured_primary_plugins_dir); char *tc_script = config_get("plugin:tc", "script to run to get tc values", command); - while(!netdata_exit) { + while(service_running(SERVICE_COLLECTORS)) { FILE *fp_child_input, *fp_child_output; struct tc_device *device = NULL; struct tc_class *class = NULL; @@ -939,13 +940,13 @@ void *tc_main(void *ptr) { fp_child_output = netdata_popen(command, (pid_t *)&tc_child_pid, &fp_child_input); if(unlikely(!fp_child_output)) { - error("TC: Cannot popen(\"%s\", \"r\").", command); + collector_error("TC: Cannot popen(\"%s\", \"r\").", command); goto cleanup; } char buffer[TC_LINE_MAX+1] = ""; while(fgets(buffer, TC_LINE_MAX, fp_child_output) != NULL) { - if(unlikely(netdata_exit)) break; + if(unlikely(!service_running(SERVICE_COLLECTORS))) break; buffer[TC_LINE_MAX] = '\0'; // debug(D_TC_LOOP, "TC: read '%s'", buffer); @@ -1162,13 +1163,13 @@ void *tc_main(void *ptr) { class = NULL; } - if(unlikely(netdata_exit)) + if(unlikely(!service_running(SERVICE_COLLECTORS))) goto cleanup; if(code == 1 || code == 127) { // 1 = DISABLE // 127 = cannot even run it - error("TC: tc-qos-helper.sh exited with code %d. Disabling it.", code); + collector_error("TC: tc-qos-helper.sh exited with code %d. Disabling it.", code); goto cleanup; } diff --git a/collectors/timex.plugin/README.md b/collectors/timex.plugin/README.md index 18665f807..ba2020752 100644 --- a/collectors/timex.plugin/README.md +++ b/collectors/timex.plugin/README.md @@ -1,7 +1,11 @@ <!-- title: "timex.plugin" description: "Monitor the system clock synchronization state." -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/timex.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/timex.plugin/README.md" +sidebar_label: "timex.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/System metrics" --> # timex.plugin @@ -19,7 +23,7 @@ An unsynchronized clock may indicate a hardware clock error, or an issue with UT ## Configuration -Edit the `netdata.conf` configuration file using [`edit-config`](/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) from the [Netdata config directory](/docs/configure/nodes.md#the-netdata-config-directory), which is typically at `/etc/netdata`. +Edit the `netdata.conf` configuration file using [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files) from the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory), which is typically at `/etc/netdata`. Scroll down to the `[plugin:timex]` section to find the available options: diff --git a/collectors/timex.plugin/plugin_timex.c b/collectors/timex.plugin/plugin_timex.c index 46cfc5796..84147c851 100644 --- a/collectors/timex.plugin/plugin_timex.c +++ b/collectors/timex.plugin/plugin_timex.c @@ -64,7 +64,7 @@ void *timex_main(void *ptr) usec_t step = update_every * USEC_PER_SEC; heartbeat_t hb; heartbeat_init(&hb); - while (!netdata_exit) { + while (service_running(SERVICE_COLLECTORS)) { worker_is_idle(); heartbeat_next(&hb, step); worker_is_busy(0); diff --git a/collectors/xenstat.plugin/README.md b/collectors/xenstat.plugin/README.md index 8cbe086fc..11c2bfdbe 100644 --- a/collectors/xenstat.plugin/README.md +++ b/collectors/xenstat.plugin/README.md @@ -1,6 +1,10 @@ <!-- title: "xenstat.plugin" -custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/xenstat.plugin/README.md +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/xenstat.plugin/README.md" +sidebar_label: "xenstat.plugin" +learn_status: "Published" +learn_topic_type: "References" +learn_rel_path: "References/Collectors references/Virtualized environments/Virtualize hosts" --> # xenstat.plugin diff --git a/collectors/xenstat.plugin/xenstat_plugin.c b/collectors/xenstat.plugin/xenstat_plugin.c index ea98b9bb1..b0cfa0b2f 100644 --- a/collectors/xenstat.plugin/xenstat_plugin.c +++ b/collectors/xenstat.plugin/xenstat_plugin.c @@ -920,6 +920,7 @@ static void xenstat_send_domain_metrics() { } int main(int argc, char **argv) { + stderror = stderr; clocks_init(); // ------------------------------------------------------------------------ |