diff options
Diffstat (limited to '')
| -rw-r--r-- | src/go/collectors/go.d.plugin/modules/pulsar/integrations/apache_pulsar.md | 279 |
1 files changed, 279 insertions, 0 deletions
diff --git a/src/go/collectors/go.d.plugin/modules/pulsar/integrations/apache_pulsar.md b/src/go/collectors/go.d.plugin/modules/pulsar/integrations/apache_pulsar.md new file mode 100644 index 000000000..c087d0086 --- /dev/null +++ b/src/go/collectors/go.d.plugin/modules/pulsar/integrations/apache_pulsar.md @@ -0,0 +1,279 @@ +<!--startmeta +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/pulsar/README.md" +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/pulsar/metadata.yaml" +sidebar_label: "Apache Pulsar" +learn_status: "Published" +learn_rel_path: "Collecting Metrics/Message Brokers" +most_popular: True +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE" +endmeta--> + +# Apache Pulsar + + +<img src="https://netdata.cloud/img/pulsar.svg" width="150"/> + + +Plugin: go.d.plugin +Module: pulsar + +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> + +## Overview + +This collector monitors Pulsar servers. + + +It collects broker statistics using Pulsar's [Prometheus endpoint](https://pulsar.apache.org/docs/en/deploy-monitoring/#broker-stats). + + +This collector is supported on all platforms. + +This collector supports collecting metrics from multiple instances of this integration, including remote instances. + + +### Default Behavior + +#### Auto-Detection + +By default, it detects Pulsar instances running on localhost. + + +#### Limits + +The default configuration for this integration does not impose any limits on data collection. + +#### Performance Impact + +The default configuration for this integration is not expected to impose a significant performance impact on the system. + + +## Metrics + +Metrics grouped by *scope*. + +The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels. + +- topic_* metrics are available when `exposeTopicLevelMetricsInPrometheus` is set to true. +- subscription_* and namespace_subscription metrics are available when `exposeTopicLevelMetricsInPrometheus` si set to true. +- replication_* and namespace_replication_* metrics are available when replication is configured and `replicationMetricsEnabled` is set to true. + + +### Per Apache Pulsar instance + +These metrics refer to the entire monitored application. + +This scope has no labels. + +Metrics: + +| Metric | Dimensions | Unit | +|:------|:----------|:----| +| pulsar.broker_components | namespaces, topics, subscriptions, producers, consumers | components | +| pulsar.messages_rate | publish, dispatch | messages/s | +| pulsar.throughput_rate | publish, dispatch | KiB/s | +| pulsar.storage_size | used | KiB | +| pulsar.storage_operations_rate | read, write | message batches/s | +| pulsar.msg_backlog | backlog | messages | +| pulsar.storage_write_latency | <=0.5ms, <=1ms, <=5ms, =10ms, <=20ms, <=50ms, <=100ms, <=200ms, <=1s, >1s | entries/s | +| pulsar.entry_size | <=128B, <=512B, <=1KB, <=2KB, <=4KB, <=16KB, <=100KB, <=1MB, >1MB | entries/s | +| pulsar.subscription_delayed | delayed | message batches | +| pulsar.subscription_msg_rate_redeliver | redelivered | messages/s | +| pulsar.subscription_blocked_on_unacked_messages | blocked | subscriptions | +| pulsar.replication_rate | in, out | messages/s | +| pulsar.replication_throughput_rate | in, out | KiB/s | +| pulsar.replication_backlog | backlog | messages | + +### Per namespace + +TBD + +This scope has no labels. + +Metrics: + +| Metric | Dimensions | Unit | +|:------|:----------|:----| +| pulsar.namespace_broker_components | topics, subscriptions, producers, consumers | components | +| pulsar.namespace_messages_rate | publish, dispatch | messages/s | +| pulsar.namespace_throughput_rate | publish, dispatch | KiB/s | +| pulsar.namespace_storage_size | used | KiB | +| pulsar.namespace_storage_operations_rate | read, write | message batches/s | +| pulsar.namespace_msg_backlog | backlog | messages | +| pulsar.namespace_storage_write_latency | <=0.5ms, <=1ms, <=5ms, =10ms, <=20ms, <=50ms, <=100ms, <=200ms, <=1s, >1s | entries/s | +| pulsar.namespace_entry_size | <=128B, <=512B, <=1KB, <=2KB, <=4KB, <=16KB, <=100KB, <=1MB, >1MB | entries/s | +| pulsar.namespace_subscription_delayed | delayed | message batches | +| pulsar.namespace_subscription_msg_rate_redeliver | redelivered | messages/s | +| pulsar.namespace_subscription_blocked_on_unacked_messages | blocked | subscriptions | +| pulsar.namespace_replication_rate | in, out | messages/s | +| pulsar.namespace_replication_throughput_rate | in, out | KiB/s | +| pulsar.namespace_replication_backlog | backlog | messages | +| pulsar.topic_producers | a dimension per topic | producers | +| pulsar.topic_subscriptions | a dimension per topic | subscriptions | +| pulsar.topic_consumers | a dimension per topic | consumers | +| pulsar.topic_messages_rate_in | a dimension per topic | publishes/s | +| pulsar.topic_messages_rate_out | a dimension per topic | dispatches/s | +| pulsar.topic_throughput_rate_in | a dimension per topic | KiB/s | +| pulsar.topic_throughput_rate_out | a dimension per topic | KiB/s | +| pulsar.topic_storage_size | a dimension per topic | KiB | +| pulsar.topic_storage_read_rate | a dimension per topic | message batches/s | +| pulsar.topic_storage_write_rate | a dimension per topic | message batches/s | +| pulsar.topic_msg_backlog | a dimension per topic | messages | +| pulsar.topic_subscription_delayed | a dimension per topic | message batches | +| pulsar.topic_subscription_msg_rate_redeliver | a dimension per topic | messages/s | +| pulsar.topic_subscription_blocked_on_unacked_messages | a dimension per topic | blocked subscriptions | +| pulsar.topic_replication_rate_in | a dimension per topic | messages/s | +| pulsar.topic_replication_rate_out | a dimension per topic | messages/s | +| pulsar.topic_replication_throughput_rate_in | a dimension per topic | messages/s | +| pulsar.topic_replication_throughput_rate_out | a dimension per topic | messages/s | +| pulsar.topic_replication_backlog | a dimension per topic | messages | + + + +## Alerts + +There are no alerts configured by default for this integration. + + +## Setup + +### Prerequisites + +No action required. + +### Configuration + +#### File + +The configuration file name for this integration is `go.d/pulsar.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config go.d/pulsar.conf +``` +#### Options + +The following options can be defined globally: update_every, autodetection_retry. + + +<details><summary>Config options</summary> + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| update_every | Data collection frequency. | 1 | no | +| autodetection_retry | Recheck interval in seconds. Zero means no recheck will be scheduled. | 0 | no | +| url | Server URL. | http://127.0.0.1:8080/metrics | yes | +| timeout | HTTP request timeout. | 1 | no | +| username | Username for basic HTTP authentication. | | no | +| password | Password for basic HTTP authentication. | | no | +| proxy_url | Proxy URL. | | no | +| proxy_username | Username for proxy basic HTTP authentication. | | no | +| proxy_password | Password for proxy basic HTTP authentication. | | no | +| method | HTTP request method. | GET | no | +| body | HTTP request body. | | no | +| headers | HTTP request headers. | | no | +| not_follow_redirects | Redirect handling policy. Controls whether the client follows redirects. | no | no | +| tls_skip_verify | Server certificate chain and hostname validation policy. Controls whether the client performs this check. | no | no | +| tls_ca | Certification authority that the client uses when verifying the server's certificates. | | no | +| tls_cert | Client TLS certificate. | | no | +| tls_key | Client TLS key. | | no | + +</details> + +#### Examples + +##### Basic + +A basic example configuration. + +```yaml +jobs: + - name: local + url: http://127.0.0.1:8080/metrics + +``` +##### HTTP authentication + +Basic HTTP authentication. + +<details><summary>Config</summary> + +```yaml +jobs: + - name: local + url: http://127.0.0.1:8080/metrics + username: username + password: password + +``` +</details> + +##### HTTPS with self-signed certificate + +Do not validate server certificate chain and hostname. + + +<details><summary>Config</summary> + +```yaml +jobs: + - name: local + url: https://127.0.0.1:8080/metrics + tls_skip_verify: yes + +``` +</details> + +##### Multi-instance + +> **Note**: When you define multiple jobs, their names must be unique. + +Collecting metrics from local and remote instances. + + +<details><summary>Config</summary> + +```yaml +jobs: + - name: local + url: http://127.0.0.1:8080/metrics + + - name: remote + url: http://192.0.2.1:8080/metrics + +``` +</details> + + + +## Troubleshooting + +### Debug Mode + +To troubleshoot issues with the `pulsar` collector, run the `go.d.plugin` with the debug option enabled. The output +should give you clues as to why the collector isn't working. + +- Navigate to the `plugins.d` directory, usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case on + your system, open `netdata.conf` and look for the `plugins` setting under `[directories]`. + + ```bash + cd /usr/libexec/netdata/plugins.d/ + ``` + +- Switch to the `netdata` user. + + ```bash + sudo -u netdata -s + ``` + +- Run the `go.d.plugin` to debug the collector: + + ```bash + ./go.d.plugin -d -m pulsar + ``` + + |