diff options
Diffstat (limited to '')
| -rw-r--r-- | collectors/python.d.plugin/tomcat/Makefile.inc | 13 | ||||
| l--------- | collectors/python.d.plugin/tomcat/README.md | 1 | ||||
| -rw-r--r-- | collectors/python.d.plugin/tomcat/integrations/tomcat.md | 203 | ||||
| -rw-r--r-- | collectors/python.d.plugin/tomcat/metadata.yaml | 200 | ||||
| -rw-r--r-- | collectors/python.d.plugin/tomcat/tomcat.chart.py | 199 | ||||
| -rw-r--r-- | collectors/python.d.plugin/tomcat/tomcat.conf | 89 |
6 files changed, 705 insertions, 0 deletions
diff --git a/collectors/python.d.plugin/tomcat/Makefile.inc b/collectors/python.d.plugin/tomcat/Makefile.inc new file mode 100644 index 00000000..940a7835 --- /dev/null +++ b/collectors/python.d.plugin/tomcat/Makefile.inc @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-3.0-or-later + +# THIS IS NOT A COMPLETE Makefile +# IT IS INCLUDED BY ITS PARENT'S Makefile.am +# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT + +# install these files +dist_python_DATA += tomcat/tomcat.chart.py +dist_pythonconfig_DATA += tomcat/tomcat.conf + +# do not install these files, but include them in the distribution +dist_noinst_DATA += tomcat/README.md tomcat/Makefile.inc + diff --git a/collectors/python.d.plugin/tomcat/README.md b/collectors/python.d.plugin/tomcat/README.md new file mode 120000 index 00000000..997090c3 --- /dev/null +++ b/collectors/python.d.plugin/tomcat/README.md @@ -0,0 +1 @@ +integrations/tomcat.md
\ No newline at end of file diff --git a/collectors/python.d.plugin/tomcat/integrations/tomcat.md b/collectors/python.d.plugin/tomcat/integrations/tomcat.md new file mode 100644 index 00000000..883f29dd --- /dev/null +++ b/collectors/python.d.plugin/tomcat/integrations/tomcat.md @@ -0,0 +1,203 @@ +<!--startmeta +custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tomcat/README.md" +meta_yaml: "https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/tomcat/metadata.yaml" +sidebar_label: "Tomcat" +learn_status: "Published" +learn_rel_path: "Data Collection/Web Servers and Web Proxies" +most_popular: False +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE" +endmeta--> + +# Tomcat + + +<img src="https://netdata.cloud/img/tomcat.svg" width="150"/> + + +Plugin: python.d.plugin +Module: tomcat + +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> + +## Overview + +This collector monitors Tomcat metrics about bandwidth, processing time, threads and more. + + +It parses the information provided by the http endpoint of the `/manager/status` in XML format + + +This collector is supported on all platforms. + +This collector supports collecting metrics from multiple instances of this integration, including remote instances. + +You need to provide the username and the password, to access the webserver's status page. Create a seperate user with read only rights for this particular endpoint + +### Default Behavior + +#### Auto-Detection + +If the Netdata Agent and the Tomcat webserver are in the same host, without configuration, module attempts to connect to http://localhost:8080/manager/status?XML=true, without any credentials. So it will probably fail. + +#### Limits + +This module is not supporting SSL communication. If you want a Netdata Agent to monitor a Tomcat deployment, you shouldnt try to monitor it via public network (public internet). Credentials are passed by Netdata in an unsecure port + +#### Performance Impact + +The default configuration for this integration is not expected to impose a significant performance impact on the system. + + +## Metrics + +Metrics grouped by *scope*. + +The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels. + + + +### Per Tomcat instance + +These metrics refer to the entire monitored application. + +This scope has no labels. + +Metrics: + +| Metric | Dimensions | Unit | +|:------|:----------|:----| +| tomcat.accesses | accesses, errors | requests/s | +| tomcat.bandwidth | sent, received | KiB/s | +| tomcat.processing_time | processing time | seconds | +| tomcat.threads | current, busy | current threads | +| tomcat.jvm | free, eden, survivor, tenured, code cache, compressed, metaspace | MiB | +| tomcat.jvm_eden | used, committed, max | MiB | +| tomcat.jvm_survivor | used, committed, max | MiB | +| tomcat.jvm_tenured | used, committed, max | MiB | + + + +## Alerts + +There are no alerts configured by default for this integration. + + +## Setup + +### Prerequisites + +#### Create a read-only `netdata` user, to monitor the `/status` endpoint. + +This is necessary for configuring the collector. + + +### Configuration + +#### File + +The configuration file name for this integration is `python.d/tomcat.conf`. + + +You can edit the configuration file using the `edit-config` script from the +Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory). + +```bash +cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata +sudo ./edit-config python.d/tomcat.conf +``` +#### Options + +There are 2 sections: + +* Global variables +* One or more JOBS that can define multiple different instances to monitor. + +The following options can be defined globally: priority, penalty, autodetection_retry, update_every, but can also be defined per JOB to override the global values.Additionally, the following collapsed table contains all the options that can be configured inside a JOB definition. + +Every configuration JOB starts with a `job_name` value which will appear in the dashboard, unless a `name` parameter is specified. + + +<details><summary>Config options per job</summary> + +| Name | Description | Default | Required | +|:----|:-----------|:-------|:--------:| +| update_every | Sets the default data collection frequency. | 5 | no | +| priority | Controls the order of charts at the netdata dashboard. | 60000 | no | +| autodetection_retry | Sets the job re-check interval in seconds. | 0 | no | +| penalty | Indicates whether to apply penalty to update_every in case of failures. | yes | no | +| url | The URL of the Tomcat server's status endpoint. Always add the suffix ?XML=true. | no | yes | +| user | A valid user with read permission to access the /manager/status endpoint of the server. Required if the endpoint is password protected | no | no | +| pass | A valid password for the user in question. Required if the endpoint is password protected | no | no | +| connector_name | The connector component that communicates with a web connector via the AJP protocol, e.g ajp-bio-8009 | | no | + +</details> + +#### Examples + +##### Basic + +A basic example configuration + +```yaml +localhost: + name : 'local' + url : 'http://localhost:8080/manager/status?XML=true' + +``` +##### Using an IPv4 endpoint + +A typical configuration using an IPv4 endpoint + +<details><summary>Config</summary> + +```yaml +local_ipv4: + name : 'local' + url : 'http://127.0.0.1:8080/manager/status?XML=true' + +``` +</details> + +##### Using an IPv6 endpoint + +A typical configuration using an IPv6 endpoint + +<details><summary>Config</summary> + +```yaml +local_ipv6: + name : 'local' + url : 'http://[::1]:8080/manager/status?XML=true' + +``` +</details> + + + +## Troubleshooting + +### Debug Mode + +To troubleshoot issues with the `tomcat` collector, run the `python.d.plugin` with the debug option enabled. The output +should give you clues as to why the collector isn't working. + +- Navigate to the `plugins.d` directory, usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case on + your system, open `netdata.conf` and look for the `plugins` setting under `[directories]`. + + ```bash + cd /usr/libexec/netdata/plugins.d/ + ``` + +- Switch to the `netdata` user. + + ```bash + sudo -u netdata -s + ``` + +- Run the `python.d.plugin` to debug the collector: + + ```bash + ./python.d.plugin tomcat debug trace + ``` + + diff --git a/collectors/python.d.plugin/tomcat/metadata.yaml b/collectors/python.d.plugin/tomcat/metadata.yaml new file mode 100644 index 00000000..e6852607 --- /dev/null +++ b/collectors/python.d.plugin/tomcat/metadata.yaml @@ -0,0 +1,200 @@ +plugin_name: python.d.plugin +modules: + - meta: + plugin_name: python.d.plugin + module_name: tomcat + monitored_instance: + name: Tomcat + link: "https://tomcat.apache.org/" + categories: + - data-collection.web-servers-and-web-proxies + icon_filename: "tomcat.svg" + related_resources: + integrations: + list: [] + info_provided_to_referring_integrations: + description: "" + keywords: + - apache + - tomcat + - webserver + - websocket + - jakarta + - javaEE + most_popular: false + overview: + data_collection: + metrics_description: | + This collector monitors Tomcat metrics about bandwidth, processing time, threads and more. + method_description: | + It parses the information provided by the http endpoint of the `/manager/status` in XML format + supported_platforms: + include: [] + exclude: [] + multi_instance: true + additional_permissions: + description: "You need to provide the username and the password, to access the webserver's status page. Create a seperate user with read only rights for this particular endpoint" + default_behavior: + auto_detection: + description: "If the Netdata Agent and the Tomcat webserver are in the same host, without configuration, module attempts to connect to http://localhost:8080/manager/status?XML=true, without any credentials. So it will probably fail." + limits: + description: "This module is not supporting SSL communication. If you want a Netdata Agent to monitor a Tomcat deployment, you shouldnt try to monitor it via public network (public internet). Credentials are passed by Netdata in an unsecure port" + performance_impact: + description: "" + setup: + prerequisites: + list: + - title: Create a read-only `netdata` user, to monitor the `/status` endpoint. + description: This is necessary for configuring the collector. + configuration: + file: + name: "python.d/tomcat.conf" + options: + description: | + There are 2 sections: + + * Global variables + * One or more JOBS that can define multiple different instances to monitor. + + The following options can be defined globally: priority, penalty, autodetection_retry, update_every, but can also be defined per JOB to override the global values.Additionally, the following collapsed table contains all the options that can be configured inside a JOB definition. + + Every configuration JOB starts with a `job_name` value which will appear in the dashboard, unless a `name` parameter is specified. + folding: + title: "Config options per job" + enabled: true + list: + - name: update_every + description: Sets the default data collection frequency. + default_value: 5 + required: false + - name: priority + description: Controls the order of charts at the netdata dashboard. + default_value: 60000 + required: false + - name: autodetection_retry + description: Sets the job re-check interval in seconds. + default_value: 0 + required: false + - name: penalty + description: Indicates whether to apply penalty to update_every in case of failures. + default_value: yes + required: false + - name: url + description: The URL of the Tomcat server's status endpoint. Always add the suffix ?XML=true. + default_value: no + required: true + - name: user + description: A valid user with read permission to access the /manager/status endpoint of the server. Required if the endpoint is password protected + default_value: no + required: false + - name: pass + description: A valid password for the user in question. Required if the endpoint is password protected + default_value: no + required: false + - name: connector_name + description: The connector component that communicates with a web connector via the AJP protocol, e.g ajp-bio-8009 + default_value: "" + required: false + examples: + folding: + enabled: true + title: "Config" + list: + - name: Basic + folding: + enabled: false + description: A basic example configuration + config: | + localhost: + name : 'local' + url : 'http://localhost:8080/manager/status?XML=true' + - name: Using an IPv4 endpoint + description: A typical configuration using an IPv4 endpoint + config: | + local_ipv4: + name : 'local' + url : 'http://127.0.0.1:8080/manager/status?XML=true' + - name: Using an IPv6 endpoint + description: A typical configuration using an IPv6 endpoint + config: | + local_ipv6: + name : 'local' + url : 'http://[::1]:8080/manager/status?XML=true' + troubleshooting: + problems: + list: [] + alerts: [] + metrics: + folding: + title: Metrics + enabled: false + description: "" + availability: [] + scopes: + - name: global + description: "These metrics refer to the entire monitored application." + labels: [] + metrics: + - name: tomcat.accesses + description: Requests + unit: "requests/s" + chart_type: area + dimensions: + - name: accesses + - name: errors + - name: tomcat.bandwidth + description: Bandwidth + unit: "KiB/s" + chart_type: area + dimensions: + - name: sent + - name: received + - name: tomcat.processing_time + description: processing time + unit: "seconds" + chart_type: area + dimensions: + - name: processing time + - name: tomcat.threads + description: Threads + unit: "current threads" + chart_type: area + dimensions: + - name: current + - name: busy + - name: tomcat.jvm + description: JVM Memory Pool Usage + unit: "MiB" + chart_type: stacked + dimensions: + - name: free + - name: eden + - name: survivor + - name: tenured + - name: code cache + - name: compressed + - name: metaspace + - name: tomcat.jvm_eden + description: Eden Memory Usage + unit: "MiB" + chart_type: area + dimensions: + - name: used + - name: committed + - name: max + - name: tomcat.jvm_survivor + description: Survivor Memory Usage + unit: "MiB" + chart_type: area + dimensions: + - name: used + - name: committed + - name: max + - name: tomcat.jvm_tenured + description: Tenured Memory Usage + unit: "MiB" + chart_type: area + dimensions: + - name: used + - name: committed + - name: max diff --git a/collectors/python.d.plugin/tomcat/tomcat.chart.py b/collectors/python.d.plugin/tomcat/tomcat.chart.py new file mode 100644 index 00000000..90315f8c --- /dev/null +++ b/collectors/python.d.plugin/tomcat/tomcat.chart.py @@ -0,0 +1,199 @@ +# -*- coding: utf-8 -*- +# Description: tomcat netdata python.d module +# Author: Pawel Krupa (paulfantom) +# Author: Wei He (Wing924) +# SPDX-License-Identifier: GPL-3.0-or-later + +import re +import xml.etree.ElementTree as ET + +from bases.FrameworkServices.UrlService import UrlService + +MiB = 1 << 20 + +# Regex fix for Tomcat single quote XML attributes +# affecting Tomcat < 8.5.24 & 9.0.2 running with Java > 9 +# cf. https://bz.apache.org/bugzilla/show_bug.cgi?id=61603 +single_quote_regex = re.compile(r"='([^']+)'([^']+)''") + +ORDER = [ + 'accesses', + 'bandwidth', + 'processing_time', + 'threads', + 'jvm', + 'jvm_eden', + 'jvm_survivor', + 'jvm_tenured', +] + +CHARTS = { + 'accesses': { + 'options': [None, 'Requests', 'requests/s', 'statistics', 'tomcat.accesses', 'area'], + 'lines': [ + ['requestCount', 'accesses', 'incremental'], + ['errorCount', 'errors', 'incremental'], + ] + }, + 'bandwidth': { + 'options': [None, 'Bandwidth', 'KiB/s', 'statistics', 'tomcat.bandwidth', 'area'], + 'lines': [ + ['bytesSent', 'sent', 'incremental', 1, 1024], + ['bytesReceived', 'received', 'incremental', 1, 1024], + ] + }, + 'processing_time': { + 'options': [None, 'processing time', 'seconds', 'statistics', 'tomcat.processing_time', 'area'], + 'lines': [ + ['processingTime', 'processing time', 'incremental', 1, 1000] + ] + }, + 'threads': { + 'options': [None, 'Threads', 'current threads', 'statistics', 'tomcat.threads', 'area'], + 'lines': [ + ['currentThreadCount', 'current', 'absolute'], + ['currentThreadsBusy', 'busy', 'absolute'] + ] + }, + 'jvm': { + 'options': [None, 'JVM Memory Pool Usage', 'MiB', 'memory', 'tomcat.jvm', 'stacked'], + 'lines': [ + ['free', 'free', 'absolute', 1, MiB], + ['eden_used', 'eden', 'absolute', 1, MiB], + ['survivor_used', 'survivor', 'absolute', 1, MiB], + ['tenured_used', 'tenured', 'absolute', 1, MiB], + ['code_cache_used', 'code cache', 'absolute', 1, MiB], + ['compressed_used', 'compressed', 'absolute', 1, MiB], + ['metaspace_used', 'metaspace', 'absolute', 1, MiB], + ] + }, + 'jvm_eden': { + 'options': [None, 'Eden Memory Usage', 'MiB', 'memory', 'tomcat.jvm_eden', 'area'], + 'lines': [ + ['eden_used', 'used', 'absolute', 1, MiB], + ['eden_committed', 'committed', 'absolute', 1, MiB], + ['eden_max', 'max', 'absolute', 1, MiB] + ] + }, + 'jvm_survivor': { + 'options': [None, 'Survivor Memory Usage', 'MiB', 'memory', 'tomcat.jvm_survivor', 'area'], + 'lines': [ + ['survivor_used', 'used', 'absolute', 1, MiB], + ['survivor_committed', 'committed', 'absolute', 1, MiB], + ['survivor_max', 'max', 'absolute', 1, MiB], + ] + }, + 'jvm_tenured': { + 'options': [None, 'Tenured Memory Usage', 'MiB', 'memory', 'tomcat.jvm_tenured', 'area'], + 'lines': [ + ['tenured_used', 'used', 'absolute', 1, MiB], + ['tenured_committed', 'committed', 'absolute', 1, MiB], + ['tenured_max', 'max', 'absolute', 1, MiB] + ] + } +} + + +class Service(UrlService): + def __init__(self, configuration=None, name=None): + UrlService.__init__(self, configuration=configuration, name=name) + self.order = ORDER + self.definitions = CHARTS + self.url = self.configuration.get('url', 'http://127.0.0.1:8080/manager/status?XML=true') + self.connector_name = self.configuration.get('connector_name', None) + self.parse = self.xml_parse + + def xml_parse(self, data): + try: + return ET.fromstring(data) + except ET.ParseError: + self.debug('%s is not a valid XML page. Please add "?XML=true" to tomcat status page.' % self.url) + return None + + def xml_single_quote_fix_parse(self, data): + data = single_quote_regex.sub(r"='\g<1>\g<2>'", data) + return self.xml_parse(data) + + def check(self): + self._manager = self._build_manager() + + raw_data = self._get_raw_data() + if not raw_data: + return False + + if single_quote_regex.search(raw_data): + self.warning('Tomcat status page is returning invalid single quote XML, please consider upgrading ' + 'your Tomcat installation. See https://bz.apache.org/bugzilla/show_bug.cgi?id=61603') + self.parse = self.xml_single_quote_fix_parse + + return self.parse(raw_data) is not None + + def _get_data(self): + """ + Format data received from http request + :return: dict + """ + data = None + raw_data = self._get_raw_data() + if raw_data: + xml = self.parse(raw_data) + if xml is None: + return None + + data = {} + + jvm = xml.find('jvm') + + connector = None + if self.connector_name: + for conn in xml.findall('connector'): + if self.connector_name in conn.get('name'): + connector = conn + break + else: + connector = xml.find('connector') + + memory = jvm.find('memory') + data['free'] = memory.get('free') + data['total'] = memory.get('total') + + for pool in jvm.findall('memorypool'): + name = pool.get('name') + if 'Eden Space' in name: + data['eden_used'] = pool.get('usageUsed') + data['eden_committed'] = pool.get('usageCommitted') + data['eden_max'] = pool.get('usageMax') + elif 'Survivor Space' in name: + data['survivor_used'] = pool.get('usageUsed') + data['survivor_committed'] = pool.get('usageCommitted') + data['survivor_max'] = pool.get('usageMax') + elif 'Tenured Gen' in name or 'Old Gen' in name: + data['tenured_used'] = pool.get('usageUsed') + data['tenured_committed'] = pool.get('usageCommitted') + data['tenured_max'] = pool.get('usageMax') + elif name == 'Code Cache': + data['code_cache_used'] = pool.get('usageUsed') + data['code_cache_committed'] = pool.get('usageCommitted') + data['code_cache_max'] = pool.get('usageMax') + elif name == 'Compressed': + data['compressed_used'] = pool.get('usageUsed') + data['compressed_committed'] = pool.get('usageCommitted') + data['compressed_max'] = pool.get('usageMax') + elif name == 'Metaspace': + data['metaspace_used'] = pool.get('usageUsed') + data['metaspace_committed'] = pool.get('usageCommitted') + data['metaspace_max'] = pool.get('usageMax') + + if connector is not None: + thread_info = connector.find('threadInfo') + data['currentThreadsBusy'] = thread_info.get('currentThreadsBusy') + data['currentThreadCount'] = thread_info.get('currentThreadCount') + + request_info = connector.find('requestInfo') + data['processingTime'] = request_info.get('processingTime') + data['requestCount'] = request_info.get('requestCount') + data['errorCount'] = request_info.get('errorCount') + data['bytesReceived'] = request_info.get('bytesReceived') + data['bytesSent'] = request_info.get('bytesSent') + + return data or None diff --git a/collectors/python.d.plugin/tomcat/tomcat.conf b/collectors/python.d.plugin/tomcat/tomcat.conf new file mode 100644 index 00000000..009591bd --- /dev/null +++ b/collectors/python.d.plugin/tomcat/tomcat.conf @@ -0,0 +1,89 @@ +# netdata python.d.plugin configuration for tomcat +# +# This file is in YaML format. Generally the format is: +# +# name: value +# +# There are 2 sections: +# - global variables +# - one or more JOBS +# +# JOBS allow you to collect values from multiple sources. +# Each source will have its own set of charts. +# +# JOB parameters have to be indented (using spaces only, example below). + +# ---------------------------------------------------------------------- +# Global Variables +# These variables set the defaults for all JOBs, however each JOB +# may define its own, overriding the defaults. + +# update_every sets the default data collection frequency. +# If unset, the python.d.plugin default is used. +# update_every: 1 + +# priority controls the order of charts at the netdata dashboard. +# Lower numbers move the charts towards the top of the page. +# If unset, the default for python.d.plugin is used. +# priority: 60000 + +# penalty indicates whether to apply penalty to update_every in case of failures. +# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes. +# penalty: yes + +# autodetection_retry sets the job re-check interval in seconds. +# The job is not deleted if check fails. +# Attempts to start the job are made once every autodetection_retry. +# This feature is disabled by default. +# autodetection_retry: 0 + +# ---------------------------------------------------------------------- +# JOBS (data collection sources) +# +# The default JOBS share the same *name*. JOBS with the same name +# are mutually exclusive. Only one of them will be allowed running at +# any time. This allows autodetection to try several alternatives and +# pick the one that works. +# +# Any number of jobs is supported. +# +# All python.d.plugin JOBS (for all its modules) support a set of +# predefined parameters. These are: +# +# job_name: +# name: myname # the JOB's name as it will appear at the +# # dashboard (by default is the job_name) +# # JOBs sharing a name are mutually exclusive +# update_every: 1 # the JOB's data collection frequency +# priority: 60000 # the JOB's order on the dashboard +# penalty: yes # the JOB's penalty +# autodetection_retry: 0 # the JOB's re-check interval in seconds +# +# Additionally to the above, tomcat also supports the following: +# +# url: 'URL' # the URL to fetch nginx's status stats +# +# if the URL is password protected, the following are supported: +# +# user: 'username' +# pass: 'password' +# +# if you have multiple connectors, the following are supported: +# +# connector_name: 'ajp-bio-8009' # default is null, which use first connector in status XML +# +# ---------------------------------------------------------------------- +# AUTO-DETECTION JOBS +# only one of them will run (they have the same name) + +localhost: + name : 'local' + url : 'http://localhost:8080/manager/status?XML=true' + +localipv4: + name : 'local' + url : 'http://127.0.0.1:8080/manager/status?XML=true' + +localipv6: + name : 'local' + url : 'http://[::1]:8080/manager/status?XML=true' |