summaryrefslogtreecommitdiffstats
path: root/collectors/python.d.plugin/mongodb
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2019-09-03 10:23:38 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2019-09-03 10:23:38 +0000
commit574098461cd45be12a497afbdac6f93c58978387 (patch)
tree9eb60a5930b7c20d42f7fde1e234cae3968ed3d9 /collectors/python.d.plugin/mongodb
parentAdding upstream version 1.16.1. (diff)
downloadnetdata-574098461cd45be12a497afbdac6f93c58978387.tar.xz
netdata-574098461cd45be12a497afbdac6f93c58978387.zip
Adding upstream version 1.17.0.upstream/1.17.0
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'collectors/python.d.plugin/mongodb')
-rw-r--r--collectors/python.d.plugin/mongodb/README.md265
-rw-r--r--collectors/python.d.plugin/mongodb/mongodb.chart.py70
-rw-r--r--collectors/python.d.plugin/mongodb/mongodb.conf10
3 files changed, 217 insertions, 128 deletions
diff --git a/collectors/python.d.plugin/mongodb/README.md b/collectors/python.d.plugin/mongodb/README.md
index 3ddf137d7..fd694c1e5 100644
--- a/collectors/python.d.plugin/mongodb/README.md
+++ b/collectors/python.d.plugin/mongodb/README.md
@@ -3,135 +3,161 @@
Module monitor mongodb performance and health metrics
**Requirements:**
- * `python-pymongo` package v2.4+.
-You need to install it manually.
+- `python-pymongo` package v2.4+.
+You need to install it manually.
Number of charts depends on mongodb version, storage engine and other features (replication):
-1. **Read requests**:
- * query
- * getmore (operation the cursor executes to get additional data from query)
-
-2. **Write requests**:
- * insert
- * delete
- * update
-
-3. **Active clients**:
- * readers (number of clients with read operations in progress or queued)
- * writers (number of clients with write operations in progress or queued)
-
-4. **Journal transactions**:
- * commits (count of transactions that have been written to the journal)
-
-5. **Data written to the journal**:
- * volume (volume of data)
-
-6. **Background flush** (MMAPv1):
- * average ms (average time taken by flushes to execute)
- * last ms (time taken by the last flush)
-
-8. **Read tickets** (WiredTiger):
- * in use (number of read tickets in use)
- * available (number of available read tickets remaining)
-
-9. **Write tickets** (WiredTiger):
- * in use (number of write tickets in use)
- * available (number of available write tickets remaining)
-
-10. **Cursors**:
- * opened (number of cursors currently opened by MongoDB for clients)
- * timedOut (number of cursors that have timed)
- * noTimeout (number of open cursors with timeout disabled)
-
-11. **Connections**:
- * connected (number of clients currently connected to the database server)
- * unused (number of unused connections available for new clients)
-
-12. **Memory usage metrics**:
- * virtual
- * resident (amount of memory used by the database process)
- * mapped
- * non mapped
-
-13. **Page faults**:
- * page faults (number of times MongoDB had to request from disk)
-
-14. **Cache metrics** (WiredTiger):
- * percentage of bytes currently in the cache (amount of space taken by cached data)
- * percantage of tracked dirty bytes in the cache (amount of space taken by dirty data)
-
-15. **Pages evicted from cache** (WiredTiger):
- * modified
- * unmodified
-
-16. **Queued requests**:
- * readers (number of read request currently queued)
- * writers (number of write request currently queued)
-
-17. **Errors**:
- * msg (number of message assertions raised)
- * warning (number of warning assertions raised)
- * regular (number of regular assertions raised)
- * user (number of assertions corresponding to errors generated by users)
-
-18. **Storage metrics** (one chart for every database)
- * dataSize (size of all documents + padding in the database)
- * indexSize (size of all indexes in the database)
- * storageSize (size of all extents in the database)
-
-19. **Documents in the database** (one chart for all databases)
- * documents (number of objects in the database among all the collections)
-
-20. **tcmalloc metrics**
- * central cache free
- * current total thread cache
- * pageheap free
- * pageheap unmapped
- * thread cache free
- * transfer cache free
- * heap size
-
-21. **Commands total/failed rate**
- * count
- * createIndex
- * delete
- * eval
- * findAndModify
- * insert
-
-22. **Locks metrics** (acquireCount metrics - number of times the lock was acquired in the specified mode)
- * Global lock
- * Database lock
- * Collection lock
- * Metadata lock
- * oplog lock
-
-23. **Replica set members state**
- * state
-
-24. **Oplog window**
- * window (interval of time between the oldest and the latest entries in the oplog)
-
-25. **Replication lag**
- * member (time when last entry from the oplog was applied for every member)
-
-26. **Replication set member heartbeat latency**
- * member (time when last heartbeat was received from replica set member)
-
-### prerequisite
-Create a read-only user for the netdata in the admin database.
-
-1. Authenticate as the admin user.
+1. **Read requests**:
+
+ - query
+ - getmore (operation the cursor executes to get additional data from query)
+
+2. **Write requests**:
+
+ - insert
+ - delete
+ - update
+
+3. **Active clients**:
+
+ - readers (number of clients with read operations in progress or queued)
+ - writers (number of clients with write operations in progress or queued)
+
+4. **Journal transactions**:
+
+ - commits (count of transactions that have been written to the journal)
+
+5. **Data written to the journal**:
+
+ - volume (volume of data)
+
+6. **Background flush** (MMAPv1):
+
+ - average ms (average time taken by flushes to execute)
+ - last ms (time taken by the last flush)
+
+7. **Read tickets** (WiredTiger):
+
+ - in use (number of read tickets in use)
+ - available (number of available read tickets remaining)
+
+8. **Write tickets** (WiredTiger):
+
+ - in use (number of write tickets in use)
+ - available (number of available write tickets remaining)
+
+9. **Cursors**:
+
+- opened (number of cursors currently opened by MongoDB for clients)
+- timedOut (number of cursors that have timed)
+- noTimeout (number of open cursors with timeout disabled)
+
+10. **Connections**:
+
+ - connected (number of clients currently connected to the database server)
+ - unused (number of unused connections available for new clients)
+
+11. **Memory usage metrics**:
+
+ - virtual
+ - resident (amount of memory used by the database process)
+ - mapped
+ - non mapped
+
+12. **Page faults**:
+
+ - page faults (number of times MongoDB had to request from disk)
+
+13. **Cache metrics** (WiredTiger):
+
+ - percentage of bytes currently in the cache (amount of space taken by cached data)
+ - percantage of tracked dirty bytes in the cache (amount of space taken by dirty data)
+
+14. **Pages evicted from cache** (WiredTiger):
+
+ - modified
+ - unmodified
+
+15. **Queued requests**:
+
+ - readers (number of read request currently queued)
+ - writers (number of write request currently queued)
+
+16. **Errors**:
+
+ - msg (number of message assertions raised)
+ - warning (number of warning assertions raised)
+ - regular (number of regular assertions raised)
+ - user (number of assertions corresponding to errors generated by users)
+
+17. **Storage metrics** (one chart for every database)
+
+ - dataSize (size of all documents + padding in the database)
+ - indexSize (size of all indexes in the database)
+ - storageSize (size of all extents in the database)
+
+18. **Documents in the database** (one chart for all databases)
+
+- documents (number of objects in the database among all the collections)
+
+19. **tcmalloc metrics**
+
+ - central cache free
+ - current total thread cache
+ - pageheap free
+ - pageheap unmapped
+ - thread cache free
+ - transfer cache free
+ - heap size
+
+20. **Commands total/failed rate**
+
+ - count
+ - createIndex
+ - delete
+ - eval
+ - findAndModify
+ - insert
+
+21. **Locks metrics** (acquireCount metrics - number of times the lock was acquired in the specified mode)
+
+ - Global lock
+ - Database lock
+ - Collection lock
+ - Metadata lock
+ - oplog lock
+
+22. **Replica set members state**
+
+ - state
+
+23. **Oplog window**
+
+ - window (interval of time between the oldest and the latest entries in the oplog)
+
+24. **Replication lag**
+
+ - member (time when last entry from the oplog was applied for every member)
+
+25. **Replication set member heartbeat latency**
+
+ - member (time when last heartbeat was received from replica set member)
+
+## prerequisite
+
+Create a read-only user for Netdata in the admin database.
+
+1. Authenticate as the admin user.
```
use admin
db.auth("admin", "<MONGODB_ADMIN_PASSWORD>")
```
-2. Create a user.
+2. Create a user.
```
# MongoDB 2.x.
@@ -161,11 +187,10 @@ local:
port : 27017
user : 'netdata'
pass : 'netdata'
-
```
If no configuration is given, module will attempt to connect to mongodb daemon on `127.0.0.1:27017` address
---
-[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fcollectors%2Fpython.d.plugin%2Fmongodb%2FREADME&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)]()
+[![analytics](https://www.google-analytics.com/collect?v=1&aip=1&t=pageview&_s=1&ds=github&dr=https%3A%2F%2Fgithub.com%2Fnetdata%2Fnetdata&dl=https%3A%2F%2Fmy-netdata.io%2Fgithub%2Fcollectors%2Fpython.d.plugin%2Fmongodb%2FREADME&_u=MAC~&cid=5792dfd7-8dc4-476b-af31-da2fdb9f93d2&tid=UA-64295674-3)](<>)
diff --git a/collectors/python.d.plugin/mongodb/mongodb.chart.py b/collectors/python.d.plugin/mongodb/mongodb.chart.py
index 5db48cb12..0dbe82ff9 100644
--- a/collectors/python.d.plugin/mongodb/mongodb.chart.py
+++ b/collectors/python.d.plugin/mongodb/mongodb.chart.py
@@ -3,6 +3,8 @@
# Author: ilyam8
# SPDX-License-Identifier: GPL-3.0-or-later
+import ssl
+
from copy import deepcopy
from datetime import datetime
from sys import exc_info
@@ -418,18 +420,31 @@ CHARTS = {
}
}
+DEFAULT_HOST = '127.0.0.1'
+DEFAULT_PORT = 27017
+DEFAULT_TIMEOUT = 100
+DEFAULT_AUTHDB = 'admin'
+
+CONN_PARAM_HOST = 'host'
+CONN_PARAM_PORT = 'port'
+CONN_PARAM_SERVER_SELECTION_TIMEOUT_MS = 'serverselectiontimeoutms'
+CONN_PARAM_SSL_SSL = 'ssl'
+CONN_PARAM_SSL_CERT_REQS = 'ssl_cert_reqs'
+CONN_PARAM_SSL_CA_CERTS = 'ssl_ca_certs'
+CONN_PARAM_SSL_CRL_FILE = 'ssl_crlfile'
+CONN_PARAM_SSL_CERT_FILE = 'ssl_certfile'
+CONN_PARAM_SSL_KEY_FILE = 'ssl_keyfile'
+CONN_PARAM_SSL_PEM_PASSPHRASE = 'ssl_pem_passphrase'
+
class Service(SimpleService):
def __init__(self, configuration=None, name=None):
SimpleService.__init__(self, configuration=configuration, name=name)
self.order = ORDER[:]
self.definitions = deepcopy(CHARTS)
- self.authdb = self.configuration.get('authdb', 'admin')
+ self.authdb = self.configuration.get('authdb', DEFAULT_AUTHDB)
self.user = self.configuration.get('user')
self.password = self.configuration.get('pass')
- self.host = self.configuration.get('host', '127.0.0.1')
- self.port = self.configuration.get('port', 27017)
- self.timeout = self.configuration.get('timeout', 100)
self.metrics_to_collect = deepcopy(DEFAULT_METRICS)
self.connection = None
self.do_replica = None
@@ -705,14 +720,53 @@ class Service(SimpleService):
return data
- def _create_connection(self):
- conn_vars = {'host': self.host, 'port': self.port}
+ def build_ssl_connection_params(self):
+ conf = self.configuration
+
+ def cert_req(v):
+ if v is None:
+ return None
+ if not v:
+ return ssl.CERT_NONE
+ return ssl.CERT_REQUIRED
+
+ ssl_params = {
+ CONN_PARAM_SSL_SSL: conf.get(CONN_PARAM_SSL_SSL),
+ CONN_PARAM_SSL_CERT_REQS: cert_req(conf.get(CONN_PARAM_SSL_CERT_REQS)),
+ CONN_PARAM_SSL_CA_CERTS: conf.get(CONN_PARAM_SSL_CA_CERTS),
+ CONN_PARAM_SSL_CRL_FILE: conf.get(CONN_PARAM_SSL_CRL_FILE),
+ CONN_PARAM_SSL_CERT_FILE: conf.get(CONN_PARAM_SSL_CERT_FILE),
+ CONN_PARAM_SSL_KEY_FILE: conf.get(CONN_PARAM_SSL_KEY_FILE),
+ CONN_PARAM_SSL_PEM_PASSPHRASE: conf.get(CONN_PARAM_SSL_PEM_PASSPHRASE),
+ }
+
+ ssl_params = dict((k, v) for k, v in ssl_params.items() if v is not None)
+
+ return ssl_params
+
+ def build_connection_params(self):
+ conf = self.configuration
+ params = {
+ CONN_PARAM_HOST: conf.get(CONN_PARAM_HOST, DEFAULT_HOST),
+ CONN_PARAM_PORT: conf.get(CONN_PARAM_PORT, DEFAULT_PORT),
+ }
if hasattr(MongoClient, 'server_selection_timeout'):
- conn_vars.update({'serverselectiontimeoutms': self.timeout})
+ params[CONN_PARAM_SERVER_SELECTION_TIMEOUT_MS] = conf.get('timeout', DEFAULT_TIMEOUT)
+
+ params.update(self.build_ssl_connection_params())
+ return params
+
+ def _create_connection(self):
+ params = self.build_connection_params()
+ self.debug('creating connection, connection params: {0}'.format(sorted(params)))
+
try:
- connection = MongoClient(**conn_vars)
+ connection = MongoClient(**params)
if self.user and self.password:
+ self.debug('authenticating, user: {0}, password: {1}'.format(self.user, self.password))
getattr(connection, self.authdb).authenticate(name=self.user, password=self.password)
+ else:
+ self.debug('skip authenticating, user and password are not set')
# elif self.user:
# connection.admin.authenticate(name=self.user, mechanism='MONGODB-X509')
server_status = connection.admin.command('serverStatus')
diff --git a/collectors/python.d.plugin/mongodb/mongodb.conf b/collectors/python.d.plugin/mongodb/mongodb.conf
index 2dded40ae..9f660f594 100644
--- a/collectors/python.d.plugin/mongodb/mongodb.conf
+++ b/collectors/python.d.plugin/mongodb/mongodb.conf
@@ -71,6 +71,16 @@
# user: 'username' # the mongodb username to use
# pass: 'password' # the mongodb password to use
#
+# SSL connection parameters (https://api.mongodb.com/python/current/examples/tls.html):
+#
+# ssl: yes # connect to the server using TLS
+# ssl_cert_reqs: yes # require a certificate from the server when TLS is enabled
+# ssl_ca_certs: '/path/to/ca.pem' # use a specific set of CA certificates
+# ssl_crlfile: '/path/to/crl.pem' # use a certificate revocation lists
+# ssl_certfile: '/path/to/client.pem' # use a client certificate
+# ssl_keyfile: '/path/to/key.pem' # use a specific client certificate key
+# ssl_pem_passphrase: 'passphrase' # use a passphrase to decrypt encrypted private keys
+#
# ----------------------------------------------------------------------
# to connect to the mongodb on localhost, without a password: