summaryrefslogtreecommitdiffstats
path: root/source/configuration/modules/omamqp1.rst
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--source/configuration/modules/omamqp1.rst476
1 files changed, 476 insertions, 0 deletions
diff --git a/source/configuration/modules/omamqp1.rst b/source/configuration/modules/omamqp1.rst
new file mode 100644
index 0000000..c5e2e07
--- /dev/null
+++ b/source/configuration/modules/omamqp1.rst
@@ -0,0 +1,476 @@
+*****************************************
+omamqp1: AMQP 1.0 Messaging Output Module
+*****************************************
+
+=========================== ===========================================================================
+**Module Name:** **omamqp1**
+**Available Since:** **8.17.0**
+**Original Author:** Ken Giusti <kgiusti@gmail.com>
+=========================== ===========================================================================
+
+
+Purpose
+=======
+
+This module provides the ability to send logging via an AMQP 1.0
+compliant message bus. It puts the log messages into an AMQP
+message and sends the message to a destination on the bus.
+
+
+Notable Features
+================
+
+- :ref:`omamqp1-message-format`
+- :ref:`omamqp1-interoperability`
+
+
+Configuration Parameters
+========================
+
+.. note::
+
+ Parameter names are case-insensitive.
+
+
+Action Parameters
+-----------------
+
+Host
+^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "word", "5672", "yes", "none"
+
+The address of the message bus in *host[:port]* format.
+The port defaults to 5672 if absent. Examples: *"localhost"*,
+*"127.0.0.1:9999"*, *"bus.someplace.org"*
+
+
+Target
+^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "word", "none", "yes", "none"
+
+The destination for the generated messages. This can be
+the name of a queue or topic. On some messages buses it may be
+necessary to create this target manually. Example: *"amq.topic"*
+
+
+Username
+^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "word", "none", "no", "none"
+
+Used by SASL to authenticate with the message bus.
+
+
+Password
+^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "word", "none", "no", "none"
+
+Used by SASL to authenticate with the message bus.
+
+
+Template
+^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "word", "RSYSLOG_FileFormat", "no", "none"
+
+Format for the log messages.
+
+
+idleTimeout
+^^^^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "integer", "0", "no", "none"
+
+The idle timeout in seconds. This enables connection
+heartbeats and is used to detect a failed connection to the message
+bus. Set to zero to disable.
+
+
+reconnectDelay
+^^^^^^^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "integer", "5", "no", "none"
+
+The time in seconds this module will delay before
+attempting to re-established a failed connection to the message bus.
+
+
+MaxRetries
+^^^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "integer", "10", "no", "none"
+
+The number of times an undeliverable message is
+re-sent to the message bus before it is dropped. This is unrelated
+to rsyslog's action.resumeRetryCount. Once the connection to the
+message bus is active this module is ready to receive log messages
+from rsyslog (i.e. the module has 'resumed'). Even though the
+connection is active, any particular message may be rejected by the
+message bus (e.g. 'unrouteable'). The module will retry
+(e.g. 'suspend') for up to *maxRetries* attempts before discarding
+the message as undeliverable. Setting this to zero disables the
+limit and unrouteable messages will be retried as long as the
+connection stays up. You probably do not want that to
+happen.
+
+
+DisableSASL
+^^^^^^^^^^^
+
+.. csv-table::
+ :header: "type", "default", "mandatory", "|FmtObsoleteName| directive"
+ :widths: auto
+ :class: parameter-table
+
+ "integer", "0", "no", "none"
+
+Setting this to a non-zero value will disable SASL
+negotiation. Only necessary if the message bus does not offer SASL
+support.
+
+
+Dependencies
+============
+
+* `libqpid-proton <http://qpid.apache.org/proton>`_
+
+Configure
+=========
+
+.. code-block:: none
+
+ ./configure --enable-omamqp1
+
+
+.. _omamqp1-message-format:
+
+Message Format
+==============
+
+Messages sent from this module to the message bus contain an AMQP List
+in the message body. This list contains one or more log messages as
+AMQP String types. Each string entry is a single log message. The
+list is ordered such that the oldest log appears at the front of the
+list (e.g. list index 0), whilst the most recent log is at the end of
+the list.
+
+
+.. _omamqp1-interoperability:
+
+Interoperability
+================
+
+The output plugin has been tested against the following messaging systems:
+
+* `QPID C++ Message Broker <http://qpid.apache.org/components/cpp-broker>`_
+* `QPID Dispatch Message Router <http://qpid.apache.org/components/dispatch-router>`_
+
+
+TODO
+====
+
+- Add support for SSL connections.
+
+
+Examples
+========
+
+Example 1
+---------
+
+This example shows a minimal configuration. The module will attempt
+to connect to a QPID broker at *broker.amqp.org*. Messages are
+sent to the *amq.topic* topic, which exists on the broker by default:
+
+.. code-block:: none
+
+ module(load="omamqp1")
+ action(type="omamqp1"
+ host="broker.amqp.org"
+ target="amq.topic")
+
+
+Example 2
+---------
+
+This example forces rsyslogd to authenticate with the message bus.
+The message bus must be provisioned such that user *joe* is allowed to
+send to the message bus. All messages are sent to *log-queue*. It is
+assumed that *log-queue* has already been provisioned:
+
+.. code-block:: none
+
+ module(load="omamqp1")
+
+ action(type="omamqp1"
+ host="bus.amqp.org"
+ target="log-queue"
+ username="joe"
+ password="trustno1")
+
+
+Notes on use with the QPID C++ broker (qpidd)
+=============================================
+
+*Note well*: These notes assume use of version 0.34 of the QPID C++
+broker. Previous versions may not be fully compatible.
+
+To use the Apache QPID C++ broker **qpidd** as the message bus, a
+version of qpidd that supports the AMQP 1.0 protocol must be used.
+
+Since qpidd can be packaged without AMQP 1.0 support you should verify
+AMQP 1.0 has been enabled by checking for AMQP 1.0 related options in
+the qpidd help text. For example:
+
+.. code-block:: none
+
+ qpidd --help
+
+ ...
+
+ AMQP 1.0 Options:
+ --domain DOMAIN Domain of this broker
+ --queue-patterns PATTERN Pattern for on-demand queues
+ --topic-patterns PATTERN Pattern for on-demand topics
+
+
+If no AMQP 1.0 related options appear in the help output then your
+instance of qpidd does not support AMQP 1.0 and cannot be used with
+this output module.
+
+The destination for message (target) *must* be created before log
+messages arrive. This can be done using the qpid-config tool.
+
+Example:
+
+.. code-block:: none
+
+ qpid-config add queue rsyslogd
+
+
+Alternatively the target can be created on demand by configuring a
+queue-pattern (or topic-pattern) that matches the target. To do this,
+add a *queue-patterns* or *topic_patterns* configuration directive to
+the qpidd configuration file /etc/qpid/qpidd.conf.
+
+For example to have qpidd automatically create a queue named
+*rsyslogd* add the following to the qpidd configuration file:
+
+.. code-block:: none
+
+ queue-patterns=rsyslogd
+
+
+or, if a topic behavior is desired instead of a queue:
+
+.. code-block:: none
+
+ topic-patterns=rsyslogd
+
+
+These dynamic targets are auto-delete and will be destroyed once there
+are no longer any subscribers or queue-bound messages.
+
+Versions of qpidd <= 0.34 also need to have the SASL service name set
+to *"amqp"* if SASL authentication is used. Add this to the qpidd.conf
+file:
+
+.. code-block:: none
+
+ sasl-service-name=amqp
+
+
+Notes on use with the QPID Dispatch Router (qdrouterd)
+======================================================
+
+*Note well*: These notes assume use of version 0.5 of the QPID Dispatch
+Router **qdrouterd**. Previous versions may not be fully compatible.
+
+The default qdrouterd configuration does not have SASL authentication
+turned on. If SASL authentication is required you must configure SASL
+in the qdrouter configuration file /etc/qpid-dispatch/qdrouterd.conf
+
+First create a SASL configuration file for qdrouterd. This
+configuration file is usually /etc/sasl2/qdrouterd.conf, but its
+default location may vary depending on your platform's configuration.
+
+This document assumes you understand how to properly configure Cyrus
+SASL.
+
+Here is an example qdrouterd SASL configuration file that allows the
+client to use either the **DIGEST-MD5** or **PLAIN** authentication
+mechanisms and specifies the path to the SASL user credentials
+database:
+
+.. code-block:: none
+
+ pwcheck_method: auxprop
+ auxprop_plugin: sasldb
+ sasldb_path: /var/lib/qdrouterd/qdrouterd.sasldb
+ mech_list: DIGEST-MD5 PLAIN
+
+
+Once a SASL configuration file has been set up for qdrouterd the path
+to the directory holding the configuration file and the name of the
+configuration file itself **without the '.conf' suffix** must be added
+to the /etc/qpid-dispatch/qdrouterd.conf configuration file. This is
+done by adding *saslConfigPath* and *saslConfigName* to the
+*container* section of the configuration file. For example, assuming
+the file /etc/sasl2/qdrouterd.conf holds the qdrouterd SASL
+configuration:
+
+.. code-block:: none
+
+ container {
+ workerThreads: 4
+ containerName: Qpid.Dispatch.Router.A
+ saslConfigPath: /etc/sasl2
+ saslConfigName: qdrouterd
+ }
+
+
+In addition the address used by the omamqp1 module to connect to
+qdrouterd must have SASL authentication turned on. This is done by
+adding the *authenticatePeer* attribute set to 'yes' to the
+corresponding *listener* entry:
+
+.. code-block:: none
+
+ listener {
+ addr: 0.0.0.0
+ port: amqp
+ authenticatePeer: yes
+ }
+
+
+This should complete the SASL setup needed by qdrouterd.
+
+The target address used as the destination for the log messages must
+be picked with care. qdrouterd uses the prefix of the target address
+to determine the forwarding pattern used for messages sent to that
+target address. Addresses starting with the prefix *queue* are
+distributed to only one message receiver. If there are multiple
+message consumers listening to that target address only one listener
+will receive the message - mimicking the behavior of a queue with
+competing subscribers. For example: *queue/rsyslogd*
+
+If a multicast pattern is desired - where all active listeners receive
+their own copy of the message - the target address prefix *multicast*
+may be used. For example: *multicast/rsyslogd*
+
+Note well: if there are no active receivers for the log messages the
+messages will be rejected by qdrouterd since the messages are
+undeliverable. In this case the omamqp1 module will return a
+**SUSPENDED** status to the rsyslogd main task. rsyslogd may then
+re-submit the rejected log messages to the module which will attempt
+to send them again. This retry option is configured via rsyslogd - it
+is not part of this module. Refer to the rsyslogd actions
+documentation.
+
+
+Using qdrouterd in combination with qpidd
+=========================================
+
+A qdrouterd-based message bus can use a broker as a message storage
+mechanism for those that require broker-based message services (such
+as a message store). This section explains how to configure qdrouterd
+and qpidd for this type of deployment. Please read the above notes
+for deploying qpidd and qdrouterd first.
+
+Each qdrouterd instance that is to connect the broker to the message
+bus must define a *connector* section in the qdrouterd.conf file.
+This connector contains the addressing information necessary to have
+the message bus set up a connection to the broker. For example, if a
+broker is available on host broker.host.com at port 5672:
+
+.. code-block:: none
+
+ connector {
+ name: mybroker
+ role: on-demand
+ addr: broker.host.com
+ port: 5672
+ }
+
+
+In order to route messages to and from the broker, a static *link
+route* must be configured on qdrouterd. This link route contains a
+target address prefix and the name of the connector to use for
+forwarding matching messages.
+
+For example, to have qdrouterd forward messages that have a target
+address prefixed by "Broker" to the connector defined above, the
+following link pattern must be added to the qdrouterd.conf
+configuration:
+
+.. code-block:: none
+
+ linkRoutePattern {
+ prefix: /Broker/
+ connector: mybroker
+ }
+
+
+A queue must then be created on the broker. The name of the queue
+must be prefixed by the same prefix specified in the linkRoutePattern
+entry. For example:
+
+.. code-block:: none
+
+ $ qpid-config add queue Broker/rsyslogd
+
+
+Lastly use the name of the queue for the target address for the omamqp
+module action. For example, assuming qdrouterd is listening on local
+port 5672:
+
+.. code-block:: none
+
+ action(type="omamqp1"
+ host="localhost:5672"
+ target="Broker/rsyslogd")
+
+
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-20 05:35:32 +0000