1 files changed, 178 insertions, 0 deletions
diff --git a/source/configuration/modules/mmnormalize.rst b/source/configuration/modules/mmnormalize.rst
new file mode 100644
index 0000000..7d526a5
--- /dev/null
+++ b/source/configuration/modules/mmnormalize.rst
@@ -0,0 +1,178 @@
+Log Message Normalization Module (mmnormalize)
+==============================================
+
+**Module Name:    mmnormalize**
+
+**Available since:** 6.1.2+
+
+**Author:** Rainer Gerhards <rgerhards@adiscon.com>
+
+**Description**:
+
+This module provides the capability to normalize log messages via
+`liblognorm <http://www.liblognorm.com>`_. Thanks to liblognorm,
+unstructured text, like usually found in log messages, can very quickly
+be parsed and put into a normal form. This is done so quickly, that it
+should be possible to normalize events in realtime.
+
+This module is implemented via the output module interface. This means
+that mmnormalize should be called just like an action. After it has been
+called, the normalized message properties are available and can be
+accessed. These properties are called the "CEE/lumberjack" properties,
+because liblognorm creates a format that is inspired by the
+CEE/lumberjack approach.
+
+**Please note:** CEE/lumberjack properties are different from regular
+properties. They have always "$!" prepended to the property name given
+in the rulebase. Such a property needs to be called with
+**%$!propertyname%**.
+
+Note that from a performance point of view mmnormalize should only be called
+once on each message, if possible. To do so, place all rules into a single
+rule base. If that is not possible, you can safely call mmnormalize multiple
+times. This incurs a small performance drawback.
+
+Module Parameters
+~~~~~~~~~~~~~~~~~
+
+Note: parameter names are case-insensitive.
+
+.. function:: allow_regex <boolean>
+
+   **Default**: off
+
+   Specifies if regex field-type should be allowed. Regex field-type has
+   significantly higher computational overhead compared to other fields,
+   so it should be avoided when another field-type can achieve the desired
+   effect. Needs to be "on" for regex field-type to work.
+
+Action Parameters
+~~~~~~~~~~~~~~~~~
+
+Note: parameter names are case-insensitive.
+
+.. function:: ruleBase <word>
+
+   Specifies which rulebase file is to use. If there are multiple
+   mmnormalize instances, each one can use a different file. However, a
+   single instance can use only a single file. This parameter or **rule** MUST be
+   given, because normalization can only happen based on a rulebase. It
+   is recommended that an absolute path name is given. Information on
+   how to create the rulebase can be found in the `liblognorm
+   manual <http://www.liblognorm.com/files/manual/index.html>`_.
+
+.. function:: rule <array>
+
+   *(Available since: 8.26.0)*
+
+   Contains an array of strings which will be put together as the rulebase. This parameter
+   or **rulebase** MUST be given, because normalization can only happen based on a rulebase.
+
+.. function:: useRawMsg <boolean>
+
+   **Default**: off
+
+   Specifies if the raw message should be used for normalization (on)
+   or just the MSG part of the message (off).
+
+.. function:: path <word>
+
+   **Default**: $!
+
+   Specifies the JSON path under which parsed elements should be
+   placed. By default, all parsed properties are merged into root of
+   message properties. You can place them under a subtree, instead. You
+   can place them in local variables, also, by setting path="$.".
+
+.. function:: variable <word>
+
+   *(Available since: 8.5.1)*
+
+   Specifies if a variable insteed of property 'msg' should be used for
+   normalization. A variable can be property, local variable, json-path etc.
+   Please note that **useRawMsg** overrides this parameter, so if **useRawMsg**
+   is set, **variable** will be ignored and raw message will be used.
+
+
+
+
+Legacy Configuration Parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Note: parameter names are case-insensitive.
+
+-  $mmnormalizeRuleBase <rulebase-file> - equivalent to the "ruleBase"
+   parameter.
+-  $mmnormalizeUseRawMsg <on/off> - equivalent to the "useRawMsg"
+   parameter.
+
+See Also
+~~~~~~~~
+
+-  `First steps for
+   mmnormalize <http://www.rsyslog.com/normalizer-first-steps-for-mmnormalize/>`_
+-  `Log normalization and special
+   characters <http://www.rsyslog.com/log-normalization-and-special-characters/>`_
+-  `Log normalization and the leading
+   space <http://www.rsyslog.com/log-normalization-and-the-leading-space/>`_
+-  `Using mmnormalize effectively with Adiscon
+   LogAnalyzer <http://www.rsyslog.com/using-rsyslog-mmnormalize-module-effectively-with-adiscon-loganalyzer/>`_
+
+Caveats/Known Bugs
+~~~~~~~~~~~~~~~~~~
+
+None known at this time.
+
+Example
+~~~~~~~
+
+**Sample 1:**
+
+In this sample messages are received via imtcp. Then they are normalized with the given rulebase.
+After that they are written in a file.
+
+::
+
+  module(load="mmnormalize")
+  module(load="imtcp")
+
+  input(type="imtcp" port="10514" ruleset="outp")
+
+  ruleset(name="outp") {
+  	action(type="mmnormalize" rulebase="/tmp/rules.rulebase")
+  	action(type="omfile" File="/tmp/output")
+  }
+
+**Sample 2:**
+
+In this sample messages are received via imtcp. Then they are normalized based on the given rules.
+The strings from **rule** are put together and are equal to a rulebase with the same content.
+
+::
+
+  module(load="mmnormalize")
+  module(load="imtcp")
+
+  input(type="imtcp" port="10514" ruleset="outp")
+
+  ruleset(name="outp") {
+  	action(type="mmnormalize" rule=["rule=:%host:word% %tag:char-to:\\x3a%: no longer listening on %ip:ipv4%#%port:number%", "rule=:%host:word% %ip:ipv4% user was logged out"])
+  	action(type="omfile" File="/tmp/output")
+  }
+
+**Sample 3:**
+
+This activates the module and applies normalization to all messages:
+
+::
+
+  module(load="mmnormalize")
+  action(type="mmnormalize" ruleBase="/path/to/rulebase.rb")
+
+The same in legacy format:
+
+::
+
+  $ModLoad mmnormalize
+  $mmnormalizeRuleBase /path/to/rulebase.rb
+  *.* :mmnormalize: