summaryrefslogtreecommitdiffstats
path: root/source/configuration/action/rsconf1_repeatedmsgreduction.rst
blob: 33ddeb90e5602e4f0d0572b8f4620bf03ca9a944 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
$RepeatedMsgReduction
---------------------

**Type:** global configuration parameter

**Default:** off

Description
^^^^^^^^^^^

This parameter models old sysklogd legacy. **Note that many people,
including the rsyslog authors, consider this to be a misfeature.** See
*Discussion* below to learn why.

This parameter specifies whether or not repeated messages should be
reduced (this is the "Last line repeated n times" feature). If set to
*on*, repeated messages are reduced. If kept at *off*, every message is
logged. In very early versions of rsyslog, this was controlled by the
*-e* command line option.

What is a repeated message
^^^^^^^^^^^^^^^^^^^^^^^^^^

For a message to be classified as repeated, the following properties
must be **identical**:

* msg
* hostname
* procid
* appname

Note that rate-limiters are usually applied to specific input sources
or processes. So first and foremost the input source must be the same
to classify a messages as a duplicated.

You may want to check out
`testing rsyslog ratelimiting <http://www.rsyslog.com/first-try-to-test-rate-limiting/>`_
for some extra information on the per-process ratelimiting.

Discussion
^^^^^^^^^^

* Very old versions of rsyslog did not have the ability to include the
  repeated message itself within the repeat message.

* Versions before 7.3.2 applied repeat message reduction to the output
  side. This had some implications:

  - they did not account for the actual message origin, so two processes
    emitting an equally-looking message triggered the repeated message
    reduction code

  - repeat message processing could be set on a per-action basis, which
    has switched to per-input basis for 7.3.2 and above

* While turning this feature on can save some space in logs, most log analysis
  tools need to see the repeated messages, they can't handle the
  "last message repeated" format.
* This is a feature that worked decades ago when logs were small and reviewed
  by a human, it fails badly on high volume logs processed by tools.

Sample
^^^^^^

This turns on repeated message reduction (**not** recommended):

::

 $RepeatedMsgReduction on    # do not log repeated messages