diff options
Diffstat (limited to '')
| -rw-r--r-- | source/configuration/properties.rst | 322 |
1 files changed, 322 insertions, 0 deletions
diff --git a/source/configuration/properties.rst b/source/configuration/properties.rst new file mode 100644 index 0000000..863984f --- /dev/null +++ b/source/configuration/properties.rst @@ -0,0 +1,322 @@ +rsyslog Properties +================== + +Data items in rsyslog are called "properties". They can have different +origin. The most important ones are those that stem from received +messages. But there are also others. Whenever you want to access data items, +you need to access the respective property. + +Properties are used in + +- :doc:`templates <templates>` +- conditional statements + +The property name is case-insensitive (prior to 3.17.0, they were case-sensitive). + +Note: many users refer to "rsyslog properties" as "rsyslog variables". You can treat +them as synonymous. +Read how `rsyslog lead author Rainer Gerhards explains the naming +difference <https://rainer.gerhards.net/2020/08/rsyslog-template-variables-where-to-find-them.html">`_. + +Message Properties +------------------ +These are extracted by rsyslog parsers from the original message. All message +properties start with a letter. + +The following message properties exist: + +**msg** + the MSG part of the message (aka "the message" ;)) + +**rawmsg** + the message "as is". Should be useful for debugging and also if a message + should be forwarded totally unaltered. + Please notice *EscapecontrolCharactersOnReceive* is enabled by default, so + it may be different from what was received in the socket. + +**rawmsg-after-pri** + Almost the same as **rawmsg**, but the syslog PRI is removed. + If no PRI was present, **rawmsg-after-pri** is identical to + **rawmsg**. Note that the syslog PRI is header field that + contains information on syslog facility and severity. It is + enclosed in greater-than and less-than characters, e.g. + "<191>". This field is often not written to log files, but + usually needs to be present for the receiver to properly + classify the message. There are some rare cases where one + wants the raw message, but not the PRI. You can use this + property to obtain that. In general, you should know that you + need this format, otherwise stay away from the property. + +**hostname** + hostname from the message + +**source** + alias for HOSTNAME + +**fromhost** + hostname of the system the message was received from (in a relay chain, + this is the system immediately in front of us and not necessarily the + original sender). This is a DNS-resolved name, except if that is not + possible or DNS resolution has been disabled. + +**fromhost-ip** + The same as fromhost, but always as an IP address. Local inputs (like + imklog) use 127.0.0.1 in this property. + +**syslogtag** + TAG from the message + +**programname** + the "static" part of the tag, as defined by BSD syslogd. For example, + when TAG is "named[12345]", programname is "named". + + Precisely, the programname is terminated by either (whichever occurs first): + + - end of tag + - nonprintable character + - ':' + - '[' + - '/' + + The above definition has been taken from the FreeBSD syslogd sources. + + Please note that some applications include slashes in the static part + of the tag, e.g. "app/foo[1234]". In this case, programname is "app". + If they store an absolute path name like in "/app/foo[1234]", programname + will become empty (""). If you need to actually store slashes as + part of the programname, you can use the global option + + global(parser.permitSlashInProgramName="on") + + to permit this. Then, a syslogtag of "/app/foo[1234]" will result in + programname being "/app/foo". Note: this option is available starting at + rsyslogd version 8.25.0. + +**pri** + PRI part of the message - undecoded (single value) + +**pri-text** + the PRI part of the message in a textual form with the numerical PRI + appended in brackets (e.g. "local0.err<133>") + +**iut** + the monitorware InfoUnitType - used when talking to a + `MonitorWare <https://www.monitorware.com>`_ backend (also for + `Adiscon LogAnalyzer <https://loganalyzer.adiscon.com/>`_) + +**syslogfacility** + the facility from the message - in numerical form + +**syslogfacility-text** + the facility from the message - in text form + +**syslogseverity** + severity from the message - in numerical form + +**syslogseverity-text** + severity from the message - in text form + +**syslogpriority** + an alias for syslogseverity - included for historical reasons (be + careful: it still is the severity, not PRI!) + +**syslogpriority-text** + an alias for syslogseverity-text + +**timegenerated** + timestamp when the message was RECEIVED. Always in high resolution + +**timereported** + timestamp from the message. Resolution depends on what was provided in + the message (in most cases, only seconds) + +**timestamp** + alias for timereported + +**protocol-version** + The contents of the PROTOCOL-VERSION field from IETF draft + draft-ietf-syslog-protocol + +**structured-data** + The contents of the STRUCTURED-DATA field from IETF draft + draft-ietf-syslog-protocol + +**app-name** + The contents of the APP-NAME field from IETF draft + draft-ietf-syslog-protocol + +**procid** + The contents of the PROCID field from IETF draft + draft-ietf-syslog-protocol + +**msgid** + The contents of the MSGID field from IETF draft + draft-ietf-syslog-protocol + +**inputname** + The name of the input module that generated the message (e.g. + "imuxsock", "imudp"). Note that not all modules necessarily provide this + property. If not provided, it is an empty string. Also note that the + input module may provide any value of its liking. Most importantly, it + is **not** necessarily the module input name. Internal sources can also + provide inputnames. Currently, "rsyslogd" is defined as inputname for + messages internally generated by rsyslogd, for example startup and + shutdown and error messages. This property is considered useful when + trying to filter messages based on where they originated - e.g. locally + generated messages ("rsyslogd", "imuxsock", "imklog") should go to a + different place than messages generated somewhere else. + +**uuid** + + *Only Available if rsyslog is build with --enable-uuid* + + A UUID for the message. It is not present by default, but will be created + on first read of the uuid property. Thereafter, in the local rsyslog + instance, it will always be the same value. This is also true if rsyslog + is restarted and messages stayed in an on-disk queue. + + Note well: the UUID is **not** automatically transmitted to remote + syslog servers when forwarding. If that is needed, a special template + needs to be created that contains the uuid. Likewise, the receiver must + parse that UUID from that template. + + The uuid property is most useful if you would like to track a single + message across multiple local destination. An example is messages being + written to a database as well as to local files. + +**jsonmesg** + + *Available since rsyslog 8.3.0* + + The whole message object as JSON representation. Note that the JSON + string will *not* include an LF and it will contain *all other message + properties* specified here as respective JSON containers. It also includes + all message variables in the "$!" subtree (this may be null if none are + present). + + This property is primarily meant as an interface to other systems and + tools that want access to the full property set (namely external + plugins). Note that it contains the same data items potentially multiple + times. For example, parts of the syslog tag will by contained in the + rawmsg, syslogtag, and programname properties. As such, this property + has some additional overhead. Thus, it is suggested to be used only + when there is actual need for it. + +System Properties +----------------- +These properties are provided by the rsyslog core engine. They are **not** +related to the message. All system properties start with a dollar-sign. + +Special care needs to be taken in regard to time-related system variables: + +* ``timereported`` contains the timestamp that is contained within the + message header. Ideally, it resembles the time when the message was + created at the original sender. + Depending on how long the message was in the relay chain, this + can be quite old. +* ``timegenerated`` contains the timestamp when the message was received + by the local system. Here "received" actually means the point in time + when the message was handed over from the OS to rsyslog's reception + buffers, but before any actual processing takes place. This also means + a message is "received" before it is placed into any queue. Note that + depending on the input, some minimal processing like extraction of the + actual message content from the receive buffer can happen. If multiple + messages are received via the same receive buffer (a common scenario + for example with TCP-based syslog), they bear the same ``timegenerated`` + stamp because they actually were received at the same time. +* ``$now`` is **not** from the message. It is the system time when the + message is being **processed**. There is always a small difference + between ``timegenerated`` and ``$now`` because processing always + happens after reception. If the message is sitting inside a queue + on the local system, the time difference between the two can be some + seconds (e.g. due to a message burst and in-memory queueing) up to + several hours in extreme cases where a message is sitting inside a + disk queue (e.g. due to a database outage). The ``timereported`` + property is usually older than ``timegenerated``, but may be totally + different due to differences in time and time zone configuration + between systems. + +The following system properties exist: + +**$bom** + The UTF-8 encoded Unicode byte-order mask (BOM). This may be useful in + templates for RFC5424 support, when the character set is know to be + Unicode. + +**$myhostname** + The name of the current host as it knows itself (probably useful for + filtering in a generic way) + +Time-Related System Properties +.............................. + +All of these system properties exist in a local time variant (e.g. \$now) +and a variant that emits UTC (e.g. \$now-utc). The UTC variant is always +available by appending "-utc". Note that within a single template, only +the localtime or UTC variant should be used. While it is possible to mix +both variants within a single template, it is **not** guaranteed that +they will provide exactly the same time. The technical reason is that +rsyslog needs to re-query system time when the variant is changed. Because +of this, we strongly recommend not mixing both variants in the same +template. + +Note that use in different templates will generate a consistent timestamp +within each template. However, as $now always provides local system time +at time of using it, time may advance and consequently different templates +may have different time stamp. To avoid this, use *timegenerated* instead. + +**$now** + The current date stamp in the format YYYY-MM-DD + +**$year** + The current year (4-digit) + +**$month** + The current month (2-digit) + +**$day** + The current day of the month (2-digit) + +**$wday** + The current week day as defined by 'gmtime()'. 0=Sunday, ..., 6=Saturday + +**$hour** + The current hour in military (24 hour) time (2-digit) + +**$hhour** + The current half hour we are in. From minute 0 to 29, this is always 0 + while from 30 to 59 it is always 1. + +**$qhour** + The current quarter hour we are in. Much like $HHOUR, but values range + from 0 to 3 (for the four quarter hours that are in each hour) + +**$minute** + The current minute (2-digit) + +**$now-unixtimestamp** + The current time as a unix timestamp (seconds since epoch). This actually + is a monotonically increasing counter and as such can also be used for any + other use cases that require such counters. This is an example of how + to use it for rate-limiting:: + + # Get Unix timestamp of current message + set $.tnow = $$now-unixtimestamp + + # Rate limit info to 5 every 60 seconds + if ($!severity == 6 and $!facility == 17) then { + if (($.tnow - $/trate) > 60) then { + # 5 seconds window expired, allow more messages + set $/trate = $.tnow; + set $/ratecount = 0; + } + if ($/ratecount > 5) then { + # discard message + stop + } else { + set $/ratecount = $/ratecount + 1; + } + } + + NOTE: by definition, there is no "UTC equivalent" of the + $now-unixtimestamp property. |