diff options
Diffstat (limited to 'source/rainerscript/functions')
38 files changed, 1501 insertions, 0 deletions
diff --git a/source/rainerscript/functions/idx_built-in_functions.rst b/source/rainerscript/functions/idx_built-in_functions.rst new file mode 100644 index 0000000..d2be538 --- /dev/null +++ b/source/rainerscript/functions/idx_built-in_functions.rst @@ -0,0 +1,11 @@ +****************** +Built-in Functions +****************** + +Following functions are available in Rainerscript. + +.. toctree:: + :glob: + :maxdepth: 1 + + rs* diff --git a/source/rainerscript/functions/idx_module_functions.rst b/source/rainerscript/functions/idx_module_functions.rst new file mode 100644 index 0000000..1998b56 --- /dev/null +++ b/source/rainerscript/functions/idx_module_functions.rst @@ -0,0 +1,16 @@ +**************** +Module Functions +**************** + +You can make module functions accessible for the configuration by loading the +corresponding module. Once they are loaded, you can use them like any other +rainerscript function. If more than one function are part of the same module, +all functions will be available once the module is loaded. + + +.. toctree:: + :glob: + :maxdepth: 1 + + mo* + diff --git a/source/rainerscript/functions/index.rst b/source/rainerscript/functions/index.rst new file mode 100644 index 0000000..a40e166 --- /dev/null +++ b/source/rainerscript/functions/index.rst @@ -0,0 +1,27 @@ +********* +Functions +********* + +There are two types of RainerScript functions: built-ins and modules. Built-in +functions can always be called in the configuration. To use module functions, +you have to load the corresponding module first. To do this, add the following +line to your configuration: + +:: + + module(load="<name of module>") + +If more than one function with the same name is present, the function of the +first module loaded will be used. Also, an error message stating this will be +generated. However, the configuration will not abort. Please note that built-in +functions will always be automatically loaded before any modules. Because of this, you +are unable to override any of the built-in functions, since their names are already +in use. The name of a function module starts with fm. + + +.. toctree:: + :glob: + :maxdepth: 2 + + idx_built-in_functions + idx_module_functions diff --git a/source/rainerscript/functions/mo-ffaup.rst b/source/rainerscript/functions/mo-ffaup.rst new file mode 100644 index 0000000..5291c74 --- /dev/null +++ b/source/rainerscript/functions/mo-ffaup.rst @@ -0,0 +1,103 @@ +************ +Faup +************ + + +Description +=========== + +These new functions allow anybody to parse any variable containing URLs, hostnames, DNS queries and such to extract: + - the *scheme* + - the *credentials* (if present) + - the *tld* (with support for second-level TLDs) + - the *domain* (with and without the tld) + - the *subdomain* + - the *full hostname* + - the *port* (if present) + - the *resource* path (if present) + - the *query string* parameters (if present) + - the *fragment* (if present) + +HowTo +----- +The module functions are fairly simple to use, and are divided in 2 classes: +* `faup()` allows to parse the entire URL and return all parts in a complete JSON +* `faup_<field>()` allows to parse the entire URL, but only returns the (potential) value of the requested field as string + +Examples +^^^^^^^^ +Using the `faup()` function +""""""""""""""""""""""""""""""" +The `faup()` is the simplest function to use, simply provide a value or variable (any type) as the only parameter, and the function returns a json object containing every element of the URL. + +*example code:* + +.. code-block:: none + + set $!url = "https://user:pass@www.rsyslog.com:443/doc/v8-stable/rainerscript/functions/mo-faup.html?param=value#faup"; + set $.faup = faup($!url); + + +*$.faup will contain:* + +.. code-block:: none + + { + "scheme": "https", + "credential": "user:pass", + "subdomain": "www", + "domain": "rsyslog.com", + "domain_without_tld": "rsyslog", + "host": "www.rsyslog.com", + "tld": "com", + "port": "443", + "resource_path": "\/doc\/v8-stable\/rainerscript\/functions\/mo-ffaup.html", + "query_string": "?param=value", + "fragment": "#faup" + } + +.. note:: + + This is a classic rsyslog variable, and you can access every sub-key with `$.faup!domain`, `$.faup!resource_path`, etc... + + +Using the `faup_<field>()` functions +"""""""""""""""""""""""""""""""""""""""" +Using the field functions is even simpler: for each field returned by the `faup()` function, there exists a corresponding function to get only that one field. + +For example, if the goal is to recover the domain without the tld, the example above could be modified as follows: + +*example code:* + +.. code-block:: none + + set $!url = "https://user:pass@www.rsyslog.com:443/doc/v8-stable/rainerscript/functions/mo-faup.html?param=value#faup"; + set $.faup = faup_domain_without_tld($!url); + +*$.faup will contain:* + +.. code-block:: none + + rsyslog + +.. note:: + + The returned value is no longer a json object, but a simple string + + +Requirements +============ +This module relies on the `faup <https://github.com/stricaud/faup>`_ library. + +The library should be compiled (see link for instructions on how to compile) and installed on build and runtime machines. + +.. warning:: + + Even if faup is statically compiled to rsyslog, the library still needs an additional file to work properly: the mozilla.tlds stored by the libfaup library in /usr/local/share/faup. It permits to properly match second-level TLDs and allow URLs such as www.rsyslog.co.uk to be correctly parsed into \<rsyslog:domain\>.\<co.uk:tld\> and not \<rsyslog:subdomain\>.\<co:domain\>.\<uk:tld\> + + +Motivations +=========== +Those functions are the answer to a growing problem encountered in Rsyslog when using modules to enrich logs : some mechanics (like lookup tables or external module calls) require "strict" URL/hostname formats that are often not formatted correctly, resulting in lookup failures/misses. + +This ensures getting stable inputs to provide to lookups/modules to enrich logs.
\ No newline at end of file diff --git a/source/rainerscript/functions/mo-hashXX.rst b/source/rainerscript/functions/mo-hashXX.rst new file mode 100644 index 0000000..61bb69a --- /dev/null +++ b/source/rainerscript/functions/mo-hashXX.rst @@ -0,0 +1,42 @@ +****** +HashXX +****** + +Purpose +======= + +Generates hash for a given string. + +hash32(literal_string) / hash32(literal_string, seed) +----------------------------------------------------- + + Generates a 32 bit hash of the given string. + - Seed is an optional parameter with default = 0. + - If seed is not a valid number, then 0 is returned. + +hash64(literal_string) / hash64(literal_string, seed) +----------------------------------------------------- + + Generates a 64 bit hash of the given string. + - Seed is an optional parameter with default = 0. + - If seed is not a valid number, then 0 is returned. + +.. warning:: + + - Default hash implementation is non-crypto. + - To use xxhash enable compile time flag. + + +Example +======= + +.. code-block:: none + + module(load="fmhash") + + set $.hash = hash64($!msg!field_1 & $!msg!field_2 & $!msg!field_3) + set $!tag= $syslogtag & $.hash; + //send out + +.. seealso:: + :doc:`Hash based sampling<../../tutorials/hash_sampling>` diff --git a/source/rainerscript/functions/mo-hashXXmod.rst b/source/rainerscript/functions/mo-hashXXmod.rst new file mode 100644 index 0000000..8fc1d77 --- /dev/null +++ b/source/rainerscript/functions/mo-hashXXmod.rst @@ -0,0 +1,48 @@ +********* +HashXXmod +********* + +Purpose +======= + +Generates a number which is mod of given string's hash. + +hash32mod(literal_string, modulo) / hash32mod(literal_string, modulo, seed) +--------------------------------------------------------------------------- + + Generates a number which is calculated on (32 bit hash of the given string % modulo) + - If modulo is not a valid number, then 0 is returned. + - If modulo is 0, then 0 is returned. + - Seed is an optional parameter with default = 0. + - If seed is not a valid unsigned number, then 0 is returned. + + +hash64mod(literal_string, modulo) / hash64mod(literal_string, modulo, seed) +--------------------------------------------------------------------------- + + Generates a number which is calculated on (64 bit hash of the given string % modulo) + - If modulo is not a valid number, then 0 is returned. + - If modulo is 0, then 0 is returned. + - Seed is an optional parameter with default = 0. + - If seed is not a valid unsigned number, then 0 is returned. + + +.. warning:: + + - Default hash implementation is non-crypto. + - To use xxhash enable compile time flag. + + +Example +======= + +.. code-block:: none + + module(load="fmhash") + + if (hash64mod($!msg!request_id, 100) <= 30) then { + //send out + } + +.. seealso:: + :doc:`Hash based sampling<../../tutorials/hash_sampling>` diff --git a/source/rainerscript/functions/mo-http_request.rst b/source/rainerscript/functions/mo-http_request.rst new file mode 100644 index 0000000..a0b0f22 --- /dev/null +++ b/source/rainerscript/functions/mo-http_request.rst @@ -0,0 +1,39 @@ +************ +HTTP-Request +************ + +Purpose +======= + +http_request(str) + +Performs a http request to target and returns the result of said request. + +.. note:: + + this function is very slow and therefore we suggest using it only seldomly + to insure adequate performance. + + +Example +======= + +The following example performs a http request and writes the returned value +to a file. + + +.. code-block:: none + + module(load="../plugins/imtcp/.libs/imtcp") + module(load="../plugins/fmhttp/.libs/fmhttp") + input(type="imtcp" port="13514") + + template(name="outfmt" type="string" string="%$!%\n") + + if $msg contains "msgnum:" then { + set $.url = "http://www.rsyslog.com/testbench/echo-get.php?content=" & ltrim($msg); + set $!reply = http_request($.url); + action(type="omfile" file="rsyslog.out.log" template="outfmt") + } + + diff --git a/source/rainerscript/functions/mo-unflatten.rst b/source/rainerscript/functions/mo-unflatten.rst new file mode 100644 index 0000000..136251e --- /dev/null +++ b/source/rainerscript/functions/mo-unflatten.rst @@ -0,0 +1,61 @@ +************ +Unflatten +************ + +Purpose +======= + +``<result> = unflatten(<source-tree>, <key-separator-character>);`` + +This function unflattens keys in a JSON object. It provides a way to expand dot-separated fields. + +It allows for instance to produce this: + ``{ "source": { "ip": "1.2.3.4", "port": 443 } }`` + +from this source data: + ``{ "source.ip": "1.2.3.4", "source.port": 443 }`` + + +Example +======= + +Here is a sample use case: + +.. code-block:: none + + module(load="fmunflatten") + + # Say you have the following tree, obtained for instance with mmnormalize. + set $!source.ip = "1.2.3.4"; + set $!source.bytes = 3258; + set $!source.geo.country_iso_code = "FR"; + set $!destination.ip = "4.3.2.1"; + + # Now unflatten the keys in the $! tree. + set $.unflatten = unflatten($!, "."); + + # You may do this to set back safely the result in $! because the function could + # return a default dummy value of 0 (rsyslog number) if $! was not touched (it + # would evaluate to an empty rsyslog string, which is not a JSON datatype). + if (script_error() == 0) then { + unset $!; + set $! = $.unflatten; + unset $.unflatten; + } + +An output of ``$!`` would give this, in pretty-print: + +.. code-block:: none + + { + "source": { + "ip": "1.2.3.4", + "bytes": 3258, + "geo": { + "country_iso_code": "FR" + } + }, + "destination": { + "ip": "4.3.2.1" + } + } diff --git a/source/rainerscript/functions/rs-cnum.rst b/source/rainerscript/functions/rs-cnum.rst new file mode 100644 index 0000000..055a343 --- /dev/null +++ b/source/rainerscript/functions/rs-cnum.rst @@ -0,0 +1,36 @@ +****** +cnum() +****** + +Purpose +======= + +cnum(expr) + +Converts expr to a number (integer). + +.. note:: + + If the expression does not contain a numerical value, the following + rule applies: the best match as the number is returned. For example + "1x234" will return the number 1 and "Test123" will return 0. Zero is + always returned if the there is no number at the start of the string. + This also is the case for empty strings. + + +Example +======= + + + +.. code-block:: none + + cnum(3+2); + +produces + +.. code-block:: none + + 5 + + diff --git a/source/rainerscript/functions/rs-cstr.rst b/source/rainerscript/functions/rs-cstr.rst new file mode 100644 index 0000000..3cf007e --- /dev/null +++ b/source/rainerscript/functions/rs-cstr.rst @@ -0,0 +1,27 @@ +****** +cstr() +****** + +Purpose +======= + +cstr(expr) + +Converts expr to a string value. + + +Example +======= + +In the following example a string is created from the expression. + +.. code-block:: none + + cstr(5*4); + +produces + +.. code-block:: none + + "20" + diff --git a/source/rainerscript/functions/rs-dyn_inc.rst b/source/rainerscript/functions/rs-dyn_inc.rst new file mode 100644 index 0000000..67769cc --- /dev/null +++ b/source/rainerscript/functions/rs-dyn_inc.rst @@ -0,0 +1,35 @@ +********* +dyn_inc() +********* + +Purpose +======= + +dyn_inc(bucket_name_literal_string, str) + +Increments counter identified by ``str`` in dyn-stats bucket identified +by ``bucket_name_literal_string``. Returns 0 when increment is successful, +any other return value indicates increment failed. + +Counters updated here are reported by **impstats**. + +Except for special circumstances (such as memory allocation failing etc), +increment may fail due to metric-name cardinality being under-estimated. +Bucket is configured to support a maximum cardinality (to prevent abuse) +and it rejects increment-operation if it encounters a new(previously unseen) +metric-name(``str``) when full. + +**Read more about it here** :doc:`Dynamic Stats<../../configuration/dyn_stats>` + + +Example +======= + +The following example shows the counter $hostname incremented in the bucket +msg_per_host. + +.. code-block:: none + + dyn_inc("msg_per_host", $hostname) + + diff --git a/source/rainerscript/functions/rs-exec_template.rst b/source/rainerscript/functions/rs-exec_template.rst new file mode 100644 index 0000000..76b07cc --- /dev/null +++ b/source/rainerscript/functions/rs-exec_template.rst @@ -0,0 +1,25 @@ +*************** +exec_template() +*************** + +Purpose +======= + +exec_template(str) + +Sets a variable through the execution of a template. Basically this permits to easily +extract some part of a property and use it later as any other variable. + +**Read more about it here :** `<http://www.rsyslog.com/how-to-use-set-variable-and-exec_template>`_ + +Example +======= + +The following example shows the template being used to extract a part of the message. + +.. code-block:: none + + template(name="extract" type="string" string="%msg:F:5%") + set $!xyz = exec_template("extract"); + + diff --git a/source/rainerscript/functions/rs-exists.rst b/source/rainerscript/functions/rs-exists.rst new file mode 100644 index 0000000..eabffd8 --- /dev/null +++ b/source/rainerscript/functions/rs-exists.rst @@ -0,0 +1,30 @@ +******** +exists() +******** + +Purpose +======= + +exists($!path!varname) + +This function checks if the specified variable exists, in other +words: contains a value. A variable that once was set and later +on unset does also not exist. + +The function accepts a single argument, which needs to be a variable. +It returns 0 if the variable does not exist and 1 otherwise. The +function can be combined with any other expression to form more +complec expressions. + +.. versionadded:: 8.2010.10 + + +Example +======= + +.. code-block:: none + + if exists(!$path!varname) then ... + if not exists($.local!var) then ... + if exists($!triggervar) and $msg contains "something" then ... + diff --git a/source/rainerscript/functions/rs-field.rst b/source/rainerscript/functions/rs-field.rst new file mode 100644 index 0000000..bc3c08b --- /dev/null +++ b/source/rainerscript/functions/rs-field.rst @@ -0,0 +1,64 @@ +******* +field() +******* + +Purpose +======= + +field(str, delim, matchnbr) + +Returns a field-based substring. str is +the string to search, delim is the delimiter and matchnbr is the +match to search for (the first match starts at 1). This works similar +as the field based property-replacer option. Versions prior to 7.3.7 +only support a single character as delimiter character. Starting with +version 7.3.7, a full string can be used as delimiter. If a single +character is being used as delimiter, delim is the numerical ascii +value of the field delimiter character (so that non-printable +characters can by specified). If a string is used as delimiter, a +multi-character string (e.g. "#011") is to be specified. + +.. note:: + + When a single character is specified as string + ``field($msg, ",", 3)`` a string-based extraction is done, which is + more performance intensive than the equivalent single-character + ``field($msg, 44 ,3)`` extraction. + + +Example +======= + +With ascii value of the field delimiter +--------------------------------------- + +Following example returns the third field delimited by space. +In this example $msg will be "field1 field2 field3 field4". + +.. code-block:: none + + set $!usr!field = field($msg, 32, 3); + +produces + +.. code-block:: none + + "field3" + + +With a string as the field delimiter +------------------------------------ + +This example returns the second field delimited by "#011". +In this example $msg will be "field1#011field2#011field3#011field4". + +.. code-block:: none + + set $!usr!field = field($msg, "#011", 2); -- the second field, delimited by "#011" + +produces + +.. code-block:: none + + "field2" + diff --git a/source/rainerscript/functions/rs-format_time.rst b/source/rainerscript/functions/rs-format_time.rst new file mode 100644 index 0000000..aac628e --- /dev/null +++ b/source/rainerscript/functions/rs-format_time.rst @@ -0,0 +1,64 @@ +************* +format_time() +************* + +Purpose +======= + +.. note:: + + This is EXPERIMENTAL code - it may be removed or altered in + later versions than 8.30.0. Please watch the ChangeLog closely for + updates. + +Converts a UNIX timestamp to a formatted RFC 3164 or RFC 3339 date/time string. +The first parameter is expected to be an integer value representing the number of +seconds since 1970-01-01T00:00:0Z (UNIX epoch). The second parameter can be one of +``"date-rfc3164"`` or ``"date-rfc3339"``. The output is a string containing +the formatted date/time. Date/time strings are expressed in **UTC** (no time zone +conversion is provided). + +.. note:: + + If the input to the function is NOT a proper UNIX timestamp, a string + containing the original value of the parameter will be returned instead of a + formatted date/time string. + + +Example +======= + +RFC 3164 timestamp +------------------ + +In the following example the integer representing a UNIX timestamp is +formatted to a rfc-3164 date/time string. + +.. code-block:: none + + format_time(1507165811, "date-rfc3164") + +produces + +.. code-block:: none + + Oct 5 01:10:11 + + +Wrong input +----------- + +In the following example a wrong parameter is given which can't be +formatted and so it is returned unchanged. + +.. code-block:: none + + format_time("foo", "date-rfc3164") + +produces + +.. code-block:: none + + foo + + diff --git a/source/rainerscript/functions/rs-get_property.rst b/source/rainerscript/functions/rs-get_property.rst new file mode 100644 index 0000000..a1536fa --- /dev/null +++ b/source/rainerscript/functions/rs-get_property.rst @@ -0,0 +1,58 @@ +************** +get_property() +************** + +Purpose +======== + +get_property(rsyslog_variable, key_str) + +Provides ability to get a rsyslog variable or property using dynamically evaluated parameters. +The first parameter is a valid rsyslog variable or property, the second parameter is a key string, or index value. + + +Example +======== + +In the following example, a json string is parsed using parse_json(), and placed into the variable ``$!parsed``. +The get_property function is then used to get property fields from ``$!parsed``. + +.. code-block:: none + + set $.ret = parse_json("{\"offsets\": [ { \"a\": 9, \"b\": 0 },\ + { \"a\": 9, \"b\": 3 } ],\ + \"boolval\": true,\ + \"int64val\": 1234567890,\ + \"nullval\": null,\ + \"doubleval\": 12345.67890 }", "\$!parsed"); + + # get different fields from $!parsed here + if $.ret == 0 then { + # dynamically evaluate different fields in $!parsed + + # do dynamic indexing into array + $.index = 1; + # set $.ret = { "a": 9, "b": 3} + set $.ret = get_property($!parsed!offsets, $.index); + + # set $.ret = true; + set $.key = "boolval"; + set $.ret = get_property($!parsed, $.key); + + # null values are evaluated to empty string + # thus below statement will set $.ret to empty string + set $.ret = get_property($!parsed, "nullval"); + + # evaluates to 1234567890 + set $.key = "int64val"; + set $.ret = get_property($!parsed, $.key); + + # using a key variable, evaluates to 12345.67890 + set $key = "doubleval"; + set $.ret = get_property($!parsed, $key); + } + + # example of dynamically building key value + set $.foo!barval = 3; + # set $.bar = 3; + set $.bar = get_property($.foo, "bar" & "val"); diff --git a/source/rainerscript/functions/rs-getenv.rst b/source/rainerscript/functions/rs-getenv.rst new file mode 100644 index 0000000..0035a8b --- /dev/null +++ b/source/rainerscript/functions/rs-getenv.rst @@ -0,0 +1,23 @@ +******** +getenv() +******** + +Purpose +======= + +getenv(str) + +Like the OS call, returns the value of the environment variable, +if it exists. Returns an empty string if it does not exist. + + +Examples +======== + +The following example can be used to build a dynamic filter based on +some environment variable: + +.. code-block:: none + + if $msg contains getenv('TRIGGERVAR') then /path/to/errfile + diff --git a/source/rainerscript/functions/rs-int2hex.rst b/source/rainerscript/functions/rs-int2hex.rst new file mode 100644 index 0000000..29cbbdd --- /dev/null +++ b/source/rainerscript/functions/rs-int2hex.rst @@ -0,0 +1,56 @@ +********* +int2hex() +********* + +Purpose +======= + +int2hex(num) + +Returns a hexadecimal number string of a given positive integer num. + +.. note:: + + When a negative integer is given the return value will be the + `Two's complement <https://en.wikipedia.org/wiki/Two%27s_complement>`_ of + the input number. + + +Example +======= + + +Positive Integer given +---------------------- + +In the following example the hexadecimal number of 61 is returned +as a string. + +.. code-block:: none + + int2hex(61) + +produces + +.. code-block:: none + + 3d + + +Negative Integer given +---------------------- + +This example shows the result on a 64-bit system when a negative +integer such as -13 is put into the function. + +.. code-block:: none + + int2hex(-13) + +produces + +.. code-block:: none + + fffffffffffffff3 + + diff --git a/source/rainerscript/functions/rs-ipv4convert.rst b/source/rainerscript/functions/rs-ipv4convert.rst new file mode 100644 index 0000000..5ab9b16 --- /dev/null +++ b/source/rainerscript/functions/rs-ipv4convert.rst @@ -0,0 +1,52 @@ +*********************** +num2ipv4() / ipv42num() +*********************** + +Purpose +======= + +num2ipv4 +-------- + +num2ipv4(int) + +Converts an integer into an IPv4-address and returns the address as string. +Input is an integer with a value between 0 and 4294967295. The output format +is '>decimal<.>decimal<.>decimal<.>decimal<' and '-1' if the integer input is +invalid or if the function encounters a problem. + + +ipv42num +-------- + +ipv42num(str) + +Converts an IPv4-address into an integer and returns the integer. Input is +a string; the expected address format may include spaces in the beginning +and end, but must not contain any other characters in between (except dots). +If the format does include these, the function results in an error and returns -1. + + +Example +======= + +num2ipv4 +-------- + +This example shows an integer being converted to an ipv4 address. + +.. code-block:: none + + num2ipv4(123456) + + +ipv42num +-------- + +This example shows the parameter fromhost-ip being converted to an integer. + +.. code-block:: none + + ipv42num($fromhost-ip) + + diff --git a/source/rainerscript/functions/rs-is_time.rst b/source/rainerscript/functions/rs-is_time.rst new file mode 100644 index 0000000..42ca727 --- /dev/null +++ b/source/rainerscript/functions/rs-is_time.rst @@ -0,0 +1,66 @@ +********* +is_time() +********* + +Purpose +======= + +is_time(timestamp) +is_time(timestamp, format_str) + +Checks the given timestamp to see if it is a valid date/time string (RFC 3164, +or RFC 3339), or a UNIX timestamp. + +This function returns ``1`` for valid date/time strings and UNIX timestamps, +``0`` otherwise. Additionally, if the input cannot be parsed, or there is +an error, ``script_error()`` will be set to error state. + +The ``format_str`` parameter is optional, and can be one of ``"date-rfc3164"``, +``"date-rfc3339"`` or ``"date-unix"``. If this parameter is specified, the +function will only succeed if the input matches that format. If omitted, the +function will compare the input to all of the known formats (indicated above) +to see if one of them matches. + +.. note:: + + This function does not support unusual RFC 3164 dates/times that + contain year or time zone information. + + +Example +======= + +Only timestamp is given +----------------------- + +The following example shows the output when a valid timestamp is given. + +.. code-block:: none + + is_time("Oct 5 01:10:11") + is_time("2017-10-05T01:10:11+04:00") + is_time(1507165811) + +all produce + +.. code-block:: none + + 1 + + +Timestamp and Format given +-------------------------- + +The following example shows the output when a valid timestamp is given but +the format does not match. + +.. code-block:: none + + is_time("2017-10-05T01:10:11+04:00", "date-unix") + +all produce + +.. code-block:: none + + 0 + diff --git a/source/rainerscript/functions/rs-lookup.rst b/source/rainerscript/functions/rs-lookup.rst new file mode 100644 index 0000000..cdcd563 --- /dev/null +++ b/source/rainerscript/functions/rs-lookup.rst @@ -0,0 +1,33 @@ +******** +lookup() +******** + +Purpose +======= + +lookup(table_name_literal_string, key) + +Lookup tables are a powerful construct to obtain *class* information based +on message content. It works on top of a data-file which maps key (to be looked +up) to value (the result of lookup). + +The idea is to use a message properties (or derivatives of it) as an index +into a table which then returns another value. For example, $fromhost-ip +could be used as an index, with the table value representing the type of +server or the department or remote office it is located in. + +**Read more about it here** :doc:`Lookup Tables<../../configuration/lookup_tables>` + + +Example +======= + +In the following example the hostname is looked up in the given table and +the corresponding value is returned. + +.. code-block:: none + + lookup_table(name="host_bu" file="/var/lib/host_billing_unit_mapping.json") + set $.bu = lookup("host_bu", $hostname); + + diff --git a/source/rainerscript/functions/rs-parse_json.rst b/source/rainerscript/functions/rs-parse_json.rst new file mode 100644 index 0000000..269b587 --- /dev/null +++ b/source/rainerscript/functions/rs-parse_json.rst @@ -0,0 +1,27 @@ +************ +parse_json() +************ + +Purpose +======= + +parse_json(str, container) + +Parses the json string ``str`` and places the resulting json object +into ``container`` where container can be any valid rsyslog variable. +Returns 0 on success and something otherwise if ``str`` does **not** +contain valid json. + + +Example +======= + +In the following example the json string is placed into the variable $!parsed. +The output is placed in variable $.ret + +.. code-block:: none + + set $.ret = parse_json("{ \"c1\":\"data\" }", "\$!parsed"); + + + diff --git a/source/rainerscript/functions/rs-parse_time.rst b/source/rainerscript/functions/rs-parse_time.rst new file mode 100644 index 0000000..434df3d --- /dev/null +++ b/source/rainerscript/functions/rs-parse_time.rst @@ -0,0 +1,44 @@ +************ +parse_time() +************ + +Purpose +======= + +parse_time(timestamp) + +Converts an RFC 3164 or RFC 3339 formatted date/time string to a UNIX timestamp +(an integer value representing the number of seconds since the UNIX epoch: +1970-01-01T00:00:0Z). + +If the input to the function is not a properly formatted RFC 3164 or RFC 3339 +date/time string, or cannot be parsed, ``0`` is returned and ``script_error()`` +will be set to error state. + +.. note:: + + This function does not support unusual RFC 3164 dates/times that + contain year or time zone information. + +.. note:: + + Fractional seconds (if present) in RFC 3339 date/time strings will + be discarded. + + +Example +======= + +In the following example a timestamp is parsed into an integer. + +.. code-block:: none + + parse_time("Oct 5 01:10:11") # Assumes the current year (2017, in this example) + +produces + +.. code-block:: none + + 1507165811 + + diff --git a/source/rainerscript/functions/rs-percentile_observe.rst b/source/rainerscript/functions/rs-percentile_observe.rst new file mode 100644 index 0000000..40bc81f --- /dev/null +++ b/source/rainerscript/functions/rs-percentile_observe.rst @@ -0,0 +1,33 @@ +******************** +percentile_observe() +******************** + +Purpose +======= + +percentile_observe(bucket_name_literal_string, stats_name_literal_string, value) + +Adds an observation, identified by the ``bucket_name_literal_string`` +and ``stats_name_literal_string``, to the observation set. + +Periodically, on an **impstats** reporting interval, percentile and summary metrics are generated +based on the statistics collected. + +Except for special circumstances (such as memory allocation failing etc), +observe calls may fail due to stat-name cardinality being under-estimated. +Bucket is configured to support a maximum cardinality (to prevent abuse) +and it rejects observe-operation if it encounters a new(previously unseen) +stat-name(``str``) when full. + +**Read more about it here** :doc:`Percentile Stats<../../configuration/percentile_stats>` + + +Example +======= + +In the following example, the ``$msg_count`` value is being recorded as a ``msg_per_host`` statistic in the ``host_statistics`` bucket. + + +.. code-block:: none + + percentile_observe("host_statistics", "msg_per_host", $msg_count) diff --git a/source/rainerscript/functions/rs-previous_action_suspended.rst b/source/rainerscript/functions/rs-previous_action_suspended.rst new file mode 100644 index 0000000..b78e448 --- /dev/null +++ b/source/rainerscript/functions/rs-previous_action_suspended.rst @@ -0,0 +1,30 @@ +*************************** +previous_action_suspended() +*************************** + +Purpose +======= + +previous_action_suspended() + +This boolean function returns 1 (true) if the previous action is suspended, +0 (false) otherwise. It can be used to initiate action that shall happen if +a function failed. Please note that an action failure may not be immediately +detected, so the function return value is a bit fuzzy. It is guaranteed, however +that a suspension will be detected with the next batch of messages that is +being processed. + + +Example +======= + +In the following example the if-clause is executed if the previous action +is suspended. + +.. code-block:: none + + action(type="omfwd" protocol="tcp" target="10.1.1.1") + + if previous_action_suspended() then {} + + diff --git a/source/rainerscript/functions/rs-prifilt.rst b/source/rainerscript/functions/rs-prifilt.rst new file mode 100644 index 0000000..c8a88b7 --- /dev/null +++ b/source/rainerscript/functions/rs-prifilt.rst @@ -0,0 +1,23 @@ +********* +prifilt() +********* + +Purpose +======= + +prifilt(constant) + +Mimics a traditional PRI-based filter (like "\*.\*" or "mail.info"). +The traditional filter string must be given as a **constant string**. +Dynamic string evaluation is not permitted (for performance reasons). + + +Example +======= + +In this example true would be returned on an auth event. + +.. code-block:: none + + prifilt("auth,authpriv.*") + diff --git a/source/rainerscript/functions/rs-random.rst b/source/rainerscript/functions/rs-random.rst new file mode 100644 index 0000000..dbf1c15 --- /dev/null +++ b/source/rainerscript/functions/rs-random.rst @@ -0,0 +1,36 @@ +******** +random() +******** + +Purpose +======= + +random(max) + +Generates a random number between 0 and the number specified, though +the maximum value supported is platform specific. + +- If a number is not specified then 0 is returned. +- If 0 is provided as the maximum value, then 0 is returned. +- If the specified value is greater than the maximum supported + for the current platform, then rsyslog will log this in + the debug output and use the maximum value supported instead. + +While the original intent of this function was for load balancing, it +is generic enough to be used for other purposes as well. + +.. warning:: + The random number must not be assumed to be crypto-grade. + +.. versionadded:: 8.12.0 + + +Example +======= + +In the following example a random number between 0 and 123456 is generated. + +.. code-block:: none + + random(123456) + diff --git a/source/rainerscript/functions/rs-re_extract.rst b/source/rainerscript/functions/rs-re_extract.rst new file mode 100644 index 0000000..cd37910 --- /dev/null +++ b/source/rainerscript/functions/rs-re_extract.rst @@ -0,0 +1,38 @@ +************ +re_extract() +************ + +Purpose +======= + +re_extract(expr, re, match, submatch, no-found) + +Extracts data from a string (property) via a regular expression match. +POSIX ERE regular expressions are used. The variable "match" contains +the number of the match to use. This permits to pick up more than the +first expression match. Submatch is the submatch to match (max 50 supported). +The "no-found" parameter specifies which string is to be returned in case +when the regular expression is not found. Note that match and +submatch start with zero. It currently is not possible to extract +more than one submatch with a single call. + +This function performs case-sensitive matching. Use the otherwise-equivalent +:doc:`re_extract_i <rs-re_extract_i>` function to perform case-insensitive +matches. + +.. note:: + + Functions using regular expressions tend to be slow and other options + may be faster. + + +Example +======= + +In the following example the msg object is checked for the regex string. +Only the first match is used and if no match was found an empty string is returned. + +.. code-block:: none + + re_extract($msg,'(5[1-5][0-9]{14})',0,1,"") + diff --git a/source/rainerscript/functions/rs-re_extract_i.rst b/source/rainerscript/functions/rs-re_extract_i.rst new file mode 100644 index 0000000..1eb96c2 --- /dev/null +++ b/source/rainerscript/functions/rs-re_extract_i.rst @@ -0,0 +1,12 @@ +************** +re_extract_i() +************** + +Purpose +======= + +re_extract_i(expr, re, match, submatch, no-found) + +This function is equivalent to `re_extract()` except that +it performs case-insensitive matches. See :doc:`re_extract <rs-re_extract>` +for further details. diff --git a/source/rainerscript/functions/rs-re_match.rst b/source/rainerscript/functions/rs-re_match.rst new file mode 100644 index 0000000..f7036bd --- /dev/null +++ b/source/rainerscript/functions/rs-re_match.rst @@ -0,0 +1,28 @@ +********** +re_match() +********** + +Purpose +======= + +re_match(expr, re) + +Returns 1, if expr matches re, 0 otherwise. Uses POSIX ERE. In contrast to +`re_match_i()` the matching is case-sensitive. + +.. note:: + + Functions using regular expressions tend to be slow and other options + may be faster. + + +Example +======= + +In the following example it is checked if the msg object matches the regex string. + +.. code-block:: none + + re_match($msg,'(5[1-5][0-9]{14})') + + diff --git a/source/rainerscript/functions/rs-re_match_i.rst b/source/rainerscript/functions/rs-re_match_i.rst new file mode 100644 index 0000000..60d8cd2 --- /dev/null +++ b/source/rainerscript/functions/rs-re_match_i.rst @@ -0,0 +1,29 @@ +************ +re_match_i() +************ + +Purpose +======= + +re_match_i(expr, re) + +Returns 1, if expr matches re, 0 otherwise. Uses POSIX ERE. In contrast to +`re_match()` the matching is case-insensitive. + +.. note:: + + Functions using regular expressions tend to be slow and other options + may be faster. + + +Example +======= + +In the following example it is checked if the msg object matches the regex string. + +.. code-block:: none + + re_match($msg,'TesT') + +It matches it the message "Test", in any case ("TEST", "tEst", ...) +is contained in msg property. diff --git a/source/rainerscript/functions/rs-replace.rst b/source/rainerscript/functions/rs-replace.rst new file mode 100644 index 0000000..2d98e75 --- /dev/null +++ b/source/rainerscript/functions/rs-replace.rst @@ -0,0 +1,26 @@ +********* +replace() +********* + +Purpose +======= + +replace(str, substr_to_replace, replace_with) + +Returns new string with all instances of substr_to_replace replaced +by replace_with. + + +Example +======= + +.. code-block:: none + + replace("foo bar baz", " b", ", B") + +produces + +.. code-block:: none + + "foo, Bar, Baz" + diff --git a/source/rainerscript/functions/rs-script_error.rst b/source/rainerscript/functions/rs-script_error.rst new file mode 100644 index 0000000..88c5bff --- /dev/null +++ b/source/rainerscript/functions/rs-script_error.rst @@ -0,0 +1,30 @@ +************** +script_error() +************** + +Purpose +======= + +script_error() + +Returns the error state of functions that support it. C-Developers note that this +is similar to ``errno`` under Linux. The error state corresponds to the function +immediately called before. The next function call overrides it. + +Right now, the value 0 means that that the previous functions succeeded, any other +value that it failed. In the future, we may have more fine-grain error codes. + +Function descriptions mention if a function supports error state information. If not, +the function call will always set ``script_error()`` to 0. + + +Example +======= + +The following example shows that script_error() only needs to be called and does +not need any parameters. + +.. code-block:: none + + script_error() + diff --git a/source/rainerscript/functions/rs-strlen.rst b/source/rainerscript/functions/rs-strlen.rst new file mode 100644 index 0000000..569a955 --- /dev/null +++ b/source/rainerscript/functions/rs-strlen.rst @@ -0,0 +1,22 @@ +******** +strlen() +******** + +Purpose +======= + +strlen(str) + +Returns the length of the provided string. + + +Examples +======== + +In the following example returns the length of the message string. + +.. code-block:: none + + strlen($msg) + + diff --git a/source/rainerscript/functions/rs-substring.rst b/source/rainerscript/functions/rs-substring.rst new file mode 100644 index 0000000..b295abf --- /dev/null +++ b/source/rainerscript/functions/rs-substring.rst @@ -0,0 +1,75 @@ +*********** +substring() +*********** + +Purpose +======= + +`substring(str, start, subStringLength)` + +Creates a substring from `str`. The substring begins at `start` and is +at most `subStringLength` characters long. If `start` is higher than the +length of the string, the result will always be an empty string. The +same is the case if `subStringLength` is zero. + +.. note:: + + The first character of the string has the value 0. So if you set + start to '2', the substring will start with the third character. + +If you want to drop only some characters at the beginning of the string, +select an overlay large `subStringLength`, e.g. 1000000. As the length is +then always larger as any normal string, the function will return all but +the first part of the string. + +In order to aid removing `n` characters from the end of the string, you +can specify a **negative** `subStringLength`. This will result in an +actual `subStringLength` that is the string-length minus the specified +`subStringLength`. This is actually a shortcut for + +.. code-block:: none + + subStringLength = strlen(str) - n + +Specifying a negative `subStringLength` does not only offer a more +compact expression, it also has some performance benefits over the +use of `strlen()`. + + +Examples +======== + +In the following example a substring from the string is created, +starting with the 4th character and being 5 characters long. + +.. code-block:: none + + substring("123456789", 3, 5) + + +This example will result in the substring "45678". + +In the following example the first and the last character +of the string will be removed. + +.. code-block:: none + + substring("123456789", 1, -2) + + +This example will result in the substring "2345678". + +Note the `-2` value for `subStringLength`. As the extraction skips the +first character, you only need a substring of `strlen("123456789") - 2` +characters, as you want to remove the last character as well. In other +words: if you want to remove the first **and** last character from the +string, the substring in question is **two** characters shorter than the +original string! We know this is a bit counter-intuitive, this we +wanted to make you aware. + +As another example let us assume you want to drop the first two and the +last character. In this case you would use `subStringLength` of three: + +.. code-block:: none + + substring("123456789", 2, -3) diff --git a/source/rainerscript/functions/rs-tolower.rst b/source/rainerscript/functions/rs-tolower.rst new file mode 100644 index 0000000..904770e --- /dev/null +++ b/source/rainerscript/functions/rs-tolower.rst @@ -0,0 +1,22 @@ +********* +tolower() +********* + +Purpose +======= + +tolower(str) + +Converts the provided string into lowercase. + + +Example +======= + +The following example shows the syslogtag being converted to lower case. + +.. code-block:: none + + tolower($syslogtag) + + diff --git a/source/rainerscript/functions/rs-trim.rst b/source/rainerscript/functions/rs-trim.rst new file mode 100644 index 0000000..835b17b --- /dev/null +++ b/source/rainerscript/functions/rs-trim.rst @@ -0,0 +1,59 @@ +***************** +ltrim() / rtrim() +***************** + +Purpose +======= + +ltrim +----- + +ltrim(str) + +Removes any spaces at the start of a given string. Input is a string, output +is the same string starting with the first non-space character. + + +rtrim +----- + +rtrim(str) + +Removes any spaces at the end of a given string. Input is a string, output +is the same string ending with the last non-space character. + + +Example +======= + +ltrim +----- + +In the following example the spaces at the beginning of the string are removed. + +.. code-block:: none + + ltrim(" str ") + +produces + +.. code-block:: none + + "str " + + +rtrim +----- + +In the following example the spaces at the end of the string are removed. + +.. code-block:: none + + rtrim(" str ") + +produces + +.. code-block:: none + + " str" + diff --git a/source/rainerscript/functions/rs-wrap.rst b/source/rainerscript/functions/rs-wrap.rst new file mode 100644 index 0000000..d1dbd1c --- /dev/null +++ b/source/rainerscript/functions/rs-wrap.rst @@ -0,0 +1,51 @@ +****** +wrap() +****** + +Purpose +======= + +wrap(str, wrapper_str) +wrap(str, wrapper_str, escaper_str) + +Returns the str wrapped with wrapper_str. +In the second syntax additionally, any instances of wrapper_str appearing +in str would be replaced by the escaper_str. + + +Example +======= + +Syntax 1 +-------- + +The following example shows how the string is wrapped into +the wrapper_string. + +.. code-block:: none + + wrap("foo bar", "##") + +produces + +.. code-block:: none + + "##foo bar##" + + +Syntax 2 +-------- + +This example shows how parts of the string are being replaced. + +.. code-block:: none + + wrap("foo'bar", "'", "_") + +produces + +.. code-block:: none + + "'foo_bar'" + + |