diff options
Diffstat (limited to '')
| -rw-r--r-- | ChangeLog | 16389 |
1 files changed, 16389 insertions, 0 deletions
diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 0000000..f4d9a77 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,16389 @@ +---------------------------------------------------------------------------------------- +Scheduled Release 8.2402.0 (aka 2024.02) 2024-02-27 +- 2024-02-26: add DTLS support + This version comes with the initial implementation of imdtls and omdtls. + These modules permit secure message exchange over UDP. +- 2024-02-26: testbench: make omusrmsg-noabort test more reliable + The previous test did not always detect an abort of rsyslog/omusrmsg. + The detection method has now been improved, so it is far more + probable that an abort is detected. + While doing this, we noticed that the omusrmsg-noabort-legacy test was + now a 100% duplicate. There is no need any longer to check pure legacy + syntax, and so that test has been removed. + We also added a valgrind-based test ofr omusrmsg-noabort, which furthers + strengthens bug detection. Most importantly, it helps us to detect + potentially new memory leaks on all CI platforms (in case the lib + behaves differently depending on os/distro). + see also https://github.com/rsyslog/rsyslog/issues/5294 +- 2024-02-26: omusrmsg bugfix: potential double free, which can cause segfault + omusrmsg frees a string which points to OS/system library memory. When + the os/libs clean up, it frees the memory as well. This results in a + double free. This bug interestingly seems to go unnoticed in many cases. + But it can cause a segfault or hard-to-trace memory corruptions which + could lead to other problems later on. The outcome of this bug most + probably depdns on os/library versions. + closes https://github.com/rsyslog/rsyslog/issues/5294 +- 2024-02-26: ommysql bugfix: potential segfault on database error + Due to an invalid code path, ommysql may cause a segfault if database + transactions fail into a specific way. The main trigger is a totally + irrecoverrable database error which can lead to premature connection + close, which is not checked for in all recover code. + This was detected in a setting where a stored procedure is called that + rolls back a transaction in itself. + This patch fixes the issue. + closes https://github.com/rsyslog/rsyslog/issues/5288 +- 2024-02-26: omfile: do not carry out actual action when writing to /dev/null + In some use cases omfile is configured to write to /dev/null. This seems + primarily be done because of statistics gathering but maybe some other + scenarios. We now add conditional logic to not do any actual omfile + action when the target file is /dev/null. + Note: this check only works on static file names. When /dev/null is + evaluated as part of dynafile, it will be handled just in the regular + case like before this patch. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2312.0 (aka 2023.12) 2023-12-12 +- 2023-12-11: imjournal: Add new input module parameter 'defaulttag' + The DefaultTag option specifies the default value for the tag field. + In imjournal, this can happen when one of the following is missing: + * identifier string provided by the application (SYSLOG_IDENTIFIER) + * name of the process the journal entry originates from (_COMM) + Thanks to Attila Lakatos for the patch. +- 2023-12-08: core bugfix: rsyslog messages may not always have FQDN + Even if hostname FQDN is configured, rsyslog internal messages generated + after rsyslog startup and before the first HUP will not necessarily have + FQDN but instead only the shortname of the local host. This commit + fixes the situation. + Special thanks to github user eciii for doing a great bug analysis + and helping us considerably to fix the issue. + closes https://github.com/rsyslog/rsyslog/issues/5218 +- 2023-12-08: omlibdbi regression fix: database path was not properly used + Commit 4a072d6c93015a63716c49a6c7756df22750086a caused a regression that made + the database path unreliable to use. Depending on platform/libc version the + basename was improperly extracted, which made access to the database of sqllite + impossible. + Thanks to Flávio Tapajós for the patch. + closes: https://github.com/rsyslog/rsyslog/issues/5282 +- 2023-12-06: mazureeventhubs: Corrected handling of transport closed failures + - Added test for connection interrupts (requires root) + - Corrected handling of PN_TRANSPORT_CLOSED. + - Make sure Connection is being reestablished trough tryResume + - Enhanced Debug log output + closes: https://github.com/rsyslog/rsyslog/issues/5269 +- 2023-11-24: imkmsg: add params "readMode" and "expectedBootCompleteSeconds" + These parameters permit to control when imkmsg reads the full + kernel log upon startup. + Parameter "readMode" provides the following options: + * full-boot - (default) read full klog, but only "immediately" after + boot. "Immediately" is hereby meant in seconds of system + uptime given in "expectedBootCompleteSeconds" + * full-always - read full klog on every rsyslog startup. Most + probably causes messag duplication + * new-only - never emit existing kernel log message, read only + new ones. + Note that some message loss can happen if rsyslog is stopped + in "full-boot" and "new-only" read mode. The longer rsyslog is + inactive, the higher the message loss probability and potential + number of messages lost. For typical restart scenarios, this + should be minimal. On HUP, no message loss occurs as rsyslog + is not actually stopped. + The default value for "expectedBootCompleteSeconds" is 90. + see also https://github.com/rsyslog/rsyslog/issues/5161 +- 2023-11-10: imkmsg: add module param parseKernelTimestamp + The parameter permits to select whether or not and when kernel + timestamps shall parsed, that is be used as the actual time a + log message occurs. + This permits to work around problems with the way kernel + timestamps are represented. The reasoning is given in a sysklogd + commit by Joachim Wiberg, which we reproduce below ("QUOTE") to + have a stable reference. + The commit itself can be found for example at: + https://github.com/troglobit/sysklogd/commit/9f6fbb3301e571d8af95f8d771469291384e9e95 + The new parameter parseKernelTimestamp has three possible modes: + "startup" - uses the kernel time stamp during the initial read + loop of /dev/kmsg, but replaced it later ignores it for later reads. + This is the DEFAULT setting. + "on" - kernel timestamps are always used and no correction is tried + "off" - kernel timestamps are never used, system time is used instead + Note that there this is a slightly breaking change. Previously, imkmsg + reported similar to "off" mode, now it reports by default in "startup" + mode. We consider this acceptable, as "off" mode timestamps are not + correct for startup. After startup, the behaviour is correct. All in + all, the new default is kind of a bugfix. + ============== QUOTE =============== + The spec[1] says the /dev/kmsg timestamp is a monotonic clock and in + microseconds. After a while you realize it's also relative to the boot + of the system, that fact was probably too obvious to be put in the spec. + However, what's *not* in the spec, and what takes a while to realize, is + that this monotonic time is *not* adjusted for suspend/resume cycles ... + On a frequently used laptop this can manifest itself as follows. The + kernel is stuck on Nov 15, and for the life of me I cannot find any to + adjust for this offset: + $ dmesg -T |tail -1; date + [Mon Nov 15 01:42:08 2021] wlan0: Limiting TX power to 23 (23 - 0) dBm as advertised by 18:e8:29:55:b0:62 + Tue 23 Nov 2021 05:20:53 PM CET + Hence this patch. After initial "emptying" of /dev/kmsg when syslogd + starts up, we raise a flag (denoting done with backlog), and after this + point we ignore the kernel's idea of time and replace it with the actual + time we have now, the same that userspace messages are logged with. + Sure, there will be occasions where there's a LOT of kernel messages to + read and we won't be able to keep track. Yet, this patch is better than + the current state (where we log Nov 15). + [1]: https://www.kernel.org/doc/Documentation/ABI/testing/dev-kmsg + ===========END QUOTE =============== + closes https://github.com/rsyslog/rsyslog/issues/4561 + closes https://github.com/rsyslog/rsyslog/issues/5161 +- 2023-11-07: imfile bugfix: remove state file on file delete + The state file would remain in the working directory + after shutdown, even though deleteStateOnfileDelete is + set to "on" and the monitored file was removed. + closes https://github.com/rsyslog/rsyslog/issues/5258 + Thanks to Attila Lakatos for the patch. +- 2023-10-31: TLS subsystem: fix small memory leak on startup + This was a one-time leak of the file name that hapened if a certificate file + was not accessible. It had no operational issues, but could confuse automatted + testing. As not only a side-effect, certificate load failures are now somewhat + more verbosely reported, which we consider helpful to the user. + Thanks to Attila Lakatos for the patch. +- 2023-10-31: imklog bugfix: keepKernelTimestamp=off config param did not work + ... at least not as expected. It was only honored for kernel-level + messages and only when parseKernelTimestamp was "on". Otherwise, the + kernel timestamp was always kept inside the message. + closes https://github.com/rsyslog/rsyslog/issues/5160 +- 2023-10-26: TLS subsystem: add remote hostname to error reporting + This provides richer and easier to process logs for error and warning + cases. One goal is to enable automatic operations without the need + to consolidate multiple message to a single information. + This improves one situation in gtls driver and provides a more + generic approach in ossl driver for OpenSSL error reporting. + There is probably still room for improvement, however this patch + is at least a good starting point for further work. Please + provide feedback if you need more! + closes https://github.com/rsyslog/rsyslog/issues/5244 +- 2023-10-24: imjournal: add the ability to run multiple journal inputs + This may be useful to de-couple journal processing. + Thanks to Willy Tu for the patch. +- 2023-10-24: regression fix: forking rsyslogd on BSD did not work + Actually, this was an issue for all platforms that do not provide open file handle + detection via the /proc file system. + Tech details: After fork if the child process uses close_range to close open file + descriptors it has no way to exempt the parentPipeFD causing a failure to signal + successful startup to the parent process. This causes failures on all systems that + aren't Linux that implement close_range. + Thanks to Nathan Huff for the patch. +- 2023-10-24: omusrmsg: use logind instead of utmp for wall messages with systemd + Future SUSE versions will get rid of utmp due to a 32bit time_t counter + overflow in 2038. + See details at: + https://github.com/thkukuk/utmpx/blob/main/Y2038.md + On systemd based systems logind is an alternative to utmp. + Thanks to github user tblume for the patch. +- 2023-10-24: cleanup: rm no longer used --with-systemdsystemunitdir configure switch + This is a clean up following the removal of the service unit in + cfd07503ba055100a84d75d1a78a5c6cceb9fdab +- 2023-10-23: testbench: bump zookeeper version to match current offering + Older version can no longer be downloaded. It also makes sense to + test with mainstream version. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2310.0 (aka 2023.10) 2023-10-10 +- 2023-10-04: Add CAP_NET_RAW capability due to the omudpspoof module + The CAP_NET_RAW ensures the use of RAW and PACKET sockets, + which is utilized by the omudpspoof module, more precisely + the libnet_init function. + Thanks to Attila Lakatos for the patch. +- 2023-10-04: Add new global config option "libcapng.enable" + Defines whether rsyslog should drop capabilities at startup or not. + By default, it is set to "on". Until this point, if the project was + compiled with --enable-libcap-ng option, capabilities were + automatically dropped. This is configurable now. + Thanks to Attila Lakatos for the patch. +- 2023-10-04: tcp net subsystem: handle data race gracefully + It may happen that a socket file descriptor has been closed either + while setting up poll() et al or while being inside the system call. + This was previously treated as error and caused abort in debug + builds. However, it was essentially ignored in production builds. + This has now been fixed and now is always gracefully ignored. This + most importantly fixes some flakes in CI runs (which were caused + by this situation). +- 2023-09-29: imrelp bufgifx: avoid crash on restart in imrelp SIGTTIN handler + While existing, if at specific time rsyslog receives a SIGTTIN, it + crashes due to 2 issues. + 1. debug.unloadModules="off" a double free of pRelpEngine + 2. debug.unloadModules="on" it crashes because the signal handler has + been unmapped from memory. + This patch covers both issues. + Thanks to Ali Abdallah for the patch. +- 2023-09-28: fix startup issue on modern systemd systems + When we startup AND are told to auto-background ourselfs, we must + close all unneeded file descriptors. Not doing this has some + security implications. Traditionally, we do this by iterating + over all possible file descriptor values. This is fairly compatible, + because we need no OS-specific method. However, modern systemd configs + tend to not limit the number of fds, so there are potentially 2^30(*) + fds to close. While this is OKish, it takes some time and makes + systemd think that rsyslog did not properly start up. + We have now solved this by using the /proc filesystem to obtain our + currently open fds. This works for Linux, as well as Cygwin, NetBSD, + FreeBDS and MacOS. Where not available,and close_range() is available + on the (build) platform, we try to use it. If that fails as well, we + fall back to the traditional method. In our opionion, this fallback + is unproblematic, as on these platforms there is no systemd and in + almost all cases a decent number of fds to close. + Very special thanks go out to Brennan Kinney, who clearly described + the issue to us on github and also provided ample ways to solve it. + What we did is just implement what we think is the best fit from + rsyslog's PoV. + (*) Some details below on the number of potentially to close fds. + This is directly from a github posting from Brennan Kinney. + Just to clarify, by default since systemd v240 (2018Q4), that + should be `1024:524288` limit. As in the soft limit is the expected + `1024`. + The problem is other software shipping misconfiguration in systemd + services that overrides this to something silly like + `LimitNOFILE=infinity`. + - Which will map to the sysctl `fs.nr_open` (_a value systemd + v240 also raises from `2^20` to 2^30`, some distro like Debian are + known to opt-out via patch for the `fs.nr_open` change_). + - With the biggest issue there being that the soft limit was also + set to `infinity` instead of their software requesting to raise + the soft limit to a higher value that the hard limit permits. + `infinity` isn't at all sane though. + - The known source of this misconfiguration is container software such + as Docker and `containerd` (_which would often sync with the + systemd `.service` config from the Docker daemon `dockerd.service`_). + closes https://github.com/rsyslog/rsyslog/issues/5158 +- 2023-09-13: Add the 'batchsize' parameter to imhiredis + Parameter set to allow configuring the amount of entries imhiredis debatches at once. + Default value of '10' has been kept to avoid any side effect on existing + configurations. + Thanks to Jérémie Jourdin for the patch. +- 2023-09-13: omprog bugfix: Add CAP_DAC_OVERRIDE to the bounding set + The omprog module uses the execve() function to execute + a third party program. Some required capabilities were not + preserved in the bounding set [1]. This caused problems, e.g. + the program could not write to files even if rsyslog was + executed as root and privileges were not dropped. As of now, + only the CAP_DAC_OVERRIDE capability is added to the bounding + set. Others could be added later, if there is justification + behind that. + [1] The capability bounding set is a security mechanism that + can be used to limit the capabilities that can be gained + during an execve(2). During an execve, the capability + bounding set is ANDed with the file permitted capability + set, and the result of this operation is assigned to the + thread's permitted capability set. The capability + bounding set thus places a limit on the permitted + capabilities that may be granted by an executable file. + Thanks to Attila Lakatos for the patch. +- 2023-09-13: tcpflood bugfix: plain tcp send error not properly reported + The error code when plain tcp sending failed was improperly returned, + resulting in no meaningful error message. + Note: tcpflood is a testbench tool, not part of production rsyslog. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2308.0 (aka 2023.08) 2023-08-15 +- 2023-08-07: crypto subsystem bugfix: potential undefined behaviour + The is some potential undefined behaviour when initializting the IV for locally + encrypting log files. The issue cancels itself out, but at least causes + some confusion when using undefined behaviour sanitizer (UBSAN). However, + UBSAN seems not to detect the issue on all platforms and/or in all versions + (we were not able to reproduce this issue in our CI). + Please also note that the functionality where this can happen is extremely + rarely being used. + Thanks to Jeffrey Walton for providing the patch. +- 2023-08-02: lookup tables: fix static analyzer issue + If something goes really wrong, a lookup table's name would not + be set. That could lead to a NULL pointer access. HOWEVER, this + would require serious bugs in config parameter parsing, as the + lookup table name is a required parameter and the parser will + error out if not set. + So the bug is mostly cosmetic - but it does not hurt to handle + this case, of course. +- 2023-08-02: lookup tables bugfix: reload on HUP did not work when backgrounded + Lookup tables were only reloaded on HUP if the -n option was given + and rsyslog no backgrounded. This patch fixes the issue. + closes: https://github.com/rsyslog/rsyslog/issues/4813 +- 2023-07-30: testbench: make test more reliable + There was a race between tcpflood and rsyslog in imptpc_maxsessions.sh. + We now use the new -A tcpflood option to make the timing more + predictable, hopefully fixing test flakiness. + Note: if that does not help, we need to introduce a wait on the number + of error messages and maybe a delay before tcpflood termination. The + theory behind the latter is that rsyslog possibly does not fully + iniaitlize session which are quickly aborted before rsyslog receives + the related OS notification! We just record this info in case we + need it and are positive that this change will fix the situation. +- 2023-07-28: openssl: make connection setup more reliable by use of newer lib feature + Replaced depreceated method SSLv23_method with TLS_method. + In OpenSSL 1.1.0 and higher, SSLv23_method causes some errors + in TLS handshake from time to time. As this method is depreceated + since 1.1.0, I have replaced it with the follow up method + TLS_method which is the most generic one. + It fixes the random test failures in tests like + - sndrcv_tls_ossl_anon_rebind.sh + Also added some debug output in OpenSSL error handling, which is + useful when analysing debug files. + closes: https://github.com/rsyslog/rsyslog/issues/5201 +- 2023-07-28: testbench improvement: define state file directories for imfile tests + Not all imfile tests have state file directories or a global working + directory defined. This results in usage of the default location. + While state file names should be sufficiently different, there is still + some riks of using the same name in different tests. That becomes + problematic if tests are run in parallel (and they are run in + parallel inside the regular CI). + NOTE: NOT YET COMPLETED FOR ALL TESTS! We are considering if it makes + sense to deliberately keep some as-is. +- 2023-07-28: tcpflood bugfix: TCP sending was not implemented properly + Note: tcpflood is a testbench tool. This bug could lead to testbench + false positives. No way it can affect production deployments. + The tcpflood tool did improperly assume that a TCP sendto() call + would send messages of any size in a single shot. This is not the + case. It has now been corrected to proper behavior. + As a side-activity, some int variables which acutally needed to be + size_t have been fixed as well. +- 2023-07-28: testbench: make waiting for HUP processing more reliable + The previous approach was more or less delay based. We have now + changed the code to enable imdiag to detect if HUP is underway + and wait until it is completed. The new method still employs some + kind of timeout, but is now quite reliable. Most importantly, + it works great with long-running HUP processing, which can happen + e.g. when querying the system name takes long or some actions need + longer time to persist their HUP processing. + The new approach will most likely reduce CI flakes and also speed + up testbench runs. The speedup happens from not having to wait a + full delay in cases where we detect HUP is completed (plus reduced + timeout when we cannot clearly detect this - see code comments why + the new method is still considered more reliable than the old one). + Code note: we needed to slightly re-structure the way actual HUP + processing and the "HUP mutex" is handled. After best analysis, + this does not affect the reliability or speed in production + settings. + closes https://github.com/rsyslog/rsyslog/issues/5192 +- 2023-07-27: build system: make rsyslogd execute when --disable-inet is configured + This option is mostly useless, as network functionality depends on the + modules loaded by the config. The only real, and important, effect it + has is to control auto-load of omfwd - a feature almost all installations + depend in (backward compatibility). + This has been clarified in ./configure -help + Also, when --disable-inet is given, rsyslog now executes successfully. + The reason for the abort was that previously building of the lmnet + component was prevented, but that component is also needed by rsyslog + startup itself to query its own (correct) hostname. + Note that --disable-inet still does not compile some networking + libraries. So do not use it if you intend to load standard networking + modules like omfwd, imtcp or imudp. + closes https://github.com/rsyslog/rsyslog/issues/5188 +- 2023-07-26: testbench/CI: update zookeper download to newer version + Old version is no longer available. +- 2023-07-24: openssl: add support for new-version init function +- 2023-07-07: add CRL support for network (TLS) drivers + Thanks to Darren J Moffat for implementing the OpenSSL part. +- 2023-07-07: omazureeventhubs: Initial implementation of new output module + The output module uses Apache "Qpid Proton C API" which is a solid + AMQP protocol library implementation that can be integrated + very well into the rsyslog dev environment. + - Implemented Delivery with submitted and accepted state checking + - saving of failed messages in a failed list with support of saving + and restoring. + - Add testcases (requires ENV variables) to testbench + - Using application/octect-stream (binary) to send messages based on + Microsoft Code Sample: + https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-c-getstarted-send + * Note original Microsoft Samplecode is not working anymore, we are using + * QPID Proton Proactor based on + https://github.com/apache/qpid-proton/blob/main/c/examples/send.c + - requires QPID-PROTON Version 0.13 or higher because of the proactor API + - Add EventProperties configuration parameters + - Slow down when sender credit reaches zero (10ns). + - Add support for static library linking of qpid-proton + This is needed to build the module from source and remove + library package dependencies. + - adjusted valgrind suppressions +- 2023-07-04: core bugfix: action.resumeintervalmax parameter was not respected + Unfortunately, defining action.resumeintervalmax in the configration + did not have any effect at all. Instead, the default value was used, + which is 1800. This was caused by not having all the letters in + lower-case. + Fixes https://github.com/rsyslog/rsyslog/issues/5132 + Thanks to Attila Lakatos for the patch. +- 2023-06-29: core bugfix: do not try to drop capabilities when we don't have any + In case the process does not have any capabilities, e.g. running as regular user then + we do not have to force capability dropping. The capng_have_capabilities() returns + none if that's the case. + Fixes https://github.com/rsyslog/rsyslog/issues/5091 + Thanks to Attila Lakatos for the patch. +- 2023-06-29: imhiredis bugfix: Restore compatiblity with hiredis < v1.0.0 + RESP3 protocol wasn't implemented yet, some types weren't + available (REDIS_REPLY_DOUBLE) + Thanks to Théo Bertin (frikilax) for the patch. +- 2023-06-23: testbench: use newer zookeeper version in tests +- 2023-06-23: build system: more precise error message on too-old lib + When libcap-ng was enabled, the lib was present but did not meet the minimum version + dependency during configure, it was reported as "missing". We now emit a message + telling that it is present, but the version too old. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2306.0 (aka 2023.06) 2023-06-20 +- 2023-06-19: mmnormalize bugfix: if msg cannot be parsed, parser chain is stopped + When an parser is not able to parse a message, it should indicate this + to rsyslog core, which then activates the next parser(s) inside the + configured parser chain. + Unfortunatley, mmnormalize always tells core "success", and so no + other parsers are activated. + closes https://github.com/rsyslog/rsyslog/issues/5148 +- 2023-06-19: [i/o]mhiredis: various fixes and enhancements + please see the change log for details. Among others, suspending of the modules + has been fixed. Also a new "stream" mode has been added. + Thanks to Théo Bertin (frikilax) for the patch. +- 2023-06-19: testbench/bug: mmexternal-SegFault-empty-jroot-vg.sh fails due to typo + Fix the typo that makes the test fail. + Thanks to Paul Fertser for the patch. +- 2023-06-16: imjournal: Add FileCreateMode module parameter + FileCreateMode allows to set the default file mode bits + when creating new files. As of now, it has only impact on the state file. + Add test suite as well. + Minor indentation fix in run_journal.yml + Thanks to Attila Lakatos for the patch. +- 2023-06-16: core bugfix: potential segfault on busy systems + This was discovered by Konstantin J. Chernov in a practicaly deployment. + Here, msg object tag processing caused sporadic segfaults. We did not + hear from similiar cases, but there clearly is potential for problems + because a mutex lock had insufficient range, thus leading to a potential + race. + The patch is directly from Konstantin J. Chernov, thanks for that. + Please note that the mutex lock could be minimized as it is not strictly + needed for the pM == NULL case, but this cause is extremely exotic + and the resulting code would be harder to understand. Thus we opt + to do the locking on funtion level (as usual). + Descriptiond edited by Rainer Gerhards + closes: https://github.com/rsyslog/rsyslog/issues/5110 +- 2023-06-16: Add new global config option "libcapng.default" + Defines how rsyslog should behave in case something went wrong + when capabilities were to be dropped. Default value is "on", + in which case rsyslog exits on a libcapng related error. + Thanks to Attila Lakatos for the patch. + Closes https://github.com/rsyslog/rsyslog/issues/5096 +- 2023-06-05: imfile bugfix: file handle leak, primarily in kubernetes context + At this point there is a code imfile.c#L919 that adds an inotify observer to the + parent of the symbolic link target. But there is no such code that removes this + observer in the case when inotify events do not occur in the directory tree above. + This may be if the directory tree of the symbolic link target and the directory tree + of the symbolic link itself are divided into different subtrees somewhere at the levels + above. + For example, in the rsyslog configuration, an imfile with the + template /var/log/containers/*.log is configured and there is the following directory + tree: + /var/log/pods/pod-1/a/0.log + /var/log/containers/pod-1-a-0.log -> /var/log/pods/pod-1/a/0.log + In this example, kubernetes cron jobs will permanently delete directories at the + /var/log/pods/pod-* level. And thus, inotify observer on the parent object of the + symbolic link target (/var/log/pods/pod-1/a/0.log) looking at the directory + /var/log/pods/pod-1/a will constantly leak. + This is due to the fact that the list of active objects in the edge with path + /var/log/containers, where the parent object of the target symbolic link is added, + is not checked. Verification and deletion will occur only in the case of an inotify + event in the upper nodes of the directory tree, in /var/log and above. + Thanks to Sergey Kacheev for the patch! +- 2023-06-05: GNUTls Driver: Fix memory leaks in gtlsInitCred + Missing CA Certificate or multiple Connections caused + a memory leak in pThis->xcred as it was allocated each time in + gtlsInitCred by gnutls_certificate_allocate_credentials + closes: https://github.com/rsyslog/rsyslog/issues/5135 +- 2023-05-24: CI: update base ubuntu image for github actions +---------------------------------------------------------------------------------------- +Scheduled Release 8.2304.0 (aka 2023.04) 2023-04-18 +- 2023-04-17: imptcp bugfix: spam log on oversize message + If an oversize message was received by imptcp, imptcp reported + one error message for EACH oversize character. This could + result in a potentially very large number of similar (and + useless) messages. + This is a regression from commit f052717178. + closes https://github.com/rsyslog/rsyslog/issues/5078 +- 2023-04-17: core/bugfix: using $uuid msg prop can deadlock rsyslog on shutdown + This problem can occur if a large number of threads is used and rsyslog + cannot shut down all queues etc within the regular time interval. In this + case, it cancels some threads. That can leave the mutex guarding libuuid + calls locked and thus prevents other, not yet cancelled threads from + progressing. Assuming pthread_mutex_lock() is not a cancellation point, + this will case these other threads to hang forever and thus create a + deadlock situation. + closes https://github.com/rsyslog/rsyslog/issues/5104 +- 2023-04-17: Do not preserve capabilities when changing credentials + In configurations where $PrivDropToGroup or $PrivDropToUser are used, + rsyslogd changes uid/gid to a non-privileged user. As part of that + change, all capabilities should be lost. However, if rsyslog is + compiled with --enable-libcap-ng option, some capabilities are + preserved due to using capng_change_id() instead of setgid()and + setuid(). https://linux.die.net/man/3/capng_change_id: + This function preserves capabilities while changing uid/gid, causing + rsyslogd to run as non-root user, but with some root capabilities. + Unfortunately, rsyslogd will run with higher privileges than before. + The patch also removes CAP_SETPCAP, because the capability set does + not need to be altered at a later phase. + Thanks to Attila Lakatos for the patch. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2302.0 (aka 2023.02) 2023-02-21 +- 2023-01-27: core/template: implement negative position.to + This will easily permit to drop the last n characters from a property + without the need to know the exact length of the string. This is + especially useful as the exact length is most often not known + beforehand. +- 2023-01-18: Introduce --enable-libcap-ng configure option + The option allows to drop the capabilities to only + the necessary set, to minimize security exposure in + case there was ever a mistake in a networking + plugin or some other input resource. Moreover, it adds + ability to change uid and gid while retaining the + previously specified capabilities. + Add ability to change uid and gid while retaining the + capabilities previously specified. + closes https://github.com/rsyslog/rsyslog/issues/4986 + Thanks to Attila Lakatos for the patch. +- 2023-01-16: + - omfile: add action parameters "rotation.*" + Add new action parameters + - rotation.sizeLimit + - rotation.sizeLimitCommand + provide automatic output file rotation functionality feature-wise + equivalent to legacy $outchannel. This finally permits to use + this feature set in rscript. + - core substring function: enhancement and hardening + Now, length can have a negative value -n to denote that the + substring should be build between startpos and the character + -n chars from the end. This is a shortcut for stripping charactes + on "both ends" of the string. See doc for details on the enhanced + semantics. + Also, some hardening against invalid startpos and length has + been added. + - core bugfix: wrong type conversion in internal string class could lead to segfault + This could only happen with very unusually large strings + Thanks to Flos Lonicerae for the patch. + - QA: changed to CodeQL scanning on github as LGTM replacement + - bugfix: wrong version number on daily stable builds + - CI: use newer version of zookeeper (needed modernization) + - ffaup bugfix : memory corruption with concurrent workers + The ffaup function fails to work properly when it is used with multiple workers. + The faup_handler_t struct is not supposed to be shared between threads. + This may have caused memory corruptions and race conditions when used + inside of actions. + Thanks to Thibaud Cartegnie for the fix. + - openssl bugfix: undefined reference error on OpenSSL 1.1 or higher. + This could have prevented ossl components from being loaded/used. +- 2023-01-02: core bugfix: template system may generate invalid json + When + - a list template + - is created with option.jsonf="on" + - and the last list element is a property with onEmpty="skip" + - and that property is actually empty + invalid JSON is generated. + The JSON string in this case ends with ", " instead of "}\n". This + patch fixes the issue. + closes https://github.com/rsyslog/rsyslog/issues/5050 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2212.0 (aka 2022.12) 2022-12-06 +- 2022-12-05: testbench: make python http server based tests more reliable + Harden them against races during server port assignment. Prevents + testbench flakes. +- 2022-12-05: omprog bugfix: invalid status handling at called program startup + There is a bug when external program *startup* does not return "OK". This + can also lead to a misadressing with potentially a segfault (very unlikely). + Note that no problem exists once the initializiation phase of the external + program is finished and regular message transfer runs. + The problem basically is that for a startup failure, the control data for + that external program instance is freed on error. Unfortunately, that state + data is needed later on to detect a suspended instance. We now keep the control + data even on init failure (as we then need to do normal control options). + closes https://github.com/rsyslog/rsyslog/issues/4967 +- 2022-11-29: testbench bugfix: wrong message injection object of instance 1 + In some client-server test cases, messages are supposed to be injected into + the instance 2(client), but they are actually injected into instance 1(server), + which may lead to false negative results. This patch fixed it by replacing + 'injectmsg' with 'injectmsg2', and dealt with some minor issues. + Thanks to Guodong Zhu for the patch. +- 2022-11-21: rsyslog.conf man page bugfix: description of selectors + Document historic difference to BSD syslog selectors. +- 2022-11-18: imtcp bugfix: legacy config directives did no longer work + Many "$InputTCPServer..." config directives did no longer work + and were completely ignored (e.g. "$InputTCPServerStreamDriverMode"). + This was a regression from a08591be5d9 (May, 5th 2021). + closes https://github.com/rsyslog/rsyslog/issues/5021 +- 2022-11-16: ksi bugfix: sending of too many signing requests fixed. + As there is a bug in libksi where too many signing requests may have bene sent + out the amount of signing requests will be limited by KSI module until the fix + is implemented. + Thanks to Taavi Valjaots for the patch. +- 2022-11-14: bugfix: prevent potential segfault when switchung to queue emergency mode + When switching to Disk queue emergency mode, we destructed the in-memory + queue object. Practice has shown that this MAY cause races during + destruction which themselfs can lead to segfault. For that reason, we + now keep the disk queueu object. This will keep some ressources, + including disk space, allocated. But we prefer that over a segfault. + After all, it only happens after a serious queue error when we are + already at the edge of hard problems. + see also: https://github.com/rsyslog/rsyslog/issues/4963 +- 2022-11-08: ksi bugfix: Segmentation fault in async mode fixed + Thanks to Taavi Valjaots for the patch. +- 2022-11-02: imjournal: add second fallback to _COMM + If SYSLOG_IDENTIFIER is not present in the journal message, + then lookup the _COMM field, which stands for the name + of the process the journal entry originates from. This is + needed in order to be in compliance with the journalctl + output. + Thanks to Attila Lakatos for the patch. +- 2022-10-25: core bugfix: local hostname invalid if no global() config object given + The local hostname is invalidly set to "[localhost]" on rsyslog startup + if no global() config object is present in rsyslog.conf. Sending a HUP + corrects the hostname. + This is a regression from ba00a9f25293f + closes https://github.com/rsyslog/rsyslog/issues/4975 + closes https://github.com/rsyslog/rsyslog/issues/4825 +- 2022-10-25: testbench bugfix: fixed timing issue that sometimes lead to test failure + Timing caused a race in test tool sync and could lead to premature termination of + tools, which in turn caused test failure +---------------------------------------------------------------------------------------- +Scheduled Release 8.2210.0 (aka 2022.10) 2022-10-18 +- 2022-10-13: fix NetBSD build issue + On NetBSD, time_t has for a long time now been __int64_t. + On 32-bit CPUs, the compiler is not obliged to define + __sync_bool_compare_and_swap_8, so instead this ends up + as an undefined symbol when linking rsyslog. This makes + the code fall back to the pthread / locking method on these + systems, but at least lets the program build. + Thanks to Havard Eidnes for the patch. +- 2022-10-12: omrabbitmq: Add TLS support + Thanks to github user 21stcavenan for the patch. +- 2022-09-14: config: add "abortOnFailedQueueStartup" global config parameter + similiar to "abortONUncleanConfig", this parameter aborts rsyslog + when a queue has problems during startup. Some users perfer rsyslog + to terminate in this case. By default, nothing changes. + closes https://github.com/rsyslog/rsyslog/issues/4902 +- 2022-09-07: cor bugfix: leak in helper function SetString + A part of rsyslog runtime, SetString(), had a small memory leak when a value was + assigned multiple times. While this could potentially consume larger amounts of + memory, this did not happen in practice. The reason is that multiple assignments + to the same object occur very seldom. + Thanks to github user seuzw930 for the patch. + closes: https://github.com/rsyslog/rsyslog/issues/4961 +- 2022-09-07: core bugfix: correct local host name after config processing + rsyslog.conf may affect the host's local name. These changes were + so far only activated after the first HUP. This patch now ensures + that the configured local host name is applied correctly throughout + all processing, including early startup. + This patch causes a slight change of behaviour. However, the behaviour + was inconsitent before. Now it is consistent and according to the config. + Please note: this patch also exposes a global entry point via "regular" + dynamic loading as this makes things much easier to do. This is in-line + with ongoing simplification effort. + Finally, we also remove a CI test that we do no longer need because + the problem covered is now addressed differently and the original issue + can no longer occur. + closes https://github.com/rsyslog/rsyslog/issues/4975 +- 2022-08-31: imtcp: add option notifyonconnectionopen + Add this both as module an input parameter. Complements already-existing + config param notifyonconnectionclose and mirrors the similar feature from + imptcp. + The module parameter acts as default, similarly to notifyonconnectionclose. + Note that in contrast to imptcp, we emit IP addresses and not host + names. This sticks with the traditional semantics of imtcp. + Note that we also fixed a mislading error message in the case when a + disallowed sender tried to connect. + Thanks to John Chivian for suggesting the addition. +- 2022-08-26: openssl TLS driver: add mechanism to include extra CA files parameter + This change allows to include extra CA files so that no "unable to get issuer + certificates" issue is obtained when using chained cert files. New parameter name is + "NetstreamDriverCAExtraFiles". + Thanks to Sergio Arroutbi for the patch. + closes: https://github.com/rsyslog/rsyslog/issues/4851 +- 2022-08-19: fix compile issue with older gcc compilers + Thanks to Julien Thomas for the contribution. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2208.0 (aka 2022.08) 2022-08-09 +- 2022-08-09: ksi bugfix: request cache size and send timeout issue fixed. + Async service send timeout is not configurable and request cache size is too + small to handle large amount of signing requests with small amount of permitted + requests per aggregation round. For example user with max_requests = 4 results + cache size 5 * max_requests or at least 256. When signing 300 log files cache + will be too small resulting several unsigned blocks. When signing 200 log file + cache will be adequate, but with rate of 4 signatures per second, it is only + possible to sign 4 * 10 blocks before all requests that are not sent out will + timeout. + Fix for the issue is to make send timeout configurable and make the size of the + cache depend on the value of send timeout. New configuration value + sig.block.signtimeout="time, s" introduced that defines the time window wherein + the block has to be signed. The size of the request cache is increased to + 3 * max_requests * sign_timeout or at least 256. + Thanks to Taavi Valjaots for the patch. +- 2022-08-09: imjournal bugfix: segmentation fault in close journal + Thanks to github user t-feng for the patch. +- 2022-08-09: net subsystem: support sha256 for StreamDriverAuthMode="x509/fingerprint" + Thanks to github user codemaker219 for the patch. +- 2022-08-05: imfile bugfix: message loss/duplication when monitored file is rotated + When a to-be-monitored file is being rotated, some messages may be lost or + duplicated. In case of duplication, many file lines may be duplicated + depending on actual timing. The whole bug was primarily timing depenedent + in general. It most often was visible in practice when the monitored + file was very frequently rotated (we had some report with every few + seconds). + Note that while we try hard to not lose any messages, input file + rotation always has some loss potential. This is inevitable if + the monitored file is being truncated. + Also note that this bugfix affects imfile, only. It has nothing to do + and no relation to rsyslog output files being rotated on HUP. + closes: https://github.com/rsyslog/rsyslog/issues/4797 +- 2022-08-05: ksi bugfix: optimize processing of signer queue to fix delays. + There is a worker queue where rsyslog KSI module collects events and signing + requests. When queue is processed thread is periodically put to sleep. Previous + implementation handles signature requests well but sleeps every time after + handling new file open / close event. When several log files are opened or + closed simultaneously process is significantly slowed down. Another issue is + that thread always sleeps 1000ms that may be 2x longer than aggregation round. + This slows down overall signing process. + Fix for the issue is to simply not sleep after file open / close event if there + are next items to be processed. To speed up the signing process, rsyslog uses + KSI aggregator conf. to obtain the aggregation period that is used for the sleep + time configuration. + Thanks to Taavi Valjaots for the patch. +- 2022-08-04: ksi bugfix: possible crash fixed when several log files are opened. + KSI module in async mode used to request aggregator conf. every time a log + file was opened. When several log files were opened simultaneously + corresponding amount of pointless concurrent conf. requests were posted. + Concurrent conf. requests lead to a bug in libksi, where internal count of + pending requests was not decremented correctly causing system to crash. + Fix for the issue is to optimize the frequency of conf. requests so that only + one conf. requests is handled at once. Instead of checking conf. every time + log file is opened, conf is requested periodically after conf timeout. This will + affect both sync and async mode. + New option for KSI module introduced - sig.confinterval="time, s". + Thanks to Taavi Valjaots for the patch. +- 2022-08-04: openssl: add support to split tls commands by semicolon + - Add support to split tls commands by semicolon. + - Changed one test with multiple tls commands to use semicolon as + separator instead of newline. + closes: https://github.com/rsyslog/rsyslog/issues/4852 +- 2022-08-04: openssl subsystem bugfix: build issue on Solaris + Needed header file was added. Platforms other than Solaris did not actually need it, + so this bug was discovered late. + Thanks to Jakub Kulík for the patch. + Import <strings.h> when index() is used. +- 2022-08-04: openssl: add more details to error messages + - Avoid LogMsg outputs osslEndSess on successfull terminated + connection. Only LogMsg if the connection was terminated + unsuccessfully. + - Handle SSL_ERROR_SYSCALL in both Send / osslRecordRecv, + do not log as error if underlaying socket was terminated + (ECONNRESET). Log as information instead. + closes: https://github.com/rsyslog/rsyslog/issues/4946 +- 2022-08-04: omclickhouse: capture additional exceptions + - DB::NetException + - DB::ParsingExceptions + Thanks to Victor Kustov for the patch. +- 2022-08-04: mmanon bugfix: Simplified and fixed IPv4 digit detection. + - Fixed an issue with numbers above int64 in syntax_ipv4. + Numbers that were up to 256 above the max of an int64 + could incorrectly be detected as valid ipv4 digit. + - Simplified the IPv4 digit detection function and renamed + to isPosByte. + - added testcasse for malformed IPvc4 addresses + closes: https://github.com/rsyslog/rsyslog/issues/4940 +- 2022-07-21: imptcp: slight tuning + - reduce indirect addressing to obtain more speed + - also a fix for an annoying typo + - minor other optimizations + - modernization of one test +- 2022-07-20: template procesing/json: performance optimization +- 2022-07-19: core bugfix: memory leak when free action worker data table + During free action worker data table when action destruct, worker instance in worker + data table were not null. It resulted in memory leak. + Thanks to github user seuzw930 for the patch. +- 2022-07-13: omfile: support for zstd compression + The zstd library provides better and faster compression than zlib. + This patch integrates zstd as a dynamically-loadable functionality. + As such, no further dependencies need to be added to the rsyslog + base package. + Due to the increased performance, usage of zstd is highly recommended + for high-volume use cases. + This patch also refactor zlib compression in order to unify handling + in both compression cases. +- 2022-07-07: stream cleanup: move error message to debug log, only + This error message is most probably rooted in a kernel problem. At + least knowbody knows how it can happen. It's definitely not a + rsyslog issue. We also can recover from it for a long time now + so there is no reason to irritate users by emitteing this + "error" message. +- 2022-07-04: mmdblookup bugfix: Don't crash Rsyslog on mmdb file errors + Thanks to Théo Bertin (frikilax) for the patch. +- 2022-06-28: build error fix: libbson requires out-of-date language constructs +- 2022-06-27: OpenSSL: fix depreacted API issues for OpenSSL 3.x + - OpenSSL error strings are loaded automatically now + - Debug Callback has changed + - See for more: + https://www.openssl.org/docs/manmaster/man7/migration_guide.html + closes: https://github.com/rsyslog/rsyslog/issues/4912 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2206.0 (aka 2022.06) 2022-06-14 +- 2022-05-25: omelastisearch: allow omitting _type field + Allow omitting the _type field by setting it to an empty string. + Setting this field has been deprecated since 6.0, and support will + be removed in 8.0 + Also add testbench test for empty searchType with ES 7.0 + This checks for messages in the deprecation log and also provides + avoids deprecation messages from usage of transport.tcp.port in the + test configuration + Thanks to Jarkko Oranen for the patch. +- 2022-05-18: tcpsrv/imtcp: slight performance improvements + This change slightly improves performance for tcpsrv-based servers. + This affects imtcp and imgssapi as well as some helpers. + No other functional change is included in this change. +- 2022-05-12: imptcp bugfix: worker thread starvation on extreme traffic + When connectes were totally busy, without any pause, the assigened worker + did never terminate its reading loop. As such, it could not service any + other conenctions. If this happened multiple time and to all configured + workers, all other connections could not be processed at all. This extreme + scenario is very unlikely, as the whole issue is relatively unlikely. + In practice, the issue could lead to somewhat degraded performance and + resolved itself after some time (in practice no connection is 100% busy + for an extended period of time). + Note that this patch sets a fixed limit of 16 iterations for very busy + connections. This sounds like a good compromise between non-starvation + and performance. The exact number may be made configurable if there + is really need to. +- 2022-05-11: omelasticsearch: several support option for ElasticSearch 8 + - config params searchIndex and documentType can be empty + - support for Data Stream API + Thanks to github user EHerzog76 for these changes. + - new config param esVersion.major +- 2022-05-09: tcp receiver bugfix: delay/potential hang on some error conditions + Error were not correctly handled in some cases for imtcp and imgssapi. This could + lead to a temporary stall of some connections. For ultry-low traffic systems, this + stall could stay for a long period of time. In most cases, it was resolved very quickly. + Note that imptcp was not affected. + Thanks to Iwan Timmer for the fix. +- 2022-05-05: net bugfix: potential buffer overrun + there is heap buffer overflow vulnerability in rsyslog tcp reception components. + This can only happen in octet-counted mode, which is enabled by default. + Affected components: imtcp, imptcp, imhttp, imgssapi, imdiag when octet-counted + framing was enabled. + If the receiver ports are exposed to the public Internet AND are used + without authentication, this can lead to remote DoS and potentially to + remote code execution. It is unclear if remote code execution is + actually possible. If so, it needs a very sophisticated attack. + When syslog best practices with proper firewalling and authentication + is used, thean attack can only be carried out from within the Intranet + and authorized systems. This limits the severity of the vulnerability + considerably (it would obviously require an attacker already to be + present inside the internal network). + Credits to Peter Agten for initially reporting the issue and working + with us on the resolution. + fixes CVE-2022-24903 + Advisory: + https://github.com/rsyslog/rsyslog/security/advisories/GHSA-ggw7-xr6h-mmr8#advisory-comment-72243 +- 2022-05-05: imptcp: set OS worker thread name + We now set the worker thread names to "imptcp/<thrd nbr>" where + <thrd nbr> is the numerical index (0, 1, ...) of the worker thread. + This enables to distinguish individual worker threads in OS tools like + htop. That is useful for performance testing and system monitoring. + The choosen name format is consistant with other similar thread + names inside rsyslog. For imptcp, worker threads were not yet + given individual names. + Note: "in:imptcp" is imptcp's "main" thread, which also is used + as a worker in some scenarios. This name was not modified. +- 2022-04-26: mmanon bugfix: shortened IPv6 form not always anonymized + If the IPv6 is in non-recommended form followed by a 5 digit port number, it + is not anonymized. + A reproducer for this is: 1a00:c820:1180:c84c::ad3f:d991:ec2e:49255 + closes https://github.com/rsyslog/rsyslog/issues/4856 +- 2022-04-22: mmdblookup fix: wrong copy of buffer + ...following parse of libmaxminddb's return after a successful search sometimes + failed to return specific field from data. + Thanks to Théo Bertin for the patch. +- 2022-04-22: mmdblookup: several enhancements + - support arrays in MMDB entry + - support escaped quotes '"' in MMDB entry + - support '<' characters in MMDB entry, when in a field + - support '}' characters in MMDB entry, when in a field + Thanks to Théo Bertin for the patch. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2204.1 (aka 2022.04) 2021-05-05 +- security bugfix: potential buffer overrun in imptcp, imtcp, imgssapi and others + This addresses CVE-2022-24903 + see also https://github.com/rsyslog/rsyslog/security/advisories/GHSA-ggw7-xr6h-mmr8 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2204.0 (aka 2022.04) 2021-04-19 +- 2022-04-18: gnutls bugfix: possibility of infinite loop + There was a rare possibility that the E_AGAIN/E_INTERRUPT handling + could cause an infinite loop (100% CPU Usage), for example when a TLS + handshake is interrupted at a certain stage. + * After gnutls_record_recv is called, and E_AGAIN/E_INTERRUPT error + occurs, we need to do additional read/write direction handling + with gnutls_record_get_direction. + * After the second call of gnutls_record_recv (Expand buffer) + we needed to also check the eror codes for E_AGAIN/E_INTERRUPT + to do propper errorhandling. + * Add extra debug output based on ossl driver. + * Potential fix for 100% CPU Loop Receiveloop after gtlsRecordRecv + in doRetry call. + closes https://github.com/rsyslog/rsyslog/issues/4834 + closes https://github.com/rsyslog/rsyslog/issues/4818 + closes https://github.com/rsyslog/rsyslog/issues/4638 +- 2022-04-17: core/bugfix: errorfile could grow over max configures size + When action.errorfile.maxsize configuration option is enabled and error file + already has a certain size smaller than max size configured, it is increasing + higher than configured max size as the error file is considered to be zero in code. + This fix reads current error file size and limits the size to the maximum + size configured. + Thanks to Sergio Arroutbi for the patch. + fixes https://github.com/rsyslog/rsyslog/issues/4821 +- 2022-04-17: omkafka bugfix: potential misadressing + The `failedmsg_entry` expects a null-terminated string in `key`, but + here we allocate with malloc and copy a string-with-length-n into only + the first n bytes. If the final byte is null, this is by coincidence + only. + This was observed by means of seeing random binary data appended to + keys submitted to kafka apparently at random. This could also result + in more severe problems, inclusing a segfault. + Thanks to David Buckley for the patch. +- 2022-04-06: added new "FullJSONFmt" standard template (with addtl fields) + This comes handy for a number of use cases, especially with ElasticSearch. + Thanks to Art O Cathain for the patch. +- 2022-04-04: imfile: potential processing delay + This was mentioned by Mikko Kortelainen without exact details on what exactly + this could cause in practice. But we were confident enough that it is worth + merging (though it does not look like something that brought real problems in + practice, as we do not know any related reports). + see also: https://github.com/rsyslog/rsyslog/pull/4445 + Thanks to Mikko Kortelainen for the patch. +- 2022-04-04: bugfix: cosmetic data races + there was a more or less cosmetic data race which could happen when children + processes died in quick sequence. Even then, no real harm happened, as all + children were reaped eventually. + A similar data race exists for HUP processing. + However, these races polluted TSAN test runs, and so we fixed them +- 2022-04-01: add property options to support ISO week/year number + Thanks to Mattia Barbon for the patch. +- 2022-04-01: core bugfix: "action suspended" message was emitted even when turned off + Most messages were diasabled, but there was one part of the code that ignored the + user configuration. + Thanks to Deyneko Aleksey for the patch. +- 2022-03-31: testbench: add more tests for rscript comparison operations +- 2022-03-31: core bugfix: make internal logs emitted during HUP procesing appear quicker + After call doHUP(), probably there is a internal log in the list. However, it + will not be wrote out immediately, because the mainloop will be blocked at + pselect in wait_timeout() until a long timeout or next message occur. + More deadly, the log may be lost if the deamon exits unexpectedly. + We might as well put processImInternal() after doHUP(), so that the message + will be flushed out immediately. + Fixes: 723f6fdfa6(rsyslogd: Fix race between signals and main loop timeout) + Thanks to Yun Zhou for the patch. +- 2022-03-20: refactor: Move the parser directive to the main config + Thanks to Attila Lakatos for the patch. +- 2022-03-16: refactor: ake the main message queue part of the config + The intent of this patch is to make the main message queue part of the main config. + It will help us to proceed towards dynamic configuration reload. +- regression bugfix: rsyslog may segfault during startup + glblGetMaxLine() might be called even before the main configuration file exists + resulting unexpected behavior, most probably segmentation fault. This is addressed + by re-introducing the old default of 8KiB. The problem was introduced earlier in + 2022. +- regression fix: script string comparison did not work correctly + In rscript, comparison operations on strings did not work correctly + and returned false results. This is cause by a regression in commit + 5cec5dd634e0. While it fixed number comparisons, it introduced new + problems in string comparisons, which were not present before. Note + that most items in rsyslog are strings, so this can actually cause + some problems. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2202.0 (aka 2022.02) 2022-02-15 +- 2022-02-14: imfile bugfix: remove cause for "internal error message" (not causing harm) + When any message is output into a renamed input file, rsyslogd output the following: + message. + imfile: internal error? inotify provided watch descriptor 7 which we could not find + in our tables - ignored + When rsyslogd detects the inode change, it deletes the entry from wdmap[]. But, + the watch descriptor is not removed. Some application like sssd outputs some messages + (like "HUP signal was received!!") after HUP signal is received and before switching + into the new log file. And, the above messages can be output every log rotation. + This situation is now resolved. + Thanks to Masahiro Matsuya for the patch. +- 2022-02-04: rscript bugfix: literal numbers were not compared correctly + This problem occurred when numbers were used in rsyslog.conf in + the set statement, e.g. + set $nbr = 1234; + In this case, during comparisons, the number was actually interpreted + as a string with digits. Thus numerical comparisons lead to unexpected + results. Even more so, as in other places of the code they were + treated as native numbers. + This is now fixed. We cannot outrule that this causes, in border cases, + change of behavior to existing configs. But it is unlikely and the + previous behaviour was a clear bug and very unintuitive. This in our + opinion it is justified to risk a breaking change for an expected + very minor subset of installations, if any such exists at all. + closes https://github.com/rsyslog/rsyslog/issues/4770 +- 2022-02-04: omelasticsearch bugfix: indexSuccess impstats counter in bulkmode wrong + When bulkmode is enabled, and a batch was processed without any + failures (errors is false), the code that increments the indexSuccess + impstats counter was never reached. + closes: https://github.com/rsyslog/rsyslog/issues/4794 +- 2022-01-17: imkmsg bugfix: effectively disabled input on error reading kmsg + Due to a program bug, imkmsg could not recover from an kmsg read error. + Note that recovering is possible and was intended. + Thanks to Kailash Sethuraman for the patch. +- 2022-01-17: imtcp bugfix: worker threads were not properly terminated + Graceful shutdown of Rsyslog could lead to segmentation faults when + multiple imtcp inputs were being used. That is because the rest of the + tcpsrv threads are left behind running, while their underlying objects + are being disposed by the main thread as part of the module + de-initialization. + closes: https://github.com/rsyslog/rsyslog/issues/4776 + Thanks to Gabor Orosz <goro@goro.io> for the analysis and patch. +- 2022-01-07: omlibdbi bugfix: use-after-free bug + This occurred in when sqllite driver was used. Depending on circumstances, this had + no visible issues (often) up to rsyslog segfault. The busier rsyslog is, the more + likely a bad outcome. +- 2022-01-06: omhttp bugfix: memory leak in lokirest batchmode + A JSON object was created (valueObj) but not used and also not released causing a + memory leak. Over time, this could lead to memory overcomittent. + closes: https://github.com/rsyslog/rsyslog/issues/4766 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2112.0 (aka 2021.12) 2021-12-16 +- 2021-12-14: refactor:Deallocate outchannel resources in rsconf destructor + Thanks to Attila Lakatos for the patch. +- 2021-12-14: refactor: use runConf instead of loadConf in ratelimiting during runtime + Thanks to Attila Lakatos for the patch. +- 2021-11-22: new contribtion: URL parser module function using libfa + Thanks to Théo Bertin for the patch. +- 2021-11-18: mmanon: relax IPv6 detection - improve anonymization + We so far tried to ensure a value is really an IPv6 address, in order + to avoid to mangle with just similar-looking information elements. + However, this lead to misdetection for unusual formats, e.g. when a + port is appended to a numerical IPv6 adress given without braces []. + This has been changed now. In a sense, we now prefer to err on the + side of privacy. + BEHAVIOR CHANGE: + Previously, a suspect value was not anonymized, and thus some other + elements (like some MAC addresses) preserved. Now the opposite is + true, and we anonymize anything that looks close enough to be an + IPv6 address. This improves anonymization. + closes https://github.com/rsyslog/rsyslog/issues/4725 +- 2021-11-10: ruleset bugfix: ruleset queue was incorrectly named + The ruleset was incorrectly and unusably named. This was a regeression + from 4a63f8e9629c3c9481a8b6f9d7787e3b3304320b. + Many thanks to github user digirati82 for alerting us. + closes https://github.com/rsyslog/rsyslog/issues/4730 +- 2021-11-10: omsnmp: update module to current IP best practices + The omsnmp module uses the inet_addr() function to convert the Internet host address + from IPv4 numbers-and-dots notation into binary data in network byte order. If the input + is invalid, INADDR_NONE (usually -1) is returned. Use of this function is problematic + because -1 is a valid address (255.255.255.255). We should avoid its use in favor of + inet_aton(), inet_pton(3), or getaddrinfo(3), which provide a cleaner way to indicate + error return [1]. + This is just a request to satisfy covscan, so no error is reported at all. + Thanks to Attila Lakatos for the patch. +- 2021-10-27: ommysql: fix threading bug + When the MariaDB connection was (re)established, old or NULL handle + could be used. This is fixed now. + We need to synchronize access to the mysql handle, because multiple threads + use it and we may need to (re)init it during processing. This could lead to + races with potentially wrong addresses or NULL accesses. If this really + matters mostly depends on the MariaDB/MySQL client library. It looks like + they guard against fatal failuers. Anyhow, logging errors inside rsyslog + could happen in any case. +- 2021-10-25: testbench: false positive when impstats was not built + Test omfwd_fast_imuxsock failed when impstats was not built. This + has been corrected, test is now only executed when impstats is + present. +- 2021-10-25: imtcp: add support for permittedPeers setting at input() level + The permittedPeers settig was actually forgotten during the refactoring + of TLS input() level settings. This functionality is now added. + closes: https://github.com/rsyslog/rsyslog/issues/4706 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2110.0 (aka 2021.10) 2021-10-19 +- 2021-10-13: config bugfix: global(security.abortonidresolutionfail=) did not work + when used with rscript based configuration, it was not checked. +- 2021-10-13: config bugfix: global param $privDropToUser did not work correctly + The parameter was not implemented for rscript based configuration and + did not properly apply to legacy configuration. In essence, it almost always + did not work as expected. + see also: https://github.com/rsyslog/rsyslog/issues/4642 + see also: https://github.com/rsyslog/rsyslog/commit/cbcaf2c7e5b67e5465e47bc7cc67af2eae47bd31 +- 2021-10-12: rscript bugfix: ruleset called async when ruleset had queue.type="direct" + The call rscript statement is able to call a rule set either synchronously or + asynchronously. We did this, because practice showed that both modes + are needed. For various reasons we decided to make async + calls if the ruleset has a queue assigned and sync if not. + To know if a "queue is assigned" we just checked if queue parameters were + given. It was overlookeded the case of someone explicitly specifying a + "direct queue", aka "no queue". As such, queue="direct" triggered async + calls. That in turn meant that when a write operation to a variable was + made inside that rule set, other rulesets could or could not see the + write. While if was often not seen, this was a data race where the + change could also be seen by the outside. + This is now fixed. No matter if queue.type="direct" is specified or + left out, the call will always by synchronous. Any values written to + variables will also be seen by the "outside world" in later processing + stages. + Note that this has some potential to BREAK EXISTING CONFIGURATIONS. + We deem this acceptable because: + 1. this was racy at all, so unexpected behaviour could alwas occur + 2. it is actually unlikely that someone used the triggering conditions + in practice. But we can not outrule this, especially when the + configuration was auto-generated. + Potential compatibility issues can be solved by defining a small + array-memory queue on the ruleset in question instead of specifying + direct type. + Again, we expect that almost all users will never experience any + problems. If you do, however, please let us know: we may add an + option to re-enable the bug. +- 2021-10-12: ksi bugfix: locking bug fixed in rsksiCtxOpenFile + Thanks to Taavi Valjaots for the patch. +- 2021-10-11: core bugfix: fix typo in error message + Thanks to github user jkschulz for the patch. +- 2021-10-11: tcpsrv bugfix: compilation without exceptions + tcpsrv.c:992:1: error: label at end of compound statement + finalize_it: + ^~~~~~~~~~~ + Quoting from pthread.h: + pthread_cleanup_push and pthread_cleanup_pop are macros and must always + be used in matching pairs at the same nesting level of braces. + Amends commit bcdd220142ec9eb106550195ba331fd114adb0bd. + Thanks to Orgad Shaneh for the patch. +- 2021-10-11: mkubernetes bugfix: no connection retry to kubernetes APP + When connection to the kubernates API was not possible, mmkubernetes + did not retry. This does now happen via regular rsyslog retry + mechanism. + Thanks to github user jayme-github for the analysis and patch. + closes https://github.com/rsyslog/rsyslog/issues/4669 +- 2021-10-11: openssl bugfix: Correct gnutlsPriorityString (custom ciphers) behaviour + - Only apply default anon ciphers if gnutlsPriorityString is NULL and + Authentication Mode is set to anon. Otherwise we do not set them + as they overwrite custom Ciphers. + - Added two tests for custom cipher configuration (anon/certvalid mode). + - Add call for applyGnutlsPriorityString if gnutlsPriorityString changes. + - Merged openssl init code from Connect into osslInitSession + closes: https://github.com/rsyslog/rsyslog/issues/4686 +- 2021-10-11: build issue: handle undefined MAXPATHLEN, PATH_MAX + While we handled missing PATH_MAX, we did not handle missing MAXPATHLEN. + This happens under GNU/Hurd, because there is no official limit. However, + extremely long pathes are extremely uncommon, so we do not want to + use slow dynamic alloc each time we need to build pathes. So we + impose a limit of 4KiB, which should be fairly enough. Note that + this obviously increases stack requirements in GNU/Hurd. + As suggested by Michael Biebl, we have now implemented a generic + approach to handle this via autoconf. +- 2021-09-12: openssl: extended output information on connection failure + Now includes the remote client/server IP address in the log output. +- 2021-09-12: imhttp enhancements - query parameter ingestion & basic auth support + - Basic Authentication support & tests + * configured via imhttp option "basicAuthFile". This option should be configured + to point to your htpasswd file generated via a standard htpasswd tool. + tests: + * imhttp-post-payload-basic-auth.sh + * imhttp-post-payload-basic-auth-vg.sh + - Query parameter ingestion capability & tests + use t `addmetadata` option to inject query parameters into + metadata for imhttp input. + DISTRO PACKAGERS BEWARE: NEW DEPENDENCY FOR IMHTTP: + libaprutil (libaprutil1-dev on debian'ish, apr-util-devel on Red Hat) + Thanks to Nelson Yen for the patch. +- 2021-09-07: testbench bugfix: privdrop tests under root user did not work + When running under root, the privdrop tests did not properly work. This + patch fixes the issue and skips test where necessary. + This also includes some modernization of the related tests. + closes https://github.com/rsyslog/rsyslog/issues/4619 +- 2021-09-07: core/ratelimiting: fix rate limiting for already parsed messages + Rate limiting may not have worked if the considered message had already + been parsed (not having NEEDS_PARSING in msgFlags). + This affects also imuxsock in its default configuration + (useSpecialParser="true" and ratelimit.severity="1") +- 2021-09-07: core bugfix: use of property $wday terminates string + When $wday is used inside a template, all template parts after it + are ignored. For exmaple: + template(name="json_filename" type="string" string="/var/log/%$wday%.log") + would generate something like "/var/log/0" - the ".log" part would be + missing. For the same reason, $wday can not reliably checked in script + filters. + Thanks to Alain Thivillon for reporting the bug and providing an + excellent analysis, which essentiellay was exactly this fix here. + closes https://github.com/rsyslog/rsyslog/issues/4670 +- 2021-09-07: core/queue bugfix: potential misadressing when queue discarded messages + When a discard mark was set, the queue was very busy and discarded messages, a + NULL pointer access could happen. Depending on circumstances, several problems + could occur, including a SEGFAULT. This is now fixed. + closes: https://github.com/rsyslog/rsyslog/issues/4437 +- 2021-09-07: imdiga bugfix: iOverallQueueSize calculation could be incorrect + This issue only affects testbench and rsyslog development debugging. The active + messages counter, used for synchronizing test steps, went wrong when the queue + discarded messages on it's consumer thread. Now fixed. +- 2021-09-06: gnutls driver: SAN priority did not work correctly on server side + PrioritizeSAN was not propagated when accepting a new connection, this is now fixed. + Thanks to Attila Lakatos for the patch. +- 2021-08-24: config: implement script-equavalent for $PrivDrop* statements + closes https://github.com/rsyslog/rsyslog/issues/891 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2108.0 (aka 2021.08) 2021-08-17 +- 2021-08-16: openssl tls: Improved error message output on tls failures. + closes: https://github.com/rsyslog/rsyslog/issues/4645 +- 2021-08-16: impstats: add percentile metrics tracking functionality + Brief overview: + TO configure tracking percentile metrics in rainerscript: + User would need to define: + - which percentile to track, such as [p50, p99, etc.] + - window size - note, this correlates directly with memory usage to + track the percentiles. + To track a value, user would call built-in function `percentile_observe()` in their configurations to + record an integer value, and percentile metrics would be emitted every + impstats interval. + Thanks to Nelson Yen for the patch. +- 2021-08-12: imfile: add parameter "ignoreolderthanoption" + instructs imfile not to ingest a file that has not been modified in the + specified number of seconds. + Thanks to github user yanjunli76 for the patch (submitted from Nelson Yen) +- 2021-08-10: imklog bugfix: invalid memory adressing, could cause abort + This is a regeression from commit 94c4a87. It introduced a free() call + using an object that was no longer valid (the main pointer to the + to-be-freed object) was already freed at time of use. This could + cause various issues, including a segfault. + Note: this bug was triggerred only during late phase of rsyslog + shutdown, so it did not affect regular operation. + Special thanks to github user wxiaoguang for analyzing the issue + and providing a draft fix proposal, on which this patch builds. + see also https://github.com/rsyslog/rsyslog/pull/4629 + closes https://github.com/rsyslog/rsyslog/issues/4625 +- 2021-08-09: imfile bugfix: deleteStateOnFileDelete missed some state files + When the log file is deleted, imfile would attempt to delete the statefile but it + was missing the file_id part of the statefile name. This means the statefiles were + only removed in the log file was less than 512 characters, because for very small + files the file ID hash is not created. This lead to some state files not being + deleted. + Thanks to pearseimperva for the patch. +- 2021-08-09: imfile bugfix: hash char invalidly added in readmode != 0 + If imfile is ingesting log files with readMode set to 2 or 1, the resulting + messages all have a '#' character at the end. This patch corrects the behaviour. + Note: if some external script "supported" the bug of extra hash character at + the end of line, it may be necessary to update them. + closes https://github.com/rsyslog/rsyslog/issues/4491 +- 2021-08-09: omelasticsearch bugfix: errorFile mutex was not consistently locked + Lock the file during SIGHUPs to avoid issues with concurrent accesses by + writeDataError(). + Thanks to François Poirotte for the patch. +- 2021-08-09: imudp: add socket type (IPv4 vs. 6) to input name + Most importantly, the input name is used for stats counter names as + well. Previously, the same name was used for IPv4 and IPv6, so we had + two counters with an equal name. That left users puzzled. + Unfortunately, this change can potentially require changes to existing + analysis scripts, as the name is now slightly different. + closes https://github.com/rsyslog/rsyslog/issues/4364 +- 2021-08-06: omfwd: add capability for action-specific TLS certificate settings + This permits to override the global definitions for TLS certificates + at the action() level. +- 2021-08-06: imfile bugfix: file handle leak if "freshStartTail" was turned on +- 2021-08-05: imtcp: permit to use different certificate files per input/action + This completes the ability to override global/default TLS settings at the imtcp + input() level. Support for using multiple CAs/Certs per Connection is now provided. +- 2021-08-04: imptcp bugfix: keep alive interval was incorrectly set + The interval was accidentally set to keep alive interval. This has been + corrected. + closes https://github.com/rsyslog/rsyslog/issues/4609 +- 2021-07-08: openssl network driver bugfix: small memory leak + Fixes a static, non-growing memory leak which existed when parameter + "GnutTLSPriorityString" was used. This was primarily a cosmetic issue, + but caused some grief during development in regard to memory leak + detectors. + Note: yes, this is for openssl -- the parameter name is historical. +- 2021-07-07: psrv bugfix: abort if no listener could be started + Modules (like imtcp and imdiag) which use tcpsrv could abort or + otherwise malfunction if no listener for a specific input could + be started. + Found during implementing a new feature, no report from practice. + But could very well happen. +- 2021-07-07: mmkubernetes bugfix: apiserver error handling + - Added graceful handling of apiserver errors with unexpected responses, + i.e., anything other than 200, 404, or 429. Idea is that apiserver + transient error state will recover. We don't want mmkubernetes to miss + metadata resolution for containers that don't have cached metadata. + During these transient error states, mmkubernetes will provide basic + container file path based resolution of namespace and pod metadata for + new pods whose metadata is not yet cached. After this error state + recovers, mmkubernetes is expected to resume its metadata resolution as + expected. + - Added a unit test case for apiserver return 500 with changes to mock server + - Fixed existing unit test that was failing due to missing expected results file + - Added mmkubernetes unit tests to testbench + Thanks to Abdul Waheed for the patch (submitted from Nelson Yen). +- 2021-07-07: ommongodb bugfixes + - Fix Segmentation fault when server is down + - Add server connexion check while resuming + Thanks to Kevin Guillemot for the patch. +- 2021-06-28: omkafka improvements + - drain librdkafka queues and retry later during rsyslog restart or hup. This + re-injects messages into rsyslog's native queues. + - add statsname on per kafka instance for better visibility + - omkafka - count errors related ssl as "errors_ssl" + Thanks to Nelson Yen for the patch. +- 2021-06-23: some CI/QA improvements, Travis-CI disabled + For the time being, Travis CI is disabled because it was outdated and Travis also + changed their system. We will re-evaluate if we re-enable it. Since quite a while + the Travits tests were redundant with the rest of CI, so this does not reduce + coverage. +- 2021-06-23: omhttp bugfix: dynrestpath param in batch mode invalid + When batchmode was used, the templates could not be used to + expand dynrestpath. We are now storing the restpath param + within the batch data if we are in batch mode. + When we are in batch mode, and the restpath value changes, the + batch is submitted and reinitialized + closes: https://github.com/rsyslog/rsyslog/issues/4567 +- 2021-06-17: add predefined template RSYSLOG_SyslogRFC5424Format + This is essentially the same as RSYSLOG_SyslogProtocol23Format with + a better name and a fix to remove the unnecessary LF at the end of + the message. + The different name also enables us to fix the LF issue without + any concern about backwards compatibility. + closes https://github.com/rsyslog/rsyslog/issues/4384 +- 2021-06-17: impstats/bugfix: _sender_stats reports integer counter as string + Note that this introduces a small backwards incompatibility: in previous output + the field was of string type, now it is integer (as intended). We discussed this + on the mailing list and the overwhelming thought was that this is not a problem + because almost all analysis backends are able to cover that format change. This made + the bugfix essentially costmetic. + HOWEVER, if you still experience issues, please let us know. We can add an option + to provide the previous format, and just spared to do so because there was no + evidence it was needed. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2106.0 (aka 2021.06) 2021-06-15 +NOTE: the prime new feature is support for TLS and non-TLS connections +via imtcp in parallel. Furthermore, most TLS parameters can now be overriden +at the input() level. The notable exceptions are certificate files, something +that is due to be implemented as next step. +- 2021-06-14: new global option "parser.supportCompressionExtension" + This permits to turn off rsyslog's single-message compression extension + when it interferes with non-syslog message processing (the parser + subsystem expects syslog messages, not generic text) + closes https://github.com/rsyslog/rsyslog/issues/4598 +- 2021-05-12: imtcp: add more override config params to input() + It is now possible to override all module parameters at the input() level. Module + parameters serve as defaults. Existing configs need no modification. +- 2021-05-06: imtcp: add stream driver parameter to input() configuration + This permits to have different inputs use different stream drivers + and stream driver parameters. + closes https://github.com/rsyslog/rsyslog/issues/3727 +- 2021-04-29: imtcp: permit to run multiple inputs in parallel + Previously, a single server was used to run all imtcp inputs. This + had a couple of drawsbacks. First and foremost, we could not use + different stream drivers in the varios inputs. This patch now + provides a baseline to do that, but does still not implement the + capability (in this sense it is a staging patch). + Secondly, we now ensure that each input has at least one exclusive + thread for processing, untangling the performance of multiple + inputs from each other. + see also: https://github.com/rsyslog/rsyslog/issues/3727 +- 2021-04-27: tcpsrv bugfix: potential sluggishnes and hang on shutdown + tcpsrv is used by multiple other modules (imtcp, imdiag, imgssapi, and, + in theory, also others - even ones we do not know about). However, the + internal synchornization did not properly take multiple tcpsrv users + in consideration. + As such, a single user could hang under some circumstances. This was + caused by improperly awaking all users from a pthread condition wait. + That in turn could lead to some sluggish behaviour and, in rare cases, + a hang at shutdown. + Note: it was highly unlikely to experience real problems with the + officially provided modules. +- 2021-04-22: refactoring of syslog/tcp driver parameter passing + This has now been generalized to a parameter block, which makes it much cleaner and + also easier to add new parameters in the future. +- 2021-04-22: config script: add re_match_i() and re_extract_i() functions + This provides case-insensitive regex functionality. + closes https://github.com/rsyslog/rsyslog/issues/4429 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2104.0 (aka 2021.04) 2021-04-20 +- 2021-04-19: new contributed module imhiredis + Thanks to Théo Bertin (frikilax) for the patch. +- 2021-04-19: new built-in function get_property() to access property vars + Provides ability to evaluate a rsyslog variable using dynamically + evaluated parameters. + 1st param is the rsyslog param, 2nd param is a key, can be an array + index or key string. + Useful for accessing json sub-objects, where a key + needs to be evaluated at runtime. Can be used to access arrays as well. + Thanks to Nelson Yen for contributing this module. +- 2021-04-19: mmdblookup: add support for mmdb DB reload on HUP + Thanks to Théo Bertin (frikilax) for the patch. +- 2021-04-19: script bugfix: empty array in foreach() improperly handled + When running a foreach() loop inside a ruleset, if the json array/object iterated + over is empty but valid, the foreach will make the message processing in the + ruleset abort operation, no following operation (such as actions) will be + executed after this. + Thanks to Théo Bertin (frikilax) for the patch. +- 2021-04-19: imjournal bugfixes (handle leak, empty file) + Flush the FILE* buffer before rename & fsync in order + to not end up syncing an empty file. + Also, close WorkDir on fsync in order to prevent + file descriptor leakage. + Thanks to github user gerd-rausch for the fix. +- 2021-04-06: new contributed function module fmunflatten + This commit adds a new rainerscript function to unflatten keys in a JSON tree. It + provides a way to expand dot separated fields. + <result> = unflatten(<source-tree>, <key-separator-character>); + It allows for instance to produce this: { "source": { "ip": "1.2.3.4", "port": 443 } } + from this source data: { "source.ip": "1.2.3.4", "source.port": 443 } + Thanks to Julien Thomas for the contribution. +- 2021-02-22: test bugfix: some tests did not work with newer TLS library versions + Newer versions provide TLS versions that cannot be disabled in older versions as they + are unknown there. This is solved by setting restrictions in multiple steps. For + older library versions, the final step will error out, but the other one be applied. + This permits to achieve proper test results. + closes: https://github.com/rsyslog/rsyslog/issues/4534 +- some improvements to project CI +---------------------------------------------------------------------------------------- +Scheduled Release 8.2102.0 (aka 2021.02) 2021-02-16 +- 2021-02-15: omfwd: add stats counter for sent bytes + Thanks to John Chivian for suggesting this feature. +- 2021-02-15: omfwd: add error reporting configuration option + RSyslog on a plain TCP cannot guarantee the message delivery + without using RELP protocol. Besides that the logs may be + flooded with connection errors making the rest of messages + difficult to find. To alleviate the problem (see issue 3910), + this patch adds a configuration option that enables to reduce + the number of network errors logged and reported. + For example, if each 10th network error message should be logged, + the rsyslog configuration has to be updated as follows. + action(type="omfwd" Target="<IP_ADDR>" Port="<PORT>" Protocol="tcp" ConErrSkip="10") + Thanks to Libor Bukata for the patch. +- 2021-02-15: action stats counter bugfix: failure count was not properly incremented + In some cases the counter was not incremented, most notably with transaction-enabled + actions. + Thanks to github user thinkst-marco for the patch. +- 2021-02-15: action stats counter bugfix: resume count was not incremented + And so it always stayed at zero. + Thanks to github user thinkst-marco for the patch. +- 2021-02-15: omfwd bugfix: segfault or error if port not given + If omfwd is configured via RainerScript config format and the "port" + parameter is not given, a segfault will most likely happen on + connection establishment for TCP connections. For UDP, this is + usually not the case. + Alternatively, in any case, errors may happen. + Note that the segfault will usually happen right on restart so this + was easy to detect. + We did not receive reports from practice. Instead, we found the bug + while conducting other work. +- 2021-01-29: lookup table bugfix: data race on lookup table reload + A data race could happen when a lookup table was reloaded. We found + this while moving to newer version of TSAN, but have no matching + report from practice. However, there is a potential for this to cause + a segfault under "bad circumstances". +- 2021-01-18: testbench modernization + Bump dependency versions, use newer distro versions for some tests. + Make kafka distcheck separate to help diagnose flaky kafka tests. +- 2021-01-16: testbench: fix invalid sequence of kafka tests runs + kafka tests can not run well in parallel (mostly due to ressource + constraints on CI machines). Accidentally, this was not enforced for + one of the tests. That could lead to random failures and false positives. +- 2021-01-14: testbench: fix kafkacat issues + The kafkacat tool has an upper limit of how many messages it can send + at once. Going over that limit causes messages loss. The exact limit + seems to depend on the environment. This causes testbench false positives. + This commit fixes two related issues: + - errors during kafkacat run were not detected - this has been added + - we now have a "max messages at once" setting, after which kafkacat + is restarted for the next batch of messages. It currently is set + to 25,000 msgs per incarnation. All tests loop now to send the + required number of messages. This has been fixed at the testbench + framework level, so no need to adjust individual tests. +- 2021-01-14: testbench: fix year-dependendt clickhouse test + A test had the year value hardcoded and as such failed whenever the + year changed. This patch corrects that. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2012.0 (aka 2020.12) 2020-12-08 +- 2020-12-07: testbench bugfix: some tests did not work in make distcheck + - certificate file missing in dist tarball + - some test cases did not properly specify path to cert file + Thanks to Michael Biebl for alerting us and providing part of + the fix. + closes https://github.com/rsyslog/rsyslog/issues/4446 +- 2020-12-07: immark: rewrite with many improvements + - mark message text can now be specified + - support for rulesets + - support for using syslog API vs. regular internal interface + - support for output template system + - ability to specify is mark message flag can be set + - minor changes and improvements +- 2020-11-30: usability: re-phrase error message to help users better understand cause + see also https://github.com/rsyslog/rsyslog/issues/3910 +- 2020-11-10: add new system property $now-unixtimestamp + Among others, this may be used as a monotonic counter + for doing load-balancing and other things. + Thanks to Nicholas Brown for suggesting this feature. +- 2020-11-04: omfwd: add new rate limit option + Adding new rate limit option to omfwd for rate limiting + syslog messages sent to the remote server + ratelimit.interval: + Specifies the rate-limiting interval in seconds. + Default value is 0, which turns off rate limiting. + ratelimit.burst + Specifies the rate-limiting burst in number of messages. + closes https://github.com/rsyslog/rsyslog/issues/4423 + Thanks to Dinesh-Ramakrishnan for the patch. +- 2020-11-03: omfwd bug: param "StreamDriver.PermitExpiredCerts" is not "off" by default + The default behaviour of expired certificates of stream driver in TLS mode, should + have been that the see tcp transmission is closed due to expired certificates, and + error messages emited in rsyslog status. This was not the case. That in turn could + lead to permitting sessions which should not be permitted. + Thanks to Vincent Zhu for alerting us and providing a great problem analysis + closes: https://github.com/rsyslog/rsyslog/issues/4425 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2010.0 (aka 2020.10) 2020-10-20 +- 2020-10-13: gnutls TLS subsystem bugfix: handshake error handling + If the tls handshake does not immediatelly finish, gnutls_handShake is called in + doRetry handler again. However the error handling was not + complete in the doRetry handler. A failed gnutls_handShake call + did not abort the connection and properly caused unexpected + problems like in issues: + https://github.com/rsyslog/rsyslog/issues/4270 + https://github.com/rsyslog/rsyslog/issues/4288 +- 2020-10-13: core/msg bugfix: memory leak + There is a missing call to json_object_put(json) if the call to + jsonPathFindParent() failed. It's leaking memory. Depending on workload and config, + this leak can potentially grow large (albeit we did not see reports from practice). + Thanks to Julien Thomas for the patch. +- 2020-10-13: core/msg bugfix: segfault in jsonPathFindNext() when <root> not an object + The segfault gets happens when <bCreate> is 1 and when the <root> + container where to insert the <namebuf> key is not an object. + Here is simple reproducible test case: + // ensure we start fresh + // unnecessary if there was no previous set + unset $!; + set $! = ""; + set $!event!created = 123; + Thanks to Julien Thomas for the patch. +- 2020-10-13: openssl TLS subsystem: improvments of error and status messages + Adding error logs at the ssl handshake failure scenarios. + Adding the header "nsd_ossl:" tag to these logs to identify + the origin module from which logs are generated. + Thanks to Anusha Pai G for the patch. +- 2020-10-06: add 'exists()' script function to check if variable exists + This implements a way to check if rsyslog variables (e.g. '$!path!var') is + currently set of not. + Sample: if exists($!somevar) then ... + closes https://github.com/rsyslog/rsyslog/issues/4385 +- 2020-10-03: core bugfix: do not create empty JSON objects on non-existent key access + Performing a condition (eg: check for an empty string) on a subtree key that do not + exists (depth > 1 from the root container), creates an empty "parent" object. + Depending on your context, you may end up with (kind of...) annoying garbage when + producing object documents (for instance to index in ES). + Also fixes a hypothetical hang condition with an almost (?) unused plugin parameter + passing mode, for details see + https://github.com/rsyslog/rsyslog/issues/4436 + closes https://github.com/rsyslog/rsyslog/issues/4430 + Thanks to Julien Thomas for the patch. +- 2020-09-28: gnutls subsysem bugfix: potential hang on session closure + Some TLS servers don't reply to graceful shutdown requests "for + optimization". This results in rsyslog's omfwd+gtls client to wait + forever for a reply of the TLS server which never comes, due to shutting + down the connection with gnutls_bye(GNUTLS_SHUT_RDWR). + On systemd systems, commands such as "systemctl restart rsyslog" just + hang for 1m30 and rsyslogd gets killed upon timeout by systemd. + This is fixed by replacing the call to gnutls_bye(GNUTLS_SHUT_RDWR) by calls to + gnutls_bye(GNUTLS_SHUT_WR) which is sufficient and doesn't wait for a + server reply. + As an example, Kiwi Syslog server is known to cause this issue. + Thanks to Renaud Métrich for the patch. +- 2020-09-23: core/network bugfix: obey net.enableDNS=off when querying local hostname + Local hostname resolution used DNS queries even if the enableDNS was set to off, and + this could cause unexpected delays in the HUP signal handling if the DNS server was + not responsive. + Thanks to Samu Nuutamo for the fix. +- 2020-09-14: core bugfix: potential segfault on query of PROGRAMNAME property + A data race can happen on variable iLenProgram as it is not guarded + by the message mutex at time of query. This can lead to it being + non -1 while the buffer has not yet properly set up. + Thanks to Leo Fang for alerting us and a related + patch proposal. + replaces https://github.com/rsyslog/rsyslog/pull/4300 +- 2020-09-14: imtcp bugfix: broken connection not necessariy detected + Due to an invalid return code check, broken TCP sessions could not + necessarily be detected "right in time". This can result is the loss + of one message. + closes https://github.com/rsyslog/rsyslog/issues/4227 + Thanks to Leo Fang for the patch. +- 2020-09-14: new module: imhttp - http input + permits to receive log data via HTTP. + uses http library to provide http input. + user would need to configure an 'endpoint' as input, along + with a ruleset, defining how the input should be routed in + rsyslog. + Thanks to Nelson Yen for contributing this module. +- 2020-09-11: mmdarwin bugfix: potential zero uuid when reusing existing one + - fix a use-after-free variable during darwin uuid message extraction + - improve debug/output by logging uuid parse errors + Thanks to github user frikilax for the patch. +- 2020-09-10: imdocker bugfix: build issue on some platforms + An invalid variable type was used, leading to compile errors at least on + all platform that use gcc 10 and above. Otherwise, however, it looks like the + issue caused no real harm. +- 2020-09-07: omudpspoof bugfix: make compatbile with Solaris build + Thanks to Dagobert Michelsen for the patch. +- 2020-09-03: testbench fix: python 3 incompatibility +- 2020-09-02: core bugfix: segfault if disk-queue file cannot be created + When using Disk Queue and a queue.filename that can not be created + by rsyslog, the service does not switch to another queue type as + supposed to and crashes at a later step. + closes: https://github.com/rsyslog/rsyslog/issues/4282 +- 2020-08-26: cosmetic: fix dummy module name in debug output + When we have optional components (like imjournal) a dummy module + is used. It's sole purpose is to emit "this module is not available". + During init, the module emitted an invalid module name into the debug + log. This has now been replaced by the generic term "dummy". + Note: it is highly unlikely that someone will ever see that message + at all, as it is unlikely for the dummy modules to be build. + see also: https://github.com/rsyslog/rsyslog/commit/84a7e3d80b80106dcc86c273ed8cf78a6c11c722#r41782830 + Thanks to Thomas D. (whissi) for the patch. +- 2020-08-26: config bugfix: intended warning emitted as error + When there are actions configured after a STOP, a warning should be + emitted. In fact, an error message is generated. This prevents the + construct, which may have some legit uses in exotic settings. It + may also break older configs, but as the message is an error + for so long now, this should be no longer of concern. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2008.0 (aka 2020.08) 2020-08-25 +- 2020-08-25: imdocker bugfix: error reporting not always correct + A wrong function to obtain the error code was used. This + could lead to invalid error messages. + Thanks to Steve Grubb for the bug report and fix proposal. + closes https://github.com/rsyslog/rsyslog/issues/4381 +- 2020-08-25: imptcp: add max sessions config parameter + The max is per-instance, not global across all instances. + There is also a bugfix where if epoll failed I think we could leave a + session linked in the list of sessions, this code unlinks it. + Thank to Alfred Perlstein for the patch. +- 2020-08-24: omelasticsearch bugfix: reply buffer reset after health check + The issue happens when more than one server is defined on the + action. On that condition a health check is made through + checkConn() before sending the POST. The replyLen should be + set back to 0 after the health check, otherwise the response + data received from the POST gets appended to the end of the + last health check. + Thanks to Julien Thomas for the patch. +- 2020-08-14: omfile: do no longer limit dynafile cache size in legacy format + When using obsolete legacy config format, omfile had a hard limit of + 1,000 dynafile cache entries. This does not play well with very + large installation. This limit is now removed and converted into + a warning if cache size > 25,000 is specified. + Note: the problem can easily be worked-around by using modern + config format (RainerScript). + closes: https://github.com/rsyslog/rsyslog/issues/4241 +- 2020-08-13: imudp: fix very small, static memory leak + When ruleset support was used, the ruleset name was not freed upon rsyslog + termination. While this has no consequences for regular runs, it generates + leak errors under memory debuggers and as such makes debugging harder than + necessary. + Thanks to github user frikilax for the patch. +- 2020-08-13: omelasticsearch: add parameter skipPipelineIfEmpty + When POST'ing a document, Elasticsearch does not allow an empty pipeline + parameter value. This patch introduces boolean option skipPipelineIfEmpty + to the omelasticsearch action. When set to true, the pipeline parameter + won't be posted. Default is false so we do not modify current behavior. + Thanks to Julien Thomas for the patch. +- 2020-08-12: systemd service file removed from project + This was done as distros nowadays have very different service files and it no + longer is useful to provide a "generic" (sic) example. + see also: https://github.com/rsyslog/rsyslog/issues/4333 +- 2020-08-11: gnutls TLS driver bugfix: EKU check not done properly + When the server accepted a new connection, it did not properly set the + dataTypeCheck field based on the listening socket. That resulted in + skipping ExtendedKeyUsage (EKU) check on the client. + Thanks to Daiki Ueno for the patch. +- 2020-08-06: MMDARWIN:: improve configuration flexibility and UUID fix + -t pu now able to get fields from local variables ($.) + - now able to configure a custom root container for mmdarwin fields + - now able to put nested keys ($!key1!key2) + - don't regenerate a UUID each time, but instead check if one exists before + creating it (allow successive calls without losing previous UUID) + Thanks to github user frikilax for the contribution. +- 2020-08-06: add --enable-imjournal=optional ./configure option +- 2020-08-06: IMPCAP::Fixes: segfault, memory and build corrections + * fix bug in ethernet packets parsing + * fix removes build error with gcc10: 'multiple definition of...' + * resolve memory leak during interface init failure (device not freed after post-create error) + * add test 'impcap_bug_ether' to prove ethernet parser fix is working + Thanks to github user frikilax for the contribution. + closes https://github.com/rsyslog/rsyslog/issues/4332 +- 2020-07-14: CI: add support for github actions +- 2020-07-14: imklog: add ruleset support + see also: https://github.com/rsyslog/rsyslog/issues/4344#issuecomment-658001854 + see also: https://github.com/rsyslog/rsyslog/issues/106 +- 2020-07-06: config system fix: ChkDisabled method to make config.enabled work + There was wrong negation in the method so it returned 0/1 in reverse + and also it did not mark the node to not be reported as unknown at all + times which is needed after all. + Thanks to Jiri Vymazal for the patch. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2006.0 (aka 2020.06) 2020-06-23 +- 2020-06-22: queue: permit ability to double size at shutdown + This prevents message loss due to "queue full" when re-enqueueing data + under quite exotic settings. + see also https://github.com/rsyslog/rsyslog/issues/3941#issuecomment-549765813 + closes https://github.com/rsyslog/rsyslog/issues/4020 +- 2020-06-22:Fixing imfile segfaulting on selinux denial + If imfile is denied access to file watched trough symlink there is + unchecked condition resulting in access to not initialized memory. +- 2020-06-22: openssl: Fixed memory leak when tls handshake failed. + closes: https://github.com/rsyslog/rsyslog/issues/4319 +- 2020-06-22: change systemd service file to wait for network + now that rsyslog is usually only installed for real syslog servers, + we should assume that some network listening or forwarding happens + on start. As such we need to start a bit later, after the network. + This poses no problem as systemd nowadays comes with journal which + is in almost all cases configured to buffer log data while + rsyslog is not yet running. + see also https://github.com/rsyslog/rsyslog-pkg-rhel-centos/issues/72 +- 2020-06-22: NEW INPUT MODULE:: impcap, network packets input parser + Thanks to github user frikilax for the contribution. +- 2020-06-22: ksi bugfix: Optimized code in KSI module initialization fixed. + KSI module initialization will not stuck in infinite loop when code is + built with optimization -O2. +- 2020-06-05: operatingstatefile bugfix: month was given too low + The month was printed with the range 0 (January) to 11 (December). + This has now been corrected. + closes https://github.com/rsyslog/rsyslog/issues/4292 +- 2020-06-05: build system: add "optional" build functionality to some components + Nameley: + --enable-libdbi=optional + --enable-mmdblookup=optional + --enable-imkafka=optional + --enable-omkafka=optional + If used, builds a dummy module which just emits a "module not supported + on this platform" error message when loaded. + Primary use case for this system is Debian-ish builds on SUSE OBS, + where we prefer to have a single package definition for all versions + (else things get much more complicated). +- 2020-05-23: config system bugfix: backticks cat segfault if file cannot be opened + when a `cat <filename>` construct is used in rsyslog.conf and <filename> can not + be accessed (does not exist, no permissions, ...), rsyslog segfaults. + Thanks to Michael Skeffington for notifying us and providing root cause analysis. + closes https://github.com/rsyslog/rsyslog/issues/4290 +- 2020-05-15: imtcp bugfix: octet framing/stuffing problem with discardTruncatedMsg on + When "discardTruncatedMsg" was enabled in imtcp, messages were incorrectly + skipped if the last character before the truncation was the LFdelimiter. + Also adds two testbench tests for this case. + closes: https://github.com/rsyslog/rsyslog/issues/4281 +- 2020-05-12: ompipe bugfix: race during HUP + When HUP was received, the write mutex was not acquired. This could + lead to unexpected invalidation of the output file descriptor. + Thanks to Julien Thomas for alerting us on this issue. + see also https://github.com/rsyslog/rsyslog/pull/4136#issuecomment-578326278 +- 2020-05-12: ompipe: add action parameter tryResumeReopen + Sometimes we need to reopen a pipe after an ompipe action gets + suspended. Sending an HUP signal to rsyslog does the job but requires + an interraction with rsyslog. The patch adds support for a new boolean + option, tryResumeReopen, for the ompipe action. It mimics what an HUP + signal would do. + Thanks to Julien Thomas for the patch. +- 2020-05-12: imjournal: remove strcat call + Thanks to Jeff Marckel for the patch. +- 2020-05-12: build system: libzcmq version requirement needs to be bumped + Thanks to Thomas Deutschmann for pointing this out. + closes https://github.com/rsyslog/rsyslog/issues/3957 +- 2020-05-12: testbench: download ElasticSearch binaries from rsyslog.com + The official ElasticSearch download site sometimes denies the download. +- 2020-05-11: openssl netstream driver bugfix: context leak + The context object was not properly freed. + Thanks to Michael Zimmermann for the fix. +- 2020-05-11: omhttp: Add support for multiple http headers + Allows the inclusion of multiple http headers on the REST call. + Thanks to callmegar for the patch. +- 2020-04-29: core bugfix: group id could not be obtained for very large groups + Thanks to github user emilbart for the patch. +- 2020-04-29: testbench additions (relp broken connection test) +- 2020-04-29: omudpspoof bugfix: issues with oversized messages + First issue was an incorrect packet length in UDP Header. It has to be the FULL UDP Packet + regardless of the MTU Setting. As a result regardless of IP fragmentation, the MTU setting + also limited the siizmax size of the UDP message. + The second issue was incorrect calculation of the UDP Checksum with libnet if + IP fragmentation was used (Based on MTU Setting). As a result, the network packets were + dropped by the tcp stack before they even could reach there target. The workarround for this + problem is, that we set the UDP Checksum to 0x0000 which allows skipping of the checksum + test. Fixing the problem by calculating the correct UDP Checksum would require some + code changes in the libnet. + Also fixed the omudpspoof bigmsg test and increased the testing size to 16KB. +- 2020-04-29: omprog: fix assert failed on HUP with output flag + If the 'output' setting of omprog was used and rsyslog received a HUP + signal just after starting (and before the omprog action received the + first log to process), an internal assertion could fail, causing + rsyslog to terminate. The failure message was "rsyslogd: omprog.c:660: + closeOutputFile: Assertion `pCtx->bIsRunning' failed." + The failure could also occur if rsyslog received a HUP signal during + the shutdown sequence. + This bug was introduced in v8.2004 by PR https://github.com/rsyslog/rsyslog/pull/4255 + Although a test already existed that checked the interaction of HUPs + with the 'output' setting, it didn't always fail in this particular case + due to timing conditions. The test has been improved to cover this case + more reliably. + Thanks to Joan Sala Isern for the patch. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2004.0 (aka 2020.04) 2020-04-28 +- 2020-04-28: ksi bugfix: When KSI module is suddenly closed, files are finalized + In async. mode all pending signature requests are closed immediately and + unsigned block marker is attached with message about sudden closure. + Similar approach is used for blocks that already contain some records. + Empty blocks are just closed without any metadata. + Thanks to Taavi Väljaots for the patch. +- 2020-04-28: ksi bugfix: Signer thread initialization is verified before usage. + When signer thread is created in rsksiInitModule thread successful + initialization is verified before returning the function. This will + prevent adding records to not initialized module and in case of an + error signature files opened will contain only magic bytes. + Thread flags replaced with thread state. + When init module fails, module is disabled. + Thanks to Taavi Väljaots for the patch. +- 2020-04-28: ksi bugfix: Hardcoded default hash algorithm replaced with 'default' + Instead of hardcoded SHA-256 KSI_getHashAlgorithmByName("default") + is used to get default hash function. + Function rsksiSetHashFunction and SetCnfParam updated. + Thanks to Taavi Väljaots for the patch. +- 2020-04-28: imfile bugfix: poential segfault in stream object on file read + - if cstrLen(pThis->prevMsgSegment) > maxMsgSize then len calculation + become negative if cstrLen(thisLine) < cstrLen(pThis->prevMsgSegment) + This causes illegal access to memory location and thus causing segfault. + - assigning len = 0 if cstrLen(pThis->prevMsgSegment) > maxMsgSize so that + it access the correct memory location. + Thanks to github user jaankit +- 2020-04-28: openssl TLS drivers: made more reliable for older openssl versions + OpenSSL can retry some failed operations, but older versions need an explicit + opt-in to do so. This is now done. +- 2020-04-28: omprog: fix bad fd errors in daemon mode + When omprog was used with the 'forceSingleInstance=on' option, and/or + the 'output' setting, "bad file descriptor" errors occurred, which + prevented the external program to be executed and/or the program output + to be correctly captured. The bug could also manifest as "resource + temporarily unavailable" errors, or other errors related to the use of + invalid/reassigned file descriptors. These errors only happened when + rsyslog ran in daemon mode (i.e. they didn't happen if rsyslogd was + run with the '-n' option). + The cause of the bug was that omprog opened the pipe fds needed by + these flags during the configuration load phase (in the 'newActInst' + module entrypoint). This is a bad place since the fork of the daemon + occurs after this phase, and all fds are closed when the daemon process + is started (see 'initAll' in rsyslogd.c), hence invalidating the + previously opened fds. + To correct this, the single child process and the output capture thread + are now started later, when the first log message is received by the + first worker thread. (Note: the 'activateCnf' module entrypoint, despite + being invoked after the fork, cannot be used for this purpose, since it + is invoked per module, not per action instance.) + Currently no automated test exists for this use case since the testbench + always runs rsyslog in non-daemon mode. + Affected versions: v8.38 and later + closes: https://github.com/rsyslog/rsyslog/issues/4247 + Thanks to Joan Sala Isern for the patch. +- 2020-04-28: omfile bugfix: $outchannel split log lines at rotation time +- 2020-04-17: openssl: add support for libreSSL + Disable use of "@SECLEVEL" in default cipher string and + avoid SSL_CONF_CTX_set_flags() API when LIBRESSL is used. + This means tlscommands will not work. + closes: https://github.com/rsyslog/rsyslog/issues/4210 +- 2020-03-04: imudp bugfix: build problems on some Linux kernel versions + Thanks to Wen Yang for the patch. +- 2020-03-02: conf output bugfix: -o produces missing space between call and rulename + Thanks to Tetiana Ohnieva for the patch. + closes https://github.com/rsyslog/rsyslog/issues/3761 +---------------------------------------------------------------------------------------- +Scheduled Release 8.2002.0 (aka 2020.02) 2020-02-25 +- 2020-02-25: imfile: add per minute rate limiting + Add MaxBytesPerMinute and MaxLinesPerMinute options. + These take integer values and, respectively, limit the number + of bytes or lines that may be sent in a minute. + This can be used to put a limit on the count or volume of logs + that may be sent for an imfile. + Thanks to Greg Farrell for the patch. +- 2020-02-24: core: add global parameter "security.abortOnIDResolutionFail" + This parameter controls whether or not rsyslog aborts when a name ID + lookup fails (for user and group names). This is necessary as a security + measure, as otherwise the wrong permissions can be assigned or privileges + are not dropped. + CHANGE OF BEHAVIOR + The default for this parameter is "on". In previous versions, the default + was "off" (by virtue of this parameter not existing). As such, existing + configurations may now error out. + We have decided to accept this change of behavior because of the potential + security implications. + closes https://github.com/rsyslog/rsyslog/issues/4164 +- 2020-02-24: openssl TLS driver bugfix: chained certificates were not accepted + This was supported since always inside GnuTLS driver, but was missing for openssl one. +- 2020-02-24: core bugfix: too early parsing of incoming messages + In theory, rsyslog should call parsers on the queue worker threads whenever + possible. This enables the parsers to be executed in parallel. There are + some cases where parsers needs to be called earlier, namely when parsed + data is needed for rate-limiting. + The logic to do this previously did not work correctly and was fixed six + years ago (!) by b51dd22. Unfortunately, b51dd22 was overly agressive: + it actually makes the early parser call now mandatory, effectively moving + parsing to the input side where there is no to little concurrency. + We still do not need to call the parser when all messages, regardless of + severity, need to be rate-limited. This is the default and very frequent + case. This patch introduces support for this and as such makes parsers + able to run in parallel in the frequent case again. + closes https://github.com/rsyslog/rsyslog/issues/4187 +- 2020-02-20: testbench bugfix: two minor issues in omkafkadynakey.sh test + lead to false positives during test runs (depending on circumstances) + closes: https://github.com/rsyslog/rsyslog/issues/4134 +- 2020-02-20: testbench: set max extra data length for tcpflood from 200 to 512KiB + Added a imrelp test for big messages (256KB). + closes: https://github.com/rsyslog/rsyslog/issues/4158 +- 2020-02-20: config system bugfix: 'config.enabled' directive oddities + Previously the directive was processed way too late which caused false + errors whenever it was set to 'off' and possibly other problems. + Thanks to Jiri Vymazal for the patch. +- 2020-02-09: imfile bugfix: timeout did not work on very busy system + The timeout feature was soley based on timeouts of the poll() + system call. On a very busy system, this would probably happen + very seldomly. Moreover, the timeout could occur later than + expected on any system with high load. + The issue was not reported from practice but discovered during + CI system improvements. +- 2020-01-30: build system: change --enable-imfile-tests default to "yes" + This was accidentally set to "no" some time ago (actual commit unknown). Tests for + imfile should by default run when imfile is enabled. + see also https://github.com/rsyslog/rsyslog/issues/4120 +- 2020-01-27: build system: add option --enable-gnutls-tests + This enables us to build GNUtls support but not necessarily + test it in CI. This is useful for some specialised subcomponent + test. The default is enabled if gnutls is enabled and disabled if not. +- 2020-01-26: testbench: new test for loadbalancing via global vars + This is a popular functionality which had not been routinely tested + in the past. +- 2020-01-26: mmdblookup bugfix: invalid data returned when no entry found + Since the upgrade of the package libmaxminddb on FreeBSD (1.3.2_2 -> 1.4.2), + the module mmdblookup returns the first entry of the mmdb database even if the entry + is not found. After some debug, I found the solution in the official maxminddb + repository : to check if the entry is in database, we must check the found_entry + attribute, otherwise the function MMDB_get_entry_data_list will return the first + entry of the database if the entry is not found in it. + Thanks to Kevin Guillemot for the patch. +- 2020-01-23: oversize message log bugfix: do not close fd -1 + The oversize message log fd is always closed on HUP, even if it never + was opened (and thus has -1 value). This patch corrects the issue. + The bug had no know-bad effect in practice other than getting an + (ignored) error status from close(). However, it introduced warnings + in test runs (e.g. when running under valgrind). +- 2020-01-22: imfile bugfix: saving of old file_id for statefiles + Previously we saved old file_id unconditionally, which led to not + deleting old statefiles if files changes without rsyslog running. + Now it should work correctly. + Thanks to Jiri Vymazal for the patch. +- 2020-01-22: imfile bugfix: misadressing and potential segfault + Commit 3f72e8c introduced an invalid memory allocation size. This lead to + too-short alloc and thus to overwrite of non-owned memory. That in turn + could lead to segfaults or other hard to find problems. + The issue was detected by our upgraded CI system. We did not receive + any problem reports in practice. Nevertheless, the problem is real and + people should update affected versions to patched ones. + The bug was present in scheduled stable release 8.1911.0 and 8.2001.0. + see also: https://github.com/rsyslog/rsyslog/issues/4120 + see also: https://github.com/rsyslog/rsyslog/pull/4141 +- 2020-01-20: core bugfix: potential race during HUP + when rsyslog is HUPed immediately after startup and before it is fully + initialized, there is a potential race with the list of loaded modules. + This patch ensures no bad things can happen in that case. + Detected by LLVM TSAN, not seen in practice. +- 2020-01-20: testbench improvements and fixes + modernize tests, reduce robustness against slow machines, provide some + test framework functional enhancements, and optimize some tests. + Also includes some code changes to C testing components. Among others, + tests have slightly been speeded up by reducing the wait time at queue + shutdown. This is possible because of better overall completion checks. +---------------------------------------------------------------------------------------- +Scheduled Release 8.2001.0 (aka 2020.01) 2020-01-14 +- 2020-01-12: core bugfix: race condition related to libfastjson when using DA queue + Rsyslogd aborts when writing to disk queue from multiple workers simultaneously. + It is assumed that libfastjson is not thread-safe. + Resolve libfastjson race condition when writing to disk queue. + see also https://github.com/rsyslog/rsyslog/issues/4041 + Thanks to MIZUTA Takeshi for the fix. +- 2020-01-12: omfwd bugfix: parameter streamdriver.permitexpiredcerts did not work + closes https://github.com/rsyslog/rsyslog/issues/4098 +- 2020-01-11: Bugfix: KSI module + dynafile in asynchronous mode fixed + Thanks to Taavi Valjaots for the patch +- 2020-01-08: tls driver: add support to configure certificate verify depth + Support added in omfwd as instance parameter: + streamdriver.TlsVerifyDepth + Support added in imtcp as module parameter: + streamdriver.TlsVerifyDepth + Can be 2 or higher. + Support added into ossl driver + Support added into gtls driver + Added testcases for both drivers. + closes: https://github.com/rsyslog/rsyslog/issues/4035 +- 2020-01-08: modernization of testbench + moved some tests to newer standards, hardened them against slow testbench machines, + kafka component download improvements, and prevent dangling left-over test tool + instances from aborted tests +- 2020-01-07: tls subsystem bugfix: default for permitExpiredCerts was invalidly "on" + The problem occurred with commit 3d9b8df in December 2018 and went into + scheduled stable 8.1901.0. Unfortunately, the change in default was not detected + until a year later. This commit re-enables the previous default ("off"), which is + also the only sensible default from a security PoV. Unfortunately, new 2019 + deployments may begin to see connection rejection when usin expired certs. As + expired certs should not be used, this hopefully will not cause problems in + practice. + Thanks to Jiri Vymazal for the patch. +- 2020-01-01: testbench: improve ElasticSearch test speed + We now support re-using suitable running ES instances, which reduces the + number of restarts. +- 2019-12-31: omelasticsearch: improve curl reply buffer handling + The curl reply buffer (pWrkrData->reply) was allocated, realloced and freed with + each request. This has now been reduced to once per module, slightly increasing + overall performance. + closes https://github.com/rsyslog/rsyslog/issues/1964 +- 2019-12-31: config system: emit proper error message on $ in double-quoted string + closes https://github.com/rsyslog/rsyslog/issues/2869 +- 2019-12-30: core bugfix: rsyslog aborts when config parse error is detected + In defaut settings, rsyslog tries to continue to run, but some data + structures are not properly initialized due to the config parsing error. + This causes a segfault. + In the following tracker, this is the root cause of the abort: + see also https://github.com/rsyslog/rsyslog/issues/2869 +- 2019-12-30: fix some alignment issues + So far, this worked everywhere (for years). But it may still have + caused issues on some platforms. + closes https://github.com/rsyslog/rsyslog/issues/2608 +- 2019-12-27: core bugfix: APP-NAME fields could become empty + RFC 5424 specifies that an empty APP-NAME needs to be indicated by + "-". Instead, the field could become empty under certain conditions. + If so, outgoing 5424 messages were invalidly formatted. + This happened under quite unusual conditions, but could be seen + in practice. + closes https://github.com/rsyslog/rsyslog/issues/4043 +- 2019-12-27: core bugfix: reopen /dev/urandom file descriptor after fork on Linux + This patch updates prepareBackground() in tools/rsyslogd.c to reopen any file + descriptors used for random number generation in the child process. This fixes + an issue on Linux systems where the file descriptor obtained for /dev/urandom + by seedRandomNumber() in runtime/srutils.c was left closed after the fork. This + could be observed in procfs, where /proc/fd/ would show no open descriptors to + /dev/urandom in the forked process. /dev/urandom is reopened as the child may be + be operating in a jail, and so should not continue to use file descriptors from + outside the jail (i.e. inherited from the parent process). + I found that this issue led to rsyslog intermittently hanging during seedIV() + in runtime/libgcry.c. After the fork, the closed file descriptor number tended + to get re-assigned. randomNumber() would then read from an incorrect (although + still valid) file descriptor, and could block (depending on the state of that + file descriptor). This gave rise to the intermittent hang that I observed. + Thanks to Simon Haggett for the patch. +- 2019-12-20: imdocker bugfix: did not compile without atomic operations +- 2019-12-20: omclickhouse: new parameter "timeout" + Thanks to Pavlo Bashynskiy for the patch. +- 2019-12-20: omhiredis: add 'set' mode plus some fixes + - new mode 'set' to send SET/SETEX commands + - new parameter 'expiration' to send SETEX instead of SET commands (only applicable to 'set' mode) + - fixes to missing frees + Thanks to github user frikilax for the patch. +- 2019-12-18: relp: Add support setting openssl configuration commands. + Add new configuration parameter tls.tlscfgcmd to omrelp and imrelp. + (Using relpSrvSetTlsConfigCmd and relpCltSetTlsConfigCmd) + OpenSSL Version 1.0.2 or higher is required for this feature. + A list of possible commands and their valid values can be found in the + documentation: https://www.openssl.org/docs/man1.0.2/man3/SSL_CONF_cmd.html + The setting can be single or multiline, each configuration command is + separated by linefeed (n). Command and value are separated by + equal sign (=). Here are a few samples: + tls.tlscfgcmd="Protocol=ALL,-SSLv2,-SSLv3,-TLSv1,-TLSv1.2" + tls.tlscfgcmd="Protocol=ALL,-SSLv2,-SSLv3,-TLSv1 + MinProtocol=TLSv1.2" + Add to new testcases for librelp and tlscfgcmd. + closes https://github.com/rsyslog/rsyslog/issues/3959 +- 2019-12-18: bugfix core: potential segfault in template engine + under some circumstances (not entirely clear right now), memory + was freed but later re-used as state-tracking structures were not + properly maintained. Github issue mentioned below has full details. + Thanks to github user snaix for analyzing this issue and providing + a patch. I am committing as myself as snaix did not disclose his or + her identity. + closes https://github.com/rsyslog/rsyslog/issues/3019 + closes https://github.com/rsyslog/rsyslog/issues/4040 +- 2019-12-18: fixed some minor issues detected by clang static analyzer 9 +- 2019-12-10: core/config bugfix: false error msg when config.enabled="on" is used + When the 'config.enabled="on"' config parameter an invalid error message + was emitted that this parameter is not supported. However, it was still + applied properly. This commit removes the invalid error message. + closes https://github.com/rsyslog/rsyslog/issues/4011 +- 2019-12-03: omsnmp bugfix: "traptype" parameter invalidly rejected value 6 + "Traptype" needs to support values 0 to 6. + However, if value 6(ENTERPRISESPECIFIC) was set, an invalid error message + was emitted. Otherwise processing was correct. + This could lead to problems with automatic config deployment, + as valid configurations were invalidly reported as incorrect. + That in turn could make a deployment fail. + closes https://github.com/rsyslog/rsyslog/issues/3973 +- 2019-12-03: omsnmp: add new parameter "snmpv1dynsource" + If set, the source field from SNMPv1 trap can be overwritten + with a template, default is "%fromhost-ip%". The content should be a + valid IPv4 Address that can be passed to inet_addr(). If the content + is not a valid IPv4 Address, the source will not be set. + closes: https://github.com/rsyslog/rsyslog/issues/3991 +- 2019-12-02: imfile bugfix: state file renaming sometimes did not work properly + Now checking if file-id changes and renaming - cleaning state file + accordingly and always checking and cleaning old inode-only style + state files. + Thanks to Jiri Vymazal for the patch. +- 2019-12-02: ratelimit: increase rate limit interval parameter max value + The burst parameter in the ratelimit was increased to an unsigned int + but the interval remained an unsigned short. While it may be unusual, + there is possibly a chance to need to represent an interval longer than + about 3/4 of a day. + While here, go through and normalize all the various incarnations of + rate limiting to be explicitly unsigned int for the burst and interval. + Thanks to github user frikilax for the patch. +- 2019-12-02: ommongodb: Add other supported formats for 'time' and 'date' fields + Thanks to github user frikilax for the patch. +- 2019-12-02: imjournal bugfix: too many messages in error case + Under certain error conditions, `ignorePreviousMessages="on"` could be ignored + an existing messages be processed. + Thanks to github user 3chas3 for the patch. +- 2019-11-27: core bugfix: action on retry mangles messages + When a failed action goes into retry, template content is rendered + invalid if the action uses more than 1 template. + closes https://github.com/rsyslog/rsyslog/issues/3898 + Thanks to Mikko Kortelainen for the patch. +- 2019-11-27: testbench: improve mysql testing support + tests can now run in parallel and are hardened against several glitches +- 2019-11-22: omhttp: add basic support for Loki Rest + Loki is a new message indexer and querier from Grafana Labs. See + https://github.com/grafana/loki for details on Loki. + This change provides the initial message structure to send bulk message + payloads to the Loki Rest endpoint. omhttp, received a new bulk message + format called lokirest. Additionally, the plugin relies on the user to + provide the correct "stream" read message format. + A loki template must be json compatible and include a "stream" key of + key value tags, and a values key of an array of 2 element arrays, where + each 2 element array is the unix epoch in nanoseconds followed by an + unstructured message. + An example: + template(name="array_loki" type="string" string="{\"stream\":{\"host\":\"%HOSTNAME%\",\"facility\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"syslogtag\":\"%syslogtag%\"},\"values\": [[ \"%timegenerated:::date-unixtimestamp%000000000\", \"%msg%\" ]]}") +- 2019-11-22: testbench: obtain python binary path via AM_PATH_PYTHON + see also https://github.com/rsyslog/rsyslog/issues/3853 +- 2019-11-22: omprog: detect violation of interface protocol + The spec for the omprog interaction with the program it calls specifies + that the program receives one message via one line. In other words: + it must be a string terminated by LF. + However, omprog does currently rely on a proper template to fulfill this + requirement, If the template does not provide for the LF, it is never + written. For the called program, this looks like it does not receive any + input at all. Even if it finally reads data (e.g. due to full buffer), + it will not properly be able to discern the messages. + This handling is improved with this commit. + We cannot just check the template, because at the end of the template + may by a non-constant value. As such, we do not know at config load + time if there is this problem or not. + So the correct approach is to, during runtime, check if each message + is properly terminated. For those that are not: + * we append a LF, because anything else makes matters worse + * log a warning message, at least for a sample of the messages + The warning is useful in the (expected most often) case that the template + is simply missing the LF. While appending works, it slows down processing. + As such the user should be given a chance to correct the config bug. + To avoid clutter, the warning is emitted at most once every 30 seconds. + This value is hardcoded as we do not envision a need to adjust it. Usually + users should quickly fix the template. + closes https://github.com/rsyslog/rsyslog/issues/3975 +- 2019-11-19: core queue: emit warning if parameters are set for direct queue + Direct queues do not apply queue parameters because they are actually + no physical queue. As such, any parameter set is ignored. This can + lead to unintentional results. + The new code detects this case and warns the user. + closes https://github.com/rsyslog/rsyslog/issues/77 +- 2019-11-19: imjournal bugfix: do not wait too long on recovery try + When trying to recover journal errors, imjournal waited a hardcoded + period of 10s between tries. This was pretty long and could lead to + loss of journal data. + This commit adjust it to 100ms, which should still be fully sufficient + to prevent the journal from "hammering" the CPU. + It may be worth considering to make this setting configurable - but + let's first see if there is real demand to actually do that. + closes https://github.com/rsyslog/rsyslog/issues/3969 +- 2019-11-19: mmutf8fix: enhance handling of incorrect UTF-8 sequences + 1. Invalid utf8 detection didn't handle 3 and 4-byte overlong encodings (2 + byte overlong encodings were handled explicitly by rejection E0 and E1 + start bytes). Unified checks for overlong encodings. + 2. Surrogates U+D800..U+DFFF are not valid codepoints (Unicode Standard, D92) + 3. Replacement of characters in invalid 3 or 4-bytes encodings was too + eager. It must not replace bytes which are valid UTF-8 sequences. For + example, in [0xE0 0xC2 0xA7] sequence the 0xC2 is invalid as a continuation + byte, but it starts a valid UTF8 symbol [0xC2 0xA7]. That is, with current + code processing the sequence will result in "???" but the correct result is "?§" + (provided that the replacement character is "?"). + 4. Various tests for UTF-8 invalid/valid sequences. + Thanks to Sergei Turchanov for the patch. +- 2019-11-14: imfile: add new input parameter escapeLF.replacement + The new parameter permits to specify a replacement to be configured + when "escapeLF" is set to "on". Previously, a fixed replacement string + was used ("#012"/"\n") depending on circumstances. If the parameter is + set to an empty string, the LF is simply discarded. + closes https://github.com/rsyslog/rsyslog/issues/3889 +---------------------------------------------------------------------------------------- +Scheduled Release 8.1911.0 (aka 2019.11) 2019-11-12 +- 2019-11-12: core queue: add config param "queue.takeFlowCtlFromMsg" + This is a fine-tuning option which permits to control whether or not + rsyslog shall alays take the flow control setting from the message. If + so, non-primary queues may also block when reaching high water mark. + This permits to add some synchronous processing to rsyslog core engine. + However, it is dangerous, as improper use may make the core engine + stall. As such, enabling this option requires very careful planning + of the rsyslog configuration and deep understanding of the consequences. + Note that the option is applied to individual queues, so a configuration + with a large number of queues can (and must if use) be fine-tuned to + the exact use case. + The rsyslog team strongly recommends to let the option turned off, + which is the default setting. + see also https://github.com/rsyslog/rsyslog/issues/3941 +- 2019-11-12: imrelp: add new config parameter "flowcontrol" + This permits to fine-tune the flowControl parameter. Possible values are + "no", "light", and "full". With light being the default and previously + only value. + Changing the flow control setting may be useful for some rare applications, + but be sure to know exactly what you are doing when changing this setting. + Most importantly, whole rsyslog may block and become unresponsive if you + change flowcontrol to "full". While this may be a desired effect when + intentionally trying to make it most unlikely that rsyslog needs to + lose/discard messages, usually this is not what you want. + see also https://github.com/rsyslog/rsyslog/issues/3941 +- 2019-11-11: imrelp: remove unsafe debug instrumentation + dbgprintf, which is not signal safe, was called from a signal handler + to get better understanding during debugging. While this usually works, + it can occasionally (5%) lead to a hang during shutdown. We have now + removed that debug info as it is no longer vital. + Note: this could only happen during debug runs. Production mode was + not affected. As such, this fix is only relevant to developers. + However, it caused some confusion in the following issue tracker. + see also https://github.com/rsyslog/rsyslog/issues/3941 +- 2019-11-06: ossl driver bugfix: fix wrong OpenSSL Version check + Fix OpenSSL Version check in: + - SetGnutlsPriorityString function in nsd_ossl.c + - initTLS() function tcpflood.c + See https://www.openssl.org/docs/man1.1.0/man3/OPENSSL_VERSION_NUMBER.html + for more. + This bug lead to not enabling some functionality correctly. + Removed "MinProtocol=TLSv1.1" from two testcases because MinProtocol + is only supported by OpenSSl 1.1.0 or higher and was not really + necessary for the testcases. + closes https://github.com/rsyslog/rsyslog/issues/3939 +- 2019-11-05: mmdarwin: Optimizations, new parameters, update to protocol header + - use permanent worker-dependent buffers to avoid malloc/free for each entry + - move socket structures to worker data, remove global mutex + - add log lines for parameters and general workflow + - don't send body if empty/incomplete (see new parameters) + - don't close/reopen socket every time -> let session open or create new every X + entry (see new parameters) + - clean up code + - added 'send_partial', to let mmdarwin send body if not all fields were + retrieved, or not; default false = only send complete bodies + - added 'socket_max_use' to open new session every X packet, useful for + some versions of Darwin (prior to 1.1) + default is 0 = do not open new session/keep only one + - added 'evt_id' to the darwin header (Darwin v1+ compatibility) + Note: mmdarwin is a contributed module + Thanks to github user frikilax for the patch. +- 2019-11-01: mmkubernetes bugfix: improper use of realloc() + could cause problems under extreme memory shortage - very unlikely + credits to LGTM.COM for detecting this +- 2019-10-31: imjournal: set the journal data threshold to MaxMessageSize + When data is read from the journal using sd_journal_get_data it may be + truncated to a certain threshold (64K by default). + If the rsyslog MaxMessageSize is larger than the threshold, there is a + chance rsyslog will receive incomplete messages from the journal. + Empirically, this appears to happen reliably when XZ compression is + used by journald. Systems where journald uses LZ4 compression do not + appear to suffer this issue reliably--if at all. + This change sets the threshold to the MaxMessageSize when the + journal is opened. + Thanks to Robert Winslow Dalpe for the patch. +- 2019-10-30: improg bugfix: allow improg to handle multi-line inputs + miscellaneous bug fixes in improg: + * properly truncate string after an input event is submitted + * set msgoffset to 0. + * tests added to check above fixes + Thanks to Nelson Yen for the fix. +- 2019-10-30: mmdblookup bugfix: missing space in city name + This fixes the issue that spaces in city names are dropped. However, the + fix is more or less a work-around. As it turns out, the libmaxminddb API + is not correctly used. In the somewhat longer term, we should fix this. + see also https://github.com/maxmind/libmaxminddb/issues/218 + closes https://github.com/rsyslog/rsyslog/issues/1650 +- 2019-10-30: core/queue: provide ability to run diskqueue on multiple threads + Up until this release, disk queues could only use a single thread, + what limited their performance with outputs like ElasticSearch. + Now disk queues can utilize multiple threads just like any other + queue type. Most importantly, the disk queue part of a DA queue + now inherits the max number of threads from its memory queue + counterpart. + NOTE: the new multi-threaded DA disk queue is actually a change of + behavior. We have not guarded it by a new config switch as we + assume the new behavior is most often exactly within user + expectations. In any case, we cannot see any harm from running + the disk queue on multiple threads. + see also https://github.com/rsyslog/rsyslog/issues/3543 + closes https://github.com/rsyslog/rsyslog/issues/3833 +- 2019-10-25: omfile bugfix: file handle leak + The stream class does not close re-opened file descriptors. + This lead to leaking file handles and ultimately to the inability + to open any files/sockets/etc as rsyslog ran out of handles. + The bug was depending on timing. This involved different OS + thread scheduler timing as well as workload. The bug was more + common under the following conditions: + - async writing of files + - dynafiles + - not committing file data at end of transaction + However it could be triggered under other conditions as well. + The refactoring done in 8.1908 increased the likelihood of + experiencing this bug. But it was not a real regression, the new + code was valid, but changed the timing so that the race was more + likely. + Thanks to Michael Biebl for reporting this bug and helping to + analyze it. + closes https://github.com/rsyslog/rsyslog/issues/3885 +- 2019-10-22: imfile bugfix: improper use of calloc() + could cause problems under extreme memory shortage - very unlikely + credits to LGTM.COM for detecting this +- 2019-10-22: TLS driver bugfix: improper use of calloc() + can cause problems under extreme memory shortage - very unlikely + credits to LGTM.COM for detecting this +- 2019-10-22: imuxsock bugfix: improper use of calloc() + can cause problems under extreme memory shortage - very unlikely + credits to LGTM.COM for detecting this +- 2019-10-17: build system bugfix: incorrect default in ./configure help text + closes https://github.com/rsyslog/rsyslog/issues/3904 + Thanks to Michael Biebl for pointing this out. +- 2019-10-17: mmkubernetes bugfix: improper use of calloc() + can cause problems under extreme memory shortage - very unlikely + credits to LGTM.COM for detecting this +- 2019-10-16: core queue bugfix: propagate batch size to DA queue + This was a long-standing bug where the DA queue always had a fixed small batch + size because the setting was not propagated from the memory queue. This also + removes a needless and counter-productive "debug aid" which seemed to be in + the code for quite some while. It did not cause harm because of the batch + size issue. +- 2019-10-16: testbench: fix unreliable gzipwrite test + The test was timing-sensitive as we did not properly check all data + was output to the output file - we just relied on sleep periods. + This has been changed. Also, we made some changes to the testing + framework to fully support sequence checking of multiple ZIP files. +- 2019-10-16: core queue bugfix: handle multi-queue-file delete correctly + Rsyslog may leave some dangling disk queue files under the following + conditions: + - batch sizes and/or messages are large + - queue files are comparatively small + - a batch spans more than two queue files (from n to n+m with m>1) + In this case, queue files n+1 to (n+m-1) are not deleted. This can + lead to problems when the queue is re-opened again. In extreme cases + this can also lead to stalled processing when the max disk space is + used up by such left-over queue files. + Using defaults this scenario is very unlikely, but it can happen, + especially when large messages are being processed. +- 2019-10-16: imjournal: fix regression from yesterday's patch + commit 78976a9bc059 introduced a regression that caused writing + the journal state file to fail. This happens when the state file + is given as relative file name and the working directory is also + a relative path. This situation is very uncommon. So most deployments + will never experience it. We discovered the issue during CI runs + where the trigger condition is given. Note that it also takes + multiple times of loading the journal to actually see the bug. + see also https://github.com/rsyslog/rsyslog/pull/3878 +- 2019-10-15: imjournal plugin code restructuring, added remote option + Decomposed ReadJournal() a bit, also now coupling journald + variables in one struct, added few warning messages and debug + prints to help with bug hunts in future, also got rid of two + needless journald calls. WorkAroundJournalBug now deprecated. + Added option to pull journald records from outside local machine. + Thanks to Jiri Vymazal for the patch. +- 2019-10-11: core bugfix: potential abort on very long action name + The action name is stored in modified form for the debug header and + some messages. If it is extremely long, a buffer can be overrun, + resulting in misaddressing and potential segfault for rsyslog. This + can also happen if the action is NOT named, but a custom path to + the output module is given and that path is very long. This triggers + the same issue because by default the module load path is included + in the action name. + This patch corrects the problem and truncates overly long names + when being used for name generation. + The problem was detected during testbench work. We did never receive + a bug report from practice. +- 2019-10-10: testbench: add test for mmpstrucdata with RFC5424 escape sequences +---------------------------------------------------------------------------------------- +Scheduled Release 8.1910.0 (aka 2019.10) 2019-10-01 +- 2019-10-01: core bugfix: incorrect error message on duplicate module load + A Null-pointer was passed to printf instead of the module name. + On some platforms this may lead to a segfault. On most platforms + printf check's for NULL pointers and uses the string "(null)" + instead. In any case, the module name is missing from the error message. +- 2019-10-01: imczmq nitfix: potential NULL ptr in printf on out-of-memory condition + very unlikely to happen but if it does without any real issue on most platforms. +- 2019-10-01: work around some compiler warning messages induced by pthreads API +- 2019-10-01: core ratelimiting: more verbose message when rate-limiting happens + When messages are rate-limited, the error message now also contains the + rate limiter setting. This enables the user to more quickly understand what + the problem is (especially if default values apply). + Thanks to Jiri Vymazal for the patch. +- 2019-10-01: openssl TLS driver: do not emit unnecessary error message + On older openssl versions, an API was missing to set user-defined parameters. If we + had such an older version, rsyslog emitted an error message even if the user did + not configure such parameters. This has been corrected, so that a message is only + emitted if there really is a problem. Based on user feedback the severity has also + been downgraded to "warning". +- 2019-10-01: pmcisconames (contributed module) bugfix: potential misaddressing +- 2019-09-30: pmaixforwardedfrom (contributed module) bugfix: potential misaddressing +- 2019-09-30: pmdb2diag (contributed module) bugfix: Out of bounds issue + Add a new sanity check after determining the level len. + Thanks to Philippe Duveau for the patch. + see also: https://nvd.nist.gov/vuln/detail/CVE-2019-17040 +- 2019-09-02: ability to set stricter TLS operation modes + - checking of extendedKeyUsage certificate field + - stricter checking of certificate name/addresses + Thanks to Jiri Vymazal for the patch. +- 2019-08-21: testbench: add basic test for immark +- 2019-08-20: core: do not unnecessarily set hostname on each HUP +- 2019-08-20: build system: support cross-platform build for mysql/mariadb + rsyslog fails to cross build from source, because it uses mysql_config + and mysql_config is unfixably broken for cross compilation. It would be + better to use pkg-config. The attached patch makes rsyslog try + pkg-config first and fall back to mysql_config. + Thanks to Helmut Grohne for providing a base patch. +- 2019-08-20: core/tcpsrv: potential race on startup/shutdown + if the tcpsrv component is started and quickly terminated, it may hang + for a short period of time. Also a very small amount of memory is leaked + immediately before shutdown. While this leak is irrelevant in practice + (the OS clean up the process anyways), it leads to CI failures. The hang, + however, can lead to longer than expected shutdown times for rsyslog. + The problem can be experienced via imtcp, imgssapi and imdiag (users + of affected core component). +---------------------------------------------------------------------------------------- +Scheduled Release 8.1908.0 (aka 2019.08) 2019-08-20 +- 2019-08-19: testbench: add test for $allowedSender functionality +- 2019-08-19: testbench: harden some tests against very slow CI machines +- 2019-08-16: testbench: make most tests use a port file and assign listen port 0 + This makes the test much more robust against heavily loaded test systems. +- 2019-08-16: core/action: guard action.externalstate.file content against whitespace + remove trailing whitespace before checking the status string. This is + most important as a line usually ends with \n, which is considered + trailing whitespace. Accepting this increases usability. +- 2019-08-16: imtcp bugfix: multiple listenerPortFile parameter did not work + ... because they were treated as module-global. If we had multiple imtcp + listeners with multiple port files, only the last filename was always used. + closes https://github.com/rsyslog/rsyslog/issues/3817 +- 2019-08-16: testbench: improve testbench plumbing for gzip and fail cases + We have added new capabilities to the testbench plumbing to automatically + deal with gzip-compressed files. This also permits to use the wait_seq_check + function to work for gzip tests as well. The known-timing-sensitive + gzipwr_large test now makes use of the new capabilities. This enables us + to more reliably detect when we can savely shutdown the tested instance. + This commit also adds an ability to "abort" the full testbench run on + first test failure. This is especially useful during CI. +- 2019-08-13: testbench: add test for imuxsock legacy format + This was never tested. Ensures we don't accidentally break existing + configurations. +- 2019-08-13: omelasticsearch bugfix: segfault on unknown retryRuleset + omelasticsearch does some "interesting tricks" for an output module. + This causes a segfault if the retryRuleset is now known. + The action module interface currently expects that all config errors + be detected during instance creation. Instead omelasticsearch defers + the retry ruleset check to a later state. The reason is that it wants + to support the use the same rulesetname it is defined in - and this + is not yet available at action parsing. + We fix this by ensuring that any deleted instance is properly unlinked + from the instance list. One may argue the module interface should get + upgrade for such cases, but this is a longer-term approach. + closes https://github.com/rsyslog/rsyslog/pull/3796 +- 2019-08-12: imptcp bugfix: port="0" parameter did not work as expected + when multiple interfaces and/or protocols could be bound, each of + them used a different listener ports were assigned. While this is + basically correct, it makes things unusable, especially as + listenPortFileName will only contain the port number used for + the latest listener. + This patch now follows the model of nsd_ptcp.c to assign only + the first port randomly and then use that port consistently. +- 2019-08-10: omelasticsearch bugfix: potential resource leak with "rebindinterval" + If the "rebindInterval" parameter was used connections could be linked. This + was especially the case with small intervals (such as "2"). This is fixed by + forcing libcurl to close the connection on rebind. + Thanks to Noriko Hosoi for providing the patch. +- 2019-08-10: imjournal bugfix: state file close with fsync() was incorrect + This lead to fsync() now always applied where expected. + Thanks to Jiri Vymazal for the patch. +- 2019-08-10: testbench: add addtl test for multithreading and HUP +- 2019-08-10: imptcp bugfix: received bytes counter improperly maintained + imptcp counts the number of bytes received. However, receives + happen on different worker thread. The access to the counter + was not synchronized, which can cause loss of updates. Also, + thread debuggers validly flag this as an error, which creates + problems under CI. + This commit fixes the situation via atomic operations and + falls back to mutex calls if they are not available. + Detected by LLVM thread sanitizer. + closes https://github.com/rsyslog/rsyslog/issues/3798 +- 2019-08-07: testbench: add basic tests for omusrmsg +- 2019-08-05: omhttp bugfix: enable checkpath configuration parameter + omhttp, 'checkpath' option, was not configurable in the past. + - add 'checkpath' to the cnfparamdescr table. + - fix issue with checkpath passing extra garbage characters in string. + - add 'checkpath' into unit test - omhttp-retry.sh + Thanks to Nelson Yen for the fix. +- 2019-08-05: testbench bugfix: some tests were executed when req module was missing + In actual case if --enable-impstats was not given some other tests failed. +- 2019-08-03: iminternal bugfix: race on termination + This could in theory lead to loss of shutdown messages, but was mostly a + cosmetic issues. We primarily fixed it to get TSAN-clean so that we can + utilize LLVM TSAN in CI. +- 2019-08-02: testbench: new test for omfile outchannel functionality +- 2019-08-02: core/janitor bugfix: properly maintain dynafile cache + When the janitor cleans out timed-out files, it does not + properly indicate the entry is gone. Especially when running + in async mode this can lead to use-after-free and thus + memory corruption or segfault. + see also https://github.com/rsyslog/rsyslog/issues/3756 +- 2019-08-01: omfile bugfix: race file when async writing is enabled + This seems to be a long-standing bug, introduced around 7 years ago. + It became more visible by properly closing files during HUP, which + was done in 8.1905.0 (and was another bugfix). Note that due to this + race a memory corruption can occur under bad circumstances. As such, + this may have also caused segfaults or system hangs (mutexes could + have been affected). + closes https://github.com/rsyslog/rsyslog/issues/3772 +- 2019-08-01: testbench: additional tests for HUP +- 2019-07-31: imrelp bugfix: hang after HUP + termination condition was not properly checked; this lead to + premature termination after patch 1c8712415b9 was applied. + It is open to debate if patch 1c8712415b9 changed the module + interface. Actually it looks like this was previously not + well thought out. + closes https://github.com/rsyslog/rsyslog/issues/3760 +- 2019-07-24: mmdarwin: add new module + This is a contributed module. For details see doc. + Thanks to the Advens team for contributing it. +- 2019-07-23 iminternal bugfix: suppress mutex double-unlock + If there is a burst of log messages during a time when rsyslog is unable + to output (either during log rotation, an out-of-space condition, or + some other similar condition), rsyslog can SEGFAULT due to a mutex + double-unlock. +- 2019-07-23 imtcp: enable listenPortFileName parameter + this parameter was added, but it had no effect as it was not + passed down to the driver layer. This has been fixed. That also + now enables us to use dynamically-assigned port, which are + very useful for further testbench stabilization. Quite some + false positives occurred because the pre-selected port was + already in use again when rsyslog started. +- 2019-07-19 imtcp: enable listenPortFileName parameter + this parameter was added, but it had no effect as it was not + passed down to the driver layer. This has been fixed. That also + now enables us to use dynamically-assigned port, which are + very useful for further testbench stabilization. Quite some + false positives occurred because the pre-selected port was + already in use again when rsyslog started. +- 2019-07-18 core/action: no error file written if act suspended on TX commit + when an action was already disabled while the action was tried to be + committed, no error file was written. Note that this state is highly + unlikely to happen. Most probably, it can only happen if parameter + action.externalstate.file is used. +---------------------------------------------------------------------------------------- +Version 8.1907.0 (aka 2019.07) 2019-07-09 +NOTE TO MAINTAINERS: libee is not used by rsyslog for quite some while. +However, we never included this info into the changelog. So if you still +make rsyslog depend on libee (some do this), you should stop doing so now. +Libee is dead and no longer been maintained nor hosted by us. Old versions +can still be found at github for those in need. + +GENERAL NOTE: during 8.1907 scheduled release timeframe we changed the ChangeLog +format to include the date a change went into master branch. This is to provide +an easy way to identify which changes went into the respective daily stable. + +- 2019-07-05 imuxsock: support FreeBSD 12 out of the box + FreeBSD 12 uses RFC5424 on the system log socket by default. This + format is not supported by the special parser used in imuxsock. + Thus for FreeBSD the default needs to be changed to use the + regular parser chain by default. That is all this commit does. + closes https://github.com/rsyslog/rsyslog/issues/3694 +- 2019-07-05 function bugfix: "ipv42num" misspelled as "ip42mum" (without "v") + To fix the issue but keep compatible with existing deployments + both function names are now supported. + closes https://github.com/rsyslog/rsyslog/issues/3676 +- 2019-07-04 fix leading double space in rsyslog startup messages + see also https://github.com/rsyslog/rsyslog/issues/2979 +- omamqp1: port to latest api, add tests + This brings omamqp1 up-to-date with the latest qpid-proton-c + api version. This also adds a test for the plugin, to test + the basic functionality. The test requires the user to + install qdrouterd and the python qpid-proton library in order + to use the simple_recv.py test program. + Thanks to Richard Megginson for the patch. +- omclickhouse bugfix: potential segfault on omclickhouse batchmode + segfault happened when the template did not contain the string + "VALUES". + Thanks to github user wdjwxh for the fix. +- core bugfix: message duplication copied incorrect timestamp + MsgDup() placed timereported into timegenerated property, resulting + in invalid property values. Original timegenerated was lost. This + occurred always when a message needed to be duplicated. Most + importantly this is the case when queues are used. + closes https://github.com/rsyslog/rsyslog/issues/3716 +- core bugfix: segfault on startup depending on queue file names + rsyslog will segfault on startup when a main queue file name has + been set and at least on other queue contains a file name. This + was cased by too-early freeing config error-detection data + structures. It is a regression caused by commit e22fb205a3. + Thanks to Wade Simmons for reporting this issue and providing + detailed analysis. That greatly helps fixing it quickly. + closes https://github.com/rsyslog/rsyslog/issues/3681 +- core "bugfix": alignment issue + This was not a hard error on current platforms, but a + to-be-considered compiler warning regarding invalid alignment. + While it works well on current platforms, alignment issues may + turn into real issues in future platforms. So we try to fix them + if possible. As not only a side-effect this resolves compiler + warnings even on current platforms. + This fix has some regression potential. If so, the problems + may occur during IP address resolution. + see also https://github.com/rsyslog/rsyslog/issues/2608 +- omfile bugfix: potential hang/segfault on HUP of dynafile action + when omfile was HUPed it did not sufficiently clear all dynafile + cache maintenance data structures. This usually lead to misaddressing + and could result in various issues, including a hang of rsyslog + processing or segfaults. It could also have "no effect" by pure + luck of not hitting anything important. This actually seems to + have been the most frequent case. + This seems to be a long-standing bug, but the likelihood of its + appearance seems to have been increased by commit 62fbef7 + introduced in 8.1905. Note: the commit itself has no regression, + just increases the likelihood to trigger the pre-existing bug. + special thanks to Alexandre Guédon for his help in analyzing + the issue - without him, we would probably still not know + what actually went wrong. + closes https://github.com/rsyslog/rsyslog/issues/3686 +- imjournal bugfix: potential message duplication + When journal was preloaded from previously saved cursor it was not advanced + to next entry so reading begun from last message which was therefore + duplicated. + Thanks to Jiri Vymazal for the patch. +- rfc5424 parser bugfix: leading space sometimes lost + if structured data is present a leading space in MSG field is lost +- queue subsystem bugfix: oversize queue warning message shown as error + The warning message was emitted as an error message, which is misleading + and may also break some automated procedures. +- core bugfix: HUP did not work reliable on all platforms + most notably not on FreeBSD, maybe others. The reason was obviously + different handling of signals in respect to multiple threads. +- build system bugfix: missing files in distribution tarball +- testbench + * fixed "make distcheck" settings which were missing some modules + This lead to incomplete "make distcheck" run; some errors were not + detected due to that. + * testbench framework: use ip tool instead of outdated ifconfig + The framework now first checks if "ip" is available and falls back + to "ifconfig" only if this is not the case. + Thanks to Michael Biebl for the suggestion. + closes https://github.com/rsyslog/rsyslog/issues/3682 +------------------------------------------------------------------------------ +Version 8.1905.0 (aka 2019.05) 2019-05-28 +- templates: add datatype template option for JSON generation + The new "datatype" and "onEmpty" template options permits to + generate non-string data rather easily. It works together with + jsonf formatting, which is what people should use nowadays. + closes https://github.com/rsyslog/rsyslog/issues/2827 +- config processing: check disk queue file is unique + If the same name is specified for multiple queues, the queue files + will become corrupted. This commit adds a check during config parsing. + If duplicate names are detected the config parser errors out and the + related object is not created. + Note: this may look to a change-of-behavior to some users. However, + this never worked and it was pure luck that these users did not run + into big problems (e.g. DA queues were never going to disk at the + same time). So it is acceptable to error out in this hard error case. + closes https://github.com/rsyslog/rsyslog/issues/1385 +- global config: new parameters for ruleset queue defaults + specifically: + * default.ruleset.queue.timeoutshutdown + * default.ruleset.queue.timeoutactioncompletion + * default.ruleset.queue.timeoutenqueue + * default.ruleset.queue.timeoutworkerthreadshutdown + closes https://github.com/rsyslog/rsyslog/issues/3656 +- add capability to write full config file (-o cmdline option) + Introduces the capability to create an output config file that explodes + all "includes" into a single file. This provides a much better overview + of how exactly the configuration is crafted. That could often be a great + troubleshooting aid. + This commit also contains some slight not-really-related cleanup. + closes https://github.com/rsyslog/rsyslog/issues/3634 +- queue subsystem: permit to disable "light delay mark" + New semantic: if lightDelayMark is 0, it is set to the max queue + size, effectively disabling the "light delay" functionality. + Thanks to Yury Bushmelev to mentioning issues related to light + delay mark and proposing the solution (which actually is what + this commit does). + closes https://github.com/rsyslog/rsyslog/issues/1778 +- queue subsystem: provide better user status messages + The queue subsystem now provides additional information messages which + may help a regular user to maintain system health. Most importantly, + DA queues now output when they persist queue data at end of run and + when they restart the queue based on persisted data. +- core: emit a warning message for ultra-large queue size definitions + We see error reports from users who have configured excessively large queues + and receive an OOM condition or other problems. + With that patch we generate a warning message if a queue is configured very + large. "Very large" is defined to be in excess of 500000 messages. + see also https://github.com/rsyslog/rsyslog/issues/3314 + closes https://github.com/rsyslog/rsyslog/issues/3334 +- new global config parameter "internalmsg.severity" + permits to specify a severity filter for internal message. Only + messages with this severity level or more severe are logged. + Originally this was done in rsyslog.conf as usual: you can filter + rsyslog messages on severity, just like any other. But with systemd, + we now emit primarily to the journal, and this is outside of rsyslog's + rule engine and so regular filters do not apply (at least in regard + to the journal). Logging to journal is good, because finally + folks begin to see the messages (traditional distro configs discard + them, for whatever is the reason). + This commit implements a global setting for a severity-based filter + for internal messages, before submitted to journal. So it's not 100% + of what rsyslog can do, but at least some way to customize. + see also https://github.com/rsyslog/rsyslog/issues/3639 +- config processing bugfix: error messages if config.enabled="off" is used + Using config.enabled="off" could lead to error messages on + "parameter xxx not known", which were invalid. They occurred + because the config handler expected them to be used, which + was not the case due to being disabled. + This commit fixes that issue. + closes https://github.com/rsyslog/rsyslog/issues/2520 +- core portability bugfix: harden shutdown processing on FreeBSD + On FreeBSD, rsyslog does not always terminate immediately on SIGTERM. + Root cause seems to be that SIGTERM is delivered differently under + FreeBSD. This causes the main thread to not be awaken, and so it + takes until the next janitor interval to come back to life - which + can be far too long. Fixed this bug explicitly awaking the main + thread. +- imtcp bugfix: oversize message truncation causes log to be garbled + The actual problem is in the tcpserver component. However, the prime user + is imtcp and so users will likely experience this as imtcp problem. + When a too-long message is truncated, the byte after the truncation + position becomes the first byte of the next message. This will garble + the next messages and in almost all cases render it is syslog-noncompliant. + The same problem does NOT occur when the message is split. + This commit fixes the issue. It also includes a testbench fix. + Unfortunately the test for exactly this feature was not properly + crafted and so could not detect the problem. + closes https://github.com/rsyslog/rsyslog/issues/3580 +- omfile bugfix: FlushOnTXEnd does not work reliably with dynafiles + The flush was only done to the last dynafile in use at end of + transactions. Dynafiles that were also modified during the + transaction were not flushed. + Special thanks to Duy Nguyen for pointing us to the bug and + suggesting a solution. + This commit also contains a bit of cosmetic cleanup inside + the file stream class. + closes https://github.com/rsyslog/rsyslog/issues/2502 +- lmcry_gcry build bugfix: was not always properly build + Due to an invalid definition in build system this seems to have not + been correctly build on at least some platforms (but it worked on + others as it passed CI testing). This has now been corrected. + Thanks to Remi Locherer for the patch. +- dnscache bugfix: very unlikely memory leak + This fixes a memory leak that can only occur under OOM conditions. + Detected by Coverity Scan, CID 203717 +- testbench bugfix: wrong parameter check in diag.sh (tcpflood()) + When first parameter is check_only, the tcpflood funtion shall not + abort the test itself (The fail is intended if this option is set). + closes issue #3625 +- testbench bugfix: imfile-symlink test failed w/ parallel test run + The test sometimes failed. It used a symlink to a hardcoded name + rsyslog-link.*.log. This symlink was created but then disappears. + The reason is that upon (every!) test exit, rsyslog-link.*.log is + deleted. So a parallel test running the exit procedure just at the + "right" time can removed that file. + The bug is that the file name should be created using the tests's + dynamic name. This is done now. + closes https://github.com/rsyslog/rsyslog/issues/3550 +------------------------------------------------------------------------------ +Version 8.1904.0 (aka 2019.04) 2019-04-16 +- omfile: provide more helpful error message on file write errors + now contains actual file name plus a link to probable causes for this type + of problem +- imfile: emit error on startup if no working directory is set + When the work directory has not been set or is invalid, state files + are created in the root of the file system. This is neither expected + nor desirable. We now complain loudly about this fact. For backwards + compatibility reasons, we still need to support running imfile in + this case. + closes https://github.com/rsyslog/rsyslog/issues/1296 +- dnscache: add global parameter dnscache.default.ttl + This permits to control default TTL for cache entries. If set + to 0, the DNS cache is effectively disabled. + closes https://github.com/rsyslog/rsyslog/issues/49 + closes https://github.com/rsyslog/rsyslog/issues/1487 +- omelasticsearch: new parameter rebindinterval + Thanks to Richard Megginson for the patch. +- omelasticsearch: new parameter skipverifyhost + Add ability to specify the libcurl CURLOPT_SSL_VERIFYHOST + option to skip verification of the hostname in the peer cert. + WARNING: This option is insecure, and should only be used + for testing. The default value is off, meaning, the hostname + will be verified by default. + Thanks to Richard Megginson for the patch. +- omelasticsearch: set rawmsg to data from original request + Previously, when constructing the message to submit for a retry + for an original request, if the original request did not contain + the field `message`, the system property `rawmsg` was set to + the entire metadata + data from the original request. This was + causing problems with Elasticsearch. This patch changes + the code so that the `rawmsg` will be set to only the data part + of the original request if there is no `message` field. + closes https://github.com/rsyslog/rsyslog/issues/3573 + Thanks to Richard Megginson for the patch. +- mmkubernetes - support for metadata cache expiration + New parameters for mmkubernetes (module and action): + * `cacheexpireinterval` + If `cacheexpireinterval` is -1, then do not check for cache expiration. + If `cacheexpireinterval` is 0, then check for cache expiration. + If `cacheexpireinterval` is greater than 0, check for cache expiration + if the last time we checked was more than this many seconds ago. + * `cacheentryttl` - maximum age in seconds for cache entries + New statistics counters: + * `podcachenumentries` - the number of entries in the pod metadata cache. + * `namespacecachenumentries` - the number of entries in the namespace + metadata cache. + * `podcachehits` - the number of times a requested entry was found in the + pod metadata cache. + * `namespacecachehits` - the number of times a requested entry was found + in the namespace metadata cache. + * `podcachemisses` - the number of times a requested entry was not found + in the pod metadata cache, and had to be requested from Kubernetes. + * `namespacecachemisses` - the number of times a requested entry was not + found in the namespace metadata cache, and had to be requested from + Kubernetes. +- imdocker: new contributed module + imdocker will get (docker) container logs from a host as well as filling + out some basic container metadata as id, name, image, labels. + Thanks to Nelson Yen for the contribution. +- mmtaghostname: new contributed module + This module allows one to force hostname after parsing to the localhostname of + rsyslog and/or add a tag to messages received from input modules without + tag parameter. + Thanks to Philippe Duveau for the contribution. +- imbatchreport: new contributed input module + This input module manage batches' reports : complete file as a single log. + Thanks to Philippe Duveau for the contribution. +- imtuxedolog: new contributed input module for Tuxedo ULOG + Thanks to Philippe Duveau for the contribution. +- openssl network driver: Added support setting openssl configcommands + We are using the gnutlsPriorityString setting variable, to pass + configuration commands to openssl. + closes: https://github.com/rsyslog/rsyslog/issues/3605 +- omkafka: drop messages rejected due to being too large + Drop messages that were rejected due to + 'RD_KAFKA_RESP_ERR_MSG_SIZE_TOO_LARGE' error + Thanks to Nelson Yen for the patch +- core/action: implement capability to resume/suspend via external file + It has been reported that some TCP receivers exists that accept syslog tcp + messages at any rate, even if they do not manage to actually process them. + Instead, they silently drop the message. This behavior is not configurable. + All in all, it can lead to considerate message loss. + To support such use cases, we need to provide an ability to externally + trigger actions suspension and resumption. + We do this via a configured file which contains the status of the action. + Rsyslog periodically reads the file and if it contains "SUSPEND", it + suspend the action (and likewise for resume). + closes https://github.com/rsyslog/rsyslog/issues/2924 +- improg bugfix: some memory leaks + Thanks to Philippe Duveau for the contribution. +- msg object bugfix: regression from 1255a67 + closes https://github.com/rsyslog/rsyslog/issues/3570 +- pmnormalize: fix memory leaks, improve tests + This patch fixes a set of problems plus provides more and enhanced + tests for the module. + Most important problem was a memory leak that occurred when a message + could not be passed at all. For each message that could not be parsed + memory of at least the size the message is leaked. Depending on + traffic pattern this can quickly lead to OOM. Note, however, that + this leak was never reported - it was discovered as part of code + review. + closes https://github.com/rsyslog/rsyslog/issues/2007 +- omkafka bugfix: build failure due to inconsistent type + fails depending on platform and settings; was somehow undetected by CI +- imjournal bugfix: potential segfault on some API failure returns + In one case there was possibility of free()'d value of journal + cursor not being reset, causing double-free and crash later on. + closes https://github.com/rsyslog/rsyslog/issues/3537 +- openssl subsystem bugfix: better error handling + Handling of SSL_ERROR_SYSCALL has been hardened. + Handling for SSL_Shutdown errors has been corrected. + Also fixed SSL Shutdown handling in tcpflood (openssl code). + If SSL_Shutdown returns error, we call SSL_read as described in + the documentation to do a bidirectional shutdown. + Closes https://github.com/rsyslog/rsyslog/issues/3561 +- imjournal bugfix: Fetching journal cursor only for valid journal + The sd_journal_get_cursor() got called regardless of previous + retcodes from other journal calls which flooded logs with journald + errors. Now skipping the call in case of previous journal call + non-zero result. Fixed success checking of get_cursor() call + to eliminate double-free possibility. + Also, making WorkAroundJournalBug true by default, as there were no + confirmed performance regressions for a quite long time. + Thanks to Jiri Vymazal for the patch. +- omamqp: fix build errors + They occur on some, newer, platforms. We do not really fix them, but rather + make the compiler ignore them. This is not really good, but the module is + contributed and so that's for now the best thing we can do. +- testbench: change manytcp.sh to use a larger connection count again + not sure why it was reduced, maybe related to + https://github.com/rsyslog/rsyslog/issues/1108 + also, modernize this and another test +- tcpflood bugfix: make soft connection limit work again + It looks like the soft limit became defunct when tcpflood was enhanced to + request more open file handles from OS. + closes https://github.com/rsyslog/rsyslog/issues/1108 +- testbench bugfix: omhttp tests were not run during "make distcheck" +- build system bugfix: omhttp test files were not included in dist tarball + Thanks to Thomas D. (whissi) for the patch. +------------------------------------------------------------------------------ +Version 8.1903.0 (aka 2019.03) 2019-03-05 +- omrabbitmq: add features (RabbitMQ HA management, templatize routing_key, + populate amqp message headers, delivery_mode and expiration parameters) +- improg: create input module to use external program as input datas +- imtuxedoulog: create input module to consume Tuxedo ULOG files +- omhttp: rewritten with large feature enhancements + Many thanks to Gabriel Intrator for this work. Gabriel also has adopted the + module and plans to support it in the future. +- pmdb2diag: create parser module for DB2 diag logs +- TLS subsystem: add support for certless communication + both openssl and GnuTLS drivers have been updated to support certless + communications. In this case e.g. Diffie-Helman is used. + NOTE: this is an insecure mode, as it does NOT guard against + man-in-the-middle attacks. We implemented it because of the large demand, + not because we think it makes sense to use this mode. We strongly recommend + against it. + closes https://github.com/rsyslog/rsyslog/issues/1068 +- imrelp/omrelp: add capability to specify tlslib for librelp + closes https://github.com/rsyslog/rsyslog/issues/3451 +- build system: introduce a better way to handle compiler pragmas + we now use macros and _Pragma(). This requires less code lines and is more + portable. +- omkafka: add support for dynamic keys + A new configuration property "dynaKey" is added that, when "on", changes the + value of property "key" to a template names instead of a constant value. + This is similar in approach to the DynaTopic implementation. + Thanks to Ludo Brands for the patch. +- AIX port: add AIX linking extensions on many plugins and contributions to + allow building them on this os. +- template: add Time-Related System Property $wday which is the day of week + This allows one to get a week based rotation of log as AIX does. +- ksi subsystem: add high availability mode + Note: ksi subsystem now REQUIRES libksi 3.19.0 or above + Thanks to Allan Park for the patch. + closes https://github.com/rsyslog/rsyslog/issues/3338 +- imfile bugfix: file reader could get stuck + State file handling was invalid. When a file was moved and re-created + rsyslog could use the file_id if the new file to write the old files' + state file. This could make the file reader stuck until it reached the + previous offset. Depending on file sizes this could never happen AND + would cause large message loss. This situation was timing dependent + (a race) and most frequently occurred under log rotation. In polling + mode the bug was less likely, but could also occur. + closes: https://github.com/rsyslog/rsyslog/issues/3465 + closes: https://github.com/rsyslog/rsyslog/issues/3468 +- imfile bugfix: potential segfault when working with directories or symlinks + see also https://github.com/rsyslog/rsyslog/pull/3496 + Thanks to Nelson Yen for the patch +- omhttp bugfix: header items could not have spaces in them + Thanks to Nathan Brown for the patch. +- core bugfix: enlarged msg offset types for bigger structured messages + using a large enough (dozens of kBs) structured message + it is possible to overflow the signed short type which leads + to rsyslog crash. (applies to msg.c, the message object) + Thanks to Jiri Vymazal for the patch. +- core bugfix for AIX: timeval2syslogTime now handle the bias according to + local time zone as documented by IBM. +- imfile feature: add configuration parameter to force parsing of read logs +- imczmq bugfix: + Release zframe following read from socket + Make the 0MQ frame pointer local to the receive loop and destroy the + frame as soon as the contents have been copied. This avoids: + * a memory leak should the receive loop execute more than once + * referencing an un-initialized value during cleanup (finalize_it) + Thanks to Mark Gillott for the patch. +- omclickhouse bugfix: default template unusable + STDSQL option added to the default template used in output module of clickhouse + Thanks to gagandeep trivedi for the patch. +- omclickhouse "bugfix": work-around failed error detection + omclickhouse uses a questionable method to check if a request generated + an error. We have seen the method to fail when we slightly upgraded clickhouse + server in CI testing. + This commit makes the method a bit more reliable without really fixing it. + But it's at least a short-term solution. + This should be changed to a proper status check. I assume such is possible. + see also https://github.com/rsyslog/rsyslog/issues/3485 +- imptcp bugfix: overly long socket bind path can lead to segfault + if the `path` input parameter is overly long (e.g. more than 108 + characters on some platforms) a non-terminated string is generated + and then passed to OS API. This can lead to all sorts of problems + including segfault. + We detected that based on gcc-8 warnings during code inspection. + No real-world problem case is known. +- ommongodb bugfix: improper stpncpy() calls +- testbench tcpflood: add new transport option relp-tls + Tcpflood can now send messages via relp with tls support. + closes https://github.com/rsyslog/rsyslog/issues/3448 +- testbench: mmdb valgrind tests failed is srcdir env was not set +- testbench: add omclickhouse tests +- testbench bugfix: some long-running tests had too low runtime allowance + closes https://github.com/rsyslog/rsyslog/issues/3493 +- testbench bugfix: daqueue-dirty-shutdown test + This test occasionally failed with left-over spool files. As far as we + have analyzed, this is due to the use of an invalid shutdown timeout + (very short) in the second phase of the test. It looks like this is + actually a copy&paste error from phase one. Behavior of rsyslog was + correct, but the test itself created a false positive. + We have corrected the timeout now and also modernized the test + a bit. + closes https://github.com/rsyslog/rsyslog/issues/2122 +- testbench bugfix: some omhttp tests had compatibility issues with Python 3 + Thanks to Thomas D. (whissi) for the patch. +------------------------------------------------------------------------------ +Version 8.1901.0 (aka 2019.01) 2019-01-22 +- new version scheme: 8.yymm.0 - version now depends on release date + see also https://rainer.gerhards.net/2018/12/rsyslog-version-numbering-change.html +- queue: add support for minimum batch sizes + closes https://github.com/rsyslog/rsyslog/issues/495 +- change queue.timeoutshutdown default to 10 for action queues + The previous default of 0 gave action queues no real chance to + shutdown - at the time they were applied, they were usually already + expired (computing the absolute timeout took a small amount of time). + So we change this now to 10ms, which still is very quick but gives + the queue at least a chance to shutdown itself. That in turn + smoothes the whole shutdown process. + If a very large number of action queues is used this may lead + to a very slightly longer shutdown time, albeit this is very + improbable. +- omclickhouse: new output module for clickhouse + This output module adds the possibility to send + INSERT querys to a Clickhouse database. See doc for details. + The messages are sent via a REST interface. + This commit also adds support of the testbench + for clickhouse tests, as well as various tests. + Closes https://github.com/rsyslog/rsyslog/issues/2272 +- omkafka: Add ability to dump librdkafka statistics to a file + Use statsFile to specify statistics output file; also requires + setting statistics.interval.ms confparam to a non-zero value. + Thanks to github user pcullen65 for the contribution. +- tls(ossl/gtls): add new Option "StreamDriver.PermitExpiredCerts" + The new Option can have one of the following values: + on = Expired certificates are allowed + off = Expired certificates are not allowed + warn = Expired certificates are allowed but warning will be logged (Default) + Includes necessary tests to validate new code. + closes https://github.com/rsyslog/rsyslog/issues/3364 +- action: add "action.resumeIntervalMax" parameter + This parameter permits to set an upper limit on the growth of the + retry interval. This is most useful when a target has extended + outage, in which case retries can happen very infrequently. + closes https://github.com/rsyslog/rsyslog/issues/3401 +- report child process exit status according to config parameter + Add new global setting 'reportChildProcessExits' with possible values + 'none|errors|all' (default 'errors'), and new global function + 'glblReportChildProcessExit' to report the exit status of a child + process according to the setting. + Invoke the report function whenever rsyslog reaps a child, namely in: + - rsyslogd.c (SIGCHLD signal handler) + - omprog + - mmexternal + - srutils.c (execProg function, invoked from stream.c and omshell) + Remove redundant "reaped by main loop" info log in omprog. + Promote debug message in mmexternal indicating that the child has + terminated prematurely to a warning log, like in omprog. + closes https://github.com/rsyslog/rsyslog/issues/3281 + Thanks to Joan Sala for contributing this. +- build system: add capability to turn off helgrind tests + we add configure switch --enable-helgrind. We need to turn helgrind off + when we use clang coverage instrumentation. The instrumentation injects + mt-unsafe counter updates which we seem to be unable to suppress. + Note: for gcc this was possible, because they all occurred in a utility + function. For clang, they are inlined so we get many -and changing- violations. + see also https://github.com/rsyslog/rsyslog/issues/3361#issuecomment-450502569 +- imzmq3/omzmq3: remove modules + according to @brianknox (their author) these modules are outdated: + https://twitter.com/taotetek/status/931860786959540224 + They are replaced by imczmq/omczmq and are no longer maintained. We put a + depreciation notice into the modules a year ago, and now it finally is time + to remove them. They do NOT build in any case, except if very old versions + of the 0mq ecosystem are used. + see also https://github.com/rsyslog/rsyslog/issues/2100 + closes https://github.com/rsyslog/rsyslog/issues/2103 +- bugfix omusrmsg: don't overwrite previous set _PATH_DEV value + Since commit 56ace5e418d149af27586c7c1264fccfbc6badf1, omusrmsg was broken + because "memcpy()" is not a suitable substitute for "strncat()" in this + context, it is actually replacing the previous added content. + Bug: https://bugs.gentoo.org/673004 + Closes: https://github.com/rsyslog/rsyslog/issues/3346 + Thanks to Thomas D. (whissi) for the patch. +- bugfix ossl TLS driver: fixed authentication mode anon + authentication mode "anon" was not properly supported in ossl TLS + driver; if selected, did still require a full certificate. + closes: https://github.com/rsyslog/rsyslog/issues/3037 +- bugfix tls subsystem: Receiver hang due to insufficient TLS buffersize. + gtls and ossl driver used a default buffersize of 8KiB to store received + TLS packets. When tls read returned more than buffersize, the additional + buffer was not processed until new data arrived on the socket again. + TLS RFCs require up to 16KiB+1 buffer size for a single TLS record. + closes https://github.com/rsyslog/rsyslog/issues/3325 +- bugfix pmpanngfw: build issue due to non-matching data types in comparison + Thanks to Narasimha Datta for the patch. +- omfile: work-around for "Bad file descriptor" errors + This works-around an issue we can reproduce e.g. via the + imtcp-tls-ossl-x509fingerprint.sh test. Here, omfile gets a write + error with reason EBADF. So far, I was not able to see an actual + coding error. However I traced this down to a multithreaded race + on open and close calls. I am very surprised to see this type + of issue, as I think the kernel guarantees that it does not happen. + Here is what I see in strace -f: + openssl accepts a socket: + [pid 66386] accept(4, {sa_family=AF_INET, sin_port=htons(59054), sin_addr=inet_addr("127.0.0.1")}, [128->16]) = 10 + then, it works a bit with that socket, detects a failure and shuts it down. Sometimes, at the very same instant omfile on another thread tries to open on output file. Then the following happens: + [pid 66386] close(10) = 0 + [pid 66389] openat(AT_FDCWD, "./rstb_356100_31fa9d20.out.log", O_WRONLY|O_CREAT|O_NOCTTY|O_APPEND|O_CLOEXEC, 0644 <unfinished ...> + [pid 66386] close(10 <unfinished ...> + [pid 66389] <... openat resumed> ) = 10 + [pid 66386] <... close resumed> ) = 0 + [pid 66386] poll([{fd=4, events=POLLIN}, {fd=5, events=POLLIN}], 2, -1 <unfinished ...> + [pid 66389] write(2, "file './rstb_356100_31fa9d20.out"..., 66file './rstb_356100_31fa9d20.out.log' opened as #10 with mode 420 + ) = 66 + [pid 66389] ioctl(10, TCGETS, 0x7f59aeb89540) = -1 EBADF (Bad file descriptor) + This is **literally** from the log, without deleting or reordering + lines. I read it so that there is a race between `open` and `close` + where fd 10 is reused, but seemingly closed - resulting in the `EBADF` + While it smells like a kernel issue, it may be a well-hidden program + bug - if so, one I currently do not find. HOWEVER, this commit + works around the issue by reopening the file when we receive EBADF. + That's the best thing to do in that case, especially if it really is + a kernel bug. Data loss should not occur, as the previous writes + succeeded in that case. + The backdraw of this work-around is that it only "fixes" omfile. In + theory every part of rsyslog can be affected by this issues (queue + files, for example). So this is not to be considered a final solution + of the root issues (but a big step forward for known problem cases). + see also https://github.com/rsyslog/rsyslog/issues/3404 +- omhttp bugfix: segfault due to NULL pointer access + many thanks to Gerardo Puerta for the patch +- omkafka bugfix: segfault when running in debug mode using dynamic topics + This should only affect test environments, as debug mode is not + suitable for production (and really does not work when running for + extended period of time). +- testbench bugfix: TLS syslog tests for "anon" mode were broken + They did not detect when "anon" mode was not properly supported by the + drivers. +- test tooling bugfix: correct tcpflood error messages + it looks like tcpflood's openssl code stems partly back to tcpdump, at + least the error messages indicate this. Thankfully tcpdump is BSD licensed, + so this should not be a big issue. Nevertheless, the incorrect program name + in error messages needs to be corrected, and this is what this commit does. +- tcpflood bugfix: tool did not terminate on certificate error + when tcpflood detected a certificate error, it reported an + error message but did not abort. This could make errors undetectable + during CI runs. + also fix tests which did not properly provide CA cert (which than + caused the error). +- testbench: fix issues with journal testing + The configure/Makefile checks were not correct, leading to the + build of journal components when not necessary, even if not + supported by the platform. Thus lead to invalid build and test + failures. +- testbench: add tests for "certless" tcp/tls + This adds a test to ensure that a client without certificate can + connect to a server with certificates. So it is not exactly + "certless". + The prime intent of this test is to match config suggestions given + by log hosting companies (like loggly) and so ensure that we do + not accidentally break them. This is especially important as the + capability for certless clients was not properly documented and + also become forgotten by the rsyslog team. + see also https://github.com/rsyslog/rsyslog/issues/3413 +- CI + - further improve testbench robustness against slow machines + - testbench: add tests for parser.EscapeControlCharacterTab global option + - testbench: Updated all expired x.509 certs + Closes https://github.com/rsyslog/rsyslog/issues/3348 + - fix a potential race in CI debug mode which can lead to segfault + only when instructed to do so, rsyslog may emit a "final worker thread shutdown" + messages. This is usually only enabled in CI and/or other testing. If enabled, + the code has a race on the pWti object which can lead to segfault or abort. + Only system which explicitly enable this CI aid are affected (running in debug + mode alone is NOT sufficient). + This is a regression from 8.40.0. + - testbench: improve robustness against slow CI, gen. improvements + * add an overall timeout value for tests - if running longer, + testbench framework tries to FAIL and end test. Note that + this is not bullet-proof and not intended to be so. + * guard against hanging rsyslog instances via a new imdiag + feature to abort after n number of seconds; among others, + this guards as against timeout-cancel in CI, which is always + pretty hard to diagnose - now we see these errors in test-suite.log + * fix a bug in tcp zip test, which actually did not use zip mode + * experimentally add debug output to better understand + shutdown_when_empty operation; goal is to improve understanding + and then remove that code again. + * improve shutdown predicate for a couple of tests + * made travis run make check with two parallel threads, for which + we seem ready now. Nevertheless, it's still experimental and we + may roll this back if required. + * testbench: disable omprog tests that hang under coverage instrumentation + When gcc coverage instrumentation is used, these tests hang. They work + with clang coverage instrumentation, but for some reason clang does not + give us full reports (at least not when used together with CodeCov.io). + We have tried to troubleshoot this for hours and hours - now is time to + give up until someone comes up with a bright idea. So we make the affected + tests skip themselves when they detect gcc with coverage instrumentation. + * testbench: add new test for imfile and logrotate in copytruncate mode + * testbench: add new omkafka tests for dynamic topics + * travis: do no longer run 0mq tests + This often causes trouble when the packages are rebuild by the 0mq project + (which happens frequently). We already do intensive testing of the 0mq + components in the buildbot infrastructure, where we use dedicated containers. + This is reliable, as the containers already contain everything needed and so + do not need to reach out to the 0mq package archives. In the light of this, + let's save us the trouble of Travis failures. The only downside is that + users cannot pre-test with their local Travis when modifying 0mq modules, + which is quite acceptable. +------------------------------------------------------------------------------ +Version 8.40.0 [v8-stable] 2018-12-11 +- mmkubernetes: add support for sslpartialchain for openssl + If `"on"`, this will set the OpenSSL certificate store flag + `X509_V_FLAG_PARTIAL_CHAIN`. This will allow you to verify the Kubernetes API + server cert with only an intermediate CA cert in your local trust store, rather + than having to have the entire intermediate CA + root CA chain in your local + trust store. See also `man s_client` - the `-partial_chain` flag. + This option is only available if rsyslog was built with support for OpenSSL and + only if the `X509_V_FLAG_PARTIAL_CHAIN` flag is available. If you attempt to + set this parameter on other platforms, you will get an `INFO` level log + message. This was done so that you could use the same configuration on + different platforms. +- openssl driver: improved error messages + also fixes misleading wording of some error messages + closes https://github.com/rsyslog/rsyslog/issues/3238 +- imfile: disable file vs directory error on symlinks + The file/directory node-object alignment now ignores symlinks. Previously + it reported error on each directory symlink spamming user error logs. + Thanks to Jiri Vymazal for the patch. +- cleanup: remove no longer needed --enable-rtinst code + configure option --enable-rtinst is gone-away since a while, but there were + still some supporting code left. It required careful analysis what could + actually be removed. This is now done and the code fully cleaned up. This + greatly simplifies the code and also makes it better readable for + developers which are not deep inside the rsyslog code base. + As a positive side effect, we could eliminate mutex calls inside + the debug system. This means we are more likely to reproduce race + conditions in runs with debugging enabled. + closes https://github.com/rsyslog/rsyslog/issues/2211 +- bugfix imfile: rsyslog re-sends data for files larger 2GiB + This occurs always if and only if + - reopenOnTruncate="on" is set + - file grows over 2GiB in size + Then, the data is continuously re-sent until the file becomes smaller + 2GiB (due to truncation) or is deleted. + It is a regression introduced by 2d15cbc8221e385c5aa821e4a851d7498ed81850 + closes https://github.com/rsyslog/rsyslog/issues/3249 +- config: fix segfault in backticks "echo" expansion of undefined variables + The bug was introduced in commit abe0434 (config: enhance backticks "echo" + capability). The getenv() result passed to strlen() and es_addBuf() may be + NULL if the environment variable does not exist, resulting in a segfault. + Thanks to Julien Thomas for the patch. + fixes https://github.com/rsyslog/rsyslog/issues/3006 +- bugfix imsolaris: message timestamps on Solaris + On Solaris messages don't have their time directly in the raw body but in + a separate log_ctl structure which is currently not used. + When message is logged and processed, rsyslogd gives it current time because + it ignores the actual one. That means that old messages (e.g. from system + reboot) get timestamp of processing instead of the reboot itself (it is + not a problem for live logging where now is used anyway). + Thanks to Jakub Kulik for the patch. +- bugfix build system: "make distcheck" did not work for mysql tests +- bugfix build system: don't link liblogging-stdlog when available but not enabled + When liblogging-stdlog was available but configure option "--disable-liblogging-stdlog" + was set, rsyslog was still linking against liblogging-stdlog. + This commit will ensure that rsyslog will only link against liblogging-stdlog when + "--enable-liblogging-stdlog" was set. + see also: https://bugs.gentoo.org/667836 +- bugfix RainerScript: abs() could return negative value, now in range [0..max] + Thanks to Harshvardhan Shrivastava for providing the patch +- bugfix debug output: date property options output wrongly + inside debug logging, the date property options were not all + properly converted into strings. Some of the newer ones were + invalidly flagged as "UNKNOWN". This is primarily a cosmetic + problem and has no effect other than puzzling folks looking at + the debug log. +- bugfix omhttp: did not compile on some platforms +- CI + * made mysql-based tests (ommysql and omlibdbi) work inside containers + * bugfix testbench: do not execute libgcrypt tests if disabled + closes https://github.com/rsyslog/rsyslog/issues/3228 + * testbench: grep failed when string starting with "-" was used + The search term was mistakenly interpreted as an option. + * testbench: support auto-start/-stop of mysqld + This is required to run mysql/mariadb tests inside containers. + closes https://github.com/rsyslog/rsyslog/issues/3223 + * improve bash coding style and fix a some bug in testbench + - duplicate diag.sh init call was not detected due to typo + - queue-persists test did not work correctly + - some general testbench framework improvements + issues found be shellcheck, fixes brought up other work to do + * testbench: improve journal tests and testbench framework + improving both style and reliability of journal tests; along that way + also improve testbench framework: + - do cleanup on error_exit and skip + - explicit skip handler (vs exit 77) + this permits us to do better cleanup + - new testbench functions for journal-specific functionality + reduce code duplication and make things easier to maintain in the + future + - provide a way to do valgrind and non-valgrind tests with a single + test file + see also https://github.com/rsyslog/rsyslog/issues/2564 + * testbench: improve framework, harden rscript http test + - the test now tries to detect unavailable http server, which + should not result in test failure + - equivalent valgrind test changed to new method, removing code + duplication + - testbench supports + * new exit code 177, which indicates environment error, makes + test SKIP but still reports the failure + * new exitcode, logurl stats reporting fields + * report buildbot builder (if provided) in failure report + * testbench: add test for mmjsonparse with unparsable data + * testbench: make es-bulk-retry test more reliable + We now no longer depend on a fixed 'sleep' command but rather + check the output file for what we expect. This is much more + robust on slow test machines. + We believe this closes the below-mentioned issue. If not, it + should be re-opened. + closes https://github.com/rsyslog/rsyslog/issues/3104 + * testbench: suppress valgrind error caused by pthreads lib + finally I give up and honestly think this is a problem in pthreads and + not in rsyslog code. See issue below and previous commit for more + information. + Unfortunately, this will also mask off cases where we do not properly + call pthread_join() albeit it is needed. Nevertheless, this bug is + causing so much CI grief that it is definitely worth it. + closes https://github.com/rsyslog/rsyslog/issues/2902 + * testbench: made a couple of (unnamed due to too many) test more robust + against slow (CI) machines +------------------------------------------------------------------------------ +Version 8.39.0 [v8-stable] 2018-10-30 +- imfile: improve truncation detection + previously, truncation was only detected at end of file. Especially with + busy files that could cause loss of data and possibly also stall imfile + reading. The new code now also checks during each read. Obviously, there + is some additional overhead associated with that, but this is unavoidable. + It still is highly recommended NOT to turn on "reopenOnTruncate" in imfile. + Note that there are also inherent reliability issues. There is no way to + "fix" these, as they are caused by races between the process(es) who truncate + and rsyslog reading the file. But with the new code, the "problem window" + should be much smaller and, more importantly, imfile should not stall. + see also https://github.com/rsyslog/rsyslog/issues/2659 + see also https://github.com/rsyslog/rsyslog/issues/1605 +- imjournal: work around journald excessive reloading behavior + This is workaround for possible imjournal interaction with systemd + where journal invalidate fix is not present. The code tries to + detect SD_JOURNAL_INVALIDATE loop and not reload after each call. + Thanks to Jiri Vymazal for the patch. +- errmsg: remove no longer needed code + refactored code (over a long time) so that object-ish style is no longer + needed and could now finally be removed; We also refactored the last + component (omhttp contrib module) that used the old interface. + closes https://github.com/rsyslog/rsyslog/issues/1684 +- queue bugfix: invalid error message on queue startup + due to some old regression (commit not exactly identified, but for + sure a regression, 9 years ago it was correct) an error message + is emitted when no .qi file exists on startup of the queue, which + is a normal condition. + Actually, the code should not have tried to open the .qi file in + the first place because it detected that it did not exist. That + (necessary) shortcut had been removed a while ago. + closes https://github.com/rsyslog/rsyslog/issues/3117 +- bugfix imrelp: regression with legacy configuration startup fail + Startup of a relp listener failed if legacy configuration was used. + caused by commit: 32b71daa8aadb8f16fe0ca2945e54d593f47a824 + closes https://github.com/rsyslog/rsyslog/issues/3106 +- bugfix imudp: stall of connection and/or potential segfault + There was a regression in 493279b790a8cdace8ccbc2c5136985e820dd2fa. + This regression may cause stop (or delay) of reception from some systems + and may also cause a segfault. Triggering condition is that at least + one listener could not be created. + Thanks to Jens Låås for the patch. +- bugfix gcry crypto driver: small memleak + If a crypto key is specified directly via the key="" parameter, + the storage for that key is not freed, causing a small memleak. + Note that the problem occurs only once per context, so this + should not cause real issues. Even more so, as specifying a + key directly is meant only for testing purposes and is strongly + discouraged for production use. + Detected by internal testing, no actual fail case known. +- fix potential misaddressing in encryption subsystem + could happen if e.g. disk queues were encrypted + not seen in practice but caught by testbench test +- ksi subsystem changes + * enhance debug logging + * disable unsafe SHA1 algorithm + Thanks to Allan Park for the patch. +- bugfix core: regex compile error messages could be incorrect +- bugfix core: potential hang on rsyslog termination + The root cause was a deadlock during worker startup. This could + happen for example when a DA queue needed to persist data during + shutdown. + Fail condition: + * startup request for a new worker + * initialization of that worker + * immediate detection that the worker can or must shutdown + * main thread waiting for worker running state, which it skips, + and so the main thread hangs inside a loop + closes https://github.com/rsyslog/rsyslog/issues/3094 +- bugfix imkafka: system hang when backgrounded + imkafka initializes librdkafka too early (before the fork). This leads + to hangs in various parts of the system - not only im imkafka but + other functions as well (e.g. getaddrinfo() calls). + closes https://github.com/rsyslog/rsyslog/issues/3180 +- bugfix imfile: file change was not reliably detected + A change in the inode was not detected under all circumstances, + most importantly not in some logrotate cases. + Includes new tests made by Andre Lorbach. They now use the + logrotate tool natively to reproduce the issue. + closes https://github.com/rsyslog/rsyslog/issues/2659 + closes https://github.com/rsyslog/rsyslog/issues/1605 +- bugfix imrelp: do not fail build if librelp does not have relpSrvSetLstnAddr + closes https://github.com/rsyslog/rsyslog/issues/2938 +- bugfix queue subsystem: DA queue did ignore encryption settings + closes https://github.com/rsyslog/rsyslog/issues/3066 + closes https://github.com/rsyslog/rsyslog/issues/2575 +- bugfix KSI: lmsig-ksils12 module skips signing the last block + Thanks to Allan Park for the patch. + closes https://github.com/rsyslog/rsyslog/issues/3105 +- bugfix fmhash: function hash64mod sometimes returned wrong result + Thanks to Harshvardhan Shrivastava for providing the patch + closes https://github.com/rsyslog/rsyslog/issues/3025 +- bugfix core/debug: data written to random fd 2 under some debug settings + This happens only during auto-backgrounding, where we cannot any longer + access stderr. Whatever is opened with fd2 receives some debug messages. + Note that the specific feature is usually turned on only in CI runs. +- cleanup: removed no longer needed code + Code that was unused for quite a while or did not really belong to the + project identified and removed. +- overall code cleanup + e.g. remove unused code, replace bad bash constructs, etc... +- CI: + * some small improvements in testbench plumbing + e.g. (`cmd` replaced by $(cmd), removed useless use of cat, ...) + * testbench: improve plumbing for kafka tests + - Removed all sleeps where possible. + - Moved all kafka start/stop/download logic into functions. + - Moved kafka/zookeeper stop into error_exit and exit_test. + - Kafka/Zookeeper cleanup only done on success now. + - Kafka/Zookeeper logfiles automatically dumped on error_exit only now. + - Added cleanup for Kafka/Zookeeper instances into CI/buildbot_cleanup.sh + - added new tests + * testbench: fix incompatibility of one omprog test with Python3 + Python3 writes to stderr immediately, and this caused the + captured output to differ with respect to Python2. Simplified + the test to do a single write to stderr. Also a cast to int + was needed when calculating 'numRepeats'. + closes https://github.com/rsyslog/rsyslog/issues/3030 + * testbench: fixed imfile parallel issues + - Fixed timing issues in some imfile wildcard/regex tests + - Added touch command in imfile wildcard tests to make sure directories + exist before files are created in it if IO is under stress. + - changed content checking in some tests to use "content_check_with_count" + with check timeouts instead of using fixed sleeptimes. + * testbench: new basic tests + These ensure that for some modules that did not have any tests at all + we have at least a minimal coverage (module loads, activates, is able + to emit error messages). Of course, further improvements would make + much sense. Modules: + - ommail + - testbench: new tests for disk queue encryption + - testbench: improved auto-diagnostics for hanging instance + - testbench: hardened kafka test against failing kafka subsystem, + not in 100% of the cases, but at least in some that frequently occur + - failing tests now report failure status so that we can get stats + on unreliable tests + - testbench tooling: fix incorrect tcpflood TLS parameter check + could lead to segfault when started + - bugfix testbench tooling: tcpflood invalid type in calloc (openssl mode) + It is unlikely that this has caused a real issue, as long as pointers + are all of the same size (what is highly probable). + detected by cppcheck via Codacy.com +------------------------------------------------------------------------------ +Version 8.38.0 [v8-stable] 2018-09-18 +- AIX: make basic modules work again +- make rsyslog build on AIX again + ... at least for a limited set of default modules +- imfile: support for endmsg.regex + This adds support for endmsg.regex. It is similar to + startmsg.regex except that it matches the line that denotes + the end of the message, rather than the start of the next message. + This is primarily for container log file use cases such as this: + date stdout P start of message + date stdout P middle of message + date stdout F end of message + The `F` means this is the line which contains the final part of + the message. The fully assembled message should be + `start of message middle of message end of message`. + `startmsg.regex="^[^ ]+ stdout F "` will match. + Thanks to Richard Megginson for the patch. +- imkafka: add parameter "parseHostName" + This enables imkafka to parse the hostname from log message. + Previously that was not possible. It was most likely a bug, but + one that users may count on. The new parameter "ParseHostName" + (default is off) controls this behavior. Default is to NOT + parse the hostname. + Thanks to github user snaix for the contribution. +- im[p]tcp: improve error message on connect failure + Now a message with the actual OS error is emitted, making things far + easier to troubleshoot. +- imkafka: implement multithreading support for kafka consumers. + Each consumer runs in it's own consumer thread now. New tests have also + been added for this. +- omelasticsearch: write all header metadata to $.omes for retries + Write all of the original request metadata fields to $.omes for + the retry, if present. This may include all of the following: + _index, _type, _id, _parent, pipeline + This is in addition to the fields from the response. If the same + field name exists in the request metadata and the response, the + field from the request will be used, in order to facilitate + retrying the exact same request. + Thanks to Richard Megginson for the patch. +- core: improve error message on module load fail + The error message now lists all dlopen() errors in depth. This is + especially useful if the error is due to missing symbols or file + format errors. +- core/queue: add error message if queue file cannot be accessed + When having a disk-assisted queue without permission to write to the specified + queue file an error will now be generated. + closes https://github.com/rsyslog/rsyslog/issues/323 +- imtcp/imudp: new option preservecase for managing the case of FROMHOST value + default is left at current behavior + see also https://github.com/rsyslog/rsyslog/pull/2774 + see also https://bugzilla.redhat.com/show_bug.cgi?id=1309698 +- omprog: add feedback timeout and keep-alive feature + - Restart the program if it does not respond within timeout. + - New setting 'confirmTimeout' (default 10 seconds). + - Allow the program to provide keep-alive feedback when a + message requires long-running processing. + - Improve efficiency when reading feedback line (use buffer). + Retry interrupted writes/reads to/from pipe. + - New setting 'reportFailures' for reporting error messages + from the program. + - Report child termination when writing to pipe. + - Minor refactor: renamed writePipe function to sendMessage, + renamed readPipe to readStatus. + Thanks to Joan Sala for contributing this. +- omprog: fix forceSingleInstance configuration option + The forceSingleInstance option did not work as intended. Even + if set multiple instances were spawned. This most probably + was a regression from 0453b1670fc34c96d31ee7c9a370f0f5ec24744a + The code was broken roughly 3.5yrs ago, so it looks like the + issue was little-noticed. This also means that potentially some users + may see the bugfix as change of behavior. If so, just remove + the option. + closes https://github.com/rsyslog/rsyslog/issues/2813 + closes https://github.com/rsyslog/rsyslog/issues/2468 + Thanks to Joan Sala for contributing this. +- imfile: implement file-id, used in state file + This ensures that files with the same inodes are not accidentally treated + as equal, at least within the limits of the file id hash (see doc for + details). + We use the siphash reference implementation to generate our non-cryptographic + hash. + closes https://github.com/rsyslog/rsyslog/issues/2530 + closes https://github.com/rsyslog/rsyslog/issues/2231 +- imfile: experimental input throttling feature + The new input parameter delay.message has been added. It specifies + a delay in microseconds after each line read. + closes https://github.com/rsyslog/rsyslog/issues/2960 +- core: emit TZ warning on startup not on Linux non-container + On Linux it seems common that the TZ variable is NOT properly set. + There are some concerns that the warning related to rsyslog correcting + this confuses users. It also seems that the corrective action rsyslog + takes is right, and so there is no hard need to inform users on that. + In Linux containers, however, the warning seems to be useful as the + timezone setup there seems to be frequently-enough different and + rsyslog's corrective action may not be correct. + So we now check if we are running under Linux and not within a container. + If so, we do not emit the warning. In all other case, we do. This is + based on the assumption that other unixoid systems still should have + TZ properly set. + closes https://github.com/rsyslog/rsyslog/issues/2994 +- omkafka: + * better debug information + * Fixed minor issue in omkafka producing wrong kafka timestamps when + msgTimestamp was NULL. + * Setting RD_KAFKA_V_KEY(NULL, 0) in rd_kafka_producev now when KEY is not + configured. + * Fixed minor issue when rsyslog is compiled with --enable-debug and + librdkafka is too old. +- omfile bugfix: errant error message when dynafile param needed + also fixes related message in contributed module omfile-hardened + closes https://github.com/rsyslog/rsyslog/issues/2975 + Thanks to Frank Bicknell for the patch +- omhttp: new contributed module + Thanks to Christian Tramnitz for contributing it. + Some more info at https://github.com/rsyslog/rsyslog/pull/2782 +- mmkubernetes: action fails preparation cycle if kubernetes API ... + ... destroys resource during bootup sequence + The plugin was not handling 404 Not Found correctly when looking + up pods and namespaces. In this case, we assume the pod/namespace + was deleted, annotate the record with whatever metadata we have, + and cache the fact that the pod/namespace is missing so we don't + attempt to look it up again. + In addition, the plugin was not handling error 429 Busy correctly. + In this case, it should also annotate the record with whatever + metadata it has, and _not_ cache anything. By default the plugin + will retry every 5 seconds to connect to Kubernetes. This + behavior is controlled by the new config param `busyretryinterval`. + This commit also adds impstats counters so that admins can + view the state of the plugin to see if the lookups are working + or are returning errors. The stats are reported per-instance + or per-action to facilitate using multiple different actions + for different Kubernetes servers. + This commit also adds support for client cert auth to + Kubernetes via the two new config params `tls.mycert` and + `tls.myprivkey`. + Thanks to Richard Megginson for the patch. +- bugfix pmnormalize/core: several memory leaks, invld property handling + - major memory leak which occurred once per message processed + So this could lead to OOM. Caused by improper free of json + structure + - another two major leaks of similar magnitude could occur if + "fromhost-ip" and/or "fromhost" properties were set + - minor leaks upon termination. these were unproblematic as + static and only occurred immediately before shutdown. + But they triggered memory debugger errors. + - fixed test which did not check for mem leaks albeit it should + - core invalid handling of the "fromhost" property, if set via + the MsgSetPropsViaJSON() call. This was primarily of concern + for pmnormalize and mmexternal, and only if these properties + were used by either the rulebase or the external program + response. + Actually, most of the leaks go back to rsyslog core, but that + core functionality was not used by other modules in the same + way. But if some other would have used it, the effects would + have been the same (so be aware if you wrote custom modules). +- bugfix imptcp: fixed pointers for session counting + imptcp open, failedopen, and closed pstats counters were assigned the wrong + name, thus pstats values did provide a totally wrong picture of what was + going on. + Thanks to github user jeverakes for the patch. +- bugfix omprog: invalid memory access on partial writes to pipe + When sending logs to the program, in case of a partial write to the pipe, + invalid data was sent, or an invalid memory access could occur. (A + partial write can occur if the syscall is interrupted or the pipe is full.) + Thanks to Joan Sala for contributing this. +- bugfix omprog: rsyslog's environment was not passed to script + closes https://github.com/rsyslog/rsyslog/issues/2921 +- bugfix omprog: severity of some log messages in waitForChild corrected + Log some messages related to child process termination as info/warn + instead of error. +- bugfix imfile: files which were loaded via symlink were not always followed + They were stopped watching after being rotated. + closes https://github.com/rsyslog/rsyslog/issues/2229 + Thanks to Jiri Vymazal for the patch. +- bugfix imfile: potential misaddressing when processing symlinks + Fixed parent name when processing symlinks. Detected during code review. + There was a garbage byte left before which could cause errors down the + road. + Thanks to Jiri Vymazal for the patch. +- bugfix ommongodb: build issue if mongo-c-driver is not compiled with TLS + Let ommongodb module works even if mongo-c-driver is not compiled with SSL support. + Thanks to Jérémie Jourdin for the patch. + closes https://github.com/rsyslog/rsyslog/issues/2907 +- CI: + * many changes with the goal to support parallel test execution, e.g. + use dynamic ports and file names, changes to testing tools, etc. + * kafka tests re-enabled, as they should now no longer be racy. However, + this has yet to be proven in practice. + * upgrading kafka server version to current + * Fixed server configuration issues holding the kafka tests back from working + * Fixed some config issues in all sndrcv kafka tests. + * Generating dynamically kafka topics now for each kafka test. + * Reenabled kafka_multi test which runs a test on 3 kafka/zookeeper instances +------------------------------------------------------------------------------ +Version 8.37.0 [v8-stable] 2018-08-07 +- build system: add --enable-default-tests ./configure option + This permits to control the "default tests" in testbench runs. These + are those tests that do not need a special configure option. There are + some situations where we really want to turn them of so that we can + run tests only for a specific component (e.g. ElasticSearch). + This commit also removes the --enable-testbench[12] configure switches, + which were introduced just to work-around travis runtime restrictions. + With the new CI setup and new options we could reduce the Travis runtime + dramatically and so we do not need them any longer. +- overall adaptation to gcc 8 which emits new warnings +- fix some build warnings on 32bit systems, namely armhf architecture +- ommail change of behavior: "enable.body" default now "on" + This was always documented to be "on", but actually was "off". Usually, we + fix the doc, but after long discussion the agreement was that in this + specific case it was actually better to change the default. + see also: https://github.com/rsyslog/rsyslog/pull/2791 +- core/omfile: race in async writing mode + mutex was not properly locked at all times when the async writing buffer + was flushed + Thanks to Radovan Sroka for the patch. +- core: provide a somewhat better default action name + We now include the module name (e.g. "omelasticsearch" or "builtin:omfile") + as part of the name. This is still not perfect, but hopefully a bit + easier to grasp. + see also https://github.com/rsyslog/rsyslog/issues/342 +- new global() parameter "abortOnUncleanConfig" + This provides a new-style alternative to $AbortOnUncleanConfig. + closes https://github.com/rsyslog/rsyslog/issues/2744 +- tcpflood no longer links with -lgrcypt + as this is no longer necessary for GnuTLS + Thanks to Michael Biebl for the patch. +- imjournal: add journal-specific impstats counters + these provide some additional insight into journal operations + Thanks to Abdul Waheed for the patch. +- imjournal: fixed startup on missing state file + When starting rsyslog with imjournal for first time it outputs + an error and plugin does not run because no state file exists yet. + Now it skips the loading and creates state file on first persist. + Thanks to Jiri Vymazal for the patch. +- imjournal: fetching cursor on readJournal() and simplified pollJournal() + Fetching journal cursor in persistJournal could cause us to save + invalid cursor leading to duplicating messages further on, when new + WorkAroundJournalBug option is set we are saving it on each + readJournal() where we now that the state is good. + pollJournal() is now cleaner and faster, correctly handles INVALIDATE + status from journald and is able to continue polling after journal + flush. Also reduced POLL_TIMEOUT a bit as it caused rsyslog to exit + with error in corner cases for some ppc when left at full second. + re-factored imjournal CI tests with journal_print tool to have more + detailed error reporting. + Thanks to Jiri Vymazal for the patch. +- config: enhance backticks "echo" capability + This is now more along the lines of what bash does. We now support + multiple environment variable expansions as well as constant text + between them. + example: + env SOMEPATH is set to "/var/log/custompath" + config is: param=`echo $SOMEPATH/myfile` + param than is expanded to "/var/log/custompath/myfile" + among others, this is also needed inside the testbench to properly + support "make distcheck". + Note: testbench tests follows via separate commit. There will be + no special test, as the testbench itself requires the functionality + at several places, so the coverage will be very good even without + a dedicated test. +- imrelp: add support for setting address to bind to (#894) + This adds a new optional `address` parameter to `imrelp` inputs in order + to specify an address to bind to. + Based on support added by rsyslog/librelp@96eb5be + Thanks to Simon Wachter for the patch. +- omrelp: permit all authmodes; updated tests + omrelp for some time limited authentication modes to those + that were known. While this was OK, it prevented the easy + introduction of new auth modes into librel. + This has now been changed; omrelp now checks the validity of + the authmode directly via librelp by doing some librelp calls + upon processing the configuration. + Also, some tests have been updated to check this feature and + also ensure that the new librelp mode "certvalid" works + (if it is available). +- regexp.c: reduce lock contention when using glibc. + When using glibc, we enable per-thread regex to avoid lock contention. + See: + - https://github.com/rsyslog/rsyslog/issues/2759 + - https://github.com/rsyslog/rsyslog/pull/2786 + - https://sourceware.org/bugzilla/show_bug.cgi?id=11159 + This should not affect BSD as they don't seem to take a lock in regexec. + NOTE: it is assumed that we can craft an even better solution than + this patch, but it improves the situation and we do not have time to + craft more. So we decided to merge. For details see + https://github.com/rsyslog/rsyslog/pull/2786 +- mmpstrucdata: better error message, support $! in var names + see also https://github.com/rsyslog/rsyslog/issues/1262#issuecomment-404773495 +- more explicit error msg with message modification mod on queue + Message modification modules do not work if used with a non-direct queue. + We now make this more explicit in the config parsing error message. + closes https://github.com/rsyslog/rsyslog/issues/1323 +- omrabbitmq: improve high-load performance + A different pthread mutex is created for each connection (action) + instead of a single one shared by all connections. This will + improve performance when using multiple concurrent connections + to a single (or multiple) RabbitMQ instance(s) (e.g. for load balancing) + Thanks to github user micoq for contributing the patch. +- imudp: replace select() calls by poll() + This improves reliability in extreme cases (more than 1024 fds open when + imudp begins to listen) and potentially improves performance a little. +- ommysql: support mysql unix domain socket: + via action(.. socket="/tmp/mysqld.sock" ..) + Thanks to JoungKyun Kim for contributing this. +- impstats: emit warning if log.syslog="off" and ruleset name given + With this config, "ruleset" is silently ignored, what probably is + not obvious to a user. + closes https://github.com/rsyslog/rsyslog/issues/2821 +- build system cleanup: remove no longer needed --enable-memcheck + This was used for a very old testing capability, no longer functional but + causes build to fail if enabled. Replaced by ASAN/valgrind. + Issue detected while testing some other CI settings. +- tools: Updated python based statslog analyzer sample scripts +- developer tools: make devcontainer tool more developer friendly + slight improvement for easy interactive use +- enable better testing via "make distcheck" + Also a couple of changes to testbench worth mentioning: + * use cp -f to ensure files can be overwritten in VBUILD + * fix issue of missing include test file in EXTRA_DIST + * new suppressions + * testbench: try to use local system dependency cache + avoid going to Internet repos if not absolutely necessary. For + development containers, they should be pre-populated with the + important dependencies. + * do not enable libfaketime if ASAN is selected + unfortunately, libfaketime does not work in that case + Note: for modules with non-standard dependencies (e.g. databases), + "make distcheck" only enables what on the original ./configure line + was enabled. This is done in order to ensure that "distcheck" adapts + to what is actually available on the system in question. Rsyslog's + own CI system installs the maximum set of possible dependencies and + so tries the maximum set "make distcheck" can support on a platform. + see also https://github.com/rsyslog/rsyslog/issues/174 +- add new global config parameter "inputs.timeout.shutdown" +- omusrmsg: do not fall back to max username length of 8 + This happens if utmp.h and friends are not available and stems back to + the original syslogd. Nowadas, 32 is more appropriate and now being used + in that (now very unlikely) case. The detection logic for UT_NAMESIZE has + also been streamlined. + closes https://github.com/rsyslog/rsyslog/issues/2834 +- bugfix build system: fix race in parallel builds + If libgcry.la is built later than lmcry_gcry.la, there is a failure: + [snip] + |../aarch64-wrs-linux-libtool --tag=CC --mode=link aarch64-wrs-linux-gcc + -o lmcry_gcry.la lmcry_gcry_la-lmcry_gcry.lo libgcry.la -lgcrypt + |aarch64-wrs-linux-libtool: error: cannot find the library 'libgcry.la' + or unhandled argument 'libgcry.la' + |Makefile:1049: recipe for target 'lmcry_gcry.la' failed + |make[2]: *** [lmcry_gcry.la] Error 1 + [snip] + The LIBADD of lmcry_gcry.la contains libgcry.la, we should also add libgcry.la + to lmcry_gcry.la's DEPENDENCIES. + Thanks to Hongxu Jia for the patch. +- bugfix imfile: memory leak upon shutdown (cosmetic) + When rsyslog shuts down and imfile is inside a change polling loop, + it does not properly free memory returned by glob(). This is a cosmetic + bug as the process terminates within the next few milliseconds. However, + it causes memory analyzer reports and thus makes CI fail. +- bugfix core msg: potential deadlock (and rsyslog hang) + can happen e.g. with headerless messages when app-name + property is used + closes https://github.com/rsyslog/rsyslog/issues/3135 +- bugfix core: do not abort startup on problems setting scheduling policy + rsyslog creates a default scheduling policy on startup. This code + invalidly used CHKiRet (our exception handler) to check pthreads + return codes, what this macro cannot do. This lead to hard to + diagnose startup problems in cases where there were problems + setting the scheduling defaults (e.g. when rsyslog is set to run + at idle priority). Even more so, this blocked startup altogether, + which is not the right thing to do. Actually, this can be considered + a regression from commit 7742b21. That commit was 8 years ago, so + in general this cannot be a big issues ;-) + The code now emits proper error messages (to stderr, as at this point + no other output is available as it is during the initial state of + rsyslog initialization) and continues the startup. + closes https://github.com/rsyslog/rsyslog/issues/2855 +- bugfix core: input shutdown timeout not properly applied + The timeout could be reduced by mutex wait time, which was not the + intended behavior and could lead the the input thread being + cancelled while it would have been perfectly legal to shut it down + cleanly. + Noticed during working on the CI system. May explain some testbench + instability and may have caused trouble with state files (not) + properly being written by inputs. +- bugfix config optimizer: error in constant folding + did not work properly if a string and a number were to be folded. + Detected by gcc 8. +- build: fix improper function casts + no real issue, but generated warnings under gcc 8 and thus + broke CI +- bugfix omlibdbi: fix potential small memory leak + detected by clang static analyzer +- bugfix ommysql: unsafe use of strncpy() + also now reports oversize names as user error vs. silent truncation + overly long names only could affect config load phase +- bugfix omhttpfs: fix insecure usage if strncmp() + consequences not evaluated as this is a contributed module. + Detected by gcc 8. +- bugfix mmgrok: cosmetic build issue - compiler warnings + caused build under gcc 7 to fail with warning +- bugfix mmkubernetes: stops working with non-kubernetes container names + When mmkubernetes encounters a record with a CONTAINER_NAME field, + but the value does not match the rulebase, mmkubernetes returns + an error, and mmkubernetes does not do any further processing + of any records. + The fix is to check the return value of ln_normalize to see if + it is a "hard" error or a "does not match" error. + This also adds a test for pod names with dots in them. + Thanks to Richard Megginson for the patch. +- bugfix mmkubernetes: potential NULL pointer access + If token file could not be opened, fclose() was passed a NULL pointer. + Thanks to github user jvymazal for finding and Richard Megginson + for fixing the issue. +- bugfix omsnmp: invalid traptype was not detected + this could leave config errors unreported and cause unexpected + behavior +- bugfix mmkubernetes: default rules use container_name_and_id + also include rulebase files in dist and fix rule so that dot inside + pod name is supported. + Thanks to Richard Megginson for fixing the issue. +- bugfix omelasticsearch: build regression + Commit 6d4635efbb13907bf651b1a6e5a545effe84d9d9 introduced some compile + problems, which were only detected on CentOS6, which unfortunately did + not compile omelasticsearch during CI runs +- bugfix ommongodb: do not force MongoDB to use "PLAIN" auth mechanism + ... which also seems not to be handled by current MongoDB. + Remove ?authMechanism=PLAIN URI part to let the mongo library chooses the + default mechanism. One can force a specific authentication mechanism by + adding ?authMechanism=XXX into the uristr argument of the module + Thanks to Jérémie Jourdin for the fix. + closes https://github.com/rsyslog/rsyslog/issues/2753 +- build system: do not disable tests via --disable-liblogging-stdlog + This setting controlled both the actual rsyslog functionality as well + as some testbench tests, which use liblogging-stdlog to provide some + specific functionality. This meant those tests were not run since + changing the default. Now untangling the dependency. +- CI: + * most test refactored to use newer testbench plumbing + while no functional change, this permits further enhancements + * ElasticSearch startup timeout in tests increased to care for + slower test systems + * imjournal: fixed tests to actually test plugin functionality + Thanks to Jiri Vymazal for the patch. + * new test for gnutls priority string in librelp + Thanks to github user jvymazal for the patch + * testbench: relax hanging instance detection + This does not work reliably if multiple instances of rsyslog + builds run on a single machine. We need to improve, but this + commit makes conflict less likely and provides some diagnostic + info to help guide us towards a final solution. + * testbench: fix tests that look awfully wrong + These tests indicated they terminate rsyslog forcefully without + draining the queues, but then checked if they were drained (all + messages processed). That does not make sense, and we cannot + envision why this was written the first place. So we assume some + copy&paste problem was the root of that. + * testbench: refactor tests which used "nettester" tool + Some old tests are carried out via the nettester tool. This was + our initial shot at a testbench a couple of years ago. While it + worked back then, the testbench framework has been much enhanced. + These old tests are nowadays very hard to handle, as they miss + debug support etc. So it is time to refactor them to new style. + As a side-activity, the testbench plumbing has been enhanced to + support some operations commonly needed by these tests. Contrary + to pre-existing plumbing, these new operations are now crafted + using bash functions, which we consider superior to the current + method. So this is also the start of converting the older-style + functionality into bash functions. We just did this now because + it was required and we entangled it into the test refactoring + because it was really needed. Else we had to write old-style + operations and convert them in another commit, which would + have been a waste of time. + Special thanks to Pascal Withopf for the initial step of taking + old tests and putting config as well as test data together into + the refactored tests, on which Rainer Gerhards than could build + to create the new tests and update testbench plumbing. + * testbench: ensure uxsock test leaves no dangling listener instances + ..in case the test aborts. We utilize the timeout utility for now + to prevent this. + * testbench: make port for imdiag dynamic + This is prep work to support parallel test runs +------------------------------------------------------------------------------ +Version 8.36.0 [v8-stable] 2018-06-26 +- build system change: + Liblogging-stdlog was introduced to provide a broader ability to send rsyslog + internal logs to different sources. However, most distros did not pick up + that capability and so instead we do a regular syslog() call. We assume that + the actual functionality is never used in practice, so we plan to retire it. + That makes building rsyslog from source easier. + The plan is to disable use of liblogging-stdlog by default during + configure. So users (and distros!) can still opt-in to have it enabled if + they desire. + A couple of releases later, we want to completely remove the functionality, + except if there has desire been shown in the meantime which justifies to keep + liblogging-stdlog. + This version disabled liblogging-stdlog by default. We now also + emit a warning message ("liblogging-stdlog will go away") so that users + know what is going on and my react. + closes https://github.com/rsyslog/rsyslog/issues/2705 + see also https://github.com/rsyslog/rsyslog/issues/2706 +- add openssl driver alongside GnuTLS one for TLS communication + The openssl driver is currently experimental. It will become the new preferred + driver as it permits us to provide much better end-user error message than + we could provide with GnuTLS. It is also less picky with certificate files + and provides specific error messages if there are certificate problems. + closes: https://github.com/rsyslog/rsyslog/issues/1390 + closes: https://github.com/rsyslog/rsyslog/issues/1840 + closes: https://github.com/rsyslog/rsyslog/issues/1352 + closes: https://github.com/rsyslog/rsyslog/issues/1702 + closes: https://github.com/rsyslog/rsyslog/issues/2547 +- GnuTLS TLS driver: support intermediate certificates + this is necessary for certificate chains + Thanks to Arne Nordmark for providing the patch. + closes https://github.com/rsyslog/rsyslog/issues/2762 +- omelasticsearch: write op types; bulk rejection retries + * Add support for a 'create' write operation type in addition to + the default 'index'. Using create allows specifying a unique id + for each record, and allows duplicate document detection. + * Add support for checking each record returned in a bulk index + request response. Allow specifying a ruleset to send each failed + record to. Add a local variable `omes` which contains the + information in the error response, so that users can control how + to handle responses e.g. retry, or send to an error file. + * Add support for response stats - count successes, duplicates, and + different types of failures. + * Add testing for bulk index rejections. + Thanks to Richard Megginson for the patch. +- lookup tables: reload message now with "info" severity (was "error") + thanks to Adam Chalkley for the patch +- imptcp: add support for regex-based framing + for complex multi-line messages (XML in particular), the multiLine method + does not work well. We now have a capability to specify via a regex when + a frame starts (and the previous thus ends). + adds imptcp input parameter "framing.delimiter.regex" +- imjournal: add statistics counter + following statistics counter are now supported by imjournal + - submitted = total number of messages submitted for processing + closes https://github.com/rsyslog/rsyslog/issues/2549 +- config: permit 4-digit file creation modes + permit 4-digit file creation modes (actually 5 with the leading zero) so + that the setgid bit can also be set (and anything else on that position. + closes https://github.com/rsyslog/rsyslog/issues/1092 +- ommongodb: add possibility to ignore some insertion error code + new config parameter "allowed_error_codes", which will be ignored if + they happen. For example, 11000 DuplicateKey in case of collection + containing a unique field. + Thanks to Hugo Soszynski for contributing this work +- omprog: simplify 'plugin-with-feedback.py' example + Make the skeleton easier to understand by removing transaction support. + Also, transaction failures did not work as explained in the skeleton, + because of issue #2420. In the future, a 'plugin-with-transactions.py' + example can be added, ideally once the issue is solved. + Thanks to Joan Sala for contributing this. +- core: misaddressing when writing disk queue files + when writing disk queue files during shutdown, access to freed + memory can occur under these circumstances: + - action A is processing data, but could not complete it + most importantly, the current in-process batch needs not to + be totally completed. Most probable cause for this scenario + is a suspended action in retry mode. + - action A is called from a ruleset RA which + - does not have a queue assigned + - where RA is called from a ruleset RO which is bound + to the input from which the message originated + - RO must be defined before RA inside the expanded config + - Disk queues (or the disk part of a DA queue) must be utilized by A + When re-injecting the unprocessed messages from A into the disk queue, the + name of ruleset RO is accessed (for persisting to disk). However, RO is + already destructed at this point in time. + The patch changes the shutdown processing of rulesets, so that all + shutdown processing is done before any ruleset data is destructed. This + ensures that all data items which potentially need to be accessed + remain valid as long as some part may potentially try to access them. + This follows a the approach used in + https://github.com/rsyslog/rsyslog/pull/1857 + where obviously that part of the problem was not noticed. + see also https://github.com/rsyslog/rsyslog/issues/1122 + closes https://github.com/rsyslog/rsyslog/issues/2742 +- core: fix message loss on target unavailability during shutdown + Triggering condition: + - action queue in disk mode (or DA) + - batch is being processed by failed action in retry mode + - rsyslog is shut down without resuming action + In these cases messages may be lost by not properly writing them + back to the disk queue. + closes https://github.com/rsyslog/rsyslog/issues/2760 +- imrelp bugfix: error message "librelp too old" is always emitted ... + ... even if librelp is current. The condition check was actually missing. + This commit adds it. + closes https://github.com/rsyslog/rsyslog/issues/2712 +- imrelp: segfault on startup when cert without priv key is configured + closes https://github.com/rsyslog/rsyslog/issues/2747 +- omrelp bugfix: segfault on first message sent when authmode was wrong + A segfault could occur if the authmode was configured to an invalid value. + This is now caught during config processing and an error is reported. + closes https://github.com/rsyslog/rsyslog/issues/2743 +- imfile bugfix: double-free on module shutdown + detected by code review, not seen in practice +- imfile/core bugfix: potential misaddressing in string copy routine + This can be exposed via imfile, as follows: + - use a regex to process multiline messages + - configure timeouts + - make sure imfile reads a partial message + - wait so that at least one timeout occurs + - add the message termination sequence + This leads to a misaddressing, which may have no obvious effects potentially + up to a segfault. + closes https://github.com/rsyslog/rsyslog/issues/2661 +- imfile bugfix: if freshStartTail is set some initial file lines missing + When the option is set and a new file is created after rsyslog startup, + freshStartTail is also applied to it. That is data written quickly to it + (before rsyslog can process it) will potentially be discarded. If so, + and how much, depends on the timing between rsyslog and the logging process. + This problem is most likely to be seen in polling mode, where a relatively + long time may be required for rsyslog to find the new file. + This is changed so that now freshStartTail only applies to files that + are already-existing during rsyslog's initial processing of the file + monitors. HOWEVER, depending on the number and location (network?) of + existing files, this initial startup processing may take some time as + well. If another process creates a new file at exactly the time of + startup processing and writes data to it, rsyslog might detect this + file and it's data as prexisting and may skip it. This race is inevitable. + So when freshStartTail is used, some risk of data loss exists. The same + holds true if between the last shutdown of rsyslog and its restart log + file content has been added. This is no rsyslog bug if it occurs. + As such, the rsyslog team advises against activating the freshStartTail + option. + closes https://github.com/rsyslog/rsyslog/issues/2464 +- core: fix undefined behavior (unsigned computation may lead to value < 0) + This was detected by LLVM UBSAN. On some platforms re-setting the rawmsg + inside the message object could lead to invalid computation due to the + fact the the computation was carried out as unsigned and only then + converted to integer. + No known problem in practice. +- CI/QA: + - improved Elasticsearch tests so they can now be run without system- + installed ES service; also enables us to specify specific ES versions + and should now make the tests executable inside a container +------------------------------------------------------------------------------ +Version 8.35.0 [v8-stable] 2018-05-15 +- imptcp: add ability to configure socket backlog + this can be useful under heavy load. + For a detailed discussion see https://github.com/rsyslog/rsyslog/pull/2561 + Thanks to Maxime Graff for implementing this. +- omfile: do not permit filename that only consists of whitespace +- fmhash: new hash function module + implements hash32() and hash64() functions + Thanks to Harshvardhan Shrivastava for implementing these +- some better error messages +- imklog: add ratelimiting capability + On Linux kernel logs are ratelimited only for messages using + printk_ratelimit(). Some logs do not use this facility, so + we ratelimit kernel ourselves. + Thanks to Berend De Schouwer for the patch. +- omkafka: added impstats counters for librdkafka returned statistics + Adds: + * statscallback counters + * librdkafka failure and error counters + * acked message counter + Thanks to Abdul Waheed for implementing this. +- imudp + * use rsyslog message rate-limiter instead of home-grown one + imudp introduced it's own (feature-limited) rate-limiting capability for + message on disallowed senders before we had central rate-limiters + inside rsyslog. Also, that code evolved from running on a single + thread to running on multiple threads, which introduced data races + and so made unreliable. + Now we removed the old rate-limiting capability and depend on the + system rate limiter for internal rsyslog messages. + closes https://github.com/rsyslog/rsyslog/issues/2467 + * add stats counter "disallowed" + counts the number of messages discarded due to being received from + disallowed senders + see also https://github.com/rsyslog/rsyslog/issues/2467 +- imrelp: add parameter "oversizeMode" + Permits to instruct librelp how to handle oversize messages. The new default + is to truncate messages. Previously, the connection was aborted, what often + lead to stuck messages at the sender side. Now, there are three options passed + down to librelp: + * abort - same behavior as previously, connection is aborted on error + * truncate - do not abort but instead truncate oversize message to + configured max size + * accept - accept all oversize messages (note: this can cause security issues, + see doc for details) + see also https://github.com/rsyslog/rsyslog/pull/1525#issuecomment-384179393 + see also https://github.com/rsyslog/rsyslog/issues/2190 + closes https://github.com/rsyslog/rsyslog/issues/2633 + closes https://github.com/rsyslog/rsyslog/issues/1741 + closes https://github.com/rsyslog/rsyslog/issues/1782 + closes https://github.com/rsyslog/rsyslog/issues/2496 +- core: consistent handling of oversize input messages + In the community we frequently discuss handling of oversize messages. + David Lang rightfully suggested to create a central capability inside + rsyslog core to handle them. + We need to make a distinction between input and output messages. Also, + input messages frequently need to have some size restrictions done at + a lower layer (e.g. protocol layer) for security reasons. Nevertheless, + we should have a central capability + * for cases where it need not be handled at a lower level + * as a safeguard when a module invalidly emits it (imfile is an example, + see https://github.com/rsyslog/rsyslog/pull/2632 for a try to fix it + on the module level - we will replace that with the new capability + described here). + The central capability works on message submission, and so cannot be + circumvented. It has these capabilities: + * oversize message handling modes: + - truncate message + - split message + this is of questionable use, but also often requested. In that mode, + the oversize message content is split into multiple messages. Usually, + this ends up with message segments where all but the first is lost + anyhow as the regular filter rules do not match the other fragments. + As it is requested, we still implemented it. + - accept message as is, even if oversize + This may be required for some cases. Most importantly, it makes + quite some sense when writing messages to file, where oversize + does not matter (accept from a DoS PoV). + * report message to a special "oversize message log file" (not via the + regular engine, as that would obviously cause another oversize message) + This commit, as the title says, handles oversize INPUT messages. + see also https://github.com/rsyslog/rsyslog/issues/2190 + closes https://github.com/rsyslog/rsyslog/issues/2681 + closes https://github.com/rsyslog/rsyslog/issues/498 + Note: this commit adds global parameters: + * "oversizemsg.errorfile", + is used to specify the location of the oversize message log file. + * "oversizemsg.report", + is used to control if an error shall be reported when an oversize + message is seen. The default it "on". + * add global parameter "oversizemsg.input.mode" + is used to specify the mode with which oversized messages will + be handled. +- omfwd: add support for bind-to-address for UDP + To allow the same source address to be used regardless of the egress + interface taken, an option is added for an address to bind the datagram + socket to. Similarly to imudp, it is necessary to add an ipfreebind + option which is set by default, so as to avoid an excess of errors at + startup before the network interface has come up. This enhancement + allows a usecase on networking devices, by which a source interface + that is typically a loopback is specified, on which an address to bind + to is configured. This is so that the same source address is used for + all packets from rsyslog. + Thanks to Mike Manning for the patch. +- template systemd service file proposes higher permitted file handle limit + Especially on busy systems the default are too low. Please keep in mind + that on a very busy system even the now-proposed setting may be too low. + Thanks to github user jvymazal for the patch. +- imuxsock: replace select() call by poll() + While extremely unlikely, imuxsock could abort if a file descriptor + > 1024 was received during the startup phase (never occurred in + practice, but theoretically could if imfile monitored a large number + of files and were loaded before imuxsock - and maybe other + strange cases). + see also https://github.com/rsyslog/rsyslog/issues/2615 +- nsdsel_ptcp: replace select() by poll() + This removes us of problems with fds > 1024. The performance will + probably also increase in most cases. + Note this is not a replacement for the epoll drivers, but a general + stability improvement when epoll() is not available for some reason. + see also https://github.com/rsyslog/rsyslog/issues/2615 + closes https://github.com/rsyslog/rsyslog/issues/1728 + closes https://github.com/rsyslog/rsyslog/issues/1459 +- omprog: refactor tests, fix child closing issues + Refactor omprog tests. Fix sync issues in these tests by + using the feedback mode (confirmMessages=on) to synchronize + the test with the external program. Closes #2403 (I hope) + Fix omprog not properly closing child process when + signalOnClose=on. Needed for the new tests. Closes #2599 + Fix omprog not waiting for the child process to terminate + when signalOnClose=off. Needed for the new tests. Closes #2600 + Close all fds before executing the child even when valgrind + is enabled (--enable-valgrind). Needed for the new tests. + Fix memory leak when the xxxTransactionMark parameters were + used. + Thanks to Joan Sala for the patch. +- core: config optimizer did not handle call_indirect + This also caused the emission of an "internal error" error message + closes https://github.com/rsyslog/rsyslog/issues/2665 +- debug support: add capability to print testbench-specific timeout reports + done by setting RSYSLOG_DEBUG_TIMEOUTS_TO_STDERR to "on" + this is by default activated inside the testbench +- mmgrok: fix potential segfault + The modules used strtok(), which is not thread-safe. So it will potentially + segfault when multiple instances are spawned (what e.g. happens on busy + systems). + This patch replaces strtok() with its thread-safe counterpart + strtok_r(). + see also https://github.com/rsyslog/rsyslog/issues/1359 +- imrelp bugfix: maxDataSize could be set lower than maxMessageSize + maxDataSize specifies the length which will still be accepted + It previously could be set to any value, including values lower than the + configured rsyslog max message size, which makes no sense. Now this is + checked an error message is emitted if the size is set too low. +- build system bugfix: build broken if liblogging-stdlog installed in custom path + Thanks to Dirk Hörner for the patch. +- core bugfix: segfault on queue shutdown + if a ruleset queue is in direct mode, a segfault can occur during + rsyslog shutdown. The root cause is that a direct queue does not + have an associated worker thread pool, but the ruleset destructor + does not anticipate that and tries to destruct the worker thread + pool. It needs to do this itself, as otherwise we get a race + between rulesets on shutdown. + This was a regression from + https://github.com/rsyslog/rsyslog/commit/3fbd901b3e6300010 + closes https://github.com/rsyslog/rsyslog/issues/2480 +- imfile bugfix: statefiles contain invalid JSON + When imfile rewrites state files, it does not truncate previous + content. If the new content is smaller than the existing one, the + existing part will not be overwritten, resulting in invalid json. + That in turn can lead to some other failures. + closes https://github.com/rsyslog/rsyslog/issues/2662 +- omfile bugfix: segfault if empty filename was given + closes https://github.com/rsyslog/rsyslog/issues/2417 +- fix build issues when atomic operations are not present + for details, see https://github.com/rsyslog/rsyslog/pull/2604 +- lmsig_ksils12 bugfix: build and static analyzer issues + The module had a couple of problems building as well as some potential + errors detected by clang static analyzer. These have been fixed. + Thanks to Allan Park for the patch. + closes https://github.com/rsyslog/rsyslog/issues/2517 +- impstats bugfix: segfault if bound to non-existing ruleset + segfault happens during shutdown; up until unload of impstats, + rsyslog works correctly, except that no pstats are emitted. This + can be considered to be expected, because the error message + indicates the default ruleset is used instead. This is what + now actually happens. + closes https://github.com/rsyslog/rsyslog/issues/2645 +- mmjsonparse bugfix: invalid container name was not detected + in debug builds, this will trigger an assertion. In production + builds, an rsyslog internal error is logged, but rsyslog + continues to run. + closes https://github.com/rsyslog/rsyslog/issues/2584 +- mmkubernetes bugfixes: fix lnrules, add defaults, add test + - Fix lnrules for CONTAINER_NAME + - Add pkg check for lognorm >= 2.0.3 so we can set the macro + to enable ln_loadSamplesFromString + - Add some reasonable default values for parameters, such as + kubernetesurl https://kubernetes.default.svc.cluster.local:443 + - Clean up sample.conf configuration file + Thanks to Richard Megginson for the patch set. +- build system bugfix: --enable-atomic-operations did not work + closes https://github.com/rsyslog/rsyslog/pull/2604 +- bugfix: rsyslog aborts on startup when specific config errors are made + The following errors must be made in rsyslog.conf: + * no action present + * a call statement is used on an undefined ruleset + In this case, rsyslog emits an error message on the missing actions and + then segfaults. Depending on memory layout, it may also continue to run + but do nothing except accepting messages as no action is configured. + This patch make rsyslog properly terminate after the error message. It + is a change in behavior, but there really is no reason why a defunct + instance should be kept running. + closes https://github.com/rsyslog/rsyslog/issues/2399 +- build system: remove no longer needed --enable-libcurl configure switch + The --enable-libcurl switch was added to be able to disable libcurl + functionality inside the rsyslog core, see 46f4f43. As libcurl is no + longer used inside the core (due to introducing function modules), + --enable-libcurl needs to be removed. + closes https://github.com/rsyslog/rsyslog/issues/2628 +- QA/CI + * fixed races in some tests; root cause was that default enq timeout was too + low - we may also see in the future that other tests also need adjustment + (note that this is not a code problem but rather slow CI environments, + so increasing the timeout to get to a stable test state is the absolutely + correct thing to do) + * enabled some additional useful compiler warnings + * new test for diskqueue hitting configured disk space limit + * new tests for omfile + * added tests for mmkubernetes + * added tests for some script functions that were missing them + * made far more test compatible with FreeBSD, so that we now have fuller + coverage there +------------------------------------------------------------------------------ +Version 8.34.0 [v8-stable] 2018-04-03 +- mmkubernetes: new module + Thanks to Richard Megginson and Peter Portante for contributing the module. +- rsyslog script: introduce loadable function modules + rsyslog scripting can now also be extended via loadable modules - they + provides functions (just like loadable input, output, ... modules) + see also http://jan.gerhards.net/2018/03/loadable-rainerscript-functions.html +- imfile: large refactoring of complete module + This commit greatly refactors imfile internal workings. It changes the + handling of inotify, FEN, and polling modes. Mostly unchanged is the + processing of the way a file is read and state files are kept. + This is about a 50% rewrite of the module. + Polling, inotify, and FEN modes now use greatly unified code. Some + differences still exists and may be changed with further commits. The + internal handling of wildcards and file detection has been completely + re-written from scratch. For example, previously when multi-level + wildcards were used these were not reliably detected. The code also + now provides much of the same functionality in all modes, most importantly + wildcards are now also supported in polling mode. + The refactoring sets ground for further enhancements and smaller + refactorings. This commit provides the same feature set that imfile + had previously and all existing CI tests pass, as do some newly + created tests. + Some specific changes: + - bugfix: module parameter "sortfiles" ignored + This parameter only works in Solaris FEN mode, but is otherwise + ignored. Most importantly it is ignored under Linux. + fixes https://github.com/rsyslog/rsyslog/issues/2528 + - bugfix: imfile did not pick up all files when not present + at startup + fixes https://github.com/rsyslog/rsyslog/issues/2241 + fixes https://github.com/rsyslog/rsyslog/issues/2230 + fixes https://github.com/rsyslog/rsyslog/issues/2354 + fixes https://github.com/rsyslog/rsyslog/issues/1716 + - bugfix: directories only support "*" wildcard, no others + fixes https://github.com/rsyslog/rsyslog/issues/2303 + - bugfix: parameter "sortfiles" did only work in FEN mode + fixes https://github.com/rsyslog/rsyslog/issues/2528 + - provides the ability to dynamically add and remove files via + multi-level wildcards + see also https://github.com/rsyslog/rsyslog/issues/1280 + - the state file name currently has been changed to inode number + This will further be worked on in upcoming PRs + see also https://github.com/rsyslog/rsyslog/issues/2231 + - some enhancements were also done to CI tests, most importantly + they were made more compatible with BSD + Note that most of the mentioned bug fixes cannot be applied to older + versions, as they fix design issues which are solved by the refactoring. + Thus there are not separate commits for them. + There are probably also a number of different issues fixed, which have + not yet been full confirmed. Especially anyone having troubles with imfile + and wildcards will benefit from the refactoring. + closes https://github.com/rsyslog/rsyslog/issues/2359 +- omelasticsearch: add support for CA cert, client cert auth + This allows omelasticsearch to perform client cert based authentication + to Elasticsearch. + adds parameters: tls.cacert, tls.mycert, tls.myprivkey + Thanks to Richard Megginson for the patch. +- omfile-hardening: new contributed module for "omfile hardened operations" + This extends omfile with features considered useful for hardening. Comes + at the expense of performance loss and changed semantics. + Thanks to Mikko Kortelainen for contributing this work. +- stream/bugfix: memory leak on stream open if filename as already generated + this can happen if imfile reads a state file. On each open, memory for the + file name can be lost. + We detected this while working on imfile refactoring, so there is no related + bug report. No specific test has been crafted, as the refactored imfile + tests catch it (as soon as they are merged). +- bugfix/omkafka: did not build on platforms without atomic operations + Thanks to github user bruce87en for the patch +- bugfix/core/ratelimiting: SystemLogRateLimitBurst was limited to 65535 + rsyslog uses unsigned short for configuration setting SystemLogRateLimitBurst. + Being just 16 bits, unsigned short cannot hold values bigger than 65535. in a + practical setting rsyslog misbehaved with SystemLogRateLimitBurst being bigger + than 65535. + Thanks to github user KaleviKolttonen for the patch. +- bugfix imfile: memory leak in readMode 0 + closes https://github.com/rsyslog/rsyslog/issues/2421 +- bugfix omfile: some error messages had parameters in wrong order + which made the message look strange, but still readable + Thanks to Hans Rakers for the patch. +- bugfix omprog: file handle leak + There was a fd leak in the feedback feature added in v8.31.0 (github PR #1753). + The leak occurred when omprog was used with the confirmMessages=on setting + and no output setting. One fd was leaked every time the external program was + started. + Thanks to Joan Sala for the patch. +- bugfix imuxsock: data alignment problems + gcc did rightly complain that the cred and ts pointers would cause + alignment problems, so they were converted to structs and the necessary + data was memcpy()'ed to them. + the aux[] buffer was also potentially misaligned, so making a union + out of it and struct cmsghdr insured it was properly aligned. + The problems were especially visible on alpha and ia64 machines. + Thanks to Jason Duerstock for the fix. +- bugfix testbench: some test were accidentally not executed + Thanks to Kasumi Hanazuki for the patch. +- bugfix tcp subsystem: keepalive settings mixed up + TCPKeepAliveIntvl and TPCKeepAliveTime were switched. This is now correct and + thus causes a CHANGE OF BeHAVIOR of these settings. We applied this change only + after careful consideration of the effect. The contributor Alexandre Pierret + explained the situation as such: + "From my side, I work with thousands of servers centralizing logs to rsyslog + servers in tcp. All of them are running RHEL 6 and 7. The default rsyslog settings + in RHEL is TCPKeepAlive off. Since there are thousands of connections on the + rsyslog servers, I had to turn the TCPKeepAlive on to setup an aggressive policy + regarding ghost connections (following firewall tcp-timeout issue). Basically, + I set up: intvl=10 - probe=5 - time=2 If intvl and time are switched, it won't + break anything. It will just send 5 more empty tcp packets for 10 seconds (5 + probe x 2 seconds), which is painless (any professional grade 100€ router can + send more than 1.000.000 PPS). For 3000 servers, it means 3000 pps for 5 + seconds (3000 servers x 2 back-and-forth / 2 seconds). Let's take another worst + example: intvl=3600 - probe=5 - time=1. It means one keepalive every hour and + a 5 retry every 1s after a network issue. If the time and intvl values are + switched, it will generate 1 keepalive every seconds, It's a LOT more, but + after 5 probe or packet, it will stop. To summarize, I think it won't + break anything: + * Keepalive is off by default is many linux distribution + * When we enable it, it's to set up an aggressive policy. And setting up a + 10-5-1, 60-5-2, 2-5-60 or 1-20-1 policy is almost the same. + Bonus: For people setting up their rsyslog from the documentation, it will + now work as expected." + This was convincing, and we actually think that the fast majority of users set + up keepalive based on the doc and did never verify it actually worked as + expected. So we think that in all those cases, rsyslog will finally work as + intended. So we consider it justified to "change the behavior" here. + full discussion in PR: https://github.com/rsyslog/rsyslog/pull/2367 + Thanks to Alexandre Pierret for analyzing the situation and providing the + patch. +- fix some cosmetic issues found by lgtm.com static code analyzer + e.g. header file guard not correctly set - if you really are interested in + details, check git log +- CI + * add build test without atomic operations - now catches missing mutex macros + * add lgtm.com static analyzer (automatically called via GitHub PR) + * improved stability of global-umask.sh test, which unnecessarily used + wildcards for test output file checking. + Thanks to Kasumi Hanazuki for the patch. + * added some test for omprog with transactional interface + Thanks to Joan Sala for the new tests. + * fixed some omjournal tests which did not properly check result +------------------------------------------------------------------------------ +Version 8.33.1 [v8-stable] 2018-03-06 +- 8.33.0 tarball release was actually pre-8.33.0 + ... and so did not contain all features. This alone made a re-release + necessary, which is what now happens with 8.33.1. + Note: the git 8.33.0 label was correctly applied, "just" the tarball + was wrong. +- devcontainer: use some more sensible defaults + and add ability to specify generic docker run options + this makes integration into CI (and other scripting) easier +- fix problems with make dist + make dist did not package everything that was needed for CI, thus + resulting in make check failures if build from tarball. + Thanks to Thomas D. (whissi), and Michael Biebl for alerting us on the + problem, providing advise and some of the patches. We also added addt'l + patches ourselves. The problem occurred as the CI check for tarball + completeness was more or less disabled a couple of weeks ago, which + unfortunately went unnoticed. We have also applied some more safeguards + to detect such problems in the future. +------------------------------------------------------------------------------ +Version 8.33.0 [v8-stable] 2018-02-20 +- auto-detect if running inside a container (as pid 1) + adjust defaults in this case to be more container-friendly +- config: add include() script object + This permits to include files (like legacy $IncludeConfig) via a + script object. Needless to say, the script object offers more + features: + - include files can now be + - required, with rsyslog aborting when not present + - required, with rsyslog emitting an error message but otherwise + continuing when not present + - optional, which means non-present include files will be + skipped without notice + This is controlled by the "mode" parameter. + - text can be included form e.g. an environment variable + --> ex: include(text=`echo $ENVVAR`) + This finally really obsoletes $IncludeConfig. + closes https://github.com/rsyslog/rsyslog/issues/2151 +- template: add option to generate json "container" + This enables easy JSON generation via template. + This commit also corrects an issue with the constant "jsonf" + format. That was recently added, and the implementation problem + only became visible when used inside a larger json object. No + officially released code is affected, thought - so it really + is just a side-note. + closes https://github.com/rsyslog/rsyslog/issues/2347 +- core/template: add format jsonf to constant template entries + closes https://github.com/rsyslog/rsyslog/issues/2348 +- config: add ability to disable config parameter ("config.enabled") + For auto-generated configs, it is useful to have the ability to disable some + config constructs even though they may be specified inside the config. This + can now be done via the ```config.enabled``` parameter, applicable to all + script objects. If set to ```on``` or not specified, the construct will be + used, if set to any other value, it will be ignored. This can be used + together with the backtick functionality to configure enable and disable + from either a file or environment variable. + closes https://github.com/rsyslog/rsyslog/issues/2431 +- script: permit to use environment variables during configuration + new constant string type "backticks", inspired by sh + (sample: `echo $VARNAME`). +- new global config parameter "shutdown.enable.ctlc" + permits to shutdown rsyslog via ctl-c; useful e.g. in containers +- config optimizer: detect totally empty "if" statements and optimize + them out +- template: constant entry can now also be formatted as json field + This enhancements permits to craft clean templates that generate JSON, + e.g. for ElasticSearch consumption (or any other REST API) +- omstdout: support for new-style configuration parameters added +- core: set TZ on startup if not already set + In theory, TZ should be set by the OS. Unfortunately, this seems + to be not the case any longer on many Linux distros. We now check + it and set it appropriate if not already given. + Thanks to github user JPvRiel for providing an excellent explanation + of the reasoning for this and how to work around it. + closes https://github.com/rsyslog/rsyslog/issues/2054 +- imjournal bugfix: file handle leak during journal rotation + Thanks to Peter Portante for the patch + see also: https://github.com/rsyslog/rsyslog/pull/2437 +- lmsig_ksils12 bugfix: dirOwner and dirGroup config was not respected +- script bugfix: replace() function worked incorrectly in some cases + If the end of the message was similar to the replacement string, parts + of the string could (not always) be missing. + Thanks to Yaroslav Bo for the patch. +- build system bugfix: --disable-libcurl did not work + Thanks to Dan Molik, Thomas D. (whissi), and Michael Biebl for the patches. +- fixed build issues on Alpine Linux +- core bugfix: misaddressing in external command parser + This parser is used whenever a module (e.g. omprog) needs to process + command lines. If command parameters were given, memory misaddressing + occurred. This could lead to a segfault. + This is a regression in 8.32.0. + closes https://github.com/rsyslog/rsyslog/issues/2408 +- core bugfix: small memory leak in external command parser + This parser is used whenever a module (e.g. omprog) needs to process + command lines. On each action definitions for actions that use the + parser a small amount of memory was leaked. It is an uncritical leak + as it only occurs during config parsing. So it leaks a couple of + KiB during startup but does not grow during actual message processing. + This is a regression in 8.32.0. +- core bugfix: string not properly terminated when RFC5424 MSGID is used + This could lead to misaddressing when the jsonmesg property was used. + closes https://github.com/rsyslog/rsyslog/issues/2396 +- bugfix: strndup() compatibility layer func copies too much + The function did not obey the upper limit, effectively becoming + a strdup(). This was only noticed when the compatibility layer + was required, most importantly on Solaris 10. +- CI system + - we now use well-defined containers for parts of the CI runs + - now also build test under Alpine Linux + - test added for omprog feedback feature +------------------------------------------------------------------------------ +Version 8.32.0 [v8-stable] 2018-01-09 +- NEW BUILD REQUIREMENTs: + * libfastjson 0.99.8 is now required; older versions lead to bugs in rsyslog + * libczmq >= 3.0.2 is now required for omczmq + This was actually required for quite some while, but not properly checked + during configure run. If the lib was older, build failed. Now configure + detects this and emits the appropriate error message. + * libcurl is now needed for rsyslog core + due to new script function http_request(). This can be turned off by the + new configure option --disable-libcurl. If so, http_request() is not + available. +- rsyslogd: add capability to specify that no pid file shall be written + Command line option -iNONE provides this capability. This utilizes the + pre-existing -i option, but uses the special name "NONE" to turn of the + pid file check feature. Turning off is useful for systems where this no + longer is needed (e.g. systemd based). + closes https://github.com/rsyslog/rsyslog/issues/2143 +- ompgsql: considerable enhancements + The PostgreSQL output module was woefully out-of-date the following + list is changes made to update the module to current Rsyslog standards. + * allow for v6 configuration syntax + * configurable ports + * support transactional interface + * push db connection into workers (libpq is threadsafe) + * enable module testing on travis + * ensure configuration syntax backwards compatibility + * formatting around postgres core templating + * use new test conventions + * add new configuration syntax test + * add valgrind tests for new and old syntax + * add threading tests + * add action queue long running tests + * add action queue valgrind test + Thanks to Dan Molik for contributing this great enhancement! +- build system: removed --enable-rtinst configure option + This was a historic debugging option which has been superseded by + newer tools like valgrind, helgrind, ASAN, TSAN, etc... +- pmrfc3164: support for headerless messages + pmrfc3164 now detects if the first non-SP, non-HT character is either + '{' or '[' and if so assume that no header (TAG, HOSTNAME, DATE) is + given. If so, it uses defaults for these values. The full message is + used as the MSG part in that case. Note that an initial PRI may still + be specified. + This follows the trend to send JSON messages via syslog without any + header. We use default header values in this case. + This right now is a bit experimental; we may roll it back if + problems show up in practice. + closes https://github.com/rsyslog/rsyslog/issues/2030 +- omhiredis: add option to use RPUSH instead of LPUSH + see also https://github.com/rsyslog/rsyslog/issues/1625 +- mmexternal improvements + * better error reporting if to-be-executed script cannot be executed + * some general cleanup + * better redirection of stdin/out/err for the executed script + * bugfix: argv[0] of the script was missing if command line parameters + were not specified +- omprog: refactored, code shared with mmexternal moved to common object +- logctl tool: refactor to support newer rsyslog standards + * Made the logctl usertool ISO C90 compliant. + * Made the logctl usertool use a homogeneous coding style. + Thanks to Hugo Soszynski for contributing this work (as well as + suggesting some workarounds related to libmongoc/libbson). +- imfile: added support for Solaris File Event notification (FEN) + also improves performance under Solaris, with FEN imfile provides + features equivalent to inotify mode +- core/action: new parameter "action.errorfile" + permits to write failed messages to an "error file" so that they + can be reviewed and potentially be reprocessed +- imfile: added new module parameter "sortFiles" + This permits to process newly created files in sorted order. +- imuxsock: improved status reporting: socket name received from systemd + Providing an indication of what we got from systemd facilitates problem + analysis. +- build system: added new testbench configure switches + now --enable-testbench1 and --enable-testbench2 exists which permit + to enable/disable parts of the testbench. By default, both are on + when --enable-testbench is given. For full testbench coverage, both + options must be given. These options are necessary because under + Travis we hit the max runtime for tests and needed to split tests + over multiple incarnations. +- mmpstrucdata: new parameter "sd_name.lowercase" + to permit preserving case for structured data identifiers + Special thanks to github user alanrobson for the initial commit that + preserves case (on which we based other work). +- omfile: add module-global option "dynafile.donotsuspend" + this permits to enable SUSPENDing dynafile actions. Traditionally, + SUSPEND mode was never entered for dynafiles as it would have blocked + overall processing flow. Default is not to suspend (and thus block). + closes https://github.com/rsyslog/rsyslog/issues/2236 +- testbench: add a capability to turn off libfaketime tests via configure + Unfortunately, libfaketime becomes more and more problematic in newer + versions and causes aborts on some platforms. This provides the ability + to turn it off via --disable-libfaketime. + In the longer term, we should consider writing our own replacement. +- testbench: name valgrind tests consistently + all valgrind tests now end in -vg.sh +- RainerScript: add function parse_json() +- RainerScript: add function substring() +- RainerScript: add function http_request() +- RainerScript: add function previous_is_suspended() + This function returns a boolean indicating if the previous action is + suspended (0 - no, 1 - yes). This is useful if arbitrary processing + (other than calling actions) should be made depending on that state. + A prime example for this is probably calling a ruleset. + closes https://github.com/rsyslog/rsyslog/issues/1939 +- Patches from BSD projects have been imported + ... as far as they still apply. Some patches that patched BSD-specific + code were broadened to be more generic. +- script bugfix: invalid function names were silently ignored + no error message was produced + thanks to Matt Ezell for providing the patch. +- rainerscript: add int2hex() function +- rainerscript: add is_time() function + Thanks to Stephen Workman for implementing this. +- RainerScript: add function script_error() and error-reporting support + This permits script functions that could fail to report errors back, so + that the user can handle them if desired. We use an errno-style of + interface. That means script_error() needs to be called after functions + that supports it. It will return either 0 (all OK) or something else + (an error condition). + The commit also modifies the parse_time() function to use the new + interface. First of all, this is useful for users, and secondly we + needed a capability to craft a testbench test. + closes https://github.com/rsyslog/rsyslog/issues/1978 +- testbench: fixed build problem of testbench tools under Alpine Linux +- added --enable-libsystemd configure option to enforce use of libsystemd + so we can fail the build on platforms where this is required +- core/glbl: remove long-unused option $optimizeforuniprocessor + This was still available, but had no effect (for ~10 years or so). The + plumbing is now removed. If someone tries to use the option, an + error message is generated. + closes https://github.com/rsyslog/rsyslog/issues/2280 +- core/queue: emit better status messages at rsyslog shutdown + this helps to diagnose issue - unfortunately we need more work to ensure + that the messages always make it to the user. This is a start and + hopefully useful at least for the testbench, possibly more. +- fixed a couple of build issues with gcc-7 (in less frequently used modules) +- fixed a couple of build issues on the arm platform (actually raspbian) +- impstats: fix invalid counter definitions for getrusage() reporting + some of the counters were defined as int (32 bit) vs. intctr_t (64 bit). + On some platforms "long" seems to be 64bit, and getrusage() provides + what we store as int via long. So this caused truncation and/or overflow. + This had undefined effects. Most often, everything worked fine + for values smaller than 2^31 but sometimes we got negative values. + closes https://github.com/rsyslog/rsyslog/issues/1517 +- imudp bugfix: potential segfault in ratelimiting + The rate-limiter inside imudp was not set to be thread safe, but was + used across multiple threads. This worked in default configuration, + but failed when RepeatedMsgReduction was set to "on". + Note that it in general is a bug to use a rate-limiter in + non-threadsafe mode across multiple threads. This also causes invalid + rate limiting counts in the default case. + closes https://github.com/rsyslog/rsyslog/issues/441 + fixes https://github.com/rsyslog/rsyslog/issues/2132 +- imptcp bugfix: access to free'ed memory + When notifyconnectionclose was on, a string buffer was accessed immediately + after it was freed (as part of the connection close operation). + Detected by LLVM ASAN. +- mmanon bugfix: fix wrong ipv6 embedded recognition + mmanon recognized IPv6 with embedded IPv4 that have too few (16 bit) fields. + example: 13:abd:45:0.0.0.0 + closes https://github.com/rsyslog/rsyslog/issues/2357 +- imfile bugfix: not detecting files in directory when wildcards are used. + When directories and files are created at the same time, + imfile may missed subdirs or file if the machine is on high load. + The handling has been enhanced to scan newly created directories ALWAYS for + matching files. + fixes https://github.com/rsyslog/rsyslog/issues/2271 + However there still exist problems with multilevel directory configurations. + Details are discussed in https://github.com/rsyslog/rsyslog/issues/2354 + Fixes for the remaining issues are expected for 8.33.0. +- script bugfix: improper string-to-number conversion for negative numbers +- core/action bugfix: 100% CPU utilization on suspension of output module + Triggering condition: + * output module using the legacy transaction interface + (e.g. omelasticsearch, omlibdbi) + * output module needs to suspend itself + In these cases, rsyslog enters a busy loop trying to resolve the + suspend condition. The bug is rooted in rsyslog core action code. + This patch fixes it by inserting a 1-second sleep during calls + to the resume handler. + Note: we cannot sleep exactly as long as tryResume needs. This + would require larger refactoring, which probably is not worth for + the legacy interface. The current solution is almost as good, as + the one second sleep has very little overhead on a real system. + Thus we have chosen that approach. + This patch now also ensures that failed messages are properly + handled and do not cause eternal hang. + closes https://github.com/rsyslog/rsyslog/issues/2113 +- core/variables bugfix: bare $! cannot be used in set statement + fixes https://github.com/rsyslog/rsyslog/issues/326 +- core bugfix: auto commit of actions improperly handled + The internal state variable bHadAutoCommit was handled in thread-unsafe way + and most probably caused (hard to see) issues when action instances were + run on multiple worker threads. It looks like the state variable was + forgotten to move over to worker state when action workers were introduced. + closes https://github.com/rsyslog/rsyslog/issues/2046 +- core bugfix: filename length limitation of 199 bytes + file names (including path names) longer than 199 bytes could not be + handled at many places. This has now been uplifted to 4KiB after careful + review for the largest size supported among all relevant platforms. +- core bugfix: undefined behavior due to integer overflow + when searching strings, we may have an (unsigned) integer overflow + which can lead to misaddressing. + Detected by clang ubsan. +- core bugfix: race on LocalHostIP property during startup + The way the default was set could lead to a race if e.g. two internal + messages were emitted at startup when the property was not yet set. This + has been seen to happen in practice. It had no bad effect except a very + small stationary memory leak, but made using thread analyzers unreliable + (as it was rightfully detected as a problem). + closes https://github.com/rsyslog/rsyslog/issues/2012 +- bugfix: potential segfault on startup + timezone info table was "sorted" even though it may be NULL. There is + no practical case known where this lead to an actual abort, but in + theory it would be possible. If so, it would happen immediately on + startup. + Detected by clang ubsan. +- omhiredis bugfix: rsyslog segfault on startup if no template is specified +- omprog bugfix: argv[0] not set when using binary without arguments + When using the omprog plugin with a binary without arguments, argv[0] (the + name of the binary) is not set, causing binaries that depend on this value + being set to crash or misbehave. + This patch also mildly refactors omprog argv[] creations, albeit some more + refactoring would be useful. + closes https://github.com/rsyslog/rsyslog/issues/1858 +- core: refactoring of rsyslog's cstr "class" + Function cstrGetSzStrNoNULL shall modified the string buffer on each call, + albeit it is considered a "read-only" function. It currently adds a '\0' + at the end. This is bad, especially when multiple threads access the same + string. As always the same data is written, it was not causing real issues + (except unnecessary cache writes), but it polluted the thread debugger and + as such prevent more elaborate automated tests. +- parent directory creation function refactored + This should not cause any change of behavior, but is worth noting in case + we see a regression not caught by the CI system. +- mmsnmptrapd bugfix: potential misaddressing + This only occurred in situations when the system was totally out of memory. +- imkafka: fix potential small resource leak + If rdkafka handle cannot fully populated, cleanup is added. Previously, we + could leak a handle if e.g. no brokers were available. Note that this was + a cosmetic leak, as proper processing is not possible in any case and the + leak is once per listener, so not growing. But we now also ensure that + proper error reporting and handling is done in any case. Previously, we + may have some misleading error messages if the defunct kafka handle was + used. + closes https://github.com/rsyslog/rsyslog/issues/2084 +- imkafka bugfix: do not emit error message on regular state + This was misunderstood as an error state and could spam the system + log considerably. Regression from 8.31.0. +- omkafka: expose operational status to user where useful + omkafka emits many useful operational status messages only to the debug + log. After careful review, we have exposed many of these as user error + and warning message (ex: librdkafka queue full, so user knows why we + suspend the plugin temporarily). This may have made the module too + chatty. If so, one can always filter out messages via configuration. And + if we really went overboard, we can undo those parts with the next + release. So it's better to give a bit more than less, as this definitely + eases troubleshooting for users. + closes https://github.com/rsyslog/rsyslog/pull/2334 +- omkafka bugfix: potential message duplication + If a message that already failed in librdkafka was resubmitted and that + resubmit also failed, it got duplicated. +- omkafka: fix multithreading + omkafka has several issue if multiple worker instances are used. This commit + actually make the module use a single worker thread at max. Reasoning: + Librdkafka creates background threads itself. So omkafka basically needs to move + memory buffers over to librdkafka, which then does the heavy hauling. As such, we + think that it is best to run max one wrkr instance of omkafka -- otherwise we just + get additional locking (contention) overhead without any real gain. As such, + we use a global mutex for doAction which ensures only one worker can be active + at any given time. That mutex is also used to guard utility functions (like + tryResume) which may also be accessed by multiple workers in parallel. + Note: shall this method be changed, the kafka connection/suspension handling needs + to be refactored. The current code assumes that all workers share state information + including librdkafka handles. + closes https://github.com/rsyslog/rsyslog/issues/2313 +- omkafka bugfix: potential misaddressing + The failed message list is improperly cleaned. This is a regression + from recent commit 4eae19e089b5a83da679fe29398c6b2c10003793, which + was introduced in 8.31.0. + This problem is more likely to happen under heavy load or bad + connectivity, when the local librdkafka queue overruns or message + delivery times out. + closes https://github.com/rsyslog/rsyslog/issues/2184 + closes https://github.com/rsyslog/rsyslog/issues/2067 +- omkafka bugfix: build fails with older versions of librdkafka + closes https://github.com/rsyslog/rsyslog/issues/2168 +- omgssapi bugfix: fix compiler warnings with gcc-7 + closes https://github.com/rsyslog/rsyslog/issues/2097 +- dnscache bugfix: entries were cached based on IP AND port number + That hash key which is used to find out already cached dns entry gets + incorrectly computed from the whole sockaddr_storage + (sockaddr_in/sockaddr_in6) structure including a sin_port (which doesn't + have a static value) instead of only an address, thus creating redundant + dns cache entries/consuming more space. This lead to potentially high memory + usage and ineffectiveness of the case. It could be considered a memory leak. + Thanks to Martin Osvald for the patch. + see also: https://github.com/rsyslog/rsyslog/pull/2160 +- omkafka bugfix: fixed memory leak + a memory leak occurred when librdkafka communicated error/status information + to omkafka. this seems to happen relatively frequently, so this leak + could be pretty problematic. +- mmdblookup bugfix: replace thread-unsafe strtok() by thread-safe counterpart + Many thanks to Will Storey (github user @horgh) for mentioning this and + his phantastic help in debugging this rsyslog problem! +- pmnormalize bugfix: remove unsave "strcat" implementation +- rainerscript bugfix: ltrim() and rtrim function misaddressing + This could lead to a segfault and was triggered by certain input data + schemes. For example, a ltrim(" a") could lead to the fault. +- imklog bugfix: local host IP was hardcoded to 127.0.0.1 + This is now taken from the global localHostIP setting, which is used + consistent across all modules. + Also, the removed (2012?) directive $klogLocalIPIF has been added + again but directly marked as removed. That way, an informative error + message is generated if someone tries to use it. + closes https://github.com/rsyslog/rsyslog/issues/2276 +- cleanup: remove obsolete pre-KSI GuardTime signature interface + this is no longer functional starting Jan 2018 as it reached end of life + closes https://github.com/rsyslog/rsyslog/issues/2128 +- cleanup: obsolete definition SOL_TCP replaced by newer IPPROTO_TCP + this should not have any effect at all except better portability, but is + worth mentioning in the ChangeLog nevertheless. +- lookup tables: fixed undefined behavior detected by UBSan +- CI testing + - ARM (via Raspberry PI) added to CI system + - Debian 9 added to CI system + - omgssapi and usertools components are now also tested in Travis + - test coverage on BSD has been enhanced +------------------------------------------------------------------------------ +Version 8.31.0 [v8-stable] 2017-11-28 +- NEW BUILD DEPENDENCY: ommongodb now requires libmongo-c + instead of deprecated libmongo-client. +- remove systemd embedded code, use libsystemd instead + Since the early days rsyslog used the original systemd embedded + interface code. This version now uses libsystemd where available. + If we do not find libsystemd, we assume the system does not use + systemd, which is a safe assumption nowadays. This ensures we use the + fresh interface code and also removes bloat from our project. + closes https://github.com/rsyslog/rsyslog/issues/1933 +- mmanon: add support for IPv6 addresses with embedded IPv4 address + While this format is uncommon, it may still be present in logs and as + such should be supported. It is configurable via individual settings, + though. Especially the number of bits to anonymize may be desired to + be different than in pure IPv6. +- ommongodb: big refactoring, more or less a feature-enhanced rewrite + New features are : + * Handle TLS connection to mongodb + * Handle MongoDB Replicasets + * Added the 'ssl_ca' and 'ssl_cert' directives to configure tls connection + * Added 'uristr' directive to configure the connection uri in the form + of 'mongodb://...' + Now uses the official mongo-c-driver library instead of the deprecated + mongo-client library + Special thanks to Hugo Soszynski and Jérémie Jourdin for there hard work + to make this a reality! + See also: https://github.com/rsyslog/rsyslog/pull/1917 +- rainerscript: add parse_time() function + Thanks to Stephen Workman for implementing this. +- omelasticsearch: add LF to every elastic response for error file + error file content was written without LF after each message, making + it hard to work with and monitor. + Thanks to Yaroslav Bo for the patch. +- omelasticsearch: add pipeline support + supports static and dynamic ElasticSearch pipeline parameter. + closes https://github.com/rsyslog/rsyslog/issues/1411 + Thanks to github users scibi and WaeCo for the implementation. +- lmsig_ksi_ls12: support asynchronous mode of libksi +- omprog: added error handling and transaction support for external plugins + This permits much better integration of external output modules. + Special thanks to Joan Sala for providing this work! +- imzmq3/omzmq3: marked as deprecated, modules will be remove in v8.41 + see also: https://github.com/rsyslog/rsyslog/issues/2100 +- imzmq3/omzmq3: fixed build issues with gcc-7 +- core: emit error message on abnormal input thread termination + This indicates a serious issue of which the user should be notified. + Was not done so far and caused grief when troubleshooting issues. +- core: refactored locking for json variable access + refactored the method so that it consistent in all functions and easier + to follow. Most importantly, now an as simple as possible locking paradigm + of lock ... unlock within the function. Hopefully easier to understand + both for humans and static code analyzers. +- core: refactored creation of UDP sockets + was very old, hard to follow code; streamlined that a bit +- core/dnscache: refactor locking + keep a simple lock ... unlock scheme within the function. That is + easier to follow for both humans as well as static analyzers. + Also removes Coverity scan CID 185419 +- rainerscript: use crypto-grade random number generator for random() function + We now use /dev/urandom if available. If not, we fall back to the weak PRNG. +- imkafka: improve error reporting and cleanup refactoring +- imkafka bugfix: segfault if "broker" parameter is not specified + Now emits warning message instead and uses hardcoded default + (which usually matches where the kafka broker can be found). + fixes https://github.com/rsyslog/rsyslog/issues/2085 +- omkafka: improve error reporting +- omkafka: slight speedup do to refactoring of LIST class + double-linked list was used for temporarily undeliverable message tracking + where singly-linked list was sufficient. Changed that. +- TCP syslog: support SNI when connecting as a client + This is done if a hostname is configured and turned off if an IP is used. + Thanks to Art O Cathain for the patch. + see also https://github.com/rsyslog/rsyslog/pull/1393 +- msg variable bugfix: potential segfault on variable access + A segfault is likely to happen if during processing a variable with + more than one path component is accessed (e.g. $!path!var) and NO + variables oft hat kind (local, message, global) are defined at all. + closes https://github.com/rsyslog/rsyslog/issues/1920 +- ratelimiting bugfix: data race in Linux-like ratelimiter + access to the Linux-like ratelimiter was not properly synchronized, and + the system rate-limiter was not using it in any case. + This could lead to the rate-limit settings not being properly + respected, but no "hard error". +- core/template bugfix: potential NULL pointer access at config load + could happen if the config was loaded and the parameters could not properly + be processed. If so, this occurred only during the startup phase. + Detected by Coverity scan, CID 185318 +- core/json var subsystem bugfix: segfault in MsgSetPropsViaJSON + Invalid libfastjson API use lead to double-free. This was most importantly + triggered by mmexternal (but could potentially be triggered by other + uses as well) + closes https://github.com/rsyslog/rsyslog/issues/1822 +- core/wrkr threads bugfix: race condition + During e.g. shutdown it could happen that a worker thread was started + and immediately be requested to terminate. In such situations there was + a race the prevented proper initialization. This could lead to follow-on + problems. + We believe (but could not proof) that this also could lead to a hang of + the termination process. Thus we claim to close an issue tracker down + here below, but are not 100% certain it really closes it (we hope for + user feedback on this). In any case, the thread debugger showed invalid + operations and this has been fixed, so it is useful in any case. + closes https://github.com/rsyslog/rsyslog/issues/1959 +- core/wtp: potential hang during shutdown + when the wtp worker is cancelled in the final stage of shutting down + while the mutex is locked and there is one worker left, the system + will hang. The reason is that the cancelled thread could not free the + mutex that the other needs in order to shut down orderly. + Detected with clang thread sanitizer. +- omfwd bugfix: generate error message on connection failure +- imtcp bugfix: "streamdriver.mode" parameter could not be set to 0 +- imjournal bugfix: module was defunctional + The open function was broken by commit 92ac801, resulting in + no data being ever read from the journal. + patch bases on the idea of Radovan Sroka given here: + https://github.com/rsyslog/rsyslog/issues/1895#issuecomment-339017357 + but follows the current imjournal-paradigm of having the journal + handle inside a global variable. + see also https://github.com/rsyslog/rsyslog/issues/1895 + closes https://github.com/rsyslog/rsyslog/issues/1897 +- imjournal: refactor error handling, fix error messages + imjournal did not try to recover from errors and used the error state + returned by journal functions incorrectly, resulting in misleading + error messages. Fixed this and also increased the number of error + messages so that it now is easier to diagnose problems with this module. + Also a little bit of internal brush-up. + -mmdblookup bugfix: fix potential segfault due to threading issues + libmaxminddb seems to have issues when running under multiple threads. As + a first measure, we prevent mmdblookup now from running on more than one + thread concurrently. + see also: https://github.com/rsyslog/rsyslog/issues/1885#issuecomment-344882616 +- omelasticsearch bugfix: operational error messages not passed to user + This lead to sometimes hard to diagnose problem. Note that for obvious + reasons the amount of messages from omelasticsearch has increased; this + is not a sign of a problem in itself -- but it may reveal problems that + existed before and went unnoticed. Also note that too-verbose messages + can be filtered out via regular rsyslog methods (e.g. message discarding + based on content). +- omkafka bugfixes + * statistics counter maxoutqsize could not be reset + Thanks to github user flisky for the patch. + * potential hang condition + omkafka did not release a mutex under some error conditions (most + notably out of memory on specific alloc calls). This lead to a hang + during actively processing messages or during shutdown (at latest). + This could happen only if dynamic topics were configured. + Detected by Coverity Scan, CID 185781 (originally 185721, detected + as a different issue by Coverity after code refactoring done in regard + to 185721 -- then the problem became more obvious). + * file handle leak, which could occur when local buffering + of messages was needed + * invalid load of failedmsg file on startup if disabled + error "rsyslogd: omkafka: could not load failed messages from " + "file (null) error -2027 - failed messages will not be resend." + occurs because, despite `keepFailedMessages="off"` as a default, + omkafka still tries to check for and load a `(none)` file which + triggers an IO error of sorts according to the 2027 link above. + Obviously, it probably shouldn't try load the file if + `keepFailedMessages` is not enabled. + Thanks to github user JPvRiel for a very good error report and + analysis. + closes https://github.com/rsyslog/rsyslog/issues/1765 + * various config parameters did not work + These were not even recognized when used and lead to a config startup + error message: + ~ closeTimeout + ~ reopenOnHup + ~ resubmitOnFailure + ~ keepFailedMessages + ~ failedMsgFile + closes https://github.com/rsyslog/rsyslog/issues/2052 + * considerable memory leak + Whenever a message could (temporarily) not be delivered to kafka, + a non-trivial amount of memory was leaked. This could sum up to + quite a big memory leak. + fixes https://github.com/rsyslog/rsyslog/issues/1991 + * some small memory leaks fixed + most of them cosmetic or a few bytes statically (not growing as + omkafka was used) -- thus we do not mention each one explicitly. + For more details, see git commit log or this pull request: + https://github.com/rsyslog/rsyslog/pull/2051 +- kafka bugfix: problem on invalid kafka configuration values + omkafka ended up in an endless loop and high cpu. + imkafka tried to subscribe to a not connected kafka server. + closes https://github.com/rsyslog/rsyslog/issues/1806 +- [io]mgssapi: fix build problems (regression from 8.30.0) +- [io]czmq: fix build problems on some platforms (namely gcc 7, clang 5) +- tcpsrv bugfix: potential hang during shutdown +- queue bugfix: potential hang during shutdown +- queue bugfix: NULL pointer dereference during config processing + If the queue parameters were incorrectly given, a NULL pointer dereference + could happen during config parsing. Once post that stage, no problem could + occur. + Detected by Coverity scan, CID 185339 +- imczmq bugfix: segfault + happened in a call to + 371: zcert_destroy(&serverCert) called from rcvData(). + Thanks to ~achiketa Prachanda for the patch. +- imfile: some small performance enhancements + Thanks to Peter Portante for the patch +- omfile: handle file open error via SUSPEND mode + For a while, an open file error lead to suspension as the error was + not detected by the rule engine. This has changed with fixes + in 8.30.0. I asked users what they prefer (and expect) and + everyone meant it should still be handled via suspension. See + github tracker below for more details. + closes https://github.com/rsyslog/rsyslog/issues/1832 +- omfile bugfix: race during directory creation can lead to loop + There was a race where two threads were competing for directory creation + which could lead to none succeeding and a flood of error message like this + "error during config processing: omfile: creating parent directories for + file". This has been solved. + Thanks to Niko Kortström for the patch. +- imudp: improve error reporting + When udp listener cannot be created, an error message containing + the ip-address and port is put out. + closes https://github.com/rsyslog/rsyslog/issues/1899 +- omrelp bugfix: incorrect error handling + if librelp with TLS but without Authentication was included, librelp + did not emit the correct error message due to invalid error code + check. It also did not err-out but instead suspended itself. + Detected by Coverity scan, CID 185362 +- [io]mrelp bugfix: segfault on startup if configured cert not readable + When the certificate file specified in the omrelp/imrelp configuration + can't be accessed, e.g. because it doesn't exist or you don't have + permission to do so, a Segmentation Fault will appear when you start + Rsyslog. This commit fixes that problem. + closes https://github.com/rsyslog/rsyslog/issues/1869 +- mmanon fix: make build under gcc 7 + Thanks to William Dauchy for the patch +- mmpstrucdata bugfix: formatting error of ']' char + This was invalidly formatted as '"'. Thanks to github user + wu3396 for the error report including the patch idea. + closes https://github.com/rsyslog/rsyslog/issues/1826 +- mmexternalb bugfix: memory leak +- core/stats bugfix: memory leak if sender stats or tracking are enabled +- core bugfix: potential segfault during startup + A NULL pointer could be accessed if there was a problem with the + timezone parameters. Affects only startup, once started, no problem + existed. + Detected by Coverity scan; CID 185414 +- core bugfix: potential race in variable handling + Root of variable tree is accessed prior to locking access to it. + This introduces a race that may result in various kinds of + misaddressing. + Found while reviewing code, no bug report exists. +- core bugfix: potential segfault when shutting down rsyslog + when rulesets are nested a segfault can occur when shutting down + rsyslog. the reason is that rule sets are destructed in load order, + which means a "later" ruleset may still be active when an "earlier" + one was already destructed. In these cases, a "call" can invalidly + call into the earlier ruleset, which is destructed and so leads to + invalid memory access. If a segfault actually happens depends on the + OS, but it is highly probable. + The cure is to split the queue shutdown sequence. In a first step, + all worker threads are terminated and the queue set to enqOnly. + While some are terminated, it is still possible that the others + enqueue messages into the queue (which are then just placed into the + queue, not processed). After this happens, a call can no longer + be issued (as there are no more workers). So then we can destruct + the rulesets in any order. + closes https://github.com/rsyslog/rsyslog/issues/1122 +- core/action bugfix: potential misaddressing when processing hard errors + For batches that did fail in an output module, the rsyslog core + tries to find offending messages that generate hard (non-recoverable) + errors. During this process, the action templates for each message + are copied into a temporary buffer. That copy was invalidly sized, + actually copying only the first template string. As such, outputs + that requested more template strings AND had errors in batch submission + received uninitialized pointers. This could lead to all sorts of + problems. + see also https://github.com/rsyslog/rsyslog/issues/1885 + closes https://github.com/rsyslog/rsyslog/issues/1906 +- template object bugfix: NULL pointer access on invalid parameters + could happen only during startup + Detected by Coverity scan, CID 185376 +- omjournal bugfix: NULL pointer access on invalid parameters + could happen only during startup +- omelasticsearch bugfix: configured credentials not used during health check + Authentication credentials were not applied during health check, + permission to use unsigned CERTS was not applied to regular data post. + closes https://github.com/rsyslog/rsyslog/issues/1949 +- omelasticsearch bugfix: abort on unavailable ES server + Depending on the state of unavailability (libcurl return code), + omelasticsearch tries to process a NULL return message, what + leads to a segfault. + This fixes the problem and introduces better error handling and + better error messages. + see also https://github.com/rsyslog/rsyslog/issues/1885 +- omelasticsearch: fix memory leak and potential misaddressing + Commit 96b5fce introduced regressions, leading to potential misaddressing + and a very probable memory leak. This commit fixes the issues and + hardens the module to better detect some error cases in the + future. + It also adds valgrind-based testbench tests which ensure that no + pointer errors exist. If they had been in place, the regressions + would never have been undetected. + Note that commit 96b5fce was introduced in 8.23.0 on Nov, 15th 2016. + Many thanks to Yaroslav Bo for alerting me on the root problem and + providing a very good analysis and guidance. + see also https://github.com/rsyslog/rsyslog/issues/1906 + see also https://github.com/rsyslog/rsyslog/issues/1964 + closes https://github.com/rsyslog/rsyslog/issues/1962 +- omelasticsearch bugfix: output from libcurl to stdout + omelasticsearch made libcurl output messages to stdout. This + commit fixes that. It also automatically enables libcurl verbose + mode during debug runs - it needs to be seen if this is smart or + not (previously, code needed to be commented in). + closes https://github.com/rsyslog/rsyslog/issues/1909 +- iczmq bugfix: potential memory leak +- imptcp bugfix: potential misaddressing + When during a connection request the remote peer could not be + identified, imptcp could misaddress memory if it is configured + to emit messages on connection open. + Detected by clang 5.0 static analyzer. +- imptcp: potential buffer overflow + if the local hostname or IP is larger than NI_MAXHOST-1, an internal + buffer is overflowed. This is irrespective of any input data. + Detected by Coverity scan, CID 185390 +- core/nsd_gtls: fix potential uninitialized data access + could occur during certificate check + found by clang 5.0 static analyzer +- stats bugfix: potential program hang + due to missing unlock. This could only occur if pstats was set to + CEE-format logging (very uncommon) AND if the system runs out of + memory (in which case other things go pretty bad as well). + found by Coverity scan +- omfwd bugfix: memory leak if network namespaces are used + very seldom used feature, occurs only during error case + found by Coverity scan. +- core: potential misaddressing when accessing JSON properties + When a JSON property is accessed in template processing, memory may + have been misaddressed or a double-free may occur while obtaining the + property. + This was detected by a Coverity scan. +- gcry crypto provider bugfixes: potential misaddressing and memory leak + If the config parameters were invalid, a misaddressing could occur. If so, + this happens only during startup. + Also, a memory leak existed when the crypto provider errored out. This could + build up if it were used in frequently-changing dynafiles. This was + detected by Coverity scan, CID 185360. +- core/file stream object bugfix: memory leak + If a multiline read errored out, a line buffer could be leaked. + Detected by Coverity scan, CID 185328 +- imdiag bugfix: double mutex unlock when working with stats + Note: while this could potentially lead to a program hang, it affected + only testbench execution as imdiag is a testbench-only tool. + Detected by Coverity scan, CID 185348 and 185350 +- fixed several minor and cosmetic issues found by Coverity scan + including false positives. For details see "$ git log". All noteworthy + issues are separately mentioned in this ChangeLog. The ones not mentioned + are for example problems that can only occur during out of memory + conditions, under which it is extremely likely tha the rsyslog process + will be killed in any case +- testbench: + * added compile-check for [io]mgssapi, mmcount + * harden tests against hanging previous instances + * re-enable RELP tests on Solaris + * added basic test for imjournal + * added threading tests via valgrind's helgrind tool + * added valgrind test for kafka modules + * added capability to run elasticsearch tests with + a) different ElasticSearch versions + b) independently from OS-installed version + This also sets base to enable more elaborate ES tests + * further relaxed timing of dynstats tests, as they tend to create + false positives on slow machines +- CI: improved test coverage on FreeBSD +- Travis: clang static analyzer 5.0 now run over all source files +- build: make compile warning-free under gcc 7 +------------------------------------------------------------------------------ +Version 8.30.0 [v8-stable] 2017-10-17 +- NEW BUILD REQUIREMENTS + * libfastjson 0.99.7 is now mandatory + the new version is required to support case-insensitive variable + comparisons, which are now the default + * when building imjournal, libsystemd-journal >= 234 is now recommended + This is to support the imjournal enhancement. Note that it is possible + to build with lower version, but this will degrade imjournal functionality. +- CHANGE OF BEHAVIOR: all variables are now case-insensitive by default + Formerly, JSON based variables ($!, $., $/) were case-sensitive. + Turn old default back on: global(variables.casesensitive="on") + See ChangeLog entry below for more details. +- core: handle (JSON) variables in case-insensitive way + The variable system inside rsyslog is JSON based (for easy consumption + of JSON input, the prime source of structured data). In JSON, keys + ("variable names") are case-sensitive. This causes constant problems + inside rsyslog configurations. A major nit is that case-insensitivity + option inside templates (even if turned on) does not work with JSON + keys because they of inner workings*1. + It is much more natural to treat keys in a case-INsensitive way (e.g. + "$!Key" and "$!key" are the same). We do not expect any real problems + out of this, as key names only differing in case is highly unlikely. + However, as it is possible we provide a way to enable case-sensitivity + via the global(variables.casesensitive="on") global config object. + Note that the default is to do case-insensitive matches. The reason + is that this is the right thing to do in almost all cases, and we do + not envision any problems at all in existing deployments. + *1 Note: case-insensitivity in rsyslog is achieved by converting all + names to lower case. So that the higher speed of strcmp() can be used. + The template option does actually that, convert the template keys to + lower case. Unfortunately, this does not work with JSON, as JSON keys + are NOT converted to lower case. + closes https://github.com/rsyslog/rsyslog/issues/1805 +- imjournal: made switching to persistent journal in runtime possible + with this patch imjournal can continue logging after switch to + persistent journal without need to restart rsyslog service + Thanks to github user jvymazal for the patch +- mmanon: complete refactor and enhancements + - add pseudonymization mode + - add address randomization mode + - add support for IPv6 (this also supports various replacement modes) + closes https://github.com/rsyslog/rsyslog/issues/1614 + also fixes bugs + - in IPv4 address recognition + closes https://github.com/rsyslog/rsyslog/issues/1720 + - in IPv4 simple mode to-be-anonymized bits can get wrong + closes https://github.com/rsyslog/rsyslog/issues/1717 +- imfile: add "fileoffset" metadata + This permits to send the offset from which the message was read together + with the message text. + Thanks to github user derekjsonar for the initial implementation which + we extended to use the message start offset. +- RainerScript: add ltrim and rtrim functions + closes https://github.com/rsyslog/rsyslog/issues/1711 +- core: report module name when suspending action + Thanks to Orion Poplawski for the patch. +- core: add ability to limit number of error messages going to stderr + This permits to put a hard limit on the number of messages that can + go to stderr. If for nothing else, this capability is helpful for the + testbench. It permits to reduce spamming the test log while still + providing the ability to see initial error messages. Might also be + useful for some practical deployments. + global parameter: global(errorMessagesToStderr.maxNumber) +- tcpsrv subsystem: improve clarity of some error messages + operating system error message are added to some messages, providing + better advise of what actually is the error cause +- imptcp: include module name in error msg +- imtcp: include module name in error msg +- tls improvement: better error message if certificate file cannot be read +- omfwd: slightly improved error messages during config parsing + They now contain config file/line number information where this was missing. +- ommysql improvements + * Return specific code for unrecoverable errors. This makes retry processing + more performant and robust. + * error messages improved + * Update to utilize native v8 transactional interface. Previously, it used + the v7 interface with v8 emulation. + * treat server and client-generated messages differently + Server-generated error messages are considered non-recoverable, while + client generated once point into connection problems (which simply can + be retried). This is part of the improvements towards better + message-induced errors. Previous commits did only handle SQL parsing + errors, now we aim to address all of the message-induced error. We assume + that all server-generated errors are such - mysql API unfortunately does + not provide a clear indication of what type if error this is and it is + out of question to check for hundreds of error codes. + closes https://github.com/rsyslog/rsyslog/issues/1830 +- ommysql bugfix: do not duplicate entries on failed transaction + If a multi-message batch contains data errors, messages may be + duplicated as connection close is implicit commit (not rollback). + This patch introduces a specific rollback request in those cases. + closes https://github.com/rsyslog/rsyslog/issues/1829 +- imtcp bugfix: parameter priorityString was ignored + defaults were always used +- template/bugfix: invalid template option conflict detection + This prevented "option.casesensitive" to be used with the SQL and JSON + options. +- core/actions: fix handling of data-induced errors + Rsyslog core should try to detect data-induced (unrecoverable) errors + during action processing. An example of such is invalid SQL statements. + If the action returns a data error, rsyslog shall retry individual + messages from a batch in an effort to log those without errors. The others + shall be dropped. + This logic was temporarily disabled after the switch to the new v8 + transaction interface. Now this bug is fixed and the testbench has been + amended to detect problems in the future. + closes https://github.com/rsyslog/rsyslog/issues/974 +- core/action bugfix: no "action suspended" message during retry processing + The action engine does not emit "action suspended" messages but "resumed" + ones in retry processing. This is wrong, as they are a strong indication + that something does not work correctly. Nevertheless, "resumed" messages + were emitted, which was very puzzling for the user. + This patch fixes it so that "suspend" messages are given during retry + processing. These do not contain a retry timestamp, providing evidence + that a retry is currently being tried. + closes https://github.com/rsyslog/rsyslog/issues/1069 +- core/ratelimit bugfix: race can lead to segfault + There was a race in iminternalAddMsg(), where the mutex is + released and after that the passed message object is accessed. + If the mainloop iterates in between, the msg may have already + been deleted by this time, resulting in a segfault. + Most importantly, there is no need to release the mutex lock + early, as suggested by current code. Inside mainloop the mutex + is acquired when it is time to do so, so at worst we'll have a + very slight wait there (which really does not matter at all). + This only happens if a large number of internal messages are emitted. + closes https://github.com/rsyslog/rsyslog/issues/1828 +- core bugfix: rsyslog aborts if errmsg is generated in early startup + Note that the segfault can occur only during early startup. Once + rsyslog has started, everything works reliably. This condition can + especially be triggered by specifying invalid TLS default certificates. + closes https://github.com/rsyslog/rsyslog/issues/1783 + closes https://github.com/rsyslog/rsyslog/issues/1786 +- core bugfix: informational messages was logged with error severity + When the main loop reaped a child process (a normal action), this was + reported as an error. This caused user confusion. Now it is reported as + an informational message. +- core bugfix: --enable-debugless build was broken + This was a regression from the v8.29.0 debug enhancements + Thanks to Stephen Workman for the patch. +- queue bugfix: file write error message was incorrect + when a queue was restarted from disk file, it almost always + emitted a message claiming + "file opened for non-append write, but already contains xxx bytes" + This message was wrong and did not indicate a real error condition. + The predicate check was incorrect. + closes https://github.com/rsyslog/rsyslog/issues/170 (kind of) +- omrelp bugfix: segfault when rebindinterval parameter is used +- imudp bugfix: UDP oversize message not properly handled + When a message larger than supported by the UDP stack is to be sent, + EMSGSIZE is returned, but not specifically handled. That in turn + will lead to action suspension. However, this does not make sense + because messages over the UDP max message size simply cannot be sent. + closes https://github.com/rsyslog/rsyslog/issues/1654 +- core bugfix: memory corruption during configuration parsing + when omfwd is used with the $streamdriverpermittedpeers legacy + parameter, a memory corruption can occur. This depends on the + length of the provided strings and probably the malloc subsystem. + Once config parsing succeeds, no problem can happen. + Thanks to Brent Douglas for initially reporting this issue and + providing great analysis. + Thanks to github user bwdoll for analyzing this bug and providing + a suggested fix (which is almost what this commit includes). + closes https://github.com/rsyslog/rsyslog/issues/1408 + closes https://github.com/rsyslog/rsyslog/issues/1474 +- core bugfix: race on worker thread termination during shutdown + The testbench got some occasionally failing tests. Review of + them brought up the idea that there is a race during worker + threat termination. Further investigation showed that this + might be a long-standing issue, but so far did not really + surface as the timing was almost always correct. However, + with the new functionality to emit a message on worker + shutdown (v8.29), the timing got more complex and now this + seemed to occasionally surface. + closes https://github.com/rsyslog/rsyslog/issues/1754 +- omelasticsearch: avoid ES5 warnings while sending json in bulkmode + do this by adding proper content type header to ES request + Thanks to William Dauchy for the patch +- omelasticsearch bugfix: incompatibility with newer ElasticSearch version + ElasticSearch changed its API in newer versions. When "bulkmode" is enabled + in omelasticsearch, rsyslog seems to consider all responses from Elasticsearch + as errors, even the successful ones. As a consequence, every single request + ends up being logged into the error file. + closes https://github.com/rsyslog/rsyslog/issues/1731 + Thanks to Vincent Quéméner for the patch. +- imptcp bugfix: invalid mutex addressing on some platforms + code did not compile on platforms without atomic instructions + Thanks to github user petiepooo for the patch +- imptcp bugfix: do not accept missing port in legacy listener definition + If legacy listener definition was used, a missing port was accepted during + the config read phase but errored out upon listener activation. This now + errors out directly when processing the config directive. +------------------------------------------------------------------------------ +Version 8.29.0 [v8-stable] 2017-08-08 +- imptcp: add experimental parameter "multiline" + This enables a heuristic to support multiline messages on raw tcp syslog + connections. +- imptcp: framing-related error messages now also indicate remote peer + This is the case e.g. for oversize messages. +- imtcp: framing-related error messages now also indicate remote peer + This is the case e.g. for oversize messages. +- imptcp: add session statistics counter + - session.opened + - session.openfailed + - session.closed +- imtcp: add ability to specify GnuTLS priority string + This permits to set cipher details on a very low level. +- impstats: add new resource counter "openfiles" +- pmnormalize: new parser module + Parser module that uses liblognorm to parse incoming messages. +- core/queue: provide informational messages on thread startup and shutdown + This provides better insight into operational state of rsyslog and is useful + in monitoring system health. Note that this causes the emission of messages + not previously seen. They are of syslog.informational priority. +- omfwd/udp: improve error reporting, deprecate maxerrormessages parameter + Generally improved udp-related error messages (e.g. they now contain the + socket number, which makes it easier to related them to errors reported by + net.c subsystem). + We also deprecated (removed) the "maxerrormessages" configuration parameters. + It provided some very rough rate-limiting capabilities and was introduced + before we had native rate-limiters. The default was that only the first 5 + error messages were actually reported. For long-running instances, that + meant that in many cases no errors were ever reported. We now use the default + internal message rate limiter, which works far better and ensures that also + long-running instances will be able to emit error messages after prolonged + runtime. In contrast, this also means that users will see more error + messages from rsyslog, but that should actually improve the end user + experience. +- core: add parameters debug.file and debug.whitelist + allows one to generate debug log output only of specific files + Background information available at: + https://www.linkedin.com/pulse/improving-rsyslog-debug-output-jan-gerhards +- core/net.c: improve UDP socket creation error messages +- omfwd/udp: add "udp.sendbuf" parameter +- core: make rsyslog internal message rate-limiter configurable + New parameters "internalmsg.ratelimit.interval" and "internalmsg.ratelimit.burst" + have been added. +- omelasticsearch bugfixes and changed ES5 API support: + * avoid 404 during health check + Omelasticsearch responds differently to HTTP HEAD and GET requests and + returns correct state only on GET requests. This patch works around + that ES bug and always does a GET request even when technically a HEAD + request would be sufficient. + * avoid ES5 warnings while sending json + ES5 is generating warnings when sending json without the proper header: + $ curl -i -H "Content-Type: text/json" -XGET 'http://elasticsearch5:9200/' \ + -d '{}\n' + HTTP/1.1 200 OK + Warning: 299 Elasticsearch-5.4.3-eed30a8 "Content type detection for rest + requests is deprecated. Specify the content type using the [Content-Type] + header." "Wed, 26 Jul 2017 14:33:28 GMT" + no issue on previous version. + Now, the header is set as application/json. It works for all versions + (tested on ES2 and ES5) we also handle the bulkmode where it should be + set to application/x-ndjson + closes https://github.com/rsyslog/rsyslog/issues/1546 + * bugfix for memory leak while writing error file + Thanks to William Dauchy for providing the patches +- imfile bugfix: wildcard detection issue on path wildcards + Wildcards mode was not properly detected when wildcards + were only used in a directory name on startup. + This caused imfile not to create a proper dynamic filewatch. + closes: https://github.com/rsyslog/rsyslog/issues/1672 +- omfwd bugfix: always give reason for suspension + In many cases, no error message were emitted when omfwd + went into action suspension, which was confusing for end + users. This patch enables explicit error messages in all + those cases. + closes https://github.com/rsyslog/rsyslog/issues/782 +- omfwd bugfix: configured compression level was not properly used + Thanks to Harshvardhan Shrivastava for the patch. +- imptcp bugfix: potential socket leak on session shutdown + imptcp first tries to remove a to-be-shut-down socket from the + epoll set, and errors out if that does not work. In that case, the + underlying socket will be leaked. + This patch refactors the code; most importantly, it is not necessary + to remove the socket from the epoll set, as this happens automatically + on close. As such, we simply remove that part of the code, which + also removes the root cause of the socket leak. +- omfwd/omudpspoof bugfix: switch udp client sockets to nonblocking mode + On very busy systems, we see "udp send error 11" inside the logs, and the requesting + action is being suspended (and later resumed). During the suspension period (in + default configuration), messages are lost. Error 11 translates to EAGAIN and the + cause of this problem is that the system is running out of UDP buffer space. This + can happen on very busy systems (with busy networks). + It is not an error per se. Doing a short wait will resolve the issue. The real root + cause of the issue is that omfwd uses a nonblocking socket for sending. If it were + blocking, the OS would block until the situation is resolved. The need for a + non-blocking sockets is a purely historical one. In the days of single-threaded + processing (pre v3), everything needed to be done by multiplexing, and blocking was + not permitted. Since then, the engine has dramatically changed. Actions now run on + their own thread(s). As such, there is no longer a hard need to use non-blocking i/o + for sending data. Many other output plugins also do blocking wait (e.g. omelasticsearch). + As such, the real root cause of the trouble is unnecessarily using non-blocking mode, + and consequently the right solution is to change that. + Note that using blocking i/o might change some timing inside rsyslog, especially + during shutdown. So theoretical there is regression potential in that area. However, + the core is designed to handle that situation (e.g. there is special shutdown code to + handle the blocking case), so this does not stand against the "proper" solution. + This patch applies the change on the rsyslog core level, within net.c. The only + users of the changed functionality are omfwd and omudpspoof. Imudp is unaffected as + it requests server sockets. + Note that according to the sendto() man page, there is a second cause for the EAGAIN + error, this is when the system temporarily runs out of ephemeral ports. It is not + 100% clear if this can also happen in the blocking case. However, if so, we can argue + this is a case where we really want the default retry logic. So for the time being, + it is appropriate to not handle EAGAIN in a special case any longer. + closes https://github.com/rsyslog/rsyslog/issues/1665 +- imklog: fix permitnonkernelfacility not working +- impstats bugfix: impstats does not handle HUP + If the parameter "log.file" is specified, impstats writes its own + log file. However, HUP is not handled for this file, which makes + the functionality unusable with log rotation. It is also counter- + intuitive for users. + This patch enables correct HUP processing. As a sideline, it also + introduces a generic HUP processing framework for non-action type + of loadable modules. + closes https://github.com/rsyslog/rsyslog/issues/1662 + closes https://github.com/rsyslog/rsyslog/issues/1663 +- core bugfix: segfault after configuration errors +- core/queue bugfixes: + * Fix behavior of PersistStateInterval + If PersistStateInterval=1, then each log line read should cause the state file + to be updated, but this was not happening because nRecords was being post-increment. + Thanks to Anthony Howe for the patch. + * potential problem during deserialization + if queue object deserialization fails, invalid memory blocks might be + free'ed. + For more information see https://github.com/rsyslog/rsyslog/pull/1647 + Thanks to Derek Smith for the patch. +- core bugfix: message garbled after message modification + The MsgDup() function will return a garbled message object under these + conditions: The message was originally created with message length equal or + larger to CONF_RAWMSG_BUFSIZE. This makes rsyslog store the message in + dynamically allocated buffer space. Then, a component reduces the message + size to a size lower than CONGF_RAWMSG_BUFSIZE. A frequent sample is the + parser removing a known-bad LF at the end of the messages. Then, MsgDup is + executed. It checks the message size and finds that it is below + CONF_RAWMSG_BUFSIZE, which make it copy the msg object internal buffer + instead of the dynamically allocated one. That buffer was not written to in + the first place, so uninitialized data is copied. Note that no segfault can + happen, as the copied location was properly allocated, just not used in + this processing flow. In the end result, the new message object contains + garbage data. Whenever the new object is used (e.g. in a async ruleset or + action) that garbage will be used. Whenever the old object is accessed, + correct data will be used. Both types of access can happen inside the + same processing flow, which makes the problem appear to be random. + closes https://github.com/rsyslog/rsyslog/issues/1658 +- lmsig_ksi: removed pre-KSI_LS12 components + As of GuardTime, the provider, these no longer work due to backend + changes. The lmsig_ksi_ls12 module shall be used instead. This is + available since 8.27.0. +- testbench bugfix: hang in tests if omstdout is not present + Many tests depend on omstdout. Given the fact that omstdout + is really only useful for the testbench (at least that's the intent), + we now require --enable-omstdout if --enable-testbench is given. + The alternative would have been to disable all those tests that + need it, which would have lead to considerable less testbench + coverage. + closes https://github.com/rsyslog/rsyslog/issues/1649 +------------------------------------------------------------------------------ +Version 8.28.0 [v8-stable] 2017-06-27 +- NEW BUILD REQUIREMENT: librelp 1.2.14 (to build relp components) + This was necessary because imrelp requires an API introduced in 1.2.14. +- omfwd: add parameter "tcp_frameDelimiter" +- omkafka: large refactor of kafka subsystem + This offers improvements and greatly increases reliability. + Closes https://github.com/rsyslog/rsyslog/issues/1559 + Closes https://github.com/rsyslog/rsyslog/issues/1584 + Closes https://github.com/rsyslog/rsyslog/issues/1515 + Closes https://github.com/rsyslog/rsyslog/issues/1052 + May fix https://github.com/rsyslog/rsyslog/issues/1230 +- imfile: improved handling of atomically renamed file (w/ wildcards) + if a file is atomically renamed, the state file is also being renamed, + so processing continues as if the original name were kept. + see also: https://github.com/rsyslog/rsyslog/issues/1417 +- imfile: add capability to truncate oversize messages or split into multiple + also in this case an error message is emitted. Both of these actions are + configurable. This also solves memory issues when an endregex does not + match for prolonged time. In that case, large parts of the file were + previously buffered, which could cause big problems in case e.g. the + endregex was simply wrong and never matched. For the later, see also + https://github.com/rsyslog/rsyslog/issues/1552 +- mmdblookup + * upgraded from "contrib" to "fully supported" state + * refactored and simplified code + * added ability to specify custom names for extracted fields + * added ability to specify container name for extracted fields + * bugfix: fixed multiple memory leaks +- imptcp: add new parameter "flowControl" +- imrelp: add "maxDataSize" config parameter + Thanks to Nathan Brown for the patch. +- multiple modules: gtls: improve error if certificate file can't be opened +- omsnare: allow different tab escapes + Thanks to Shane P. Lawrence for the patch. +- omelasticsearch: converted to use libfastjson instead of json-c + json-c was used for historical purposes, and it's source included + within the rsyslog source tree. We now use libfastjson inside all + components. +- imjournal: _PID fallback + * added fallback for _PID property when SYSLOG_PID is not available + * introduced new option "usepid" which sets which property should + rsyslog use, it has 3 states system|syslog|both, default is both + * deprecated "usepidfromsystem" option, still can be used + and override the "usepid" + * it is possible to revert previous default with usepid="syslog" + Thanks to Radovan Sroka for the patch +- multiple modules: add better error messages when regcomp is failing +- omhiredis: fix build warnings + Thanks to Brian Knox for the fix. +- imfile bugfix: files mv-ed in into directory were not handled + Thanks to Zachary M. Zhao for the patch. + see also https://github.com/rsyslog/rsyslog/issues/1588 +- omprog bugfix: execve() incorrectly called + this caused failures on some but not all platforms + Thanks to 張君天(Chun-Tien Chang) and Matthew Seaman for the patch. +- imfile bugfix: multiline timeout did not work if state file exists + The timeout feature for multiline reads does not correctly work for + files for which a state file existed. This is usually the case for files + that had been processed by a previous run and that still exist on the + new start. For all other files, especially those monitored by a + wildcard and newly created after the rsyslog start, timeout worked as + expected. + closes https://github.com/rsyslog/rsyslog/issues/1445 +- lmsig_ksi-ls12 bugfix: build problems on some platforms +- core bugfix: invalid object type assertion + This lead to aborts due to failing assertion. Note that this could only + happen during debugging runs which includes runtime instrumentation, + something that never should be done in a stable production build. + So this did not affect ordinary users, only developers in with + deep debugging settings. +- regression fix: local hostname was not always detected properly... + ... on early start (w/o network). Introduced in 8.27.0. + Thanks to github user jvymazal for the patch and whissi for + reporting and helping with the analysis. +- bugfix: format security issues in zmq3 modules + see also: https://github.com/rsyslog/rsyslog/pull/1565 + Thanks to Thomas D. (whissi) for the patch. +- bugfix build system: add libksi only to those binaries that need it + Thanks to Allan Park for the patch. +- bugfix KSI ls12 components: invalid tree height calculation + Thanks to Allan Park for the patch. +- testbench/CI enhancements + * re-enable and add kafka tests + Kafka tests were disabled in 8.27.0 (a regression from imkafka). + * better testbench coverage for mmdblookup + * lmsig_ksi-ls12 is now being built at least on Centos7 +------------------------------------------------------------------------------ +Version 8.27.0 [v8-stable] 2017-05-16 +- imkafka: add module +- imptcp enhancements: + * optionally emit an error message if incoming messages are truncated + * optionally emit connection tracking message (on connection create and + close) + * add "maxFrameSize" parameter to specify the maximum size permitted + in octet-counted mode + * add parameter "discardTruncatedMsg" to permit truncation of + oversize messages + * improve octect-counted mode detection: if the octet count is larger + then the set frame size (or overly large in general), it is now + assumed that octet-stuffing mode is used. This probably solves a + number of issues seen in real deployments. +- imtcp enhancements: + * add parameter "discardTruncatedMsg" to permit truncation of + oversize messages + * add "maxFrameSize" parameter to specify the maximum size permitted + in octet-counted mode +- imfile bugfix: "file not found error" repeatedly being reported + for configured non-existing file. In polling mode, this message + appeared once in each polling cycle, causing a potentially very large + amount of error messages. Note that they were usually emitted too + infrequently to trigger the error message rate limiter, albeit often + enough to be a major annoyance. +- imfile: in inotify mode, add error message if configured file cannot + be found +- imfile: add parameter "fileNotFoundError" to optionally disable + "file not found" error messages +- core: replaced gethostbyname() with getaddrinfo() call + Gethostbyname() is generally considered obsolete, is not reentrant and + cannot really work with IPv6. Changed the only place in rsyslog where + this call remained. + Thanks to github user jvymazal for the patch +- omkafka: add "origin" field to stats output + See also https://github.com/rsyslog/rsyslog/issues/1508 + Thanks to Yury Bushmelev for providing the patch. +- imuxsock: rate-limiting also uses process name + both for the actual limit processing as well as warning messages emitted + see also https://github.com/rsyslog/rsyslog/pull/1520 + Thanks to github user jvymazal for the patch +- Added new module: KSI log signing ver. 1.2 (lmsig_ksi_ls12) +- rsyslog base functionality now builds on osx (Mac) + Thanks to github user hdatma for his help in getting this done. +- build now works on solaris again +- imfile: fix cross-platform build issue + see also https://github.com/rsyslog/rsyslog/issues/1494 + Thanks to Felix Janda for bug report and solution suggestion. +- bugfix core: segfault when no parser could parse message + This could happen if the default parser chain was changed and the + RFC3164 parser was not included. Never seen in practice, just by + experimenting in lab. +- bugfix core: rate-limit internal messages when going to external log system + Rate-limiting was only applied to messages processed internally. + While the external logging system probably also applies rate-limiting, + it would be preferable that rsyslog applies the same policies on + internal messages, no matter where they go. This is now the case. +- bugfix core: when obtaining local hostname, a NULL pointer could be + accessed. This depends on many factors, among them that no local host + name is configured in rsyslog.conf AND the local system configuration + also is set to an empty hostname. + Thanks to github user jvymazal for the patch. +- bugfix core: on shutdown, stderr was written to, even if already closed + This lead to messages going to whatever was associated with file + descriptor two. + Thanks to Allan Park for the patch. +- bugfix core: perform MainqObj destruction only when not NULL already + This affects the config object; in theory may lead to misaddressing during + config parsing. + Thanks to github user jvymazal for the patch +- bugfix core: memory leak when internal messages not processed internally + In this case, the message object is not destructed, resulting in + a memory leak. Usually, this is no problem due to the low number + of internal message, but it can become an issue if a large number + of messages is emitted. + closes https://github.com/rsyslog/rsyslog/issues/1548 + closes https://github.com/rsyslog/rsyslog/issues/1531 +- bugfix imptcp: potential overflow in octet count computation + when a very large octet count was specified, the counter could overflow +------------------------------------------------------------------------------ +Version 8.26.0 [v8-stable] 2017-04-04 +- NEW BUILD REQUIREMENT: liblognorm 2.0.3 is required for mmnormalize + If mmnormalize is not built, the build requirements are unchanged. + The new version is necessary because it contains an enhanced API for a + new mmnormalize feature. +- enable internal error messages at all times + This is an important change to the design of the rsyslog core internal + error message system. Previous code could deadlock if internal messages were + issued inside queue processing code, which effectively limited error-reporting + capabilities. This change makes it possible to call error messages from any + part of the code at any time. + As a side-effect, this also fixes an issue where rsyslog could deadlock if + imuxsock submited messages to the system log socket when that socket blocked. + This was a rare race, albeit consistently reproducible and also seen in + practice. The work-around for this issue was to set + global(processInternalMessages="on") + in rsyslog.conf. With the new code, this race can never happen again. The new + code also sets stage for emitting better error messages, especially in places + where we previously refrained from doing so and messages went only to the + debug log. For some file output and queue subsystem related messages, this + is already done, but there is still further work required. + Note well: this is a redesign of an important core component. While intensely + tested, this may still have some larger regression potential than usual code + changes. +- core: added logging name of source of rate-limited messages + This adds the name to the rate-limiting message itself, making it easier + to identify the actual source of "spam" messages. + Thanks to github user jvymazal for the patch. +- omfwd: omfwd: add support for network namespaces + Thanks to Bastian Stender for the patch. +- imrelp: honor input name if provided when submitting to impstats + Thanks to Jon Henry for the patch. +- imptcp: add ability to set owner/group/access perms for uds + Thanks to Nathan Brown for implementing this feature. +- mmnormalize: add ability to load a rulebase from action() parameter + This is especially useful for small rulebases, as it avoids having + a separate rulebase file. + closes https://github.com/rsyslog/rsyslog/issues/625 +- pmrfc3164 improvements + - permit to ignore leading space in MSG + - permit to use at-sign in host names + - permit to require tag to end in colon + Thanks to github user bdolez for the contribution +- add new global parameter "umask" + This is equivalent to "$umask" and permits to convert that construct + to new-style config format. + closes https://github.com/rsyslog/rsyslog/issues/1382 +- core: make use of -T command line option more secure + When the -T option is used, a chdir is now done right after chroot. It must + be noted, though, that -T is a testing option and has no intent to provide + real security. So this change does not mean it actually is sufficiently + secure. + Thanks to github user jvymazal for the patch. +- omfile: add error if both file and dynafile are set +- bugfix: build problem on MacOS (not a supported platform) + Thanks to FX Coudert for the fix. +- regression fix: in 8.25, str2num() returned error on empty string + past behavior was to return 0 in this case; past behavior was reinstantiated + Thanks to github user jvymazal for the patch. +- bugfix omsnmp: improper handling of new-style configuration parameters + Thanks to Radovan Sroka for the patch. +- bugfix: rsyslog identifies itself as "liblogging-stdlog" in internal messages + This occurred when liblogging-stdlog was used, and was used by default (without + explicit configuration). This is a regression of the new default, which does + not correctly call stdlog_open() in the default case. + closes https://github.com/rsyslog/rsyslog/issues/1442 +- bugfix imfile: wrong files were monitored when using multiple imfile inputs + The bug was introduced by the changes for the multilevel wildcard feature + in 8.25.0. We have to handle FileCreate events different if the directory + watch is configured or added dynamically. + closes https://github.com/rsyslog/rsyslog/issues/1452 +- bugfix: setting net.aclResolveHostname/net.acladdhostnameonfail segfaults + When compiling using hardened gcc (gentoo), specifying net.aclResolveHostname + or net.acladdhostnameonfail results in rsyslogd segfaulting on startup. + Thanks to Radovan Sroka for the patch. +- bugfix: immark emitted error messages with text "imuxsock" + Thanks to Chris Pilkington for the patch. +- bugfix tcpflood: build failed if RELP was disabled +- fix gcc6 compiler warnings + This also fixes a small bug with incorrectly working deprecated -u + command line option. + Thanks to Nikola Pajkovsky for the patch. +- the output module array passing interface has been removed + It wasn't functional since the v8 update, and the only user was omoracle, + which is a contributed module that is no longer maintained. So we + removed that interface to streamline the code. Should it ever be needed + again, we could check the 8.25 code base. Note, though, that that code + still needs to be adjusted to the v8 engine. +- testbench: + * tcpflood now automatically enters silent mode during Travis CI testing + This reduces testbench output, which is limited under Travis. + * the libqpid-proton package is no longer available for Ubuntu trusty. As + such, we disabled its use in Travis on this platform. Right now, this + means omamqp1 module is no longer tested on trusty. +------------------------------------------------------------------------------ +Version 8.25.0 [v8-stable] 2017-02-21 +- imfile: add support for wildcards in directory names + This now permits to monitor newly created directories without altering + the configuration. +- add new global option "parser.PermitSlashInProgramname" +- mmdblookup: fix build issues, code cleanup + Thanks to Dan Molik for the patch. +- improved debug output for queue corruption cases +- an error message is now displayed when a directory owner cannot be set + This mostly happens with omfile and dynafile. The new messages + facilitates troubleshooting. +- rainerscript: + * add new function ipv42num + * add new function num2ipv4 +- bugfix: ratelimiter does not work correctly is time is set back + Thanks to github user hese10 for the patch. + see also https://github.com/rsyslog/rsyslog/issues/1354 +- core: fix potential message loss in old-style transactional interface + This was experienced for example by omrelp. Could loose one message per + broken connection, iff that message did not make it to the unacked list. +- bugfix queue subsystem: queue corrupted if certain msg props are used + The core issues was in the msg object deserializer, which had the wrong + deserialization sequence. That in turn lead to queue corruption issues. + Corruption of disk queue (or disk part of DA queue) always happens if + the "json" property (message variables) is present and "structured-data" + property is also present. This causes rsyslog to serialize to the + queue in wrong property sequence, which will lead to error -2308 on + deserialization. + Seems to be a long-standing bug. Depending on version used, some or + all messages in disk queue may be lost. + closes https://github.com/rsyslog/rsyslog/issues/1404 +- bugfix imjournal: fixed situation when time goes backwards + This is resolving the situation when system is after reboot and + boot_id doesn't match so cursor pointing into "future". + Usually sd_journal_next jump to head of journal due to journal + approximation, but when system time goes backwards and cursor is + still invalid, rsyslog stops logging. + We use sd_journal_get_cursor to validate our cursor. + When cursor is invalid we are trying to jump to the head of journal + This problem with time should not affect persistent journal, + but if cursor has been intentionally compromised it could stop + logging even with persistent journal. +- bugfix: bFlushOnTxEnd == 0 not honored when writing w/o async writer + If bFlushOnTXEnd is set, we need to flush on transaction end - in + any case. It is not relevant if this is using background writes + (which then become pretty slow) or not. And, similarly, no flush + happens when it is not set. + see also https://github.com/rsyslog/rsyslog/issues/1297 +- bugfix core: str2num mishandling empty strings + If str2num() receives an empty string, misaddressing happens. + This theoretically can lead to a segfault if a RainerScript function + is used inside the configuration which calls code that could trigger + this bug. + closes https://github.com/rsyslog/rsyslog/issues/1412 +- bugfix rainerscript: set/unset statement do not check variable name validity + Only JSON-based variables can be use with set and unset. Unfortunately, + this restriction is not checked. If an invalid variable is given + (e.g. $invalid), this is not detected upon config processing on + startup. During execution phase, this can lead to a segfault, a + memory leak or other types of problems. + Thanks to github user mostolog for reporting and helping to analyze + this issue. + see also https://github.com/rsyslog/rsyslog/issues/1376 + closes https://github.com/rsyslog/rsyslog/issues/1377 +- bugfix mmrm1stspace: last character of rawmsg property was doubled +- bugfix: rsyslog loops on freebsd when trying to write /dev/console + Rsyslog 8.23.0 loops on FreeBSD when trying to access a (now revoked) + /dev/console file descriptor, as per Alexandre's original bug report [1]. + The original patch fixes the problem when tryTTYRecover() sees errno 6 ENXIO. + Running FreeBSD 10-stable here and getting errno 5 EIO, same as Xavier gets + in his 2016 bug report [2]. + New patch [3] includes errno 5 to tryTTYRecover() in runtime/stream.c and + fixes the problem for me, on multiple machines. + [1] https://github.com/rsyslog/rsyslog/issues/371 + [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211033 + [3] https://bz-attachments.freebsd.org/attachment.cgi?id=178452 + closes https://github.com/rsyslog/rsyslog/issues/1351 + Thanks to Damien Fleuriot for the patch. +- bugfix imtcp: fix very small (cosmetic) memory leak + For each listener, the name of an assigned ruleset is not freed. This + is cosmetic, because it is a very small static leak AND it needs to + be kept until end of run anyways (and at end of run the OS frees it). + However, the leak breaks memleak checks in the testbench. +- fix build issues on some platforms (detected on newer Fedora) +------------------------------------------------------------------------------ +Version 8.24.0 [v8-stable] 2017-01-10 +- rsyslog now builds on AIX + see also: https://github.com/rsyslog/rsyslog/pull/1247 + Thanks to github user purnimam1 and the team at IBM + Note: the rsyslog project has no AIX platform to ensure that future versions + will remain buildable on AIX. If you would like to contribute resources, + please contact the rsyslog team. +- mmdblookup: new maxminddb lookup message modify plugin + Thanks to 饶琛琳 (github user chenryn) for the contribution +- mmrm1stspace: new module; removes first space in MSG if present +- KSI signature provider: file permissions can now be specified + This happens via parameters equal to those used by omfile itself. + Note that KSI files can have different permissions/owner than the log + files themself. + Thanks to Allan Park for the patch. +- omzmq: new features + Thanks to Brian Knox for the patch. +- change: when the hostname is empty, we now use "localhost-empty-hostname" + In 8.23.0, "localhost" was used in this case, but that could be misleading. + The new name makes the error condition (gethostname() should always return + a non-empty name) more obvious. +- omelasticsearch: remove "asyncrepl" config parameter + The _bulk index endpoint on ElasticSearch 5.0 no longer + ignores the ?replication=async query parameter. It was deprecated + since 1.5 and silently ignored in 2.x but passing it to a 5.x + instance breaks omelasticsearch with a 400 response. + closes https://github.com/rsyslog/rsyslog/issues/1250 +- omfwd: Add support for bind-to-device (see below on same for imudp) +- imudp: Add support for bind-to-device + Add support for bind-to-device option to omfwd and imudp modules. + Configured using device="name". Only new syntax format is supported. + e.g., + input(type="imudp" port=["10514"] device="eth0" name="udp") + Action(type="omfwd" Target="192.168.1.23" Port="10514" Device="eth0") + see also https://github.com/rsyslog/rsyslog/pull/1261 + Thanks to David Ahern for the patch. +- imudp: limit rcvbufsize parameter to max 1GiB +- rainerscript: implement new "call_indirect" statement +- bugfix imjournal: make state file handling more robust + There is a bug in rsyslog which is caused by not very atomic writes of + imjournal statefile. It's hardly reproducible but there is a way. + fscanf error appears only when rsyslog reads an empty statefile which + causes that imjournal is stopped so no logging from journal is + performed. When the statefile contains random bytes error appears + again but from journal and imjournal is stopped too. + In this patch Rsyslog writes imjournal statefile more atomically and + secure. Reading the statefile is more robust and doesn't affect + imjournal module so when corrupted statefile is read imjournal + ignores statefile, continues with logging and it doesn't stop. Logger + can be used as a test if it's logging or not. + Patch introduces a new option with both old and new config format + "IgnoreNonValidStateFile" which is "on" by default and it can turn + off ignorance of non valid statefile. + Thanks to github user tomassykora for the patch. +- bugfix core: lookup table reload was not properly integrated + The opcode was not handled, what lead to misleading messages + in debug log. Since we run debug builds more strictly, it also + causes an assertion to trigger, thus aborting the test +- bugfix core: potential deadlock on shutdown + could happen when rsyslog was started and quickly shut down OR when + coincidentally a new thread was spawend "with bad timing" around the time + of shutdown. + See also https://github.com/rsyslog/rsyslog/pull/1274 + Thanks to github user tomassykora for the final patch and Rado Sroka for + the analysis and an initial patch. +- bugfix ommongodb: did not work in v8 due to invalid indirection + Thanks to Benoit Dolez for the patch. +- bugfix ommongodb: fix tryResume handling + To make tryResume working, the connection to mongodb need to be closed. + Thus close it on "insert error". + Thanks to Benoit Dolez for the patch. +- bugfix omfwd: retry processing was not done correctly, could stall + see also https://github.com/rsyslog/rsyslog/pull/1261 + Thanks to David Ahern for the patch. +- bugfix imuxsock: segfault non shutdown when $OmitLocalLogging is on + Imuxsock tries to close socket on index 0 which ends with segfault. + Thanks to Tomas Sykora for the patch. +- testbench: + * empty-hostname test did not work correctly + * improve debugging by better output +------------------------------------------------------------------------------ +Version 8.23.0 [v8-stable] 2016-11-15 +- NEW BUILD REQUIREMENT: libfastjson 0.99.3 + This was introduced in 8.20.0 as a suggestion and has now become a hard + requirement. See 8.20.0 ChangeLog entry for more details. +- KSI signatures: removed SHA2-224 hash algorithm + This is considered insecure and no longer supported by the underlying + KSI library. If still used within a configuration, a descriptive error + message is emitted during config processing. + Thanks to Henri Lakk for the initial patch. +- imfile: new timeout feature for multi-line reads + When using startmsg.regex, messages are held until the next one is written. + We now provide a "readTimeout" parameter family (see doc) to timeout such + reads and ensure messages are not held for a very long time. + see also https://github.com/rsyslog/rsyslog/issues/1133 +- omfile: improve robustness against network file system failures + in case of failure, a close and re-open is tried, which often solves the + issue (and wasn't handle before this patch). + see also https://github.com/rsyslog/rsyslog/pull/1161 + Thanks to github user hese10 for the patch. +- pmaixforwardedfrom: support for AIX syslogd -s option + if syslog in AIX started with "-s" option, AIX syslog adds only "From " + instead of "Message forwarded from ". With this patch, both are now + detected. + Thanks to github user patelritesh for the patch. +- omelasticsearch: add ability to specify max http request size + This permits to keep batches below ES-configured limits. + Thanks to github user barakshechter for the patch. +- omelasticsearch: high availability addressing of ElasticSearch cluster + allow one to specify an array of servers, which is tried until a working + one is found (and given up only if none works). + Thanks to github user barakshechter for the patch. +- omelasticsearch: make compatible with ElasticSearch 2.x and 5.x + fixes omelasticsearch logs response from ElasticSearch 5.0 _bulk + endpoint as error + See also https://github.com/rsyslog/rsyslog/pull/1253 + Thanks to Christian Rodemeyer for the patch. +- omhiredis: add dynakey attribute. + If set to on, this causes omhiredis to treat the key attribute as the + name of a template so that you can build a dynamic redis queue name + or list key. + see also: https://github.com/rsyslog/rsyslog/pull/1218 + Thanks to github user bobthemighty for the patch +- omtcl: new contributed module + see also https://github.com/rsyslog/rsyslog/pull/1041 + Please note: contributed modules are not directly supported by the + project. You might want to contact the author for questions. + Thanks to Francisco Castro for contributing it. +- RainerScript: provide a capability to set environment variables + via 'global(environment="var=val")' config statement. + This is most importantly for things like HTTP_PROXY. + see also https://github.com/rsyslog/rsyslog/issues/1209 +- lookup tables: improved error checking + Thanks to Janmejay Singh for the patch. +- queue subsystem: add configuration parameter "queue.samplinginterval" + Supports sampling of messages (as often used in data transmission). + Thanks to Zachary M. Zhao for the patch. +- bugfix core: errmsg.LogError now switches to dfltErrLogger just before shutdown + Thanks to Janmejay Singh for the patches. +- bugfix core: fixed un-freed memory in non-transactional action using string-passing + closes https://github.com/rsyslog/rsyslog/issues/968 + Thanks to Janmejay Singh for the patches. +- rsgtutil: option to specify KSI publications file certificate constraints + see also https://github.com/rsyslog/rsyslog/issues/1207 +- omprog: bugfixes and enhancements + - omprog resource leak fix (fd leak) + - omprog got ability to force-kill process if it doesn't die in 5 seconds + (linux specific) + - child-process lifecycle debugging aid (in form of logs) (mainLoop and + omprog cleanup both log pid at child-death, mainLoop reaping is now + visible to user, as opposed to being a mystery, because omprog didn't + seem to anticipate it in terms of code) + Thanks to Janmejay Singh for the patches. + see also https://github.com/rsyslog/rsyslog/pull/1155 +- bugfix imfile: ReopenOnTruncate processing, file open processing + This fixes + * ReopenOnTruncate was only honored when a state file existed + see https://github.com/rsyslog/rsyslog/issues/1090 + * open processing could run into a loop + see https://github.com/rsyslog/rsyslog/issues/1174 + This is done via refactoring the open file handling, which provides + overall cleaner and easier-to-follow code. + Thanks to Owen Smith for analyzing the problem and providing a + prototype PR which greatly helped towards the final solution. +- bugfix omlibdbi: libdbi-driver-sqlite3/2 requires to provide a path to + database split into two strings: + * absolute path, where the database file sits + * database filename itself. + This was previously not done. + Thanks to github user aleek for the patch. +- bugfix RainerScript: issue in prifilt() function + Initialize func-data(and to-be-freed flag) correctly for prifilt + function-node + Thanks to Janmejay Singh for the patch. +- bugfix omrelp: invalid module name imrelp was used in some error messages + Thanks to Chris Pilkington for the patch. +- bugfix core: abort when persisting object state + This causes a segfault. It happens whenever an object state larger + than 4095 byte is persisted. Then, incorrectly a try to rollover to + a new state file is tried, which will lead to a division by zero + as the necessary variables for this operation are not set because we + are NOT in circular mode. + This problem can happen wherever state files are written. It has been + experienced with imfile and queue files. + Many thanks to github user mostolog for his help in reproducing the issue, + which was very important to finally nail down this long-standing bug. + closes https://github.com/rsyslog/rsyslog/issues/1239 + closes https://github.com/rsyslog/rsyslog/issues/1162 + closes https://github.com/rsyslog/rsyslog/issues/1074 +- bugfix: segfault if hostname is unset on system + happens when gethostname() returns empty string. This will cause + the createon of the localhostname prop_t to fail, which in turn + leads to a NULL pointer dereference when emitting local messages. + As we emit a startup message by default, this had usually lead + to a segfault on startup. + Thanks to Erik Potter and github user mostolog for their help + in analyzing this problem. + closes https://github.com/rsyslog/rsyslog/issues/1040 + closes https://github.com/rsyslog/rsyslog/issues/335 +- bugfix external module perl skeleton: did not work properly + Thanks to github user Igorjan666 for the patch. +- bugfix build system: Fix detection of pthread_setschedparam() on platforms + such as FreeBSD + see also https://github.com/rsyslog/rsyslog/pull/1147 + Thanks to Matthew Seaman for the patch. +- bugfix omelasticsearch: modifies constant memory under some circumstances + Function computeBaseUrl may modify its serverParam parameter, but + this may contain the constant string "localhost". Depending on the + platform, this can lead to a segfault. + Noticed while working on compiler warnings, not seen in practice. +- "bugfix": theoretical queue file corruption when more than MAX_INT files + closes https://github.com/rsyslog/rsyslog/issues/1202 +- bug fix/KSI: LOGSIG11 missing in the beginning of KSI log signature file + When logging with KSI is not working properly for whatever reason, an + empty .ksisig file is created (which by itself is not an issue). However, + later it looks like this file is re-used, but it is not checked whether it + already contains the magic LOGSIG11 in the beginning of the file. This leads + to a log signature file which has correct content but is missing the + LOGSIG11 magic in the beginning. +- bugfix template processor: missing escaping of backslash in json mode + Thanks to github user mostolog for providing the patch. +- build environment: --enable-debug now defaults to new "auto" mode + previously, DEBUG mode (and thus assert() macros) was disabled by default + and explicitly needed to be enabled by providing the --enable-debug + ./configure switch. Now, a new --enable-debug=auto mode has been added + and made the default. It enables DEBUG mode if we build from git and only + disables it when a release build is done (from the release tarball). This + aims at better error checking during testbench runs and developer testing. +- testbench improvements + * improved testbench file generation tool + Thanks to Pascal Withopf for the patch. + * added some plumbing for extended tests which work by overriding OS APIs + * imfile ReopenOnTruncate option is now being tested + * the CI environment now runs most tests in debug mode, but some in + release mode to cover potential release-mode races + * template json option is now being tested + * object state persisting received a basic test + * added test for empty hostnames + * added tests for omprog +------------------------------------------------------------------------------ +Version 8.22.0 [v8-stable] 2016-10-04 +- ompgsql: add template support + Thanks to Radu Gheorghe for implementing this. +- generate somewhat better error message on config file syntax error + a common case (object at invalid location) has received it's own error + message; for the rest we still rely on the generic flex/bison handler +- bugfix:omhiredis reconnects after failure + previously it could loose messages under such conditions. + Thanks to Bob Gregory for the patch. +- general cleanup and code improvement + mostly guided by compiler warnings induced by newer opensuse buildbot + environment +------------------------------------------------------------------------------ +Version 8.21.0 [v8-stable] 2016-08-23 +- CHANGE OF BEHAVIOR: + by default, internal messages are no longer logged via the internal + bridge to rsyslog but via the syslog() API call [either directly or + via liblogging). For the typical single-rsyslogd-instance installation this + is mostly unnoticeable (except for some additional latency). If multiple + instances are run, only the "main" (the one processing system log messages) + will see all messages. To return to the old behavior, do either of those + two: + 1) add in rsyslog.conf: + global(processInternalMessages="on") + 2) export the environment variable RSYSLOG_DFLT_LOG_INTERNAL=1 + This will set a new default - the value can still be overwritten via + rsyslog.conf (method 1). Note that the environment variable must be + set in your **startup script**. + For more information, please visit + https://www.rsyslog.com/rsyslog-error-reporting-improved/ +- slightly improved TLS syslog error messages +- queue subsystem: improved robustness + The .qi file is now persisted whenever an existing queue file is fully + written and a new file is begun. This helps with rsyslog aborts, including + the common case where the OS issues kill -9 because of insufficiently + configured termination timeout (this is an OS config error, but a frequent + one). Also, a situation where an orphaned empty file could be left in the + queue work directory has been fixed. We expect that this change causes + fewer permanent queue failures. +- bugfix: build failed on some platforms due to missing include files +------------------------------------------------------------------------------ +Version 8.20.0 [v8-stable] 2016-07-12 +- NEW BUILD REQUIREMENT: librelp, was 1.2.5, now is 1.2.12 + This is only needed if --enable-relp is used. The new version is needed + to support the new timeout parameter in omrelp. +- NEW BUILD SUGGESTION: libfastjson 0.99.3 + while not strictly necessary, previous versions of libfastjson have a bug + in unicode processing that can result in non US-ASCII characters to be + improperly encoded and may (very unlikely) also cause a segfault. + This version will become mandatory in rsyslog 8.23.0 +- omrelp: add configurable connection timeout + Thanks to Nathan Brown for implementing this feature. +- pmrfc3164: add support for slashes in hostname + added parameter "permit.slashesinhostname" to support this, off by default + [Note that the RFC5424 always supported this, as 5424 is a different + standard] +- bugfix omfile: handle chown() failure correctly + If the file creation succeeds, but chown() failed, the file was + still writen, even if the user requested that this should be treated + as a failure case. This is corrected now. + Also, some refactoring was done to create better error messages. +- omfile now better conveys status of unwritable files back to core +- config files recursively including themselves are now detected + and an error message is emitted in that case; Previously, this + misconfiguration resulted in rsyslog loop and abort during startup. + closes https://github.com/rsyslog/rsyslog/issues/1058 +- refactored code to not emit compiler warnings in "strict mode" + We changed the compiler warning settings to be rather strict and cleaned up + the code to work without generating any warning messages. + This results in an overall even more improved code quality, which will now + also be enforced via our CI systems. +- bugfix: fix some issues with action CommitTransaction() handling + An action that returns an error from CommitTransaction() caused a + loop in rsyslog action processing. Similarly, retry processing was not + properly handled in regard to CommitTransaction(). + This is a first shot at fixing the situation. It solves the + immediate problems, but does not implement the full desired + functionality (like error file). + see also https://github.com/rsyslog/rsyslog/issues/974 + see also https://github.com/rsyslog/rsyslog/issues/500 +- bugfix omqmqp1: connecting to the message bus fails on nonstandard port + Thanks to Ken Giusti for the patch. + see also: https://github.com/rsyslog/rsyslog/pull/1064 +- testbench/CI enhancements + * new tests for RELP components + * new tests for core action processing and retry + * travis tests now also run against all unstable versions of supporting + libraries. This helps to track interdependency problems early. + * new tests for hostname parsing + * new tests for RainerScript comparisons +------------------------------------------------------------------------------ +Version 8.19.0 [v8-stable] 2016-05-31 +- NEW BUILD REQUIREMENT: autoconf-archive +- omelasticsearch: add option to permit unsigned certs (experimentally) + This adds plumbing as suggested by Joerg Heinemann and Radu Gheorghe, + but is otherwise untested. Chances are good it works. If you use it, + please let us know your experience and most importantly any bug + reports you may have. + closes https://github.com/rsyslog/rsyslog/issues/89 +- imrelp: better error codes on unavailability of TLS options + Most importantly, we will tell the user in clear words if specific TLS + options are not available due to too-old GnuTLS. + closes https://github.com/rsyslog/rsyslog/issues/1019 +- default stack size for inputs has been explicitly set to 4MiB + for most platforms, this means a reduction from the default of 10MiB, however + it may mean an increase for micro-libc's (some may have as low as 80KiB by + default). +- testbench: We are now using libfaketime instead of faketime command line + tool. Make sure you have installed the library and not just the binary! +- refactor stringbuf + * use only a single string buffer + ... both for the internal representation as well as the C-String one. + The module originally tried to support embedded NUL characters, which + over time has proven to be not necessary. Rsyslog always encodes + NUL into escape sequences. + Also, the dual buffers were used inconsistently, which could lead to + subtle bugs. With the single buffer, this does no longer happen and + we also get some improved performance (should be noticeable) + and reduced memory use (a bit). + closes https://github.com/rsyslog/rsyslog/issues/1033 + * removed no longer used code + * internal API changes to reflect new needs + * performance improvements + * miscellaneous minor cleanup +- fix: potential misaddressing in template config processing + This could cause segfault on startup. Happens when template name shorter + than two chars and outname is not set. Once we are over startup, things + work reliably. +- bugfix omfile: async output file writing does not respect flushing + neither parameter flushInterval nor flushOnTXEnd="on" was respected. + closes https://github.com/rsyslog/rsyslog/issues/1054 +- bugfix imfile: corrupted multi-line message when state data was persisted + see also https://github.com/rsyslog/rsyslog/issues/874 + Thanks to Magnus Hyllander for the analysis and a patch suggestion. +- bugfix imfile: missing newline after first line of multiline message + see also https://github.com/rsyslog/rsyslog/issues/843 + Thanks to Magnus Hyllander for the patch. +- bugfix: dynstats unusedMetricTtl bug + Thanks to Janmejay Singh for fixing this. +- bugfix build system: build was broken on SunOS + Thanks to Filip Hajny for the patch. +- bugfix: afterRun entry point not correctly called + The entry point was called at the wrong spot, only when the thread + had not already terminated by itself. This could cause various + cleanup to not be done. This affected e.g. imjournal. + closes https://github.com/rsyslog/rsyslog/issues/882 +- bugfix dynstats: do not leak file handles + Thanks to Janmejay Singh for the patch. +- bugfix omelasticsearch: disable libCURL signal handling + previously, this could lead to segfaults on connection timeout + see also https://github.com/rsyslog/rsyslog/pull/1007 + Thanks to Sai Ke WANG for the patch. +- bugfix omelasticsearch: some regressions were fixed + * error file was no longer written + * fix for some potential misaddressings +- improved wording: gnutls error message points to potential cause + What GnutTLS returns us is very unspecific and somewhat misleading, so + we point to what it most probably is (broken connect). + see also https://github.com/rsyslog/rsyslog/issues/846 +- some general code improvements + * "fixed" cosmetic memory leaks at shutdown +- build system bugfix: configure can't find gss_acquire_cred on Solaris + Thanks to github user vlmarek for the patch. +- improvements to the CI environment + * improvements on the non-raciness of some tests + * imdiag: avoid races in detecting queue empty status + This resolves cases where the testbench terminated rsyslog too early, + resulting in potential message loss and test failure. + * omkafka has now dynamic tests + Thanks to Janmejay Singh for implementing them. + * try to merge PR to master and run tests; this guards against cross-PR + regressions and wasn't caught previously. Note that we skip this test + if we cannot successfully merge. So this is not a replacement for a + daily full "all-project integration test run". + * travis has finally enabled elasticsearch tests + ES was unfortunately not being regularly tested for quite a while due to + missing environment. This lead to some regressions becoming undetected. + These were now discovered thanks to the new support on travis. Also, this + guards against future regressions. + * imfile has now additional tests and overall better coverage + * omfile has now additional tests +------------------------------------------------------------------------------ +Version 8.18.0 [v8-stable] 2016-04-19 +- testbench: When running privdrop tests testbench tries to drop + user to "rsyslog", "syslog" or "daemon" when running as root and + you don't explicit set RSYSLOG_TESTUSER environment variable. + Make sure the unprivileged testuser can write into tests/ dir! +- templates: add option to convert timestamps to UTC + closes https://github.com/rsyslog/rsyslog/issues/730 +- omjournal: fix segfault (regression in 8.17.0) +- imptcp: added AF_UNIX support + Thanks to Nathan Brown for implementing this feature. +- new template options + * compressSpace + * date-utc +- redis: support for authentication + Thanks to Manohar Ht for the patch +- omkafka: makes kafka-producer on-HUP restart optional + As of now, omkafka kills and re-creates kafka-producer on HUP. This + is not always desirable. This change introduces an action param + (reopenOnHup="on|off") which allows user to control re-cycling of + kafka-producer. + It defaults to on (for backward compatibility). Off allows user to + ignore HUP as far as kafka-producer is concerned. + Thanks to Janmejay Singh for implementing this feature +- imfile: new "FreshStartTail" input parameter + Thanks to Curu Wong for implementing this. +- omjournal: fix libfastjson API issues + This module accessed private data members of libfastjson +- ommongodb: fix json API issues + This module accessed private data members of libfastjson +- testbench improvements (more tests and more thorough tests) + among others: + - tests for omjournal added + - tests for KSI subsystem + - tests for privilege drop statements + - basic test for RELP with TLS + - some previously disabled tests have been re-enabled +- dynamic stats subsystem: a couple of smaller changes + they also involve the format, which is slightly incompatible to + previous version. As this was out only very recently (last version), + we considered this as acceptable. + Thanks to Janmejay Singh for developing this. +- foreach loop: now also iterates over objects (not just arrays) + Thanks to Janmejay Singh for developing this. +- improvements to the CI environment +- enhancement: queue subsystem is more robst in regard to some corruptions + It is now detected if a .qi file states that the queue contains more + records than there are actually inside the queue files. Previously this + resulted in an emergency switch to direct mode, now the problem is only + reported but processing continues. +- enhancement: Allow rsyslog to bind UDP ports even w/out specific + interface being up at the moment. + Alternatively, rsyslog could be ordered after networking, however, + that might have some negative side effects. Also IP_FREEBIND is + recommended by systemd documentation. + Thanks to Nirmoy Das and Marius Tomaschewski for the patch. +- cleanup: removed no longer needed json-c compatibility layer + as we now always use libfastjson, we do not need to support old + versions of json-c (libfastjson was based on the newest json-c + version at the time of the fork, which is the newest in regard + to the compatibility layer) +- new External plugin for sending metrics to SPM Monitoring SaaS + Thanks to Radu Gheorghe for the patch. +- bugfix imfile: fix memory corruption bug when appending @cee + Thanks to Brian Knox for the patch. +- bugfix: memory misallocation if position.from and position.to is used + a negative amount of memory is tried to be allocated if position.from + is smaller than the buffer size (at least with json variables). This + usually leads to a segfault. + closes https://github.com/rsyslog/rsyslog/issues/915 +- bugfix: fix potential memleak in TCP allowed sender definition + depending on circumstances, a very small leak could happen on each + HUP. This was caused by an invalid macro definition which did not rule + out side effects. +- bugfix: $PrivDropToGroupID actually did a name lookup + ... instead of using the provided ID +- bugfix: small memory leak in imfile + Thanks to Tomas Heinrich for the patch. +- bugfix: double free in jsonmesg template + There has to be actual json data in the message (from mmjsonparse, + mmnormalize, imjournal, ...) to trigger the crash. + Thanks to Tomas Heinrich for the patch. +- bugfix: incorrect formatting of stats when CEE/Json format is used + This lead to ill-formed json being generated +- bugfix omfwd: new-style keepalive action parameters did not work + due to being inconsistently spelled inside the code. Note that legacy + parameters $keepalive... always worked + see also: https://github.com/rsyslog/rsyslog/issues/916 + Thanks to Devin Christensen for alerting us and an analysis of the + root cause. +- bugfix: memory leaks in logctl utility + Detected by clang static analyzer. Note that these leaks CAN happen in + practice and may even be pretty large. This was probably never detected + because the tool is not often used. +- bugfix omrelp: fix segfault if no port action parameter was given + closes https://github.com/rsyslog/rsyslog/issues/911 +- bugfix imtcp: Messages not terminated by a NL were discarded + ... upon connection termination. + Thanks to Tomas Heinrich for the patch. +------------------------------------------------------------------------------ +Version 8.17.0 [v8-stable] 2016-03-08 +- NEW REQUIREMENT: libfastjson + see also: + http://blog.gerhards.net/2015/12/rsyslog-and-liblognorm-will-switch-to.html +- new testbench requirement: faketime command line tool + This is used to generate a controlled environment for time-based tests; if + not available, tests will gracefully be skipped. +- improve json variable performance + We use libfastjson's alternative hash function, which has been + proven to be much faster than the default one (which stems + back to libjson-c). This should bring an overall performance + improvement for all operations involving variable processing. + closes https://github.com/rsyslog/rsyslog/issues/848 +- new experimental feature: lookup table support + Note that at this time, this is an experimental feature which is not yet + fully supported by the rsyslog team. It is introduced in order to gain + more feedback and to make it available as early as possible because many + people consider it useful. + Thanks to Janmejay Singh for implementing this feature +- new feature: dynamic statistics counters + which may be changed during rule processing + Thanks to Janmejay Singh for suggesting and implementing this feature +- new contributed plugin: omamqp1 for AMQP 1.0-compliant brokers + Thanks to Ken Giusti for this module +- new set of UTC-based $now family of variables ($now-utc, $year-utc, ...) +- simplified locking when accessing message and local variables + this simplifies the code and slightly increases performance if such + variables are heavily accessed. +- new global parameter "debug.unloadModules" + This permits to disable unloading of modules, e.g. to make valgrind + reports more useful (without a need to recompile). +- timestamp handling: guard against invalid dates + We do not permit dates outside of the year 1970..2100 + interval. Note that network-receivers do already guard + against this, so the new guard only guards against invalid + system time. +- imfile: add "trimlineoverbytes" input parameter + Thanks to github user JindongChen for the patch. +- ommongodb: add support for extended json format for dates + Thanks to Florian Bücklers for the patch. +- omjournal: add support for templates + see also: https://github.com/rsyslog/rsyslog/pull/770 + Thanks to github user bobthemighty for the patch +- imuxsock: add "ruleset" input parameter +- testbench: framework improvement: configs can be included in test file + they do no longer need to be in a separate file, which saves a bit + of work when working with them. This is supported for simple tests with + a single running rsyslog instance + Thanks to Janmejay Singh for inspiring me with a similar method in + liblognorm testbench. +- imptcp: performance improvements + Thanks to Janmejay Singh for implementing this improvement +- made build compile (almost) without warnings + still some warnings are suppressed where this is currently required +- improve interface definition in some modules, e.g. mmanon, mmsequence + This is more an internal cleanup and should have no actual affect to + the end user. +- solaris build: MAXHOSTNAMELEN properly detected +- build system improvement: ability to detect old hiredis libs + This permits to automatically build omhiredis on systems where the + hiredis libs do not provide a pkgconfig file. Previously, this + required manual configuration. + Thanks to github user jaymell for the patch. +- rsgtutil: dump mode improvements + * auto-detect signature file type + * ability to dump hash chains for log extraction files +- build system: fix build issues with clang + clang builds often failed with a missing external symbol + "rpl_malloc". This was caused by checks in configure.ac, + which checked for specific GNU semantics. As we do not need + them (we never ask malloc for zero bytes), we can safely + remove the macros. + Note that we routinely run clang static analyzer in CI and + it also detects such calls as invalid. + closes https://github.com/rsyslog/rsyslog/issues/834 +- bugfix: unixtimestamp date format was incorrectly computed + The problem happened in leap year from March til then end + of year and healed itself at the beginning of the next year. + During the problem period, the timestamp was 24 hours too low. + fixes https://github.com/rsyslog/rsyslog/issues/830 +- bugfix: date-ordinal date format was incorrectly computed + same root cause aus for unixtimestamp and same triggering + condition. During the affected perido, the ordinal was one + too less. +- bugfix: some race when shutting down input module threads + this had little, if at all, effect on real deployments as it resulted + in a small leak right before rsyslog termination. However, it caused + trouble with the testbench (and other QA tools). + Thanks to Peter Portante for the patch and both Peter and Janmejay + Singh for helping to analyze what was going on. +- bugfix tcpflood: did not handle connection drops correct in TLS case + note that tcpflood is a testbench too. The bug caused some testbench + instability, but had no effect on deployments. +- bugfix: abort if global parameter value was wrong + If so, the abort happened during startup. Once started, + all was stable. +- bugfix omkafka: fix potential NULL pointer addressing + this happened when the topic cache was full and an entry + needed to be evicted +- bugfix impstats: @cee cookie was prefixed to wrong format (json vs. cee) + Thanks to Volker Fröhlich for the fix. +- bugfix imfile: fix race during startup that could lead to some duplication + If a to-be-monitored file was created after inotify was initialized + but before startup was completed, the first chunk of data from this + file could be duplicated. This should have happened very rarely in + practice, but caused occasional testbench failures. + see also: https://github.com/rsyslog/rsyslog/issues/791 +- bugfix: potential loss of single message at queue shutdown + see also: https://github.com/rsyslog/rsyslog/issues/262 +- bugfix: potential deadlock with heavy variable access + When making heavy use of global, local and message variables, a deadlock + could occur. While it is extremely unlikely to happen, we have at least + seen one incarnation of this problem in practice. +- bugfix ommysql: on some platforms, serverport parameter had no effect + This was caused by an invalid code sequence which's outcome depends on + compiler settings. +- bugfix omelasticsearch: invalid pointer dereference + The actual practical impact is not clear. This came up when working + on compiler warnings. + Thanks to David Lang for the patch. +- bugfix omhiredis: serverport config parameter did not reliably work + depended on environment/compiler used to build +- bugfix rsgtutil: -h command line option did not work + Thanks to Henri Lakk for the patch. +- bugfix lexer: hex numbers were not properly represented + see: https://github.com/rsyslog/rsyslog/pull/771 + Thanks to Sam Hanes for the patch. +- bugfix TLS syslog: intermittent errors while sending data + Regression from commit 1394e0b. A symptom often seen was the message + "unexpected GnuTLS error -50 in nsd_gtls.c:530" +- bugfix imfile: abort on startup if no slash was present in file name param + Thanks to Brian Knox for the patch. +- bugfix rsgtutil: fixed abort when using short command line options + Thanks to Henri Lakk +- bugfix rsgtutil: invalid computation of log record extraction file + This caused verification to fail because the hash chain was actually + incorrect. Depended on the input data set. + closes https://github.com/rsyslog/rsyslog/issues/832 +- bugfix build system: KSI components could only be build if in default path +------------------------------------------------------------------------------ +Version 8.16.0 [v8-stable] 2016-01-26 +- rsgtutil: Added extraction support including loglines and hash chains. + More details on how to extract loglines can be found in the rsgtutil + manpage. See also: https://github.com/rsyslog/rsyslog/issues/561 +- clean up doAction output module interface + We started with char * pointers, but used different types of pointers + over time. This lead to alignment warnings. In practice, I think this + should never cause any problems (at least there have been no reports + in the 7 or so years we do this), but it is not clean. The interface is + now cleaned up. We do this in a way that does not require modifications + to modules that just use string parameters. For those with message + parameters, have a look at e.g. mmutf8fix to see how easy the + required change is. +- new system properties for $NOW properties based on UTC + This permits to express current system time in UTC. + See also https://github.com/rsyslog/rsyslog/issues/729 +- impstats: support broken ElasticSearch JSON implementation + ES 2.0 no longer supports valid JSON and disallows dots inside names. + This adds a new "json-elasticsearch" format option which replaces + those dots by the bang ("!") character. So "discarded.full" becomes + "discarded!full". + This is a workaround. A method that will provide more control over + replacements will be implemented some time in the future. For + details, see below-quoted issue tracker. + closes https://github.com/rsyslog/rsyslog/issues/713 +- omelasticsearch: craft better URLs + Elasticsearch is confused by url's ending in a bare '?' or '&'. While + this is valid, those are no longer produced. + Thanks to Benno Evers for the patch. +- imfile: add experimental "reopenOnTruncate" parameter + Thanks to Matthew Wang for the patch. +- bugfix imfile: proper handling of inotify initialization failure + Thanks to Zachary Zhao for the patch. +- bugfix imfile: potential segfault due to improper handling of ev var + This occurs in inotify mode, only. + Thanks to Zachary Zhao and Peter Portante for the patch. + closes https://github.com/rsyslog/rsyslog/issues/718 +- bugfix imfile: potential segfault under heavy load. + This occurs in inotify mode when using wildcards, only. + The root cause is dropped IN_IGNORED inotify events which be dropped + in circumstance of high input pressure and frequent rotation, and + according to wikipeida, they can also be dropped in other conditions. + Thanks to Zachary Zhao for the patch. + closes https://github.com/rsyslog/rsyslog/issues/723 +- bugfix ommail: invalid handling of server response + if that response was split into different read calls. Could lead to + error-termination of send operation. Problem is pretty unlikely to + occur in standard setups (requires slow connection to SMTP server). + Thank to github user haixingood for the patch. +- bugfix omelasticsearch: custom serverport was ignored on some platforms + Thanks to Benno Evers for the patch. +- bugfix: tarball did not include some testbench files + Thanks to Thomas D. (whissi) for the patch. +- bugfix: memory misaddressing during config parsing string template + This occurred if an (invalid) template option larger than 63 characters + was given. + Thanks to git hub user c6226 for the patch. +- bugfix imzmq: memory leak + Thanks to Jeremy Liang for the patch. +- bugfix imzmq: memory leak + Thanks to github user xushengping for the patch. +- bugfix omzmq: memory leak + Thanks to Jack Lin for the patch. +- some code improvement and cleanup +------------------------------------------------------------------------------ +Version 8.15.0 [v8-stable] 2015-12-15 +- KSI Lib: Updated code to run with libksi 3.4.0.5 + Also libksi 3.4.0.x is required to build rsyslog if ksi support + is enabled. New libpackages have been build as well. +- KSI utilities: Added option to ser publication url. + Since libksi 3.4.0.x, there is no default publication url anymore. + The publication url has to be set using the --publications-server + Parameter, otherwise the ksi signature cannot be verified. UserID + and UserKey can also be set by parameter now. + Closes https://github.com/rsyslog/rsyslog/issues/581 +- KSI Lib: Fixed wrong TLV container for KSI signatures from 0905 to 0906. + closes https://github.com/rsyslog/rsyslog/issues/587 +- KSI/GT Lib: Fixed multiple issues found using static analyzer +- performance improvement for configs with heavy use of JSON variables + Depending on the config, this can be a very big gain in performance. +- added pmpanngfw: contributed module for translating Palo Alto Networks logs. + see also: https://github.com/rsyslog/rsyslog/pull/573 + Thanks to Luigi Mori for the contribution. +- testbench: Changed valgrind option for imtcp-tls-basic-vg.sh + For details see: https://github.com/rsyslog/rsyslog/pull/569 +- pmciscoios: support for asterisk before timestamp added + thanks to github user c0by for the patch + see also: https://github.com/rsyslog/rsyslog/pull/583 +- solr external output plugin much enhanced + see also: https://github.com/rsyslog/rsyslog/pull/529 + Thanks to Radu Gheorghe for the patch. +- omrabbitmq: improvements + thanks to Luigi Mori for the patch + see also: https://github.com/rsyslog/rsyslog/pull/580 +- add support for libfastjson (as a replacement for json-c) +- KSI utilities: somewhat improved error messages + Thanks to Henri Lakk for the patch. + see also: https://github.com/rsyslog/rsyslog/pull/588 +- pmciscoios: support for some format variations + Thanks to github user c0by for the patch +- support grok via new contributed module mmgrok + Thanks to 饶琛琳 (github user chenryn) for the contribution. +- omkafka: new statistics counter "maxoutqsize" + Thanks to 饶琛琳 (github user chenryn) for the contribution. +- improvements for 0mq modules: + * omczmq - suspend / Retry handling - the output plugin can now recover + from some error states due to issues with plugin startup or message sending + * omczmq - refactored topic handling code for ZMQ_PUB output to be a little + more efficient + * omczmq - added ability to set a timeout for sends + * omczmq - set topics can be in separate frame (default) or part of message + frame (configurable) + * omczmq - code cleanup + * imczmq - code cleanup + * imczmq - fixed a couple of cases where vars could be used uninitialized + * imczmq - ZMQ_ROUTER support + * imczmq - Fix small memory leak from not freeing sockets when done with them + * allow creation of on demand ephemeral CurveZMQ certs for encryption. + Clients may specify clientcertpath="*" to indicate they want an on + demand generated cert. + Thanks to Brian Knox for the contributions. +- cleanup on code to unset a variable + under extreme cases (very, very unlikely), the old code could also lead + to erroneous processing +- omelasticsearch: build on FreeBSD + Thanks to github user c0by for the patch +- pmciscoios: fix some small issues clang static analyzer detected +- testbench: many improvements and some new tests + note that there still is a number of tests which are somewhat racy +- overall code improvements thanks to clang static analyzer +- gnutls fix: Added possible fix for gnutls issue #575 + see also: https://github.com/rsyslog/rsyslog/issues/575 + Thanks to Charles Southerland for the patch +- bugfix omkafka: restore ability to build on all platforms + Undo commit aea09800643343ab8b6aa205b0f10a4be676643b + because that lead to build failures on various important platforms. + This means it currently is not possible to configure the location + of librdkafka, but that will affect far fewer people. + closes: https://github.com/rsyslog/rsyslog/issues/596 +- bugfix omkafka: fix potentially negative partition number + Thanks to Tait Clarridge for providing a patch. +- bugfix: solve potential race in creation of additional action workers + Under extreme circumstances, this could lead to segfault. Note that we + detected this problem thanks to ASAN address sanitizer in combination + with a very extreme testbench test. We do not think that this issue + was ever reported in practice. +- bugfix: potential memory leak in config parsing + Thanks to github user linmujia for the patch +- bugfix: small memory leak in loading template config + This happened when a plugin was used inside the template. Then, the + memory for the template name was never freed. + Thanks to github user xushengping for the fix. +- bugfix: fix extra whitespace in property expansions + Address off-by-one issues introduced in f3bd7a2 resulting in extra + whitespace in property expansions + Thanks to Matthew Gabeler-Lee for the patch. +- bugfix: mmfields leaked memory if very large messages were processed + detected by clang static analyzer +- bugfix: mmfields could add garbage data to field + this happened when very large fields were to be processed. + Thanks to Peter Portante for reporting this. +- bugfix: omhttpfs now also compiles with older json-c lib +- bugfix: memory leak in (contributed) module omhttpfs + Thanks to git hub user c6226 for the patch. +- bugfix: parameter mismatch in error message for wrap() function +- bugfix: parameter mismatch in error message for random() function +- bugfix: divide by zero if max() function was provided zero +- bugfix: invalid mutex handling in omfile async write mode + could lead to segfault, even though highly unlikely (caught by + testbench on a single platform) +- bugfix: fix inconsistent number processing + Unfortunately, previous versions of the rule engine tried to + support oct and hex, but that wasn't really the case. + Everything based on JSON was just dec-converted. As this was/is + the norm, we fix that inconsistency by always using dec. + Luckily, oct and hex support was never documented and could + probably only have been activated by constant numbers. +- bugfix: timezone() object: fix NULL pointer dereference + This happened during startup when the offset or id parameter was not + given. Could lead to a segfault at startup. + Detected by clang static analyzer. +- bugfix omfile: memory addressing error if very long outchannel name used + Thanks to github user c6226 for the patch. +------------------------------------------------------------------------------ +Version 8.14.0 [v8-stable] 2015-11-03 +- Added possibility to customize librdkafka location + see also: https://github.com/rsyslog/rsyslog/pull/502 + Thanks to Matthew Wang for the patch. +- add property "rawmsg-after-pri" +- bugfix: potential misaddresseing in imfile + Could happen when wildcards were used. + see also https://github.com/rsyslog/rsyslog/issues/532 + see also https://github.com/rsyslog/rsyslog/issues/534 + Thanks to zhangdaoling for the bugfix. +- bugfix: re_extract RainerScript function did not work + Thanks to Janmejay Singh for the patch +------------------------------------------------------------------------------ +Version 8.13.0 [v8-stable] 2015-09-22 +- ZeroMQ enhancements: + * Added the ability to set a static publishing topic per action as an + alternative to constructing topics with templates + Contributor: Luca Bocassi + * ZMQ_PUB socket now defaults to bind and ZMQ_SUB socket now defaults to + connect - Contributor: Luca Bocassi +- Redis enhancements: + * Can now LPUSH to a Redis list in "queue" mode - Contributor: Brian Knox + * Can now PUBLISH to a Redis channel in "publish" mode + Contributor: Brian Knox +- build requirement for rsyslog/mmnormalize is now liblognorm 1.1.2 or above +- mmnormalize: liblognorm error messages are now emitted via regular + rsyslog error reporting mechanism (aka "are now logged") + This is possible due to a new API in liblognorm 1.1.2; + Note that the amount of error messages depends on the version of + liblognorm used. +- add support for TCP client side keep-alives + Thanks to github user tinselcity for the patch. +- bugfix: imtcp/TLS hangs on dropped packets + see also https://github.com/rsyslog/rsyslog/issues/318 + Thanks to github user tinselcity for the patch. +- bugfix testbench: some tests using imptcp are run if module is disabled + Thanks to Michael Biebl for reporting this + see also https://github.com/rsyslog/rsyslog/issues/524 +- bugfix omkafka: Fixes a bug not accepting new messages anymore. + see also: https://github.com/rsyslog/rsyslog/pull/472 + Thanks to Janmejay Singh +- bugfix: Parallel build issue "cannot find ../runtime/.libs/librsyslog.a: + No such file or directory" (#479) fixed. + Thanks to Thomas D. (Whissi) for the patch. +- bugfix: Added missing mmpstrucdata testfiles into makefile. + see also: https://github.com/rsyslog/rsyslog/issues/484 +- bugfix: Reverted FIX for issue #392 as it had unexpected side effects. + The new fix duplicates the Listener object for static files (like + done for dynamic files already), resolving issue #392 and #490. + see also https://github.com/rsyslog/rsyslog/pull/490 +- bugfix: issues in queue subsystem if syncqueuefiles was enabled + * Error 14 was generated on the .qi file directory handle. + As the .qi filestream does not have a directory set, fsync + was called on an empty directory causing a error 14 in debug log. + * When queue files existed on startup, the bSyncQueueFiles + strm property was not set to 1. This is now done in the + qqueueLoadPersStrmInfoFixup function. +- bugfix/testbench: tcpflood tool could abort when random data was added + see also: https://github.com/rsyslog/rsyslog/issues/506 + Thanks to Louis Bouchard for the fix +- rscryutil: Added support to decrypt a not closed log file. + Thanks to wizard1024 for the patch. +------------------------------------------------------------------------------ +Version 8.12.0 [v8-stable] 2015-08-11 +- Harmonize resetConfigVariables values and defaults + see also https://github.com/rsyslog/rsyslog/pull/413 + Thanks to Tomas Heinrich for the patch. +- GT/KSI: fix some issues in signature file format and add conversion tool + The file format is incompatible to previous format, but tools have been + upgraded to handle both and also an option been added to convert from + old to new format. +- bugfix: ommysql did not work when gnutls was enabled + as it turned out, this was due to a check for GnuTLS functions + with the side-effect that + AC_CHECK_LIB, by default, adds the lib to LIBS, if there is no + explicit action, what was the case here. So everything was now + linked against GnuTLS, which in turn made ommysql fail. + Thanks to Thomas D. (whissi) for the analysis of the ommysql/gnutls + problem and Thomas Heinrich for pointing out that AC_CHECK_LIB might + be the culprit. +- bugfix omfile: potential memory leak on file close + see also: https://github.com/rsyslog/rsyslog/pull/423 + Thanks to Robert Schiele for the patch. +- bugfix omfile: potential race in dynafile detection/creation + This could lead to a segfault. + Thanks to Tomas Heinrich for the patch. +- bugfix omfile: Fix race-condition detection in path-creation code + The affected code is used to detect a race condition in between + testing for the existence of a directory and creating it if it didn't + exist. The variable tracking the number of attempts wasn't reset for + subsequent elements in the path, thus limiting the number of + reattempts to one per the whole path, instead of one per each path + element. + This solution was provided by Martin Poole. +- bugfix parser subsystem: potential misaddressing in SanitizeMsg() + could lead to a segfault + Thanks to Tomas Heinrich for the patch. +- imfile: files moved outside of directory are now (properly) handled +- bugfix: imfile: segfault when using startmsg.regex if first log line + doesn't match + Thanks to Ciprian Hacman for the patch. +- bugfix imfile: file table was corrupted when on file deletion + This could happen when a file that was statically configured (not via an + wildcard) was deleted. +- bugfix ompgsql: transaction were improperly handled + now transaction support is solidly disabled until we have enough requests + to implement it again. Module still works fine in single insert mode. + closes https://github.com/rsyslog/rsyslog/issues/399 +- bugfix mmjsonparse: memory leak if non-cee-json message is processed + see also https://github.com/rsyslog/rsyslog/pull/383 + Thanks to Anton Matveenko for the patch +- testbench: remove raciness from UDP based tests +- testbench: added bash into all scripts making it mandatory +- bugfix testbench: Fixed problem building syslog_caller util when + liblogging-stdlog is not available. + Thanks to Louis Bouchard for the patch +- bugfix rscryutil.1: Added fix checking for generate_man_pages condition + Thanks to Radovan Sroka for the patch +- bugfix freebsd console: \n (NL) is prepended with \r (CR) in console + output on freebsd only. For more details see here: + https://github.com/rsyslog/rsyslog/issues/372 + Thanks to AlexandreFenyo for the patch +------------------------------------------------------------------------------ +Version 8.11.0 [v8-stable] 2015-06-30 +- new signature provider for Keyless Signature Infrastructure (KSI) added +- build system: re-enable use of "make distcheck" +- add new signature provider for Kesless Signature Infrastructure (KSI) + This has also been added to existing tooling; KSI is kind of v2 of + the Guardtime functionality and has been added in the appropriate + places. +- bugfix imfile: regex multiline mode ignored escapeLF option + Thanks to Ciprian Hacman for reporting the problem + closes https://github.com/rsyslog/rsyslog/issues/370 +- bugfix omkafka: fixed several concurrency issues, most of them related + to dynamic topics. + Thanks to Janmejay Singh for the patch. +- bugfix: execonlywhenpreviousissuspended did not work correctly + This especially caused problems when an action with this attribute was + configured with an action queue. +- bugfix core engine: ensured global variable atomicity + This could lead to problems in RainerScript, as well as probably in other + areas where global variables are used inside rsyslog. I wouldn't outrule + it could lead to segfaults. + Thanks to Janmejay Singh for the patch. +- bugfix imfile: segfault when using startmsg.regex because of empty log line + closes https://github.com/rsyslog/rsyslog/issues/357 + Thanks to Ciprian Hacman for the patch. +- bugfix: build problem on Solaris + Thanks to Dagobert Michelsen for reporting this and getting us up to + speed on the openCWS build farm. +- bugfix: build system strndup was used even if not present + now added compatibility function. This came up on Solaris builds. + Thanks to Dagobert Michelsen for reporting the problem. + closes https://github.com/rsyslog/rsyslog/issues/347 +- bugfix imjournal: do not pass empty messages to rsyslog core + this causes a crash of the daemon + see also https://github.com/rsyslog/rsyslog/pull/412 + Thanks to Tomas Heinrich for the patch. +- bugfix imjournal: cosmetic memory leak + very small and an shutdown only, so did not affect operations + see also https://github.com/rsyslog/rsyslog/pull/411 + Thanks to Tomas Heinrich for the patch. +------------------------------------------------------------------------------ +Version 8.10.0 [v8-stable] 2015-05-19 +- imfile: add capability to process multi-line messages based on regex + input parameter "endmsg.regex" was added for that purpose. The new + mode provides much more power in processing different multiline-formats. +- pmrfc3164: add new parameters + * "detect.yearAfterTimestamp" + This supports timestamps as generated e.g. by some Aruba Networks + equipment. + * "permit.squareBracesInHostname" + Permits to use "hostnames" in the form of "[127.0.0.1]"; also seen in + Aruba Networks equipment, but we strongly assume this can also happen + in other cases, especially with IPv6. +- supplementary groups are now set when dropping privileges + closes https://github.com/rsyslog/rsyslog/issues/296 + Thanks to Zach Lisinski for the patch. +- imfile: added brace glob expansion to wildcard + Thanks to Zach Lisinski for the patch. +- zmq: add the ability for zeromq input and outputs to advertise their + presence on UDP via the zbeacon API. + Thanks to Brian Knox for the contribution. +- added omhttpfs: contributed module for writing to HDFS via HTTP + Thanks to sskaje for the contribution. +- Configure option "--disable-debug-symbols" added which is disabled per + default. If you set the new option, configure won't set the appropriate + compiler flag to generate debug symbols anymore. +- When building from git source we now require rst2man and yacc (or a + replacement like bison). + That isn't any new requirement, we only added missing configure checks. +- Configure option "--enable-generate-man-pages" is now disabled for non git + source builds per default but enforced when building from git source. +- mmpstrucdata: some code cleanup + removed lots of early development debug outputs +- bugfix imuxsock: fix a memory leak that happened with large messages + ... when annotation was enabled. + Thanks to github user c6226 for the patch +- bugfix omhttpfs: memory leak + Thanks to github user c6226 for the patch +- bugfix imuxsock: fix a crash when setting a hostname + Setting a hostname via the legacy directive would lead to a crash + during shutdown caused by a double-free. + Thanks to Tomas Heinrich for the patch. +- bugfix: memory leak in mmpstrucdata + Thanks to Grégoire Seux for reporting this issue. + closes https://github.com/rsyslog/rsyslog/issues/310 +- bugfix (minor): default action name: assigned number was one off + see also https://github.com/rsyslog/rsyslog/pull/340 + Thanks to Tomas Heinrich for the patch. +- bugfix: memory leak in imfile + A small leak happened each time a new file was monitored based on + a wildcard. Depending on the rate of file creation, this could result + in a serious memory leak. +------------------------------------------------------------------------------ +Version 8.9.0 [v8-stable] 2015-04-07 +- omprog: add option "hup.forward" to forwards HUP to external plugins + This was suggested by David Lang so that external plugins (and other + programs) can also do HUP-specific processing. The default is not + to forward HUP, so no change of behavior by default. +- imuxsock: added capability to use regular parser chain + Previously, this was a fixed format, that was known to be spoken on + the system log socket. This also adds new parameters: + - sysSock.useSpecialParser module parameter + - sysSock.parseHostname module parameter + - useSpecialParser input parameter + - parseHostname input parameter +- 0mq: improvements in input and output modules + See module READMEs, part is to be considered experimental. + Thanks to Brian Knox for the contribution. +- imtcp: add support for ip based bind for imtcp -> param "address" + Thanks to github user crackytsi for the patch. +- bugfix: MsgDeserialize out of sync with MsgSerialize for StrucData + This lead to failure of disk queue processing when structured data was + present. Thanks to github user adrush for the fix. +- bugfix imfile: partial data loss, especially in readMode != 0 + closes https://github.com/rsyslog/rsyslog/issues/144 +- bugfix: potential large memory consumption with failed actions + see also https://github.com/rsyslog/rsyslog/issues/253 +- bugfix: omudpspoof: invalid default send template in RainerScript format + The file format template was used, which obviously does not work for + forwarding. Thanks to Christopher Racky for alerting us. + closes https://github.com/rsyslog/rsyslog/issues/268 +- bugfix: size-based legacy config statements did not work properly + on some platforms, they were incorrectly handled, resulting in all + sorts of "interesting" effects (up to segfault on startup) +- build system: added option --without-valgrind-testbench + ... which provides the capability to either enforce or turn off + valgrind use inside the testbench. Thanks to whissi for the patch. +- rsyslogd: fix misleading typos in error messages + Thanks to Ansgar Püster for the fixes. +------------------------------------------------------------------------------ +Version 8.8.0 [v8-stable] 2015-02-24 +- omkafka: add support for dynamic topics and auto partitioning + Thanks to Tait Clarridge for the patches. +- imtcp/imptcp: support for broken Cisco ASA TCP syslog framing +- omfwd: more detailed error messages in case of UDP send error +- TLS syslog: enable capability to turn on GnuTLS debug logging + This provides better diagnostics in hard-to-diagnose cases, + especially when GnuTLS is extra-picky about certificates. +- bugfix: $AbortOnUncleanConfig did not work +- improve rsyslogd -v output and error message with meta information + version number is now contained in error message and build platform in + version output. This helps to gets rid of the usual "which version" + question on mailing list, support forums, etc... +- bugfix imtcp: octet-counted framing cannot be turned off +- bugfix: build problems on Illuminos + Thanks to Andrew Stormont for the patch +- bugfix: invalid data size for iMaxLine global property + It was defined as int, but inside the config system it was declared as + size type, which uses int64_t. With legacy config statements, this could + lead to misaddressing, which usually meant the another config variable was + overwritten (depending on memory layout). + closes https://github.com/rsyslog/rsyslog/issues/205 +- bugfix: negative values for maxMessageSize global parameter were permitted +------------------------------------------------------------------------------ +Version 8.7.0 [v8-stable] 2015-01-13 +- add message metadata "system" to msg object + this permits to store metadata alongside the message +- imfile: add support for "filename" metadata + this is useful in cases where wildcards are used +- imptcp: make stats counter names consistent with what imudp, imtcp uses +- added new module "omkafka" to support writing to Apache Kafka +- omfwd: add new "udp.senddelay" parameter +- mmnormalize enhancements + Thanks to Janmejay Singh for the patch. +- RainerScript "foreach" iterator and array reading support + Thanks to Janmejay Singh for the patch. +- now requires liblognorm >= 1.0.2 +- add support for systemd >= 209 library names +- BSD "ntp" facility (value 12) is now also supported in filter + Thanks to Douglas K. Rand of Iteris, Inc. for the patch. + Note: this patch was released under ASL 2.0 (see email-conversation). +- bugfix: global(localHostName="xxx") was not respected in all modules +- bugfix: emit correct error message on config-file-not-found + closes https://github.com/rsyslog/rsyslog/issues/173 +- bugfix: impstats emitted invalid JSON format (if JSON was selected) +- bugfix: (small) memory leak in omfile's outchannel code + Thanks to Koral Ilgun for reporting this issue. +- bugfix: imuxsock did not deactivate some code not supported by platform + Among potential other problems, this caused build failure under Solaris. + Note that this build problem just made a broader problem appear that so + far always existed but was not visible. + closes https://github.com/rsyslog/rsyslog/issues/185 +------------------------------------------------------------------------------ +Version 8.6.0 [v8-stable] 2014-12-02 +NOTE: This version also incorporates all changes and enhancements made for +v8.5.0, but in a stable release. For details see immediately below. +- configuration-setting rsyslogd command line options deprecated + For most of them, there are now proper configuration objects. Some few + will be completely dropped if nobody insists on them. Additional info at + http://blog.gerhards.net/2014/11/phasing-out-legacy-command-line-options.html +- new and enhanced plugins for 0mq. These are currently experimental. + Thanks to Brian Knox who contributed the modules and is their author. +- empty rulesets have been permitted. They no longer raise a syntax error. +- add parameter -N3 to enable config check of partial config file + Use for config include files. Disables checking if any action exists at + all. +- rsyslogd -e option has finally been removed + It is deprecated since many years. +- testbench improvements + Testbench is now more robust and has additional tests. +- testbench is now by default disabled + To enable it, use --enable-testbench. This was done as the testbench now + does better checking if required modules are present and this in turn + would lead to configure error messages where non previously were if we + would leave --enable-testbench on by default. Thus we have turned it off. + This should not be an issue for those few testbench users. +- add new RainerScript functions warp() and replace() + Thanks to Singh Janmejay for the patch. +- mmnormalize can now also work on a variable + Thanks to Singh Janmejay for the patch. +- new property date options for day ordinal and week number + Thanks to github user arrjay for the patch +- remove --enable-zlib configure option, we always require it + It's hard to envision a system without zlib, so we turn this off + closes https://github.com/rsyslog/rsyslog/issues/76 +- slight source-tree restructuring: contributed modules are now in their + own ./contrib directory. The idea is to make it clearer to the end user + which plugins are supported by the rsyslog project (those in ./plugins). +- bugfix: imudp makes rsyslog hang on shutdown when more than 1 thread used + closes https://github.com/rsyslog/rsyslog/issues/126 +- bugfix: not all files closed on auto-backgrounding startup + This could happen when not running under systemd. Some low-numbered + fds were not closed in that case. +- bugfix: typo in queue configuration parameter + made parameter unusable + Thanks to Bojan Smojver for the patch. +- bugfix: uninitialized buffer off-by-one error in hostname generation + The DNS cache used uninitialized memory, which could lead to + invalid hostname generation. + Thanks to Jarrod Sayers for alerting us and providing analysis and + patch recommendations. +- bugfix imuxsock: possible segfault when SysSock.Use="off" + Thanks to alexjfisher for reporting this issue. + closes https://github.com/rsyslog/rsyslog/issues/140 +- bugfix: RainerScript: invalid ruleset names were accepted + during ruleset definition, but could of course not be used when + e.g. calling a ruleset. + IMPORTANT: this may cause existing configurations to error out on start, + as they invalid names could also be used e.g. when assigning rulesets. +- bugfix: some module entry points were not called for all modules + callbacks like endCnfLoad() were primarily being called for input + modules. This has been corrected. Note that this bugfix has some + regression potential. +- bugfix omlibdbi: connection was taken down in wrong thread + this could have consequences depending on the driver being used. In + general, it looks more like a cosmetic issue. For example, with + MySQL it lead to a small memory but also an annoying message about + a thread not properly torn down. +- imttcp was removed because it was an incomplete experimental module +- pmrfc3164sd because it was a custom module nobody used + We used to keep this as a sample inside the tree, but whoever wants + to look at it can check in older versions inside git. +- omoracle was removed because it was orphaned and did not build/work + for quite some years and nobody was interested in fixing it +--------------------------------------------------------------------------- +Version 8.5.0 [v8-stable] 2014-10-24 +- imfile greatly refactored and support for wildcards added +- PRI-handling code refactored for more clarity and robustness +- ommail: add support for RainerScript config system [action() object] + This finally adds support for the new config style. Also, we now permit + to set a constant subject text without the need to create a template for + it. +- refactored the auto-backgrounding method + The code is now more robust and also offers possibilities for enhanced + error reporting in the future. This is also assumed to fix some races + where a system startup script hang due to "hanging" rsyslogd. +- make gntls tcp syslog driver emit more error messages + Messages previously emitted only to the debug log are now emitted as + syslog error messages. It has shown that they contain information + helpful to the user for troubleshooting config issues. Note that this + change is a bit experimental, as we are not sure if there are situations + where large amounts of error messages may be emitted. +- bugfix: imfile did not complain if configured file did not exist + closes https://github.com/rsyslog/rsyslog/issues/137 +- bugfix: build failure on systems which don't have json_tokener_errors + Older versions of json-c need to use a different API (which don't exists + on newer versions, unfortunately...) + Thanks to Thomas D. for reporting this problem. +- imgssapi: log remote peer address in some error messages + Thanks to Bodik for the patch. +--------------------------------------------------------------------------- +Version 8.4.3 [v8-stable] 2014-10-?? +- ommail: minor bugfixes & improvements + * timestamps were 1 hour out when using daylight saving times when + viewing emails in most email clients due to incorrect date format + * X-Mailer header had a typo in it + * To: header was duplicated once per recipient (this is permitted, + but an address list is a better choice nowadays) + Thanks to github user cacheus for the patches. +- bugfix imkmsg: infinite loop on OpenVZ VMs + Thanks to github user PaulSD for the patch + closes https://github.com/rsyslog/rsyslog/pull/138 +- bugfix: typo in queue configuration parameter made parameter unusable + Thanks to Bojan Smojver for the patch. +- bugfix: uninitialized buffer off-by-one error in hostname generation + The DNS cache used uninitialized memory, which could lead to + invalid hostname generation. + Thanks to Jarrod Sayers for alerting us and providing analysis and + patch recommendations. +- bugfix imfile: segfault on startup in "inotify" mode + A segfault happened when more than one file was monitored. +- bugfix imfile: could make rsyslog exit in inotify mode +- bugfix: rsgtutil sometimes crashed in verify mode if file did not exist +- bugfix imklog: pri was miscalculated + actually, the pri was totally off the real value for PRIs > 9 +- bugfix imfile:file processing in inotify mode was stalled sometimes + closes https://github.com/rsyslog/rsyslog/issues/134 +- bugfix: imjournal did not build properly + The build succeeded, but the module did not load due to a type in + a support function name, which kept unresolved during load. +- bugfix: mmcount did no longer build + note that this is untested -- users of this module should file a bug if + the new (trivial) code is broken [if there are any users, thus I did not + invest time in testing...] + closes https://github.com/rsyslog/rsyslog/issues/129 +- bugfix imuxsock: possible segfault when SysSock.Use="off" + Thanks to alexjfisher for reporting this issue. + closes https://github.com/rsyslog/rsyslog/issues/140 +--------------------------------------------------------------------------- +Version 8.4.2 [v8-stable] 2014-10-02 +- bugfix: the fix for CVE-2014-3634 did not handle all cases + This is corrected now. + see also: CVE-2014-3683 +- fixed a build problem on some platforms + Thanks to Olaf for the patch +- behavior change: "msg" of messages with invalid PRI set to "rawmsg" + When the PRI is invalid, the rest of the header cannot be valid. So + we move all of it to MSG and do not try to parse it out. Note that + this is not directly related to the security issue but rather done + because it makes most sense. +--------------------------------------------------------------------------- +Version 8.4.1 [v8-stable] 2014-09-30 +- imudp: add for bracketing mode, which makes parsing stats easier +- permit at-sign in variable names + closes: https://github.com/rsyslog/rsyslog/issues/110 +- bugfix: fix syntax error in anon_cc_numbers.py script + Thanks to github user anthcourtney for the patch. + closes: https://github.com/rsyslog/rsyslog/issues/109 +- bugfix: ompgsql: don't loose uncommitted data on retry + Thanks to Jared Johnson and Axel Rau for the patch. +- bugfix: imfile: if a state file for a different file name was set, + that different file (name) was monitored instead of the configured + one. Now, the state file is deleted and the correct file monitored. + closes: https://github.com/rsyslog/rsyslog/issues/103 +- bugfix: omudpspoof: source port was invalid + Thanks to Pavel Levshin for the patch +- bugfix: build failure on systems which don't have json_tokener_errors + Older versions of json-c need to use a different API (which don't exists + on newer versions, unfortunately...) + Thanks to Thomas D. for reporting this problem. +- bugfix: omelasticsearch does not work with broken/changed ES 1.0+ API + closes: https://github.com/rsyslog/rsyslog/issues/104 +- bugfix: mmanon did not properly anonymize IP addresses starting with '9' + Thanks to defa-at-so36.net for reporting this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529 +- bugfix: build problems on SuSe Linux + Thanks Andreas Stieger for the patch +- bugfix: omelasticsearch error file did not work correctly on ES 1.0+ + due to a breaking change in the ElasticSearch API. + see also: https://github.com/rsyslog/rsyslog/issues/104 +- bugfix: potential abort when a message with PRI > 191 was processed + if the "pri-text" property was used in active templates, this could + be abused to a remote denial of service from permitted senders + see also: CVE-2014-3634 +--------------------------------------------------------------------------- +Version 8.4.0 [v8-stable] 2014-08-18 +- this is the new stable branch, which incorporates all enhancements of + rsyslog 8.3. +--------------------------------------------------------------------------- +Version 8.3.5 [v8-devel] 2014-08-05 +- mmjsonparse: support selectable cookie and target containers + This permits to put different meanings into a json formatted syslog + message, e.g. the "traditional" cee or cim data. +- bugfix: mmjsonparse did not build with json-c < 0.10 + This was a regression introduced some time in the past in order to + support API changes in json-c. Now we check for the version and use + proper code. +- omprog: emit error message via syslog() if loading binary fails + This happens after forking, so omprog has no longer access to rsyslog's + regular error reporting functions. Previously, this meant any error + message was lost. Now it is emitted via regular syslog (which may end up + in a different instance, if multiple instances run...) +- couple of patches imported from v7-stable (7.6.4) +--------------------------------------------------------------------------- +Version 8.3.4 [v8-devel] 2014-07-11 +- new pmciscoios parser supporting various Cisco IOS formats +- RFC3164 timestamp parser now accepts timezones and subsecond resolution + ... at least for some common formats and where we could do so without + running risk of breaking proper formats (or introducing regressions) +- new parser config object -- permits to define custom parser definitions +- new tzinfo config object -- permits to define time zone offsets + This is a utility object that currently is being used by some parsers. +- bugfix: mishandling of input modules not supporting new input instances + If they did not support this, accidentally the output module part of the + module union was written, leading to unpredictable results. Note: all + core modules do support this interface, but some contributed or very + old ones do not. +- bugfix: double-free when ruleset() parser parameters were used + While unlikely, this could cause stability issues even after the + config phase. +--------------------------------------------------------------------------- +Version 8.3.3 [v8-devel] 2014-06-26 +- unify input object naming + imudp now supports "name" parameter, as other inputs do. "inputname" has + been deprecated, but can still be used. Same applies to "appendport" + subparameter". Thanks to "Nick Syslog" for the suggestion. +- made the missing (contributed) modules build under v8 [import from 8.2.2] + Modules: + * mmrfc5424addhmac + * omrabbitmq + * omgssapi + * omhdfs + * omzmq3 +- added a cleanup process (janitor); permits to close omfile files after a + timeout +- make omgssapi build under v8.3 [import vom v8.2] + note that we could do this to the stable, because there is NO regression + chance at all: only omgssapi was changed, and this module did NOT work + previously. +- removed obsolete --disable-fsstnd configure option + Thanks to Thomas D. for alerting us. + Closes: https://github.com/rsyslog/rsyslog/issues/72 +--------------------------------------------------------------------------- +Version 8.3.2 [v8-devel] 2014-05-02 +- new template options for date extraction: + - year + - month + - day + - wday + - hour + - minute + - second + - tzoffshour + - tzoffsmin + - tzoffsdirection + - wdayname + For string templates, these are property options and they are + prefixed with "date-" (e.g. "date-year", "date-month", ...) + see also: https://github.com/rsyslog/rsyslog/issues/65 +- bugfix: mmexternal remove framing char before processing JSON reply + This did not have any real bad effects, but caused unnecessary + processing, as empty replies were not properly detected. Otherwise, + the bug was not noticeable from the user's PoV. +- bugfix: mmexternal segfault due to invalid free in non-json input mode + closes: https://github.com/rsyslog/rsyslog/issues/70 +- bugfix: mmexternal segfault when external plugin sent invalid reply + ... or no reply at all. This happened if the reply was improper JSON. + Now, we emit an error message in those cases. + see also: https://github.com/rsyslog/rsyslog/issues/69 +- bugfix: mmexternal did potentially pass incomplete data to restarted + external plugin + This could happen if EPIPE was returned "too late", in which case the + beginning of the data could be lost. +- bugfix: mmexternal did not properly process messages over 4KiB + The data to be passed to the external plugin was truncated after 4KiB. + see: https://github.com/rsyslog/rsyslog/issues/64 +- imrelp: added support for per-listener ruleset and inputname + see: https://github.com/rsyslog/rsyslog/pull/63 + Thanks to bobthesecurityguy github user for the patch +--------------------------------------------------------------------------- +Version 8.3.1 [v8-devel] 2014-04-24 +- external message modification interface now support modifying message PRI +- "jsonmesg" property will include uuid only if one was previously generated + This is primarily a performance optimization. Whenever the message uuid + is gotten, it is generated when not already present. As we used the + regular setter, this means that always the uuid was generated, which is + quite time-consuming. This has now been changed so that it only is + generated if it already exists. That also matches more closely the + semantics, as "jsonmesg" should not make modifications to the message. + Note that the same applies to "fulljson" passing mode for external + plugins. +- added plugin to rewrite message facility and/or severity + Name: fac-sever-rewrite.py +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- bugfix: using UUID property could cause segfault +- bugfix/mmexternal: memory leak +- bugfix: memory leak when using "jsonmesg" property +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +- bugfix: build problems with lexer.l on some platforms + For some reason, the strdup() prototype and others are missing. I admit + that I don't know why, as this happens only in 8.3.0+ and there is no + indication of changes to the affected files. In any case, we need to + fix this, and the current solution works at least as an interim one. +--------------------------------------------------------------------------- +Version 8.3.0 [v8-devel] 2014-04-10 +- new plugin for anonymizing credit card numbers + Thanks to Peter Slavov for providing the code. +- external message modification modules are now supported + They are bound via the new native module "mmexternal". Also, a sample + skeleton for an external python message modification module has been + added. +- new $jsonmesg property with JSON representation of whole message object + closes: https://github.com/rsyslog/rsyslog/issues/19 +- improved error message for invalid field extraction in string template + see also: + http://kb.monitorware.com/problem-with-field-based-extraction-t12299.html +- fix build problems on Solaris +- NOTE: a json-c API that we begun to use requires the compiler to be in + c99 mode. By default, we select it automatically. If you modify this and + use gcc, be sure to include "-std=c99" in your compiler flags. This seems + to be necessary only for older versions of gcc. +--------------------------------------------------------------------------- +Version 8.2.3 [v8-stable] 2014-??-?? +- bugfix: ommysql: handle/mem leak upon termination of worker thread + This could become bad if the (instance) worker threads are often + started and terminated. But it takes quite a while to show effect. +--------------------------------------------------------------------------- +Version 8.2.2 [v8-stable] 2014-06-02 +- made the missing (contributed) modules build under v8 + Note that we could do this to the stable, because there is NO regression + chance at all: only the modules themselves were changed, and they did + NOT work at all previously. Please also note that most of these modules + did not yet receive real testing. As we don't have the necessary + environments (easily enough available), we depend on users submitting + error reports and helping to iron out any issues that may arise. + Modules: + * mmrfc5424addhmac + * omrabbitmq + * omgssapi + * omhdfs + * omzmq3 +--------------------------------------------------------------------------- +Version 8.2.1 [v8-stable] 2014-04-17 +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- doc is no longer shipped as part of the rsyslog tarball + Instead, the rsyslog-doc project creates its own tarball. This is the + result of a mailing list discussion after the 8.2.0 release with a + tarball-in-tarball approach, which was disliked by almost all distro + maintainers. This move also has the advantage of de-coupling the + release cycles of both projects a bit (which turned out to be a bit + problematic in practice). +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +--------------------------------------------------------------------------- +Version 8.2.0 [v8-stable] 2014-04-02 +This starts a new stable branch based on 8.1.6 plus the following changes: +- we now use doc from the rsyslog-doc project + As such, the ./doc subtree has been removed. Instead, a cache of the + rsyslog-doc project's files has been included in ./rsyslog-doc.tar.gz. + Note that the exact distribution mode for the doc is still under + discussion and may change in future releases. + This was agreed upon on the rsyslog mailing list. For doc issues + and corrections, be sure to work with the rsyslog-doc project. It is + currently hosted at https://github.com/rsyslog/rsyslog-doc +- add support for specifying the liblogging-stdlog channel spec + new global parameter "stdlog.channelspec" +- add "defaultnetstreamdrivercertfile" global variable to set a default + for the certfile. + Thanks to Radu Gheorghe for the patch. +- omelasticsearch: add new "usehttps" parameter for secured connections + Thanks to Radu Gheorghe for the patch. +- "action resumed" message now also specifies module type + which makes troubleshooting a bit easier. Note that we cannot output all + the config details (like destination etc) as this would require much more + elaborate code changes, which we at least do not like to do in the + stable version. +- add capability to override GnuTLS path in build process + Thanks to Clayton Shotwell for the patch +- better and more consistent action naming, action queues now always + contain the word "queue" after the action name +- bugfix: ompipe did resume itself even when it was still in error + See: https://github.com/rsyslog/rsyslog/issues/35 + Thanks to github user schplat for reporting +- bugfix: ompipe used invalid default template + This is a regression from an old change (didn't track it down precisely, + but over a year ago). It used the Forwarding template instead of + the file template (so we have a full syslog header). This fix corrects + it back to previous behavior, but new scripts that used the wrong + format may now need to have the RSYSLOG_ForwardingFormat template + explicitly be applied. + closes: https://github.com/rsyslog/rsyslog/issues/50 +--------------------------------------------------------------------------- +Version 8.1.6 [release candidate] 2014-02-20 +- omfile: permit to set global defaults for action parameters + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/23 +- add capability to escape control characters in the C way of doing it + adds new global parameter "parser.escapeControlCharactersCStyle" + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/13 +- parser global parameters can now be set using RainerScript global() + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/23 +- omprog: guard program-to-be-executed against CTL-C + This can frequently happen in debug mode, where rsyslog is terminated + by ctl-c. In any case, SIGINT is not meant to control the child process, + so it should be blocked. +- omprog bugfix: parameter "forceSingleInstance" is NOT mandatory +- add new jsonr property replacer option + Thanks to Nathan Brown for the patch. +- added external plugin interface +- ommongodb: add authentication support (untested) + Thanks to JT for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/17 +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behavior, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitly forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +- bugfix: ommongodb's template parameter was mandatory but should have + been optional + Thanks to Alain for the analysis and the patch. +- bugfix: end of batch processing was not 100% correct. Could lead to + outputs not properly writing messages. At least omelasticsearch did not + write anything to the database due to this bug. + See: https://github.com/rsyslog/rsyslog/issues/10 + Thanks to Radu Gheorghe for reporting the issue. +--------------------------------------------------------------------------- +Version 8.1.5 [devel] 2014-01-24 +- omprog: ability to execute multiple program instances per action + It can now execute one program instance per worker thread. This is + generally a very good thing the have performance wise. Usually, this + should cause no problems with the invoked program. For that reason, + we have decided to make this the default mode of operation. If not + desired, it can be turned off via the 'forceSingleInstance="on"' + action parameter. + CHANGE OF BEHAVIOR: previous versions did always execute only one + instance per action, no matter how many workers were active. If + your program has special needs, you need to change your configuration. +- imfile now supports inotify (but must be explicitly turned on) +- imfile no longer has a limit on number of monitored files +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +--------------------------------------------------------------------------- +Version 8.1.4 [devel] 2014-01-10 +- add exec_template() RainerScript function +- imrelp: support for TCP KEEPALIVE added +- bumped librelp dependency to 1.2.2 to support new KEEPALIVE feature +- Add directives for numerically specifying GIDs/UIDs + The already present directives (FileOwner, FileGroup, DirOwner, + DirGroup) translate names to numerical IDs, which depends on the user + information being available during rsyslog's startup. This can fail if + the information is obtained over a network or from a service such as + SSSD. The new directives provide a way to specify the numerical IDs + directly and bypass the lookup. + Thanks to Tomas Heinrich for the patch. +- bugfix: action commitTransaction() processing did not properly handle + suspended actions +- bugfix: omelasticsearch fail.es stats counter was improperly maintained +--------------------------------------------------------------------------- +Version 8.1.3 [devel] 2013-12-06 + +THIS VERSION CAN BE CONSIDERED A "NORMAL" DEVEL RELEASE. It's no longer +highly experimental. This assertion is based on real-world feedback. + +- changes to the strgen module interface +- new output module interface for transactional modules +- performance improvements + * reduced number of malloc/frees due to further changes to the + output module interface + * reduced number of malloc/frees during string template processing + We now re-use once allocated string template memory for as long + as the worker thread exists. This saves us from doing new memory + allocs (and their free counterpart) when the next message is + processed. The drawback is that the cache always is the size of + the so-far largest message processed. This is not considered a + problem, as in any case a single messages' memory footprint should + be far lower than that of a whole set of messages (especially on + busy servers). + * used variable qualifiers (const, __restrict__) to hopefully help + the compiler generate somewhat faster code +- failed action detection more precisely for a number of actions + If an action uses string parameter passing but is non-transactional + it can be executed immediately, giving a quicker indication of + action failure. +- bugfix: limiting queue disk space did not work properly + * queue.maxdiskspace actually initializes queue.maxfilesize + * total size of queue files was not checked against + queue.maxdiskspace for disk assisted queues. + Thanks to Karol Jurak for the patch. +--------------------------------------------------------------------------- +Version 8.1.2 [experimental] 2013-11-28 +- support for liblognorm1 added - results in performance improvements + Thanks to Pavel Levshin for his work in this regard. +- support for jemalloc added via --enable-jemalloc + Thanks to Pavel Levshin for suggesting jemalloc + Note that build system is experimental at this stage. +- queue defaults have changed + * high water mark is now dynamically 90% of queue size + * low water makr is now dynamically 70% of queue size + * queue.discardMark is now dynamically 98% of queue size + * queue.workerThreadMinimumMessage set to queue.size / num workers + For queues with very low queue.maxSize (< 100), "emergency" defaults + will be used. +- bugfix: disk queues created files in wrong working directory + if the $WorkDirectory was changed multiple times, all queues only + used the last value set. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: mmrfc5424addhmac: "key" parameter was not properly processed +--------------------------------------------------------------------------- +Version 8.1.1 [experimental] 2013-11-19 +- bugfix: STOP/discard(~) was mostly NOT honored + This lead to execution of config code that was not meant to be executed. +- bugfix: memory leak on worker thread termination +- bugfix: potential segfault in omfile under heavy load + Thanks to Pavel Levshin for alerting us. +- bugfix: mmsequence: instance mode did not work + Thanks to Pavel Levshin for the patch +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- omhiredis: now supports v8 output module interface and works again + Thanks to Pavel Levshin for the patch +- mmaudit: now supports v8 output module interface and work again +- bugfix: potential abort on startup in debug mode + This depends on template type being used. The root cause was a + non-necessary debug output, which were at the wrong spot (leftover from + initial testing). + Thanks to Pavel Levshin for alerting us and providing a patch + proposal. +--------------------------------------------------------------------------- +Version 8.1.0 [experimental] 2013-11-15 +- rewritten core engine for higher performance and new features + In detail: + * completely rewritten rule execution engine + * completely changed output module interface + * remodelled output module interface + * enabled important output modules to support full concurrent + operation + The core engine has been considerably changed and must be considered + experimental at this stage. Note that it does not yet include all + features planned for v8, but is close to this goal. In theory, the + engine should perform much better, especially on complex configurations + and busy servers. Most importantly, actions instances can now be called + concurrently from worker threads and many important output modules + support multiple concurrent action instances natively. +- module omruleset is no longer enabled by default. + Note that it has been deprecated in v7 and been replaced by the "call" + statement. Also, it can still be build without problems, the option must + just explicitly be given. +--------------------------------------------------------------------------- +Version 7.6.8 [v7.6-stable] 2014-10-?? +- bugfix: typo in queue configuration parameter made parameter unusable + Thanks to Bojan Smojver for the patch. +- bugfix imuxsock: possible segfault when SysSock.Use="off" + Thanks to alexjfisher for reporting this issue. + closes https://github.com/rsyslog/rsyslog/issues/140 +- bugfix: uninitialized buffer off-by-one error in hostname generation + The DNS cache used uninitialized memory, which could lead to + invalid hostname generation. + Thanks to Jarrod Sayers for alerting us and providing analysis and + patch recommendations. +- remove zpipe (a testing tool) from --enable-diagtools + This tool is no longer maintained and currently not used inside the + testbench. We keep it in the source tree for the time being in case that + it may be used in the future. +- bugfix: imjournal did not build properly + The build succeeded, but the module did not load due to a type in + a support function name, which kept unresolved during load. +- bugfix imklog: pri was miscalculated + actually, the pri was totally off the real value for PRIs > 9 +- bugfix rsgtutil: sometimes crashed in verify mode if file did not exist +- bugfix rsgtutil: some errors/problems at end of file were not reported + * The verification function in rsgtutil tool did not report deletion of + whole signed blocks of lines from the end of the log file. + * The verification function in rsgtutil tool did not report extra + (unsigned) lines at the end of the log file. + Thanks to Henri Lakk for the patch. +- bugfix: error: json_tokener_errors undeclared when overriding PKGCONFIG + If PKGCONFIG settings for json-c were overridden, presence of + json_tokener_errors was not properly detected. + closes: https://github.com/rsyslog/rsyslog/issues/143 + Thanks to Alex Fisher for alerting us and the patch. +--------------------------------------------------------------------------- +Version 7.6.7 [v7.6-stable] 2014-10-02 +- bugfix: the fix for CVE-2014-3634 did not handle all cases + This is corrected now. + see also: CVE-2014-3683 +- fixed a build problem on some platforms + Thanks to Olaf for the patch +- behavior change: "msg" of messages with invalid PRI set to "rawmsg" + When the PRI is invalid, the rest of the header cannot be valid. So + we move all of it to MSG and do not try to parse it out. Note that + this is not directly related to the security issue but rather done + because it makes most sense. +--------------------------------------------------------------------------- +Version 7.6.6 [v7.6-stable] 2014-09-30 +- bugfix: potential abort when a message with PRI > 191 was processed + if the "pri-text" property was used in active templates, this could + be abused to a remote denial of service from permitted senders + see also: CVE-2014-3634 +- bugfix: potential segfault on startup on 64 bit systems + This happened immediately on startup during config processing. Once + rsyslog got past this stage, it could not happen. +- bugfix: build problems on SuSe Linux + Thanks Andreas Stieger for the patch +--------------------------------------------------------------------------- +Version 7.6.5 [v7.6-stable] 2014-09-17 +- bugfix: in 7.6.4, pri-based filters did not work correctly + messages were distributed to the wrong bins. +- bugfix: build problems on systems without atomic instructions + e.g. RHEL 5; backport from v8 +--------------------------------------------------------------------------- +Version 7.6.4 [v7.6-stable] 2014-09-12 +- add --enable-generate-man-pages configure switch (default: enabled) + This forces generation of man pages, even if cached ones exists. This + "fixes" a typical release tarball nit. While it is hackish, the + benefit is clear given the history of failed tarball releases since + we changed the cached man page handling. It was just too easy to get + that wrong. +- removed obsolete --disable-fsstnd configure option + Thanks to Thomas D. for alerting us. + Closes: https://github.com/rsyslog/rsyslog/issues/72 +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- new omfile default module parameters + * filecreatemode + * fileowner + * fileownernum + * filegroup + * filegroupnum + * dirowner + * dirownernum + * dirgroup + * dirgroupnum + Thanks to Karol Jurak for the patch. +- bugfix: memory leak in TCP TLS mode +- bugfix: imfile: if a state file for a different file name was set, + that different file (name) was monitored instead of the configured + one. Now, the state file is deleted and the correct file monitored. + closes: https://github.com/rsyslog/rsyslog/issues/103 +- bugfix: using UUID property could cause segfault +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +- bugfix: file descriptor leak with Guardtime signatures + When a .gtstate file is opened it is never closed. This is especially + bad when dynafiles frequently get evicted from dynafile cache and be + re-opened again. +- bugfix: busy loop in tcp listener when running out of file descriptors + Thanks to Susant Sahani for the patch. +- bugfix: mishandling of input modules not supporting new input instances + If they did not support this, accidentally the output module part of the + module union was written, leading to unpredictable results. Note: all + core modules do support this interface, but some contributed or very + old ones do not. +- bugfix: double-free when ruleset() parser parameters were used + While unlikely, this could cause stability issues even after the + config phase. +- bugfix: output modules with parameters with multiple passing modes + could caused strange behavior including aborts + This was due to the fact that the action module only preserved and + processed the last set passing mode. Note that this was not a problem + for the plugins provided by the rsyslog git: none of them uses different + passing modes. + Thanks to Tomas Heinrich for providing a very detailed bug report. +- various fixes after coverity scan + These do not address issues seen in practice but those seen by the tool. + Some of them may affect practical deployments. + Thanks to Tomas Heinrich for the patches. +- bugfix imuxsock: "Last message repeated..." was not emitted at shutdown + The "Last message repeated..." notice didn't get printed if rsyslog was + shut down before the repetition was broken. + Thanks to Tomas Heinrich for the patch. +- bugfix: make dist failed when GUARDTIME or LIBGCRYPT feature was disabled +- bugfix: mmjsonparse did not build with json-c < 0.10 + This was a regression introduced some time in the past in order to + support API changes in json-c. Now we check for the version and use + proper code. +- bugfix: mmanon did not properly anonymize IP addresses starting with '9' + Thanks to defa-at-so36.net for reporting this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529 +--------------------------------------------------------------------------- +Version 7.6.3 [v7.6-stable] 2014-03-27 +- add capability to override GnuTLS path in build process + Thanks to Clayton Shotwell for the patch +- support for librelp 1.2.5 + Support new return states of librelp 1.2.5 to emit better error messages + For obvious reasons, librelp 1.2.5 is now required. +- bugfix: ompipe used invalid default template + This is a regression from an old change (didn't track it down precisely, + but over a year ago). It used the Forwarding template instead of + the file template (so we have a full syslog header). This fix corrects + it back to previous behavior, but new scripts that used the wrong + format may now need to have the RSYSLOG_ForwardingFormat template + explicitly be applied. + closes: https://github.com/rsyslog/rsyslog/issues/50 +- bugfix: ompipe did emit many suspension messages for /dev/xconsole + (hopefully now) closes: https://github.com/rsyslog/rsyslog/issues/35 + When it was present, but nobody reading from it. The problem + is the way the rsyslog v7 engine tries to resolve failures in outputs. + It does some retries, and along those lines some state information gets + lost and it is close to impossible to retain it. However, the actual + root problem is that ompipe does not reliably detect if it is able to + recover. The problem here is that it actually does not know this + before it does an actual write. These two things together mess up the + logic that suppresses invalid resumption/suspension messages + (actually, the plugin switches state really that often). + Nevertheless, the prime problem with /dev/xconsole (and probably + most other pipes as well) is that it gets full. So I have now added + code that checks, during resume processing, if the pipe is writable. + If it is not, resume is deferred. That should address the case. +--------------------------------------------------------------------------- +Version 7.6.2 [v7.6-stable] 2014-03-17 +- support for librelp 1.2.4 + This was necessary due to the problems with librelp 1.2.3 API stability. + We now use the new native 1.2.4 APIs to learn about the state of + librelp's TLS support. + For obvious reasons, librelp 1.2.4 is now required. +--------------------------------------------------------------------------- +Version 7.6.1 [v7.6-stable] 2014-03-13 +- added "action.reportSuspension" action parameter + This now permits to control handling on a per-action basis rather to + the previous "global setting only". +- "action resumed" message now also specifies module type + which makes troubleshooting a bit easier. Note that we cannot output all + the config details (like destination etc) as this would require much more + elaborate code changes, which we at least do not like to do in the + stable version. +- better and more consistent action naming, action queues now always + contain the word "queue" after the action name +- add support for "tls-less" librelp + we now require librelp 1.2.3, as we need the new error code definition + See also: https://github.com/rsyslog/librelp/issues/1 +- build system improvements + * autoconf subdir option + * support for newer json-c packages + Thanks to Michael Biebl for the patches. +- imjournal enhancements: + * log entries with empty message field are no longer ignored + * invalid facility and severity values are replaced by defaults + * new config parameters to set default facility and severity + Thanks to Tomas Heinrich for implementing this +- bugfix: ompipe did resume itself even when it was still in error + See: https://github.com/rsyslog/rsyslog/issues/35 + Thanks to github user schplat for reporting +- bugfix: "action xxx suspended" did report incorrect error code +- bugfix: ommongodb's template parameter was mandatory but should have + been optional + Thanks to Alain for the analysis and the patch. +- bugfix: only partial doc was put into distribution tarball + Thanks to Michael Biebl for alerting us. + see also: https://github.com/rsyslog/rsyslog/issues/31 +- bugfix: async ruleset did process already-deleted messages + Thanks to John Novotny for the patch. +--------------------------------------------------------------------------- +Version 7.6.0 [v7.6-stable] 2014-02-12 +This starts a new stable branch based on 7.5.8 plus the following changes: +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behavior, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitly forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +--------------------------------------------------------------------------- +Version 7.5.8 [v7-release candidate] 2014-01-09 +- add exec_template() RainerScript function +- add debug.onShutdown and debug.logFile global parameters + These enable the new "debug on shutdown" mode, which can be used to + track hard to find problems that occur during system shutdown. +- Add directives for numerically specifying GIDs/UIDs + The already present directives (FileOwner, FileGroup, DirOwner, + DirGroup) translate names to numerical IDs, which depends on the user + information being available during rsyslog's startup. This can fail if + the information is obtained over a network or from a service such as + SSSD. The new directives provide a way to specify the numerical IDs + directly and bypass the lookup. + Thanks to Tomas Heinrich for the patch. +- actions now report if they suspend and resume themselves + this is by default on and controllable by the action.reportSuspension + global parameter +- bugfix: omelasticsearch fail.es stats counter was improperly maintained +- bugfix: mmrfc5424addhmac: "key" parameter was not properly processed +- add new impstats action counters: + * suspended + * suspended.duration + * resumed +--------------------------------------------------------------------------- +Version 7.5.7 [v7-devel] 2013-11-25 +- queue defaults have changed + * high water mark is now dynamically 90% of queue size + * low water makr is now dynamically 70% of queue size + * queue.discardMark is now dynamically 98% of queue size + * queue.workerThreadMinimumMessage set to queue.size / num workers + For queues with very low queue.maxSize (< 100), "emergency" defaults + will be used. +- worker thread pool handling has been improved + Among others, permits pool to actually shrink (was quite hard with + previous implementation. This will also improve performance and/or + lower system overhead on busy systems. + Thanks to Pavel Levshin for the enhancement. +- bugfix: mmpstrucdata generated inaccessible properties +- bugfix: RainerScript optimizer did not optimize PRI filters + things like "if $syslogfacility-text == "local3"" were not converted + to PRIFILT. This was a regression introduced in 7.5.6. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- bugfix: ommysql lost configfile/section parameters after first close + This means that when a connection was broken, it was probably + re-instantiated with different parameters than configured. +- bugfix: regression in template processing with subtrees in templates + Thanks to Pavel Levshin for the fix +- bugfix: regular worker threads are not properly (re)started if DA + mode is active. + This occurs only under rare conditions, but definitely is a bug that + needed to be addressed. It probably is present since version 4. + Note that this patch has not been applied to v7.4-stable, as it + is very unlikely to happen and the fix itself has some regression + potential (the fix looks very solid, but it addresses a core component). + Thanks to Pavel Levshin for the fix +- now emit warning message if om with msg passing mode uses action queue + These can modify the message, and this causes races. +- bugfix: $SystemLogUseSysTimeStamp/$SystemLogUsePIDFromSystem did not work + Thanks to Tomas Heinrich for the patch. +--------------------------------------------------------------------------- +Version 7.5.6 [devel] 2013-10-29 +- impstats: add capability to bind to a ruleset +- improved performance of RainerScript variable access + by refactoring the whole body of variable handling code. This also + solves some of the anomalies experienced in some versions of rsyslog. + All variable types are now handled in unified code, including + access via templates. +- RainerScript: make use of 64 bit for numbers where available + Thanks to Pavel Levshin for enhancement. +- slight performance optimization if GCC is used + We give branch prediction hints for the frequent RETiRet macro which is + used for error handling. Some slight performance gain is to be expected + from that. +- removed global variable support + The original idea was not well thought out and global variables, as + implemented, worked far different from what anybody would expect. As + such, we consider the current approach as an experiment that did not + work out and opt to removing it, clearing the way for a better future + solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. +- new module mmsequence, primarily used for action load balancing + Thanks to Pavel Levshin for contributing this module. +- bugfix: unset statement always worked on message var, even if local + var was given +- imudp: support for binding to ruleset added +- bugfix: segfault if variable was assigned to non-container subtree + Thanks to Pavel Levshin for the fix +- bugfix: imuxsock did not support addtl sockets if syssock was disabled + Thanks to Pavel Levshin for the fix +- bugfix: running imupd on multiple threads lead to segfault if recvmmsg + is available +- bugfix: imudp when using recvmmsg could report wrong sender IP +- bugfix: segfault if re_extract() function was used and no match found +- bugfix: omelasticsearch did not compile on platforms without atomic + instructions +- bugfix: potential misaddressing on startup if property-filter was used + This could happen if the property name was longer than 127 chars, a case + that would not happen in practice. +- bugfix: invalid property filter was not properly disabled in ruleset + Note: the cosmetic memory leak introduced with that patch in 7.4.5 is + now also fixed. +- imported bugfixes from 7.4.6 stable release +--------------------------------------------------------------------------- +Version 7.5.5 [devel] 2013-10-16 +- imfile: permit to monitor an unlimited number of files +- imptcp: add "defaultTZ" input parameter +- imudp: support for multiple receiver threads added +- imudp: add "dfltTZ" input config parameter +- bugfix: memory leak in mmnormalize +- bugfix: mmutf8fix did not properly handle invalid UTF-8 at END of message + if the very last character sequence was too long, this was not detected + Thanks to Risto Vaarandi for reporting this problem. +- mmanon: removed the check for specific "terminator characters" after + last octet. As it turned out, this didn't work in practice as there + was an enormous set of potential terminator chars -- so removing + them was the best thing to do. Note that this may change behavior of + existing installations. Yet, we still consider this an important + bugfix, that should be applied to the stable branch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477 + Thanks to Muri Cicanor for initiating the discussion +- now requires libestr 0.1.7 as early versions had a nasty bug in + string comparisons +- bugfix: mmanon did not detect all IP addresses in rewrite mode + The problem occurred if two IPs were close to each other and the first one + was shrunk. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: mmanon sometimes used invalid replacement char in simple mode + depending on configuration sequence, the replacement character was set + to 's' instead of the correct value. Most importantly, it was set to + 's' if simple mode was selected and no replacement char set. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: memory leak in mmnormalize +- bugfix: array-based ==/!= comparisons lead to invalid results + This was a regression introduced in 7.3.5 bei the PRI optimizer +--------------------------------------------------------------------------- +Version 7.5.4 [devel] 2013-10-07 +- mmpstrucdata: new module to parse RFC5424 structured data into json + message properties +- change main/ruleset queue defaults to be more enterprise-like + new defaults are queue.size 100,000 max workers 2, worker + activation after 40,000 msgs are queued, batch size 256. These settings + are much more useful for enterprises and will not hurt low-end systems + that much. This is part of our re-focus on enterprise needs. +- omfwd: new action parameter "maxErrorMessages" added +- omfile: new module parameters to set action defaults added + * dirCreateMode + * fileCreateMode +- mmutf8fix: new module to fix invalid UTF-8 sequences +- imuxsock: handle unlimited number of additional listen sockets +- doc: improve usability by linking to relevant web resources + The idea is to enable users to quickly find additional information, + samples, HOWTOs and the like on the main site. + At the same time, (very) slightly remove memory footprint when + few listeners are monitored. +- bugfix: omfwd parameter streamdrivermode was not properly handled + it was always overwritten by whatever value was set via the + legacy directive $ActionSendStreamDriverMode +- imtcp: add streamdriver.name module parameter + permits overriding the system default stream driver (gtls, ptcp) +- bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled + Thanks to Jonny Törnbom for reporting this problem +- imported bugfixes from 7.4.4 +--------------------------------------------------------------------------- +Version 7.5.3 [devel] 2013-09-11 +- imfile: support for escaping LF characters added + embedded LF in syslog messages cause a lot of trouble. imfile now has + the capability to escape them to "#012" (just like the regular control + character escape option). This requires new-style input statements to be + used. If legacy configuration statements are used, LF escaping is always + turned off to preserve compatibility. + NOTE: if input() statements were already used, there is a CHANGE OF + BEHAVIOR: starting with this version, escaping is enabled by + default. So if you do not want it, you need to add + escapeLF="off" + to the input statement. Given the trouble LFs cause and the fact + that the majority of installations still use legacy config, we + considered this behavior change acceptable and useful. + see also: http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html +- add support for global and local variables +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. +- add main_queue() configuration object to configure main message queue +- bugfix: stream compression in imptcp caused timestamp to be corrupted +- imudp: add ability to specify SO_RCVBUF size (rcvbufSize parameter) +- imudp: use inputname for statistics, if configured +- impstats: add process resource usage counters [via getrusage()] +- impstats: add parameter "resetCounters" to report delta values + possible for most, but not all, counters. See doc for details. +- librelp 1.2.0 is now required +- make use of new librelp generic error reporting facility + This leads to more error messages being passed to the user and + thus simplified troubleshooting. +- bugfix: very small memory leak in imrelp + more or less cosmetic, a single memory block was not freed, but this + only happens immediately before termination (when the OS automatically + frees all memory). Still an annoyance e.g. in valgrind. +- fix compile problem in debug build +- imported fixes from 7.4.4 +--------------------------------------------------------------------------- +Version 7.5.2 [devel] 2013-07-04 +- librelp 1.1.4 is now required + We use API extensions for better error reporting and higher performance. +- omrelp: use transactional mode to make imrelp emit bulk sends +- omrelp: add "windowSize" parameter to set custom RELP window size +- bugfix: double-free in omelasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461 + a security advisory for this bug is available at: + http://www.lsexperts.de/advisories/lse-2013-07-03.txt + CVE: CVE-2013-4758 + PLEASE NOTE: This issue only existed if omelasticsearch was used + in a non-default configuration, where the "errorfile" parameter + was specified. Without that parameter set, the bug could not + be triggered. + Thanks to Markus Vervier and Marius Ionescu for providing a detailed + bug report. Special thanks to Markus for coordinating his security + advisory with us. +- doc: fixed various typos + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=391 + Thanks to Georgi Georgiev for the patch. +--------------------------------------------------------------------------- +Version 7.5.1 [devel] 2013-06-26 +- librelp 1.1.3 is required - older versions can lead to a segfault +- add mmfields, which among others supports easy parsing of CEF messages +- omrelp: + * new parameter "compression.prioritystring" to control encryption + parameters used by GnuTLS +- imrelp: + * new parameter "compression.dhbits" to control the number of + bits being used for Diffie-Hellman key generation + * new parameter "compression.prioritystring" to control encryption + parameters used by GnuTLS + * support for impstats added + * support for setting permitted peers (client authentication) added + * bugfix: potential segfault at startup on invalid config parameters +- imjournal: imported patches from 7.4.1 +- omprog: add support for command line parameters +- added experimental TCP stream compression (imptcp only, currently) +- added BSD-specific syslog facilities + * "console" + * "bsd_security" - this is called "security" under BSD, but that name + was unfortunately already taken by some standard facility. So I + did the (hopefully) second-best thing and renamed it a little. +- imported fixes from 7.4.2 (especially build problems on FreeBSD) +- bugfix: imptcp did not properly initialize compression status variable + could lead to segfault if stream:always compression mode was selected +--------------------------------------------------------------------------- +Version 7.5.0 [devel] 2013-06-11 +- imrelp: implement "ruleset" module parameter +- imrelp/omrelp: add TLS & compression (zip) support +- omrelp: add "rebindInterval" parameter +- add -S command line option to specify IP address to use for RELP client + connections + Thanks to Axel Rau for the patch. +--------------------------------------------------------------------------- +Version 7.4.11 [v7.4-stable] *never released* +- imjournal enhancements: + * log entries with empty message field are no longer ignored + * invalid facility and severity values are replaced by defaults + * new config parameters to set default facility and severity + Thanks to Tomas Heinrich for implementing this +--------------------------------------------------------------------------- +Version 7.4.10 [v7.4-stable] 2014-02-12 +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behavior, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitly forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +--------------------------------------------------------------------------- +Version 7.4.9 [v7.4-stable] 2014-01-22 +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +- bugfix: potential double-free in RainerScript equal comparison + happens if the left-hand operand is JSON object and the right-hand + operand is a non-string that does not convert to a number (for + example, it can be another JSON object, probably the only case that + could happen in practice). This is very unlikely to be triggered. +- bugfix: some RainerScript Json(Variable)/string comparisons were wrong +--------------------------------------------------------------------------- +Version 7.4.8 [v7.4-stable] 2014-01-08 +- rsgtutil provides better error messages on unfinished signature blocks +- bugfix: guard against control characters in internal (error) messages + Thanks to Ahto Truu for alerting us. +- bugfix: immark did emit messages under kern.=info instead of syslog.=info + Note that his can potentially break existing configurations that + rely on immark sending as kern.=info. Unfortunately, we cannot leave + this unfixed as we never should emit messages under the kern facility. +--------------------------------------------------------------------------- +Version 7.4.7 [v7.4-stable] 2013-12-10 +- bugfix: limiting queue disk space did not work properly + * queue.maxdiskspace actually initializes queue.maxfilesize + * total size of queue files was not checked against + queue.maxdiskspace for disk assisted queues. + Thanks to Karol Jurak for the patch. +- bugfix: linux kernel-like ratelimiter did not work properly with all + inputs (for example, it did not work with imdup). The reason was that + the PRI value was used, but that needed parsing of the message, which + was done too late. +- bugfix: disk queues created files in wrong working directory + if the $WorkDirectory was changed multiple times, all queues only + used the last value set. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- bugfix: imuxsock: UseSysTimeStamp config parameter did not work correctly + Thanks to Tomas Heinrich for alerting us and providing a solution + suggestion. +- bugfix: $SystemLogUseSysTimeStamp/$SystemLogUsePIDFromSystem did not work + Thanks to Tomas Heinrich for the patch. +- improved checking of queue config parameters on startup +- bugfix: call to ruleset with async queue did not use the queue + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=443 +- bugfix: if imtcp is loaded and no listeners are configured (which is + uncommon), rsyslog crashes during shutdown. +--------------------------------------------------------------------------- +Version 7.4.6 [v7.4-stable] 2013-10-31 +- bugfix: potential abort during HUP + This could happen when one of imklog, imzmq3, imkmsg, impstats, + imjournal, or imuxsock were under heavy load during a HUP. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=489 + Thanks to Guy Rozendorn for reporting the problem and Peval Levshin for + his analysis. +- bugfix: imtcp flowControl parameter incorrectly defaulted to "off" + This could cause message loss on systems under heavy load and was + a change-of-behavior to previous version. This is a regression + most probably introduced in 5.9.0 (but did not try hard to find the + exact point of its introduction). +- now requires libestr 0.1.9 as earlier versions lead to problems with + number handling in RainerScript +- bugfix: memory leak in strlen() RainerScript function + Thanks to Gregoire Seux for reporting this bug. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=486 +- bugfix: buffer overrun if re_extract function was called for submatch 50 + Thanks to Pavel Levshin for reporting the problem and its location. +- bugfix: memleak in re_extract() function + Thanks to Pavel Levshin for reporting this problem. +- bugfix: potential abort in RainerScript optimizer + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=488 + Thanks to Thomas Doll for reporting the problem and Pavel Levshin for + fixing it. +- bugfix: memory leak in omhiredis + Thanks to Pavel Levshin for the fix +- bugfix: segfault if variable was assigned to non-container subtree + Thanks to Pavel Levshin for the fix +--------------------------------------------------------------------------- +Version 7.4.5 [v7.4-stable] 2013-10-22 +- mmanon: removed the check for specific "terminator characters" after + last octet. As it turned out, this didn't work in practice as there + was an enormous set of potential terminator chars -- so removing + them was the best thing to do. Note that this may change behavior of + existing installations. Yet, we still consider this an important + bugfix, that should be applied to the stable branch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477 + Thanks to Muri Cicanor for initiating the discussion +- now requires libestr 0.1.8 as early versions had a nasty bug in + string comparisons +- omelasticsearch: add failed.httprequests stats counter +- bugfix: invalid property filter was not properly disabled in ruleset + Note that this bugfix introduces a very slight memory leak, which is + cosmetic, as it just holds data until termination that is no longer + needed. It is just the part of the config that was invalid. We will + "fix" this "issue" in the devel version first, as the fix is a bit + too intrusive to do without hard need in the stable version. +- bugfix: segfault if re_extract() function was used and no match found +- bugfix: potential misaddressing on startup if property-filter was used + This could happen if the property name was longer than 127 chars, a case + that would not happen in practice. +- bugfix: omelasticsearch: correct failed.http stats counter +- bugfix: omelasticsearch: did not correctly initialize stats counters +- bugfix: omelasticsearch: failed.es counter was only maintained in bulk mode + This usually did not lead to any problems, because they are in static + memory, which is initialized to zero by the OS when the plugin is + loaded. But it may cause problems especially on systems that do not + support atomic instructions - in this case the associated mutexes also + did not get properly initialized. +- bugfix: mmanon did not detect all IP addresses in rewrite mode + The problem occurred if two IPs were close to each other and the first one + was shrunk. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: mmanon sometimes used invalid replacement char in simple mode + depending on configuration sequence, the replacement character was set + to 's' instead of the correct value. Most importantly, it was set to + 's' if simple mode was selected and no replacement char set. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: memory leak in mmnormalize +- bugfix: array-based ==/!= comparisons lead to invalid results + This was a regression introduced in 7.3.5 bei the PRI optimizer +- bugfix: omprog blocked signals to executed programs + The made it impossible to send signals to programs executed via + omprog. + Thanks to Risto Vaarandi for the analysis and a patch. +- bugfix: doc: imuxsock legacy param $SystemLogSocketParseTrusted was + misspelled + Thanks to David Lang for alerting us +- bugfix: imfile "facility" input parameter improperly handled + caused facility not to be set, and severity to be overwritten with + the facility value. + Thanks to forum user dmunny for reporting this bug. +- bugfix: small memory leak in imfile when $ResetConfigVariables was used + Thanks to Grégory Nuyttens for reporting this bug and providing a fix +- bugfix: segfault on startup if TLS was used but no CA cert set +- bugfix: segfault on startup if TCP TLS was used but no cert or key set +- bugfix: some more build problems with newer json-c versions + Thanks to Michael Biebl for mentioning the problem. +- bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled + Thanks to Jonny Törnbom for reporting this problem +--------------------------------------------------------------------------- +Version 7.4.4 [v7.4-stable] 2013-09-03 +- better error messages in GuardTime signature provider + Thanks to Ahto Truu for providing the patch. +- make rsyslog use the new json-c pkgconfig file if available + Thanks to the Gentoo team for the patches. +- bugfix: imfile parameter "persistStateInterval" was unusable + due to a case typo in imfile; work-around was to use legacy config + Thanks to Brandon Murphy for reporting this bug. +- bugfix: TLV16 flag encoding error in signature files from GT provider + This fixes a problem where the TLV16 flag was improperly encoded. + Unfortunately, existing files already have the bug and may not properly + be processed. The fix uses constants from the GuardTime API lib to + prevent such problems in the future. + Thanks to Ahto Truu for providing the patch. +- bugfix: slightly malformed SMTP handling in ommail +- bugfix: segfault in omprog if no template was provided (now dflt is used) +- bugfix: segfault in ompipe if no template was provided (now dflt is used) +- bugfix: segfault in omsnmp if no template was provided (now dflt is used) +- bugfix: some omsnmp optional config params were flagged as mandatory +- bugfix: segfault in omelasticsearch when resuming queued messages + after restarting Elasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=464 +- bugfix: imtcp addtlframedelimiter could not be set to zero + Thanks to Chris Norton for alerting us. +- doc bugfix: remove no-longer existing omtemplate from developer doc + was specifically mentioned as a sample for creating new plugins + Thanks to Yannick Brosseau for alerting us of this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=473 +--------------------------------------------------------------------------- +Version 7.4.3 [v7.4-stable] 2013-07-18 +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. +- bugfix: $QHOUR/$HHOUR were always "00" or "01" + regression some time between v5 and here + Thanks to forum user rjmcinty for reporting this bug +- bugfix: testbench tool chkseq did improperly report invalid file + This happened when permitted duplicate values existed in the very + last lines, right before end-of-file. + Thanks to Radu Gheorghe for reporting this bug. +--------------------------------------------------------------------------- +Version 7.4.3 [v7.4-stable] 2013-07-18 +- bugfix: memory leak if disk queues were used and json data present +- bugfix: CEE/json data was lost during disk queue operation +- bugfix: potential segfault during startup on invalid config + could happen if invalid actions were present, which could lead + to improper handling in optimizer. +- bugfix: 100% CPU utilization when DA queue became full +- bugfix: omlibdbi did not properly close connection on some errors + This happened to errors occurring in Begin/End Transaction entry + points. +- cosmetic bugfix: file name buffer was not freed on disk queue destruction + This was an extremely small one-time per run memleak, so nothing of + concern. However, it bugs under valgrind and similar memory debuggers. +- fix build on FreeBSD + Thanks to Christiano Rolim for the patch +--------------------------------------------------------------------------- +Version 7.4.2 [v7.4-stable] 2013-07-04 +- bugfix: in RFC5425 TLS, multiple wildcards in auth could cause segfault +- bugfix: RainerScript object required parameters were not properly + checked - this could result to segfaults on startup if parameters + were missing. +- bugfix: double-free in omelasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461 + a security advisory for this bug is available at: + http://www.lsexperts.de/advisories/lse-2013-07-03.txt + CVE: CVE-2013-4758 + PLEASE NOTE: This issue only existed if omelasticsearch was used + in a non-default configuration, where the "errorfile" parameter + was specified. Without that parameter set, the bug could not + be triggered. + Thanks to Markus Vervier and Marius Ionescu for providing a detailed + bug report. Special thanks to Markus for coordinating his security + advisory with us. +- bugfix: omrelp potential segfault at startup on invalid config parameters +- bugfix: small memory leak when $uptime property was used +- bugfix: potential segfault on rsyslog termination in imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=456 +- bugfix: lmsig_gt abort on invalid configuration parameters + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=448 + Thanks to Risto Laanoja for the patch. +- imtcp: fix typo in "listner" parameter, which is "listener" + Currently, both names are accepted. +- solved build problems on FreeBSD + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=457 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=458 + Thanks to Christiano for reporting and suggesting patches +- solved build problems on CENTOS5 +--------------------------------------------------------------------------- +Version 7.4.1 [v7.4-stable] 2013-06-17 +- imjournal: add ratelimiting capability + The original imjournal code did not support ratelimiting at all. We + now have our own ratelimiter. This can mitigate against journal + database corruption, when the journal re-sends old data. This is a + current bug in systemd journal, but we won't outrule this to happen + in the future again. So it is better to have a safeguard in place. + By default, we permit 20,000 messages within 10 minutes. This may + be a bit restrictive, but given the risk potential it seems reasonable. + Users requiring larger traffic flows can always adjust the value. +- bugfix: potential loop in rate limiting + if the message that tells about rate-limiting gets rate-limited itself, + it will potentially create and endless loop +- bugfix: potential segfault in imjournal if journal DB is corrupted +- bugfix: prevent a segfault in imjournal if state file is not defined +- bugfix imzmq3: potential segfault on startup + if no problem happened at startup, everything went fine + Thanks to Hongfei Cheng and Brian Knox for the patch +--------------------------------------------------------------------------- +Version 7.4.0 [v7.4-stable] 2013-06-06 +This starts a new stable branch based on 7.3.15 plus the following changes: +- add --enable-cached-man-pages ./configure option + permits to build rsyslog on a system where rst2man is not installed. In + that case, cached versions of the man pages are used (they were built + during "make dist", so they should be current for the version in + question. +- doc bugfix: ReadMode wrong in imfile doc, two values were swapped + Thanks to jokajak@gmail.com for mentioning this + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=450 +- imjournal: no longer do periodic wakeup +- bugfix: potential hang *in debug mode* on rsyslogd termination + This ONLY affected rsyslogd if it were running with debug output + enabled. +- bugfix: $template statement with multiple spaces lead to invalid tpl name + If multiple spaces were used in front of the template name, all but one + of them became actually part of the template name. So + $template a,"..." would be name " a", and as such "a" was not + available, e.g. in + *.* /var/log/file;a + This is a legacy config problem. As it was unreported for many years, + no backport of the fix to old versions will happen. + This is a long-standing bug that was only recently reported by forum + user mc-sim. + Reference: http://kb.monitorware.com/post23448.html +- 0mq fixes; credits to Hongfei Cheng and Brian Knox +--------------------------------------------------------------------------- +Version 7.3.15 [beta] 2013-05-15 +- bugfix: problem in build system (especially when cross-compiling) + Thanks to Tomas Heinrich and winfried_mb2@xmsnet.nl for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=445 +- bugfix: imjournal had problem with systemd journal API change +- imjournal: now obtain and include PID +- bugfix: .logsig files had tlv16 indicator bit at wrong offset +- bugfix: omrelp legacy config parameters set a timeout of zero + which lead the legacy config to be unusable. +- bugfix: segfault on startup if a disk queue was configure without file + name + Now this triggers an error message and the queue is changed to + linkedList type. +- bugfix: invalid addressing in string class (recent regression) +--------------------------------------------------------------------------- +Version 7.3.14 [beta] 2013-05-06 +- bugfix: some man pages were not properly installed + either rscryutil or rsgtutil man was installed, but not both + Thanks to Marius Tomaschewski for the patch. +- bugfix: potential segfault on startup when builtin module was specified + in module() statement. + Thanks to Marius Tomaschewski for reporting the bug. +- bugfix: segfault due to invalid dynafile cache handling + Accidentally, the old-style cache size parameter was used when the + dynafile cache was created in a RainerScript action. If the old-style + size was lower than the one actually set, this lead to misaddressing + when the size was overrun, and that could lead to all kinds of + "interesting things", often in segfaults. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=440 +--------------------------------------------------------------------------- +Version 7.3.13 [beta] 2013-04-29 +- added omrabbitmq module (contributed, untested) + Note: this is unsupported and as such was moved immediately into the + beta version. + Thanks to Vaclav Tomec for providing this module. +- bugfix: build problem when --enable-encryption was not selected + Thanks to Michael Biebl for fixing this. +- doc bugfix: omfile parameter "VeryRobustZip" was documented as + "VeryReliableZip" + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=437 + Thanks to Thomas Doll for reporting this. +--------------------------------------------------------------------------- +Version 7.3.12 [devel] 2013-04-25 +- added doc for omelasticsearch + Thanks to Radu Gheorghe for the doc contribution. +- omelasticsearch: _id field support for bulk operations + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=392 + Thanks to Jérôme Renard for the idea and patches. +- max number of templates for plugin use has been increased to five +- platform compatibility enhancement: solve compile issue with libgcrypt + do not use GCRY_CIPHER_MODE_AESWRAP where not available +- fix compile on Solaris + Thanks to Martin Carpenter for the patch. +- bugfix: off-by-one error in handling local FQDN name (regression) + A temporary buffer was allocated one byte too small. Did only + affect startup, not actual operations. Came up during routine tests, + and can have no effect once the engine runs. Bug was introduced in + 7.3.11. +- bugfix: build problems on Solaris + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=436 +- bugfix: block size limit was not properly honored +- bugfix: potential segfault in guardtime signature provider + it could segfault if an error was reported by the GuardTime API, because + an invalid free could happen then +--------------------------------------------------------------------------- +Version 7.3.11 [devel] 2013-04-23 +- added support for encrypting log files +- omhiredis: added support for redis pipeline support + Thanks to Brian Knox for the patch. +- bugfix: $PreserveFQDN is not properly working + Thanks to Louis Bouchard for the patch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=426 +- bugfix: imuxsock aborted due to problem in ratelimiting code + Thanks to Tomas Heinrich for the patch. +- bugfix: imuxsock aborted under some conditions + regression from ratelimiting enhancements - this was a different one + to the one Tomas Heinrich patched. +- bugfix: timestamp problems in imkmsg +--------------------------------------------------------------------------- +Version 7.3.10 [devel] 2013-04-10 +- added RainerScript re_extract() function +- omrelp: added support for RainerScript-based configuration +- omrelp: added ability to specify session timeout +- templates now permit substring extraction relative to end-of-string +- bugfix: failover/action suspend did not work correctly + This was experienced if the retry action took more than one second + to complete. For suspending, a cached timestamp was used, and if the + retry took longer, that timestamp was already in the past. As a + result, the action never was kept in suspended state, and as such + no failover happened. The suspend functionality now does no longer use + the cached timestamp (should not have any performance implication, as + action suspend occurs very infrequently). +- bugfix: gnutls RFC5425 driver had some undersized buffers + Thanks to Tomas Heinrich for the patch. +- bugfix: nested if/prifilt conditions did not work properly + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=415 +- bugfix: imuxsock aborted under some conditions + regression from ratelimiting enhancements +- bugfix: build problems on Solaris + Thanks to Martin Carpenter for the patches. +--------------------------------------------------------------------------- +Version 7.3.9 [devel] 2013-03-27 +- support for signing logs added +- imudp: now supports user-selectable inputname +- omlibdbi: now supports transaction interface + if recent enough lbdbi is present +- imuxsock: add ability to NOT create/delete sockets during startup and + shutdown + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=259 +- imfile: errors persisting state file are now reported + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=292 +- imfile: now detects file change when rsyslog was inactive + Previously, this case could not be detected, so if a file was overwritten + or rotated away while rsyslog was stopped, some data was missing. This + is now detected and the new file being forwarded right from the + beginning. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=228 +- updated systemd files to match current systemd source +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- bugfix: build problem on platforms without GLOB_NOMAGIC +- bugfix: build problems on non-Linux platforms +- bugfix: stdout/stderr were not closed on forking + but were closed when running in the foreground - this was just reversed + of what it should be. This is a regression of a recent change. +--------------------------------------------------------------------------- +Version 7.3.8 [devel] 2013-03-18 +- imrelp: now supports listening to IPv4/v6 only instead of always both + build now requires librelp 1.0.2 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=378 +- bugfix: mmanon did not build on some platforms (e.g. Ubuntu) +- bugfix: segfault in expression optimizer + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=423 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. +--------------------------------------------------------------------------- +Version 7.3.7 [devel] 2013-03-12 +- add support for anonymizing IPv4 addresses +- add support for writing to the Linux Journal (omjournal) +- imuxsock: add capability to ignore messages from ourselves + This helps prevent message routing loops, and is vital to have + if omjournal is used together with traditional syslog. +- field() function now supports a string as field delimiter +- added ability to configure debug system via rsyslog.conf +- bugfix: imuxsock segfault when system log socket was used +- bugfix: mmjsonparse segfault if new-style config was used +- bugfix: script == comparison did not work properly on JSON objects +- bugfix: field() function did never return "***FIELD NOT FOUND***" + instead it returned "***ERROR in field() FUNCTION***" in that case +--------------------------------------------------------------------------- +Version 7.3.6 [devel] 2013-01-28 +- greatly improved speed of large-array [N]EQ RainerScript comparisons + Thanks to David Lang for a related discussion that inspired the idea + to do this with a much simpler (yet sufficient) approach than originally + planned for. +- greatly improved speed of DNS cache for large cache sizes +- general performance improvements +- omfile: added stats counters for dynafile caches +- omfile: improved async writing, finally enabled full async write + also fixed a couple of smaller issues along that way +- impstats: added ability to write stats records to local file + and avoid going through the syslog log stream. syslog logging can now + also be turned off (see doc for details). +- bugfix: imklog issued wrong facility in error messages + ...what could lead to problems in other parts of the code +- fix compile problem in imklog +- added capability to output thread-id-to-function debug info + This is a useful debug aid, but nothing of concern for regular users. +--------------------------------------------------------------------------- +Version 7.3.5 [devel] 2012-12-19 +- ommysql: addded batching/transaction support +- enhanced script optimizer to optimize common PRI-based comparisons + These constructs are especially used in SUSE default config files, + but also by many users (as they are more readable than the equivalent + PRI-based filter). +- omudpspoof: add support for new config system +- omudpspoof: add support for packets larger than 1472 bytes + On Ethernet, they need to be transmitted in multiple fragments. While + it is known that fragmentation can cause issues, it is the best choice + to be made in that case. Also improved debug output. +- bugfix: omudpspoof failed depending on the execution environment + The v7 engine closes fds, and closed some of libnet's fds as well, what + lead to problems (unfortunately, at least some libnet versions do not + report a proper error state but still "success"...). The order of libnet + calls has been adjusted to by in sync with what the core engine does. +- bugfix: segfault on imuxsock startup if system log socket is used + and no ratelimiting supported. Happens only during initial config + read phase, once this is over, everything works stable. +- bugfix: mmnormalize build problems +- bugfix: mmnormalize could abort rsyslog if config parameter was in error +- bugfix: no error message for invalid string template parameters + rather a malformed template was generated, and error information emitted + at runtime. However, this could be quite confusing. Note that with this + "bugfix" user experience changes: formerly, rsyslog and the affected + actions properly started up, but the actions did not produce proper + data. Now, there are startup error messages and the actions are NOT + executed (due to missing template due to template error). +- bugfix[minor]: invalid error code when mmnormalize could not access + rulebase +- bugfix(kind of): script optimizer did not work for complex boolean + expressions +- doc bugfix: corrections and improvements in mmnormalize html doc page +- bugfix: some message properties could be garbled due to race condition + This happened only on very high volume systems, if the same message was + being processed by two different actions. This was a regression caused + by the new config processor, which did no longer properly enable msg + locking in multithreaded cases. The bugfix is actually a refactoring of + the msg locking code - we no longer do unlocked operations, as the use + case for it has mostly gone away. It is potentially possible only at + very low-end systems, and there the small additional overhead of doing + the locking does not really hurt. Instead, the removal of that + capability can actually slightly improve performance in common cases, + as the code path is smaller and requires slightly less memory writes. + That probably outperforms the extra locking overhead (which in the + low-end case always happens in user space, without need for kernel + support as we can always directly acquire the lock - there is no + contention at all). +- build system cleanup (thanks to Michael Biebl for this!) +- bugfix: omelasticsearch did not properly compile on some platforms + due to missing libmath. Thanks to Michael Biebl for the fix +--------------------------------------------------------------------------- +Version 7.3.4 [devel] 2012-11-23 +- further (and rather drastically) improved disk queue performance + we now save one third of the IO calls +- imklog: added ParseKernelTimestamp parameter (import from 5.10.2) + Thanks to Marius Tomaschewski for the patch. +- imklog: added KeepKernelTimestamp parameter (import from 5.10.2) + Thanks to Marius Tomaschewski for the patch. +- bugfix: improper handling of backslash in string-type template()s +- bugfix: leading quote (") in string-type template() lead to tight loop + on startup +- bugfix: no error msg on invalid field option in legacy/string template +- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds + ... actually, they are microseconds. So the fractional part of the + timestamp was not properly formatted. (import from 5.10.2) + Thanks to Marius Tomaschewski for the bug report and the patch idea. +--------------------------------------------------------------------------- +Version 7.3.3 [devel] 2012-11-07 +- improved disk queue performance +- bugfix: dynafile zip files could be corrupted + This could happen if a dynafile was destructed before the first write. + In practice, this could happen if few lines were written to a file and + it then became evicted from the dynafile cache. This would probably + look very random, because it depended on the timing in regard to + message volume and dynafile cache size. +--------------------------------------------------------------------------- +Version 7.3.2 [devel] 2012-10-30 +- mmnormalize: support for v6+ config interface added +- mmjsonparse: support for v6+ config interface added +--------------------------------------------------------------------------- +Version 7.3.2 [devel] 2012-10-30 +- totally reworked ratelimiting and "last message repeated n times" + all over rsyslog code. Each of the supported inputs now supports + linux-like ratelimiting (formerly only imuxsock did). Also, the + "last message repeated n times" is now processed at the input side + and no longer at the output side of rsyslog processing. This + provides the basis for new future additions as well as usually more + performance and a much simpler output part (which can be even further + refactored). +- imtcp: support for Linux-Type ratelimiting added +- imptcp: support for Linux-Type ratelimiting added +- imudp enhancements: + * support for input batching added (performance improvement) + * support for Linux-Type ratelimiting added +- permited action-like statements (stop, call, ...) in action lists +- bugfix: segfault on startup when modules using MSG_PASSING mode are used +- omelasticsearch: support for writing data errors to local file added +- omelasticsearch: fix check for bulk processing status response +--------------------------------------------------------------------------- +Version 7.3.1 [devel] 2012-10-19 +- optimized template processing performance, especially for $NOW family + of properties +- change lumberjack cookie to "@cee:" from "@cee: " + CEE originally specified the cookie with SP, whereas other lumberjack + tools used it without space. In order to keep interop with lumberjack, + we now use the cookie without space as well. I hope this can be changed + in CEE as well when it is released at a later time. + Thanks to Miloslav Trmač for pointing this out and a similar v7 patch. +- bugfix: imuxsock and imklog truncated head of received message + This happened only under some circumstances. Thanks to Marius + Tomaschewski, Florian Piekert and Milan Bartos for their help in + solving this issue. +- bugfix: imuxsock did not properly honor $LocalHostIPIF +--------------------------------------------------------------------------- +Version 7.3.0 [devel] 2012-10-09 +- omlibdbi improvements, added + * support for config load phases & module() parameters + * support for default templates + * driverdirectory is now cleanly a global parameter, but can no longer + be specified as an action parameter. Note that in previous versions + this parameter was ignored in all but the first action definition +- improved omfile zip writer to increase compression + This was achieved by somewhat reducing the robustness of the zip archive. + This is controlled by the new action parameter "VeryReliableZip". +---------------------------------------------------------------------------- +Version 7.2.8 [v7-stable] 2013-0?-?? +- bugfix: potential segfault on startup when builtin module was specified + in module() statement. + Thanks to Marius Tomaschewski for reporting the bug. +- bugfix: segfault due to invalid dynafile cache handling + Accidentally, the old-style cache size parameter was used when the + dynafile cache was created in a RainerScript action. If the old-style + size was lower than the one actually set, this lead to misaddressing + when the size was overrun, and that could lead to all kinds of + "interesting things", often in segfaults. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=440 +---------------------------------------------------------------------------- +Version 7.2.7 [v7-stable] 2013-04-17 +- rsyslogd startup information is now properly conveyed back to init + when privileges are being dropped + Actually, we have moved termination of the parent in front of the + priv drop. So it shall work now in all cases. See code comments in + commit for more details. +- If forking, the parent now waits for a maximum of 60 seconds for + termination by the child +- improved debugging support in forked (auto-backgrounding) mode + The rsyslog debug log file is now continued to be written across the + fork. +- updated systemd files to match current systemd source +- bugfix: failover/action suspend did not work correctly + This was experienced if the retry action took more than one second + to complete. For suspending, a cached timestamp was used, and if the + retry took longer, that timestamp was already in the past. As a + result, the action never was kept in suspended state, and as such + no failover happened. The suspend functionality now does no longer use + the cached timestamp (should not have any performance implication, as + action suspend occurs very infrequently). +- bugfix: nested if/prifilt conditions did not work properly + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=415 +- bugfix: script == comparison did not work properly on JSON objects + [backport from 7.3 branch] +- bugfix: imudp scheduling parameters did affect main thread, not imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=409 +- bugfix: imuxsock rate-limiting could not be configured via legacy conf + Rate-limiting for the system socket could not be configured via legacy + configuration directives. However, the new-style RainerScript config + options worked. + Thanks to Milan Bartos for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=390 +- bugfix: using group resolution could lead to endless loop + Thanks to Tomas Heinrich for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 +- bugfix: $mmnormalizeuseramsg parameter was specified with wrong type + Thank to Renzhong Zhang for alerting us of the problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=420 +- bugfix: RainerScript getenv() function caused segfault when var was + not found. + Thanks to Philippe Muller for the patch. +- bugfix: several issues in imkmsg + see bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=421#c8 +- bugfix: imuxsock was missing SysSock.ParseTrusted module parameter + To use that functionality, legacy rsyslog.conf syntax had to be used. + Also, the doc was missing information on the "ParseTrusted" set of + config directives. +- bugfix: parameter action.execOnlyWhenPreviousIsSuspended was accidentally + of integer-type. For obvious reasons, it needs to be boolean. Note + that this change can break existing configurations if they circumvented + the problem by using 0/1 values. +- doc bugfix: rsyslog.conf man page had invalid file format info + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=418 +---------------------------------------------------------------------------- +Version 7.2.6 [v7-stable] 2013-03-05 +- slightly improved config parser error messages when invalid escapes happen +- bugfix: include files got included in the wrong order + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=411 + This happens if an $IncludeConfig directive was done on multiple + files (e.g. the distro default of $IncludeConfig /etc/rsyslog.d/*.conf). + In that case, the order of include file processing is reversed, which + could lead to all sorts of problems. + Thanks to Nathan Stratton Treadway for his great analysis of the problem, + which made bug fixing really easy. +- bugfix: omelasticsearch failed when authentication data was provided + ... at least in most cases it emitted an error message: + "snprintf failed when trying to build auth string" + Thanks to Joerg Heinemann for alerting us. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=404 +- bugfix: some property-based filter were incorrectly parsed + This usually lead to a syntax error on startup and rsyslogd not actually + starting up. The problem was the regex, which did not care for double + quote characters to follow in the action part - unfortunately something + that can frequently happen with v6+ format. An example: + :programname, isequal, "as" {action(type="omfile" ...) } + Here, the part + :programname, isequal, "as" {action(type="omfile" + was treated as the property filter, and the rest as action part. + Obviously, this did not work out. Unfortunately, such situations usually + resulted in very hard to understand error messages. +---------------------------------------------------------------------------- +Version 7.2.5 [v7-stable] 2013-01-08 +- build system cleanup (thanks to Michael Biebl for this!) +- bugfix: omelasticsearch did not properly compile on some platforms + due to missing libmath. Thanks to Michael Biebl for the fix +- bugfix: invalid DST handling under Solaris + Thanks to Scott Severtson for the patch. +- bugfix: on termination, actions were incorrectly called + The problem was that incomplete fiter evaluation was done *during the + shutdown phase*. This affected only the LAST batches being processed. No + problem existed during the regular run. Could usually only happen on + very busy systems, which were still busy during shutdown. +- bugfix: very large memory consumption (and probably out of memory) when + FromPos was specified in template, but ToPos not. + Thanks to Radu Gheorghe for alerting us of this bug. +- bugfix: timeval2syslogTime cause problems on some platforms + due to invalid assumption on structure data types. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=394 + Thanks to David Hill for the patch [under ASL2.0 as per email conversation + 2013-01-03]. +- bugfix: compile errors in im3195 + Thanks to Martin Körper for the patch +- bugfix: doGetFileCreateMode() had invalid validity check ;) + Thanks to Chandler Latour for the patch. +- bugfix: mmjsonparse erroneously returned action error when no CEE cookie + was present. +---------------------------------------------------------------------------- +Version 7.2.4 [v7-stable] 2012-12-07 +- enhance: permit RFC3339 timestamp in local log socket messages + Thanks to Sebastien Ponce for the patch. +- imklog: added ParseKernelTimestamp parameter (import from 5.10.2) + Thanks to Marius Tomaschewski for the patch. +- fix missing functionality: ruleset(){} could not specify ruleset queue + The "queue.xxx" parameter set was not supported, and legacy ruleset + config statements did not work (by intention). The fix introduces the + "queue.xxx" parameter set. It has some regression potential, but only + for the new functionality. Note that using that interface it is possible + to specify duplicate queue file names, which will cause trouble. This + will be solved in v7.3, because there is a too-large regression + potential for the v7.2 stable branch. +- imklog: added KeepKernelTimestamp parameter (import from 5.10.2) + Thanks to Marius Tomaschewski for the patch. +- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds + ... actually, they are microseconds. So the fractional part of the + timestamp was not properly formatted. (import from 5.10.2) + Thanks to Marius Tomaschewski for the bug report and the patch idea. +- bugfix: supportoctetcountedframing parameter did not work in imptcp +- bugfix: modules not (yet) supporting new conf format were not properly + registered. This lead to a "module not found" error message instead of + the to-be-expected "module does not support new style" error message. + That invalid error message could be quite misleading and actually stop + people from addressing the real problem (aka "go nuts" ;)) +- bugfix: template "type" parameter is mandatory (but was not) +- bugfix: some message properties could be garbled due to race condition + This happened only on very high volume systems, if the same message was + being processed by two different actions. This was a regression caused + by the new config processor, which did no longer properly enable msg + locking in multithreaded cases. The bugfix is actually a refactoring of + the msg locking code - we no longer do unlocked operations, as the use + case for it has mostly gone away. It is potentially possible only at + very low-end systems, and there the small additional overhead of doing + the locking does not really hurt. Instead, the removal of that + capability can actually slightly improve performance in common cases, + as the code path is smaller and requires slightly less memory writes. + That probably outperforms the extra locking overhead (which in the + low-end case always happens in user space, without need for kernel + support as we can always directly acquire the lock - there is no + contention at all). +---------------------------------------------------------------------------- +Version 7.2.3 [v7-stable] 2012-10-21 +- regression fix: rsyslogd terminated when wild-card $IncludeConfig did not + find actual include files. For example, if this directive is present: + $IncludeConfig /etc/rsyslog.d/*.conf + and there are no *.conf files in /etc/rsyslog.d (but rsyslog.d exists), + rsyslogd will emit an error message and terminate. Previous (and expected) + behavior is that an empty file set is no problem. HOWEVER, if the + directory itself does not exist, this is flagged as an error and will + load to termination (no startup). + Unfortunately, this is often the case by default in many distros, so this + actually prevents rsyslog startup. +---------------------------------------------------------------------------- +Version 7.2.2 [v7-stable] 2012-10-16 +- doc improvements +- enabled to build without libuuid, at loss of uuid functionality + this enables smoother builds on older systems that do not support + libuuid. Loss of functionality should usually not matter too much as + uuid support has only recently been added and is very seldom used. +- bugfix: omfwd did not properly support "template" parameter +- bugfix: potential segfault when re_match() function was used + Thanks to oxpa for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=371 +- bugfix: potential abort of imtcp on rsyslogd shutdown +- bugfix: imzmq3 segfault with PULL subscription + Thanks to Martin Nilsson for the patch. +- bugfix: improper handling of backslash in string-type template()s +- bugfix: leading quote (") in string-type template() lead to tight loop + on startup +- bugfix: no error msg on invalid field option in legacy/string template +- bugfix: potential segfault due to invalid param handling in comparisons + This could happen in RainerScript comparisons (like contains); in some + cases an uninitialized variable was accessed, which could lead to an + invalid free and in turn to a segfault. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=372 + Thanks to Georgi Georgiev for reporting this bug and his great help + in solving it. +- bugfix: no error msg on unreadable $IncludeConfig path +- bugfix: $IncludeConfig did not correctly process directories + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=376 + The testbench was also enhanced to check for these cases. + Thanks to Georgi Georgiev for the bug report. +- bugfix: make rsyslog compile on kfreebsd again + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=380 + Thanks to Guillem Jover for the patch. +- bugfix: garbled message if field name was used with jsonf property option + The length for the field name was invalidly computed, resulting in either + truncated field names or including extra random data. If the random data + contained NULs, the rest of the message became unreadable. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=374 +- bugfix: potential segfault at startup with property-based filter + If the property name was followed by a space before the comma, rsyslogd + aborted on startup. Note that no segfault could happen if the initial + startup went well (this was a problem with the config parser). + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=381 +- bugfix: imfile discarded some file parts + File lines that were incomplete (LF missing) *at the time imfile polled + the file* were partially discarded. That part of the line that was read + without the LF was discarded, and the rest of the line was submitted in + the next polling cycle. This is now changed so that the partial content + is saved until the complete line is read. Note that the patch affects + only read mode 0. + Thanks to Milan Bartos for providing the base idea for the solution. +---------------------------------------------------------------------------- +Version 7.2.1 [v7-stable] 2012-10-29 +- bugfix: ruleset()-object did only support a single statement +- added -D rsyslogd option to enable config parser debug mode +- improved syntax error messages by outputting the error token +- the rsyslog core now suspends actions after 10 failures in a row + This was former the case after 1,000 failures and could cause rsyslog + to be spammed/resources misused. See the v6 compatibility doc for more + details. +- ommongodb rate-limits error messages to prevent spamming the syslog + closes (for v7.2): http://bugzilla.adiscon.com/show_bug.cgi?id=366 +---------------------------------------------------------------------------- +Version 7.2.0 [v7-stable] 2012-10-22 +This starts a new stable branch based on 7.1.12 plus the following changes: +- bugfix: imuxsock did not properly honor $LocalHostIPIF +- omruleset/omdiscard do no longer issue "deprecated" warnings, as 7.1 + grammar does not permit to use the replacements under all circumstances +---------------------------------------------------------------------------- +Version 7.1.12 [beta] 2012-10-18 +- minor updates to better support newer systemd developments + Thanks to Michael Biebl for the patches. +- build system cleanup + Thanks to Michael Biebl for the patch series. +- cleanup: removed remains of -c option (compatibility mode) + both from code & doc and emitted warning message if still used + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=361 + Thanks to Michael Biebl for reporting & suggestions +- bugfix: imklog truncated head of received message + This happened only under some circumstances. Thanks to Marius + Tomaschewski and Florian Piekert for their help in solving this issue. +---------------------------------------------------------------------------- +Version 7.1.11 [beta] 2012-10-16 +- bugfix: imuxsock truncated head of received message + This happened only under some circumstances. Thanks to Marius + Tomaschewski, Florian Piekert and Milan Bartos for their help in + solving this issue. +- bugfix: do not crash if set statement is used with date field + Thanks to Miloslav Trmač for the patch. +- change lumberjack cookie to "@cee:" from "@cee: " + CEE originally specified the cookie with SP, whereas other lumberjack + tools used it without space. In order to keep interop with lumberjack, + we now use the cookie without space as well. I hope this can be changed + in CEE as well when it is released at a later time. + Thanks to Miloslav Trmač for pointing this out and a similar v7 patch. +- added deprecated note to omruleset (plus clue to use "call") +- added deprecated note to discard action (plus clue to use "stop") +--------------------------------------------------------------------------- +Version 7.1.10 [beta] 2012-10-11 + - bugfix: m4 directory was not present in release tarball + - bugfix: small memory leak with string-type templates + - bugfix: small memory leak when template was specified in omfile + - bugfix: some config processing warning messages were treated as errors + - bugfix: small memory leak when processing action() statements + - bugfix: unknown action() parameters were not reported +--------------------------------------------------------------------------- +Version 7.1.9 [beta] 2012-10-09 +- bugfix: comments inside objects (e.g. action()) were not properly handled +- bugfix: in (non)equal comparisons the position of arrays influenced result + This behavior is OK for "contains"-type of comparisons (which have quite + different semantics), but not for == and <>, which shall be commutative. + This has been fixed now, so there is no difference any longer if the + constant string array is the left or right hand operand. We solved this + via the optimizer, as it keeps the actual script execution code small. +--------------------------------------------------------------------------- +Version 7.1.8 [beta] 2012-10-02 +- bugfix: ruleset(){} directive erroneously changed default ruleset + much like the $ruleset legacy conf statement. This potentially lead + to statements being assigned to the wrong ruleset. +- improved module doc +- added "parser" parameter to ruleset(), so that parser chain can be + configured +- implemented "continue" RainerScript statement +--------------------------------------------------------------------------- +Version 7.1.7 [devel] 2012-10-01 +- implemented RainerScript "call" statement +- implemented RainerScript array-based string comparison operations +- implemented imtcp "permittedPeers" module-global parameter +- imudp: support for specifying multiple ports via array added +--------------------------------------------------------------------------- +Version 7.1.6 [devel] 2012-09-28 +- implemented RainerScript input() statement, including support for it + in major input plugins +- implemented RainerScript ruleset() statement +--------------------------------------------------------------------------- +Version 7.1.5 [devel] 2012-09-25 +- implemented RainerScript prifield() function +- implemented RainerScript field() function +- added new module imkmsg to process structured kernel log + Thanks to Milan Bartos for contributing this module +- implemented basic RainerScript optimizer, which will speed up script + operations +- bugfix: invalid free if function re_match() was incorrectly used + if the config file parser detected that param 2 was not constant, some + data fields were not initialized. The destructor did not care about that. + This bug happened only if rsyslog startup was unclean. +--------------------------------------------------------------------------- +Version 7.1.4 [devel] 2012-09-19 +- implemented ability for CEE-based properties to be stored in disk queues +- implemented string concatenation in expressions via &-operator +- implemented json subtree copy in variable assignment +- implemented full JSON support for variable manipulation +- introduced "subtree"-type templates +- bugfix: omfile action did not respect "template" parameter + ... and used default template in all cases +- bugfix: MsgDup() did not copy CEE structure + This function was called at various places, most importantly during + "last messages repeated n times" processing and omruleset. If CEE(JSON) + data was present, it was lost as part of the copy process. +- bugfix: debug output indicated improper queue type +--------------------------------------------------------------------------- +Version 7.1.3 [devel] 2012-09-17 +- introduced "set" and "unset" config statements +- bugfix: missing support for escape sequences in RainerScript + only \' was supported. Now the usual set is supported. Note that v5 + used \x as escape where x was any character (e.g. "\n" meant "n" and NOT + LF). This also means there is some incompatibility to v5 for well-know + sequences. Better break it now than later. +- bugfix: invalid property name in property-filter could cause abort + if action chaining (& operator) was used + http://bugzilla.adiscon.com/show_bug.cgi?id=355 + Thanks to pilou@gmx.com for the bug report +--------------------------------------------------------------------------- +Version 7.1.2 [devel] 2012-09-12 +- bugfix: messages were duplicated, sometimes massively + regression from new code in 7.1.1 and reason for early release +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- bugfix: abort when invalid property name was configured + in property-based filter +- bugfix: multiple rulesets did no longer work correctly (7.1.1 regression) +--------------------------------------------------------------------------- +Version 7.1.1 [devel] 2012-09-11 +- MAJOR NEW FEATURE: ruleengine now fully supports nesting + including if ... then ... else ... constructs. This is a big change + and it obviously has a lot of bug potential. +- BSD-style (filter) blocks are no longer supported + see https://www.rsyslog.com/g/BSD for details and solution +- imuxsock now stores trusted properties by default in the CEE root + This was done in order to keep compatible with other implementations of + the lumberjack schema + Thanks to Miloslav Trmač for pointing to this. +- bugfix: string-generating templates caused abort if CEE field could not + be found +--------------------------------------------------------------------------- +Version 7.1.0 [devel] 2012-09-06 +- added support for hierarchical properties (CEE/lumberjack) +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- imported bugfixes from 6.4.1 +--------------------------------------------------------------------------- +Version 6.6.1 [v6-stable] 2012-10-?? +- bugfix: build problems on some platforms +- bugfix: misaddressing of $mmnormalizeuserawmsg parameter + On many platforms, this has no effect at all. At some, it may cause + a segfault. The problem occurs only during config phase, no segfault + happens when rsyslog has fully started. +- fix API "glitch" in some plugins + This did not affect users, but could have caused trouble in the future + for developers. +- bugfix: no error msg on invalid field option in legacy/string template +- bugfix: no error msg on unreadable $IncludeConfig path +- bugfix: $IncludeConfig did not correctly process directories + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=376 + The testbench was also enhanced to check for these cases. + Thanks to Georgi Georgiev for the bug report. +- bugfix: spurios error messages from imuxsock about (non-error) EAGAIN + Thanks to Marius Tomaschewski for the patch. +- imklog: added $klogParseKernelTimestamp option + When enabled, kernel message [timestamp] is converted for message time. + Default is to use receive time as in 5.8.x and before, because the clock + used to create the timestamp is not supposed to be as accurate as the + monotonic clock (depends on hardware and kernel) resulting in differences + between kernel and system messages which occurred at same time. + Thanks to Marius Tomaschewski for the patch. +- imklog: added $klogKeepKernelTimestamp option + When enabled, the kernel [timestamp] remains at begin of + each message, even it is used for the message time too. + Thanks to Marius Tomaschewski for the patch. +- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds + ... actually, they are microseconds. So the fractional part of the + timestamp was not properly formatted. + Thanks to Marius Tomaschewski for the bug report and the patch idea. +- bugfix: hostname set in rsyslog.conf was not picked up until HUP + which could also mean "never" or "not for a very long time". + Thanks to oxpa for providing analysis and a patch +- bugfix: some message properties could be garbled due to race condition + This happened only on very high volume systems, if the same message was + being processed by two different actions. This was a regression caused + by the new config processor, which did no longer properly enable msg + locking in multithreaded cases. The bugfix is actually a refactoring of + the msg locking code - we no longer do unlocked operations, as the use + case for it has mostly gone away. It is potentially possible only at + very low-end systems, and there the small additional overhead of doing + the locking does not really hurt. Instead, the removal of that + capability can actually slightly improve performance in common cases, + as the code path is smaller and requires slightly less memory writes. + That probably outperforms the extra locking overhead (which in the + low-end case always happens in user space, without need for kernel + support as we can always directly acquire the lock - there is no + contention at all). +- bugfix: invalid DST handling under Solaris + Thanks to Scott Severtson for the patch. +--------------------------------------------------------------------------- +Version 6.6.0 [v6-stable] 2012-10-22 +This starts a new stable branch, based on the 6.5.x series, plus: +- bugfix: imuxsock did not properly honor $LocalHostIPIF +--------------------------------------------------------------------------- +Version 6.5.1 [beta] 2012-10-11 +- added tool "logctl" to handle lumberjack logs in MongoDB +- imfile ported to new v6 config interface +- imfile now supports config parameter for maximum number of submits + which is a fine-tuning parameter in regard to input batching +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- added pure JSON output plugin parameter passing mode +- ommongodb now supports templates +- bugfix: imtcp could abort on exit due to invalid free() +- bugfix: missing support for escape sequences in RainerScript + only \' was supported. Now the usual set is supported. Note that v5 + used \x as escape where x was any character (e.g. "\n" meant "n" and NOT + LF). This also means there is some incompatibility to v5 for well-know + sequences. Better break it now than later. +- bugfix: small memory leaks in template() statements + these were one-time memory leaks during startup, so they did NOT grow + during runtime +- bugfix: config validation run did not always return correct return state +- bugfix: config errors did not always cause statement to fail + This could lead to startup with invalid parameters. +--------------------------------------------------------------------------- +Version 6.5.0 [devel] 2012-08-28 +- imrelp now supports non-cancel thread termination + (but now requires at least librelp 1.0.1) +- implemented freeCnf() module interface + This was actually not present in older versions, even though some modules + already used it. The implementation was now done, and not in 6.3/6.4 + because the resulting memory leak was ultra-slim and the new interface + handling has some potential to seriously break things. Not the kind of + thing you want to add in late beta state, if avoidable. +- added --enable-debugless configure option for very high demanding envs + This actually at compile time disables a lot of debug code, resulting + in some speedup (but serious loss of debugging capabilities) +- added new 0mq plugins (via czmq lib) + Thanks to David Kelly for contributing these modules +- bugfix: omhdfs did no longer compile +- bugfix: SystemLogSocketAnnotate did not work correctly + Thanks to Miloslav Trmač for the patch +- $SystemLogParseTrusted config file option + Thanks to Milan Bartos for the patch +- added template config directive +- added new uuid message property + Thanks to Jérôme Renard for the idea and patches. + Note: patches were released under ASL 2.0, see + http://bugzilla.adiscon.com/show_bug.cgi?id=353 +--------------------------------------------------------------------------- +Version 6.4.3 [V6-STABLE/NEVER RELEASED] 2012-??-?? +This version was never released as 6.6.0 came quickly enough. Note that +all these patches here are present in 6.6.0. +- cleanup: removed remains of -c option (compatibility mode) + both from code & doc and emitted warning message if still used + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=361 + Thanks to Michael Biebl for reporting & suggestions +- bugfix: imuxsock and imklog truncated head of received message + This happened only under some circumstances. Thanks to Marius + Tomaschewski, Florian Piekert and Milan Bartos for their help in + solving this issue. +- change lumberjack cookie to "@cee:" from "@cee: " + CEE originally specified the cookie with SP, whereas other lumberjack + tools used it without space. In order to keep interop with lumberjack, + we now use the cookie without space as well. I hope this can be changed + in CEE as well when it is released at a later time. + Thanks to Miloslav Trmač for pointing this out and a similar v7 patch. +- bugfix: comments inside objects (e.g. action()) were not properly handled +- bugfix: sysklogd-emulating standard template was no longer present in v6 + This was obviously lost during the transition to the new config format. + Thanks to Milan Bartos for alerting us and a patch! +- bugfix: some valid legacy PRI filters were flagged as erroneous + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=358 + This happened to filters of the style "local0,local1.*", where the + multiple facilities were comma-separated. +- bugfix: imuxsock did not properly honor $LocalHostIPIF +--------------------------------------------------------------------------- +Version 6.4.2 [V6-STABLE] 2012-09-20 +- bugfix: potential abort, if action queue could not be properly started + This most importantly could happen due to configuration errors. +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- bugfix: missing support for escape sequences in RainerScript + only \' was supported. Now the usual set is supported. Note that v5 + used \x as escape where x was any character (e.g. "\n" meant "n" and NOT + LF). This also means there is some incompatibility to v5 for well-know + sequences. Better break it now than later. +- bugfix: config validation run did not always return correct return state +--------------------------------------------------------------------------- +Version 6.4.1 [V6-STABLE] 2012-09-06 +- bugfix: multiple main queues with same queue file name were not detected + This lead to queue file corruption. While the root cause is a config + error, it is a bug that this important and hard to find config error + was not detected by rsyslog. +- bugfix: "jsonf" property replacer option did generate invalid JSON + in JSON, we have "fieldname":"value", but the option emitted + "fieldname"="value". Interestingly, this was accepted by a couple + of sinks, most importantly elasticsearch. Now the correct format is + emitted, which causes a remote chance that some things that relied on + the wrong format will break. + Thanks to Miloslav Trmač for the patch +- change $!all-json did emit an empty (thus non-JSON) string if no libee + data was present. It now emits {} and thus valid JSON. There is a + small risk that this may break some things that relied on the previous + inconsistency. + Thanks to Miloslav Trmač for the patch +- bugfix: omusrmsg incorrect return state & config warning handling + During config file processing, Omusrmsg often incorrectly returned a + warning status, even when no warning was present (caused by + uninitialized variable). Also, the core handled warning messages + incorrectly, and treated them as errors. As a result, omusrmsg + (most often) could not properly be loaded. Note that this only + occurs with legacy config action syntax. This was a regression + caused by an incorrect merge in to the 6.3.x codebase. + Thanks to Stefano Mason for alerting us of this bug. +- bugfix: Fixed TCP CheckConnection handling in omfwd.c. Interface needed + to be changed in lower stream classes. Syslog TCP Sending is now resumed + properly. Unfixed, that lead to non-detection of downstate of remote + hosts. +--------------------------------------------------------------------------- +Version 6.4.0 [V6-STABLE] 2012-08-20 +- THIS IS THE FIRST VERSION OF THE 6.4.x STABLE BRANCH + It includes all enhancements made in 6.3.x plus what is written in the + ChangeLog below. +- omelasticsearch: support for parameters parent & dynparent added +- bugfix: imtcp aborted when more than 2 connections were used. + Incremented pthread stack size to 4MB for imtcp, imptcp and imttcp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=342 +- bugfix: imptcp aborted when $InputPTCPServerBindRuleset was used +- bugfix: problem with cutting first 16 characters from message with + bAnnotate + Thanks to Milan Bartos for the patch. +--------------------------------------------------------------------------- +Version 6.3.12 [BETA] 2012-07-02 +- support for elasticsearch via omelasticsearch added + Note that this module has been tested quite well by a number of folks, + and this is why we merge in new functionality in a late beta stage. + Even if problems would exist, only users of omelasticsearch would + experience them, making it a pretty safe addition. +- bugfix: $ActionName was not properly honored + Thanks to Abby Edwards for alerting us +--------------------------------------------------------------------------- +Version 6.3.11 [BETA] 2012-06-18 +- bugfix: expression-based filters with AND/OR could segfault + due to a problem with boolean shortcut operations. From the user's + perspective, the segfault is almost non-deterministic (it occurs when + a shortcut is used). + Thanks to Lars Peterson for providing the initial bug report and his + support in solving it. +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +--------------------------------------------------------------------------- +Version 6.3.10 [BETA] 2012-06-04 +- bugfix: delayable source could block action queue, even if there was + a disk queue associated with it. The root cause of this problem was + that it makes no sense to delay messages once they arrive in the + action queue - the "input" that is being held in that case is the main + queue worker, what makes no sense. + Thanks to Marcin for alerting us on this problem and providing + instructions to reproduce it. +- bugfix: invalid free in imptcp could lead to abort during startup +- bugfix: if debug message could end up in log file when forking + if rsyslog was set to auto-background (thus fork, the default) and debug + mode to stdout was enabled, debug messages ended up in the first log file + opened. Currently, stdout logging is completely disabled in forking mode + (but writing to the debug log file is still possible). This is a change + in behavior, which is under review. If it causes problems to you, + please let us know. + Thanks to Tomas Heinrich for the patch. +- bugfix: --enable-smcustbindcdr configure directive did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330 + Thanks to Ultrabug for the patch. +- bugfix: made rsyslog compile when libestr ist not installed in /usr + Thanks to Miloslav Trmač for providing patches and suggestions +--------------------------------------------------------------------------- +Version 6.3.9 [BETA] 2012-05-22 +- bugfix: imtcp could cause hang during reception + this also applied to other users of core file tcpsrv.c, but imtcp was + by far the most prominent and widely-used, the rest rather exotic + (like imdiag) +- added capability to specify substrings for field extraction mode +- added the "jsonf" property replacer option (and fieldname) +- bugfix: omudpspoof did not work correctly if no spoof hostname was + configured +- bugfix: property replacer option "json" could lead to content loss + message was truncated if escaping was necessary +- bugfix: assigned ruleset was lost when using disk queues + This looked quite hard to diagnose for disk-assisted queues, as the + pure memory part worked well, but ruleset info was lost for messages + stored inside the disk queue. +- bugfix/imuxsock: solving abort if hostname was not set; configured + hostname was not used (both merge regressions) + -bugfix/omfile: template action parameter was not accepted + (and template name set to "??" if the parameter was used) + Thanks to Brian Knox for alerting us on this bug. +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unavailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- added capability to specify substrings for field extraction mode +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timeouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +--------------------------------------------------------------------------- +Version 6.3.8 [DEVEL] 2012-04-16 +- added $PStatJSON directive to permit stats records in JSON format +- added "date-unixtimestamp" property replacer option to format as a + unix timestamp (seconds since epoch) +- added "json" property replacer option to support JSON encoding on a + per-property basis +- added omhiredis (contributed module) +- added mmjsonparse to support recognizing and parsing JSON enhanced syslog + messages +- upgraded more plugins to support the new v6 config format: + - ommysql + - omlibdbi + - omsnmp +- added configuration directives to customize queue light delay marks + $MainMsgQueueLightDelayMark, $ActionQueueLightDelayMark; both + specify number of messages starting at which a delay happens. +- added message property parsesuccess to indicate if the last run + higher-level parser could successfully parse the message or not + (see property replacer html doc for details) +- bugfix: abort during startup when rsyslog.conf v6+ format was used in + a certain way +- bugfix: property $!all-json made rsyslog abort if no normalized data + was available +- bugfix: memory leak in array passing output module mode +- added configuration directives to customize queue light delay marks +- permit size modifiers (k,m,g,...) in integer config parameters + Thanks to Jo Rhett for the suggestion. +- bugfix: hostname was not requeried on HUP + Thanks to Per Jessen for reporting this bug and Marius Tomaschewski for + his help in testing the fix. +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +- added configuration directive to disable octet-counted framing + for imtcp, directive is $InputTCPServerSupportOctetCountedFraming + for imptcp, directive is $InputPTCPServerSupportOctetCountedFraming +- added capability to use a local interface IP address as fromhost-ip for + locally originating messages. New directive $LocalHostIPIF +--------------------------------------------------------------------------- +Version 6.3.7 [DEVEL] 2012-02-02 +- imported refactored v5.9.6 imklog linux driver, now combined with BSD + driver +- removed imtemplate/omtemplate template modules, as this was waste of time + The actual input/output modules are better copy templates. Instead, the + now-removed modules cost time for maintenance AND often caused confusion + on what their role was. +- added a couple of new stats objects +- improved support for new v6 config system. The build-in output modules + now all support the new config language +- bugfix: facility local<x> was not correctly interpreted in legacy filters + Was only accepted if it was the first PRI in a multi-filter PRI. + Thanks to forum user Mark for bringing this to our attention. +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch +- bugfix: legacy parsing of some filters did not work correctly +- bugfix: rsyslog aborted during startup if there is an error in loading + an action and legacy configuration mode is used +- bugfix: bsd klog driver did no longer compile +- relicensed larger parts of the code under Apache (ASL) 2.0 +--------------------------------------------------------------------------- +Version 6.3.6 [DEVEL] 2011-09-19 +- added $InputRELPServerBindRuleset directive to specify rulesets for RELP +- bugfix: config parser did not support properties with dashes in them + inside property-based filters. Thanks to Gerrit Seré for reporting this. +--------------------------------------------------------------------------- +Version 6.3.5 [DEVEL] (rgerhards/al), 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- imudp&imtcp now report error if no listener at all was defined + Thanks to Marcin for suggesting this error message. +- bugfix: potential misaddressing in property replacer +--------------------------------------------------------------------------- +Version 6.3.4 [DEVEL] (rgerhards), 2011-08-02 +- added support for action() config object + * in rsyslog core engine + * in omfile + * in omusrmsg +- bugfix: omusrmsg format usr1,usr2 was no longer supported +- bugfix: misaddressing in config handler + In theory, can cause segfault, in practice this is extremely unlikely + Thanks to Marcin for alerting me. +--------------------------------------------------------------------------- +Version 6.3.3 [DEVEL] (rgerhards), 2011-07-13 +- rsyslog.conf format: now parsed by RainerScript parser + this provides the necessary base for future enhancements as well as some + minor immediate ones. For details see: + http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html +- performance of script-based filters notably increased +- removed compatibility mode as we expect people have adjusted their + confs by now +- added support for the ":omfile:" syntax for actions +--------------------------------------------------------------------------- +Version 6.3.2 [DEVEL] (rgerhards), 2011-07-06 +- added support for the ":omusrmsg:" syntax in configuring user messages +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for obtaining timestamp for kernel message from message + If the kernel time-stamps messages, time is now take from that + timestamp instead of the system time when the message was read. This + provides much better accuracy. Thanks to Lennart Poettering for + suggesting this feature and his help during implementation. +- added support for obtaining timestamp from system for imuxsock + This permits to read the time a message was submitted to the system + log socket. Most importantly, this is provided in microsecond resolution. + So we are able to obtain high precision timestampis even for messages + that were - as is usual - not formatted with them. This also simplifies + things in regard to local time calculation in chroot environments. + Many thanks to Lennart Poettering for suggesting this feature, + providing some guidance on implementing it and coordinating getting the + necessary support into the Linux kernel. +- bugfix: timestamp was incorrectly calculated for timezones with minute + offset + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. +--------------------------------------------------------------------------- +Version 6.3.1 [DEVEL] (rgerhards), 2011-06-07 +- added a first implementation of a DNS name cache + this still has a couple of weaknesses, like no expiration of entries, + suboptimal algorithms -- but it should perform much better than + what we had previously. Implementation will be improved based on + feedback during the next couple of releases +--------------------------------------------------------------------------- +Version 6.3.0 [DEVEL] (rgerhards), 2011-06-01 +- introduced new config system + http://blog.gerhards.net/2011/06/new-rsyslog-config-system-materializes.html +--------------------------------------------------------------------------- +Version 6.2.2 [v6-stable], 2012-06-13 +- build system improvements and spec file templates + Thanks to Abby Edwards for providing these enhancements +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timeouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +- bugfix: --enable-smcustbindcdr configure directive did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330 + Thanks to Ultrabug for the patch. +- add small delay (50ms) after sending shutdown message + There seem to be cases where the shutdown message is otherwise not + processed, not even on an idle system. Thanks to Marcin for + bringing this problem up. +- support for resolving huge groups + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 + Thanks to Alec Warner for the patch +- bugfix: potential hang due to mutex deadlock + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316 + Thanks to Andreas Piesk for reporting&analyzing this bug as well as + providing patches and other help in resolving it. +- bugfix: property PROCID empty instead of proper nilvalue if not present + If it is not present, it must have the nilvalue "-" as of RFC5424 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332 + Thanks to John N for reporting this issue. +- bugfix: did not compile under solaris due to $uptime property code + For the time being, $uptime is not supported on Solaris +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +--------------------------------------------------------------------------- +Version 6.2.1 [v6-stable], 2012-05-10 +- change plugin config interface to be compatible with pre-v6.2 system + The functionality was already removed (because it is superseded by the + v6.3+ config language), but code was still present. I have now removed + those parts that affect interface. Full removal will happen in v6.3, in + order to limit potential regressions. However, it was considered useful + enough to do the interface change in v6-stable; this also eases merging + branches! +- re-licensed larger parts of the codebase under the Apache license 2.0 +- bugfix: omprog made rsyslog abort on startup if not binary to + execute was configured +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +- bugfix: stopped DA queue was never processed after a restart due to a + regression from statistics module +- bugfix: memory leak in array passing output module mode +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unavailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- bugfix/tcpflood: sending small test files did not work correctly +--------------------------------------------------------------------------- +Version 6.2.0 [v6-stable], 2012-01-09 +- bugfix (kind of): removed numerical part from pri-text + see v6 compatibility document for reasons +- bugfix: race condition when extracting program name, APPNAME, structured + data and PROCID (RFC5424 fields) could lead to invalid characters e.g. + in dynamic file names or during forwarding (general malfunction of these + fields in templates, mostly under heavy load) +- bugfix: imuxsock did no longer ignore message-provided timestamp, if + so configured (the *default*). Lead to no longer sub-second timestamps. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=281 +- bugfix: omfile returns fatal error code for things that go really wrong + previously, RS_RET_RESUME was returned, which lead to a loop inside the + rule engine as omfile could not really recover. +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch +- enhanced module loader to not rely on PATH_MAX +- imuxsock: added capability to "annotate" messages with "trusted + information", which contains some properties obtained from the system + and as such sure to not be faked. This is inspired by the similar idea + introduced in systemd. +--------------------------------------------------------------------------- +Version 6.1.12 [BETA], 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- bugfix: potential misaddressing in property replacer +- bugfix: memcpy overflow can occur in allowed sender checking + if a name is resolved to IPv4-mapped-on-IPv6 address + Found by Ismail Dönmez at suse +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +- bugfix: fixed incorrect state handling for Discard Action (transactions) + Note: This caused all messages in a batch to be set to COMMITTED, + even if they were discarded. +--------------------------------------------------------------------------- +Version 6.1.11 [BETA] (rgerhards), 2011-07-11 +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax in configuring user messages +--------------------------------------------------------------------------- +Version 6.1.10 [BETA] (rgerhards), 2011-06-22 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +--------------------------------------------------------------------------- +Version 6.1.9 [BETA] (rgerhards), 2011-06-14 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +- bugfix/improvement:$WorkDirectory now gracefully handles trailing slashes +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. [backport from v6.3] +- bugfix: $ActionFileDefaultTemplate did not work + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=262 +--------------------------------------------------------------------------- +Version 6.1.8 [BETA] (rgerhards), 2011-05-20 +- official new beta version (note that in a sense 6.1.7 was already beta, + so we may release the first stable v6 earlier than usual) +- new module mmsnmptrapd, a sample message modification module +- import of minor bug fixes from v4 & v5 +--------------------------------------------------------------------------- +Version 6.1.7 [DEVEL] (rgerhards), 2011-04-15 +- added log classification capabilities (via mmnormalize & tags) +- speeded up tcp forwarding by reducing number of API calls + this especially speeds up TLS processing +- somewhat improved documentation index +- bugfix: enhanced imudp config processing code disabled due to wrong + merge (affected UDP realtime capabilities) +- bugfix (kind of): memory leak with tcp reception epoll handler + This was an extremely unlikely leak and, if it happened, quite small. + Still it is better to handle this border case. +- bugfix: IPv6-address could not be specified in omrelp + this was due to improper parsing of ":" + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=250 +- bugfix: do not open files with full privileges, if privs will be dropped + This make the privilege drop code more bulletproof, but breaks Ubuntu's + work-around for log files created by external programs with the wrong + user and/or group. Note that it was long said that this "functionality" + would break once we go for serious privilege drop code, so hopefully + nobody still depends on it (and, if so, they lost...). +- bugfix: pipes not opened in full priv mode when privs are to be dropped +--------------------------------------------------------------------------- +Version 6.1.6 [DEVEL] (rgerhards), 2011-03-14 +- enhanced omhdfs to support batching mode. This permits to increase + performance, as we now call the HDFS API with much larger message + sizes and far more infrequently +- improved testbench + among others, life tests for ommysql (against a test database) have + been added, valgrind-based testing enhanced, ... +- bugfix: minor memory leak in omlibdbi (< 1k per instance and run) +- bugfix: (regression) omhdfs did no longer compile +- bugfix: omlibdbi did not use password from rsyslog.con + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=203 +- systemd support somewhat improved (can now take over existing log sockt) +- bugfix: discard action did not work under some circumstances + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=217 +- bugfix: file descriptor leak in gnutls netstream driver + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=222 +- fixed compile problem in imtemplate + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=235 +--------------------------------------------------------------------------- +Version 6.1.5 [DEVEL] (rgerhards), 2011-03-04 +- improved testbench +- enhanced imtcp to use a pool of worker threads to process incoming + messages. This enables higher processing rates, especially in the TLS + case (where more CPU is needed for the crypto functions) +- added support for TLS (in anon mode) to tcpflood +- improved TLS error reporting +- improved TLS startup (Diffie-Hellman bits do not need to be generated, + as we do not support full anon key exchange -- we always need certs) +- bugfix: fixed a memory leak and potential abort condition + this could happen if multiple rulesets were used and some output batches + contained messages belonging to more than one ruleset. + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=226 + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=218 +- bugfix: memory leak when $RepeatedMsgReduction on was used + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225 +- bugfix: potential abort condition when $RepeatedMsgReduction set to on + as well as potentially in a number of other places where MsgDup() was + used. This only happened when the imudp input module was used and it + depended on name resolution not yet had taken place. In other words, + this was a strange problem that could lead to hard to diagnose + instability. So if you experience instability, chances are good that + this fix will help. +--------------------------------------------------------------------------- +Version 6.1.4 [DEVEL] (rgerhards), 2011-02-18 +- bugfix/omhdfs: directive $OMHDFSFileName rendered unusable + due to a search and replace-induced bug ;) +- bugfix: minor race condition in action.c - considered cosmetic + This is considered cosmetic as multiple threads tried to write exactly + the same value into the same memory location without sync. The method + has been changed so this can no longer happen. +- added pmsnare parser module (written by David Lang) +- enhanced imfile to support non-cancel input termination +- improved systemd socket activation thanks to Marius Tomaschewski +- improved error reporting for $WorkDirectory + non-existence and other detectable problems are now reported, + and the work directory is NOT set in this case +- bugfix: pmsnare caused abort under some conditions +- bugfix: abort if imfile reads file line of more than 64KiB + Thanks to Peter Eisentraut for reporting and analyzing this problem. + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 +- bugfix: queue engine did not properly slow down inputs in FULL_DELAY mode + when in disk-assisted mode. This especially affected imfile, which + created unnecessarily queue files if a large set of input file data was + to process. +- bugfix: very long running actions could prevent shutdown under some + circumstances. This has now been solved, at least for common + situations. +- bugfix: fixed compile problem due to empty structs + this occurred only on some platforms/compilers. thanks to Dražen Kačar + for the fix +--------------------------------------------------------------------------- +Version 6.1.3 [DEVEL] (rgerhards), 2011-02-01 +- experimental support for mongodb added +- added $IMUDPSchedulingPolicy and $IMUDPSchedulingPriority config settings +- added $LocalHostName config directive +- improved tcpsrv performance by enabling multiple-entry epoll + so far, we always pulled a single event from the epoll interface. + Now 128, what should result in performance improvement (less API + calls) on busy systems. Most importantly affects imtcp. +- imptcp now supports non-cancel termination mode, a plus in stability +- imptcp speedup: multiple worker threads can now be used to read data +- new directive $InputIMPTcpHelperThreads added +- bugfix: fixed build problems on some platforms + namely those that have 32bit atomic operations but not 64 bit ones +- bugfix: local hostname was pulled too-early, so that some config + directives (namely FQDN settings) did not have any effect +- enhanced tcpflood to support multiple sender threads + this is required for some high-throughput scenarios (and necessary to + run some performance tests, because otherwise the sender is too slow). +- added some new custom parsers (snare, aix, some Cisco "specialities") + thanks to David Lang +--------------------------------------------------------------------------- +Version 6.1.2 [DEVEL] (rgerhards), 2010-12-16 +- added experimental support for log normalization (via liblognorm) + support for normalizing log messages has been added in the form of + mmnormalize. The core engine (property replacer, filter engine) has + been enhanced to support properties from normalized events. + Note: this is EXPERIMENTAL code. It is currently know that + there are issues if the functionality is used with + - disk-based queues + - asynchronous action queues + You can not use the new functionality together with these features. + This limitation will be removed in later releases. However, we + preferred to release early, so that one can experiment with the new + feature set and accepted the price that this means the full set of + functionality is not yet available. If not used together with + these features, log normalizing should be pretty stable. +- enhanced testing tool tcpflood + now supports sending via UDP and the capability to run multiple + iterations and generate statistics data records +- bugfix: potential abort when output modules with different parameter + passing modes were used in configured output modules +--------------------------------------------------------------------------- +Version 6.1.1 [DEVEL] (rgerhards), 2010-11-30 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +- support for omhdfs officially added (import from 5.7.1) +- merged imuxsock improvements from 5.7.1 (see there) +- support for systemd officially added (import from 5.7.0) +- bugfix: a couple of problems that imfile had on some platforms, namely + Ubuntu (not their fault, but occurred there) +- bugfix: imfile utilizes 32 bit to track offset. Most importantly, + this problem can not experienced on Fedora 64 bit OS (which has + 64 bit long's!) +- a number of other bugfixes from older versions imported +--------------------------------------------------------------------------- +Version 6.1.0 [DEVEL] (rgerhards), 2010-08-12 + +*********************************** NOTE ********************************** +The v6 versions of rsyslog feature a greatly redesigned config system +which, among others, supports scoping. However, the initial version does +not contain the whole new system. Rather it will evolve. So it is +expected that interfaces, even new ones, break during the initial +6.x.y releases. +*********************************** NOTE ********************************** + +- added $Begin, $End and $ScriptScoping config scope statements + (at this time for actions only). +- added imptcp, a simplified, Linux-specific and potentially fast + syslog plain tcp input plugin (NOT supporting TLS!) + [ported from v4] +--------------------------------------------------------------------------- +Version 5.10.2 [V5-STABLE], 201?-??-?? +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. +- updated systemd files to match current systemd source +- bugfix: spurios error messages from imuxsock about (non-error) EAGAIN + Thanks to Marius Tomaschewski for the patch. +- imklog: added $klogParseKernelTimestamp option + When enabled, kernel message [timestamp] is converted for message time. + Default is to use receive time as in 5.8.x and before, because the clock + used to create the timestamp is not supposed to be as accurate as the + monotonic clock (depends on hardware and kernel) resulting in differences + between kernel and system messages which occurred at same time. + Thanks to Marius Tomaschewski for the patch. +- imklog: added $klogKeepKernelTimestamp option + When enabled, the kernel [timestamp] remains at begin of + each message, even it is used for the message time too. + Thanks to Marius Tomaschewski for the patch. +- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds + ... actually, they are microseconds. So the fractional part of the + timestamp was not properly formatted. + Thanks to Marius Tomaschewski for the bug report and the patch idea. +- imklog: added $klogKeepKernelTimestamp option + When enabled, the kernel [timestamp] remains at begin of + each message, even it is used for the message time too. + Thanks to Marius Tomaschewski for the patch. +- bugfix: imklog mistakenly took kernel timestamp subseconds as nanoseconds + ... actually, they are microseconds. So the fractional part of the + timestamp was not properly formatted. + Thanks to Marius Tomaschewski for the bug report and the patch idea. +- bugfix: invalid DST handling under Solaris + Thanks to Scott Severtson for the patch. +- bugfix: invalid decrement in pm5424 could lead to log truncation + Thanks to Tomas Heinrich for the patch. +- bugfix[kind of]: omudpspoof discarded messages >1472 bytes (MTU size) + it now truncates these message, but ensures they are sent. Note that + 7.3.5+ will switch to fragmented UDP messages instead (up to 64K) +--------------------------------------------------------------------------- +Version 5.10.1 [V5-STABLE], 2012-10-17 +- bugfix: imuxsock and imklog truncated head of received message + This happened only under some circumstances. Thanks to Marius + Tomaschewski, Florian Piekert and Milan Bartos for their help in + solving this issue. +- enable DNS resolution in imrelp + Thanks to Apollon Oikonomopoulos for the patch +- bugfix: invalid property name in property-filter could cause abort + if action chaining (& operator) was used + http://bugzilla.adiscon.com/show_bug.cgi?id=355 + Thanks to pilou@gmx.com for the bug report +- bugfix: remove invalid socket option call from imuxsock + Thanks to Cristian Ionescu-Idbohrn and Jonny Törnbom +- bugfix: fixed wrong bufferlength for snprintf in tcpflood.c when using + the -f (dynafiles) option. +- fixed issues in build system (namely related to cust1 dummy plugin) +--------------------------------------------------------------------------- +Version 5.10.0 [V5-STABLE], 2012-08-23 + +NOTE: this is the new rsyslog v5-stable, incorporating all changes from the + 5.9.x series. In addition to that, it contains the fixes and + enhancements listed below in this entry. + +- bugfix: delayable source could block action queue, even if there was + a disk queue associated with it. The root cause of this problem was + that it makes no sense to delay messages once they arrive in the + action queue - the "input" that is being held in that case is the main + queue worker, what makes no sense. + Thanks to Marcin for alerting us on this problem and providing + instructions to reproduce it. +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timeouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +- add small delay (50ms) after sending shutdown message + There seem to be cases where the shutdown message is otherwise not + processed, not even on an idle system. Thanks to Marcin for + bringing this problem up. +- support for resolving huge groups + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 + Thanks to Alec Warner for the patch +- bugfix: potential hang due to mutex deadlock + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316 + Thanks to Andreas Piesk for reporting&analyzing this bug as well as + providing patches and other help in resolving it. +- bugfix: property PROCID empty instead of proper nilvalue if not present + If it is not present, it must have the nilvalue "-" as of RFC5424 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332 + Thanks to John N for reporting this issue. +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +- bugfix: multiple main queues with same queue file name was not detected + This lead to queue file corruption. While the root cause is a config + error, it is a bug that this important and hard to find config error + was not detected by rsyslog. +--------------------------------------------------------------------------- +Version 5.9.7 [V5-BETA], 2012-05-10 +- added capability to specify substrings for field extraction mode +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unavailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- bugfix/tcpflood: sending small test files did not work correctly +--------------------------------------------------------------------------- +Version 5.9.6 [V5-BETA], 2012-04-12 +- added configuration directives to customize queue light delay marks +- permit size modifiers (k,m,g,...) in integer config parameters + Thanks to Jo Rhett for the suggestion. +- bugfix: hostname was not requeried on HUP + Thanks to Per Jessen for reporting this bug and Marius Tomaschewski for + his help in testing the fix. +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +- bugfix: imptcp input name could not be set + config directive was accepted, but had no effect +- added configuration directive to disable octet-counted framing + for imtcp, directive is $InputTCPServerSupportOctetCountedFraming + for imptcp, directive is $InputPTCPServerSupportOctetCountedFraming +- added capability to use a local interface IP address as fromhost-ip for + locally originating messages. New directive $LocalHostIPIF +- added configuration directives to customize queue light delay marks + $MainMsgQueueLightDelayMark, $ActionQueueLightDelayMark; both + specify number of messages starting at which a delay happens. +--------------------------------------------------------------------------- +Version 5.9.5 [V5-DEVEL], 2012-01-27 +- improved impstats subsystem, added many new counters +- enhanced module loader to not rely on PATH_MAX +- refactored imklog linux driver, now combined with BSD driver + The Linux driver no longer supports outdated kernel symbol resolution, + which was disabled by default for very long. Also overall cleanup, + resulting in much smaller code. Linux and BSD are now covered by a + single small driver. +- $IMUXSockRateLimitInterval DEFAULT CHANGED, was 5, now 0 + The new default turns off rate limiting. This was chosen as people + experienced problems with rate-limiting activated by default. Now it + needs an explicit opt-in by setting this parameter. + Thanks to Chris Gaffney for suggesting to make it opt-in; thanks to + many unnamed others who already had complained at the time Chris made + the suggestion ;-) +--------------------------------------------------------------------------- +Version 5.9.4 [V5-DEVEL], 2011-11-29 +- imuxsock: added capability to "annotate" messages with "trusted + information", which contains some properties obtained from the system + and as such sure to not be faked. This is inspired by the similar idea + introduced in systemd. +- removed dependency on gcrypt for recently-enough GnuTLS + see: http://bugzilla.adiscon.com/show_bug.cgi?id=289 +- bugfix: imuxsock did no longer ignore message-provided timestamp, if + so configured (the *default*). Lead to no longer sub-second timestamps. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=281 +- bugfix: omfile returns fatal error code for things that go really wrong + previously, RS_RET_RESUME was returned, which lead to a loop inside the + rule engine as omfile could not really recover. +- bugfix: rsyslogd -v always said 64 atomics were not present + thanks to mono_matsuko for the patch +--------------------------------------------------------------------------- +Version 5.9.3 [V5-DEVEL], 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- added capability to emit config error location info for warnings + otherwise, omusrmsg's warning about new config format was not + accompanied by problem location. +- bugfix: potential misaddressing in property replacer +- bugfix: MSGID corruption in RFC5424 parser under some circumstances + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=275 +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +--------------------------------------------------------------------------- +Version 5.9.2 [V5-DEVEL] (rgerhards), 2011-07-11 +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions +--------------------------------------------------------------------------- +Version 5.9.1 [V5-DEVEL] (rgerhards), 2011-06-30 +- added support for obtaining timestamp for kernel message from message + If the kernel time-stamps messages, time is now take from that + timestamp instead of the system time when the message was read. This + provides much better accuracy. Thanks to Lennart Poettering for + suggesting this feature and his help during implementation. +- added support for obtaining timestamp from system for imuxsock + This permits to read the time a message was submitted to the system + log socket. Most importantly, this is provided in microsecond resolution. + So we are able to obtain high precision timestampis even for messages + that were - as is usual - not formatted with them. This also simplifies + things in regard to local time calculation in chroot environments. + Many thanks to Lennart Poettering for suggesting this feature, + providing some guidance on implementing it and coordinating getting the + necessary support into the Linux kernel. +- bugfix: timestamp was incorrectly calculated for timezones with minute + offset + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. [backport from v6] +- bugfix/improvement:$WorkDirectory now gracefully handles trailing slashes +--------------------------------------------------------------------------- +Version 5.9.0 [V5-DEVEL] (rgerhards), 2011-06-08 +- imfile: added $InputFileMaxLinesAtOnce directive +- enhanced imfile to support input batching +- added capability for imtcp and imptcp to activate keep-alive packets + at the socket layer. This has not been added to imttcp, as the latter is + only an experimental module, and one which did not prove to be useful. + reference: http://kb.monitorware.com/post20791.html +- added support to control KEEPALIVE settings in imptcp + this has not yet been added to imtcp, but could be done on request. +- $ActionName is now also used for naming of queues in impstats + as well as in the debug output +- bugfix: do not open files with full privileges, if privs will be dropped + This make the privilege drop code more bulletproof, but breaks Ubuntu's + work-around for log files created by external programs with the wrong + user and/or group. Note that it was long said that this "functionality" + would break once we go for serious privilege drop code, so hopefully + nobody still depends on it (and, if so, they lost...). +- bugfix: pipes not opened in full priv mode when privs are to be dropped +- this begins a new devel branch for v5 +- better handling of queue i/o errors in disk queues. This is kind of a + bugfix, but a very intrusive one, this it goes into the devel version + first. Right now, "file not found" is handled and leads to the new + emergency mode, in which disk action is stopped and the queue run + in direct mode. An error message is emited if this happens. +- added support for user-level PRI provided via systemd +- added new config directive $InputTCPFlowControl to select if tcp + received messages shall be flagged as light delayable or not. +- enhanced omhdfs to support batching mode. This permits to increase + performance, as we now call the HDFS API with much larger message + sizes and far more infrequently +- bugfix: failover did not work correctly if repeated msg reduction was on + affected directive was: $ActionExecOnlyWhenPreviousIsSuspended on + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=236 +--------------------------------------------------------------------------- +Version 5.8.13 [V5-stable] 2012-08-22 +- bugfix: DA queue could cause abort +- bugfix: "last message repeated n times" message was missing hostname + Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting +- bugfix "$PreserveFQDN on" was not honored in some modules + Thanks to bodik for reporting this bug. +- bugfix: randomized IP option header in omudpspoof caused problems + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=327 + Thanks to Rick Brown for helping to test out the patch. +- bugfix: potential abort if output plugin logged message during shutdown + note that none of the rsyslog-provided plugins does this + Thanks to bodik and Rohit Prasad for alerting us on this bug and + analyzing it. + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=347 +- bugfix: multiple main queues with same queue file name was not detected + This lead to queue file corruption. While the root cause is a config + error, it is a bug that this important and hard to find config error + was not detected by rsyslog. +--------------------------------------------------------------------------- +Version 5.8.12 [V5-stable] 2012-06-06 +- add small delay (50ms) after sending shutdown message + There seem to be cases where the shutdown message is otherwise not + processed, not even on an idle system. Thanks to Marcin for + bringing this problem up. +- support for resolving huge groups + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310 + Thanks to Alec Warner for the patch +- bugfix: delayable source could block action queue, even if there was + a disk queue associated with it. The root cause of this problem was + that it makes no sense to delay messages once they arrive in the + action queue - the "input" that is being held in that case is the main + queue worker, what makes no sense. + Thanks to Marcin for alerting us on this problem and providing + instructions to reproduce it. +- bugfix: disk queue was not persisted on shutdown, regression of fix to + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + The new code also handles the case of shutdown of blocking light and + full delayable sources somewhat smarter and permits, assuming sufficient + timeouts, to persist message up to the max queue capacity. Also some nits + in debug instrumentation have been fixed. +- bugfix/omudpspoof: problems, including abort, happened when run on + multiple threads. Root cause is that libnet is not thread-safe. + omudpspoof now guards libnet calls with their own mutex. +- bugfix: if debug message could end up in log file when forking + if rsyslog was set to auto-background (thus fork, the default) and debug + mode to stdout was enabled, debug messages ended up in the first log file + opened. Currently, stdout logging is completely disabled in forking mode + (but writing to the debug log file is still possible). This is a change + in behavior, which is under review. If it causes problems to you, + please let us know. + Thanks to Tomas Heinrich for the patch. +- bugfix/tcpflood: sending small test files did not work correctly +- bugfix: potential hang due to mutex deadlock + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316 + Thanks to Andreas Piesk for reporting&analyzing this bug as well as + providing patches and other help in resolving it. +- bugfix: property PROCID empty instead of proper nilvalue if not present + If it is not present, it must have the nilvalue "-" as of RFC5424 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332 + Thanks to John N for reporting this issue. +--------------------------------------------------------------------------- +Version 5.8.11 [V5-stable] 2012-05-03 +- bugfix: ommysql did not properly init/exit the mysql runtime library + this could lead to segfaults. Triggering condition: multiple action + instances using ommysql. Thanks to Tomas Heinrich for reporting this + problem and providing an initial patch (which my solution is based on, + I need to add more code to clean the mess up). +- bugfix: rsyslog did not terminate when delayable inputs were blocked + due to unavailable sources. Fixes: + http://bugzilla.adiscon.com/show_bug.cgi?id=299 + Thanks to Marcin M for bringing up this problem and Andre Lorbach + for helping to reproduce and fix it. +- bugfix: active input in "light delay state" could block rsyslog + termination, at least for prolonged period of time +- bugfix: imptcp input name could not be set + config directive was accepted, but had no effect +- bugfix: assigned ruleset was lost when using disk queues + This looked quite hard to diagnose for disk-assisted queues, as the + pure memory part worked well, but ruleset info was lost for messages + stored inside the disk queue. +- bugfix: hostname was not requeried on HUP + Thanks to Per Jessen for reporting this bug and Marius Tomaschewski for + his help in testing the fix. +- bugfix: inside queue.c, some thread cancel states were not correctly + reset. While this is a bug, we assume it did have no practical effect + because the reset as it was done was set to the state the code actually + had at this point. But better fix this... +--------------------------------------------------------------------------- +Version 5.8.10 [V5-stable] 2012-04-05 +- bugfix: segfault on startup if $actionqueuefilename was missing for disk + queue config + Thanks to Tomas Heinrich for the patch. +- bugfix: segfault if disk-queue was started up with old queue file + Thanks to Tomas Heinrich for the patch. +- bugfix: memory leak in array passing output module mode +--------------------------------------------------------------------------- +Version 5.8.9 [V5-stable] 2012-03-15 +- added tool to recover disk queue if .qi file is missing (recover_qi.pl) + Thanks to Kaiwang Chen for contributing this tool +- bugfix: stopped DA queue was never processed after a restart due to a + regression from statistics module +- added better doc for statsobj interface + Thanks to Kaiwang Chen for his suggestions and analysis in regard to the + stats subsystem. +--------------------------------------------------------------------------- +Version 5.8.8 [V5-stable] 2012-03-05 +- added capability to use a local interface IP address as fromhost-ip for + imuxsock imklog + new config directives: $IMUXSockLocalIPIF, $klogLocalIPIF +- added configuration directives to customize queue light delay marks + $MainMsgQueueLightDelayMark, $ActionQueueLightDelayMark; both + specify number of messages starting at which a delay happens. +- bugfix: omprog made rsyslog abort on startup if not binary to + execute was configured +- bugfix: imklog invalidly computed facility and severity + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313 +--------------------------------------------------------------------------- +Version 5.8.7 [V5-stable] 2012-01-17 +- bugfix: instabilities when using RFC5424 header fields + Thanks to Kaiwang Chen for the patch +- bugfix: imuxsock did truncate part of received message if it did not + contain a proper date. The truncation occurred because we removed that + part of the messages that was expected to be the date. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=295 +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch +- bugfix: stats counter were not properly initialized on creation +- FQDN hostname for multihomed host was not always set to the correct name + if multiple aliases existed. Thanks to Tomas Heinreich for the patch. +- re-licensed larger parts of the codebase under the Apache license 2.0 +--------------------------------------------------------------------------- +Version 5.8.6 [V5-stable] 2011-10-21 +- bugfix: missing whitespace after property-based filter was not detected +- bugfix: $OMFileFlushInterval period was doubled - now using correct value +- bugfix: ActionQueue could malfunction due to index error + Thanks to Vlad Grigorescu for the patch +- bugfix: $ActionExecOnlyOnce interval did not work properly + Thanks to Tomas Heinrich for the patch +- bugfix: race condition when extracting program name, APPNAME, structured + data and PROCID (RFC5424 fields) could lead to invalid characters e.g. + in dynamic file names or during forwarding (general malfunction of these + fields in templates, mostly under heavy load) +- bugfix: imuxsock did no longer ignore message-provided timestamp, if + so configured (the *default*). Lead to no longer sub-second timestamps. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=281 +- bugfix: omfile returns fatal error code for things that go really wrong + previously, RS_RET_RESUME was returned, which lead to a loop inside the + rule engine as omfile could not really recover. +- bugfix: imfile did invalid system call under some circumstances + when a file that was to be monitored did not exist BUT the state file + actually existed. Mostly a cosmetic issue. Root cause was incomplete + error checking in stream.c; so patch may affect other code areas. +- bugfix: rsyslogd -v always said 64 atomics were not present + thanks to mono_matsuko for the patch +--------------------------------------------------------------------------- +Version 5.8.5 [V5-stable] (rgerhards/al), 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: mark message processing did not work correctly +- bugfix: potential hang condition during tag emulation +- bugfix: too-early string termination during tag emulation +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +- bugfix: fixed incorrect state handling for Discard Action (transactions) + Note: This caused all messages in a batch to be set to COMMITTED, + even if they were discarded. +--------------------------------------------------------------------------- +Version 5.8.4 [V5-stable] (al), 2011-08-10 +- bugfix: potential misaddressing in property replacer +- bugfix: memcpy overflow can occur in allowed sender checking + if a name is resolved to IPv4-mapped-on-IPv6 address + Found by Ismail Dönmez at suse +- bugfix: potential misaddressing in property replacer +- bugfix: MSGID corruption in RFC5424 parser under some circumstances + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=275 +--------------------------------------------------------------------------- +Version 5.8.3 [V5-stable] (rgerhards), 2011-07-11 +- systemd support: set stdout/stderr to null - thx to Lennart for the patch +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions + Note: previous outchannel syntax will generate a warning message. This + may be surprising to some users, but it is quite urgent to alert them + of the new syntax as v6 can no longer support the previous one. +--------------------------------------------------------------------------- +Version 5.8.2 [V5-stable] (rgerhards), 2011-06-21 +- bugfix: problems in failover action handling + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 +- bugfix: mutex was invalidly left unlocked during action processing + At least one case where this can occur is during thread shutdown, which + may be initiated by lower activity. In most cases, this is quite + unlikely to happen. However, if it does, data structures may be + corrupted which could lead to fatal failure and segfault. I detected + this via a testbench test, not a user report. But I assume that some + users may have had unreproducable aborts that were cause by this bug. +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. [backport from v6] +- bugfix/improvement:$WorkDirectory now gracefully handles trailing slashes +--------------------------------------------------------------------------- +Version 5.8.1 [V5-stable] (rgerhards), 2011-05-19 +- bugfix: invalid processing in QUEUE_FULL condition + If the the multi-submit interface was used and a QUEUE_FULL condition + occurred, the failed message was properly destructed. However, the + rest of the input batch, if it existed, was not processed. So this + lead to potential loss of messages and a memory leak. The potential + loss of messages was IMHO minor, because they would have been dropped + in most cases due to the queue remaining full, but very few lucky ones + from the batch may have made it. Anyhow, this has now been changed so + that the rest of the batch is properly tried to be enqueued and, if + not possible, destructed. +- new module mmsnmptrapd, a sample message modification module + This can be useful to reformat snmptrapd messages and also serves as + a sample for how to write message modification modules using the + output module interface. Note that we introduced this new + functionality directly into the stable release, as it does not + modify the core and as such cannot have any side-effects if it is + not used (and thus the risk is solely on users requiring that + functionality). +- bugfix: rate-limiting inside imuxsock did not work 100% correct + reason was that a global config variable was invalidly accessed where a + listener variable should have been used. + Also performance-improved the case when rate limiting is turned off (this + is a very unintrusive change, thus done directly to the stable version). +- bugfix: $myhostname not available in RainerScript (and no error message) + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=233 +- bugfix: memory and file descriptor leak in stream processing + Leaks could occur under some circumstances if the file stream handler + errored out during the open call. Among others, this could cause very + big memory leaks if there were a problem with unreadable disk queue + files. In regard to the memory leak, this + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=256 +- bugfix: doc for impstats had wrong config statements + also, config statements were named a bit inconsistent, resolved that + problem by introducing an alias and only documenting the consistent + statements + Thanks to Marcin for bringing up this problem. +- bugfix: IPv6-address could not be specified in omrelp + this was due to improper parsing of ":" + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=250 +- bugfix: TCP connection invalidly aborted when messages needed to be + discarded (due to QUEUE_FULL or similar problem) +- bugfix: $LocalHostName was not honored under all circumstances + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=258 +- bugfix(minor): improper template function call in syslogd.c +--------------------------------------------------------------------------- +Version 5.8.0 [V5-stable] (rgerhards), 2011-04-12 + +This is the new v5-stable branch, importing all feature from the 5.7.x +versions. To see what has changed in regard to the previous v5-stable, +check the Changelog for 5.7.x below. + +- bugfix: race condition in deferred name resolution + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=238 + Special thanks to Marcin for his persistence in helping to solve this + bug. +- bugfix: DA queue was never shutdown once it was started + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=241 +--------------------------------------------------------------------------- +Version 5.7.10 [V5-BETA] (rgerhards), 2011-03-29 +- bugfix: ompgsql did not work properly with ANSI SQL strings + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=229 +- bugfix: rsyslog did not build with --disable-regexp configure option + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=243 +- bugfix: PRI was invalid on Solaris for message from local log socket +- enhance: added $BOM system property to ease writing byte order masks +- bugfix: RFC5424 parser confused by empty structured data + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=237 +- bugfix: error return from strgen caused abort, now causes action to be + ignored (just like a failed filter) +- new sample plugin for a strgen to generate sql statement consumable + by a database plugin +- bugfix: strgen could not be used together with database outputs + because the sql/stdsql option could not be specified. This has been + solved by permitting the strgen to include the opton inside its name. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=195 +--------------------------------------------------------------------------- +Version 5.7.9 [V5-BETA] (rgerhards), 2011-03-16 +- improved testbench + among others, life tests for ommysql (against a test database) have + been added, valgrind-based testing enhanced, ... +- enhance: fallback *at runtime* to epoll_create if epoll_create1 is not + available. Thanks to Michael Biebl for analysis and patch! +- bugfix: failover did not work correctly if repeated msg reduction was on + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=236 + affected directive was: $ActionExecOnlyWhenPreviousIsSuspended on +- bugfix: minor memory leak in omlibdbi (< 1k per instance and run) +- bugfix: (regression) omhdfs did no longer compile +- bugfix: omlibdbi did not use password from rsyslog.conf + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=203 +--------------------------------------------------------------------------- +Version 5.7.8 [V5-BETA] (rgerhards), 2011-03-09 +- systemd support somewhat improved (can now take over existing log sockt) +- bugfix: discard action did not work under some circumstances + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=217 +- bugfix: file descriptor leak in gnutls netstream driver + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=222 +--------------------------------------------------------------------------- +Version 5.7.7 [V5-BETA] (rgerhards), 2011-03-02 +- bugfix: potential abort condition when $RepeatedMsgReduction set to on + as well as potentially in a number of other places where MsgDup() was + used. This only happened when the imudp input module was used and it + depended on name resolution not yet had taken place. In other words, + this was a strange problem that could lead to hard to diagnose + instability. So if you experience instability, chances are good that + this fix will help. +--------------------------------------------------------------------------- +Version 5.7.6 [V5-BETA] (rgerhards), 2011-02-25 +- bugfix: fixed a memory leak and potential abort condition + this could happen if multiple rulesets were used and some output batches + contained messages belonging to more than one ruleset. + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=226 + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=218 +- bugfix: memory leak when $RepeatedMsgReduction on was used + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225 +--------------------------------------------------------------------------- +Version 5.7.5 [V5-BETA] (rgerhards), 2011-02-23 +- enhance: imfile did not yet support multiple rulesets, now added + we do this directly in the beta because a) it does not affect existing + functionality and b) one may argue that this missing functionality is + close to a bug. +- improved testbench, added tests for imuxsock +- bugfix: imuxsock did no longer sanitize received messages + This was a regression from the imuxsock partial rewrite. Happened + because the message is no longer run through the standard parsers. + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=224 +- bugfix: minor race condition in action.c - considered cosmetic + This is considered cosmetic as multiple threads tried to write exactly + the same value into the same memory location without sync. The method + has been changed so this can no longer happen. +--------------------------------------------------------------------------- +Version 5.7.4 [V5-BETA] (rgerhards), 2011-02-17 +- added pmsnare parser module (written by David Lang) +- enhanced imfile to support non-cancel input termination +- improved systemd socket activation thanks to Marius Tomaschewski +- improved error reporting for $WorkDirectory + non-existence and other detectable problems are now reported, + and the work directory is NOT set in this case +- bugfix: pmsnare caused abort under some conditions +- bugfix: abort if imfile reads file line of more than 64KiB + Thanks to Peter Eisentraut for reporting and analyzing this problem. + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 +- bugfix: queue engine did not properly slow down inputs in FULL_DELAY mode + when in disk-assisted mode. This especially affected imfile, which + created unnecessarily queue files if a large set of input file data was + to process. +- bugfix: very long running actions could prevent shutdown under some + circumstances. This has now been solved, at least for common + situations. +- bugfix: fixed compile problem due to empty structs + this occurred only on some platforms/compilers. thanks to Dražen Kačar + for the fix +--------------------------------------------------------------------------- +Version 5.7.3 [V5-BETA] (rgerhards), 2011-02-07 +- added support for processing multi-line messages in imfile +- added $IMUDPSchedulingPolicy and $IMUDPSchedulingPriority config settings +- added $LocalHostName config directive +- bugfix: fixed build problems on some platforms + namely those that have 32bit atomic operations but not 64 bit ones +- bugfix: local hostname was pulled too-early, so that some config + directives (namely FQDN settings) did not have any effect +- bugfix: imfile did duplicate messages under some circumstances +- added $OMMySQLConfigFile config directive +- added $OMMySQLConfigSection config directive +--------------------------------------------------------------------------- +Version 5.7.2 [V5-DEVEL] (rgerhards), 2010-11-26 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +- bugfix: imfile state file was not written when relative file name + for it was specified +- bugfix: compile failed on systems without epoll_create1() + Thanks to David Hill for providing a fix. +- bugfix: atomic increment for msg object may not work correct on all + platforms. Thanks to Chris Metcalf for the patch +- bugfix: replacements for atomic operations for non-int sized types had + problems. At least one instance of that problem could potentially lead + to abort (inside omfile). +--------------------------------------------------------------------------- +Version 5.7.1 [V5-DEVEL] (rgerhards), 2010-10-05 +- support for Hadoop's HDFS added (via omhdfs) +- imuxsock now optionally use SCM_CREDENTIALS to pull the pid from the log + socket itself + (thanks to Lennart Poettering for the suggesting this feature) +- imuxsock now optionally uses per-process input rate limiting, guarding the + user against processes spamming the system log + (thanks to Lennart Poettering for suggesting this feature) +- added new config statements + * $InputUnixListenSocketUsePIDFromSystem + * $SystemLogUsePIDFromSystem + * $SystemLogRateLimitInterval + * $SystemLogRateLimitBurst + * $SystemLogRateLimitSeverity + * $IMUxSockRateLimitInterval + * $IMUxSockRateLimitBurst + * $IMUxSockRateLimitSeverity +- imuxsock now supports up to 50 different sockets for input +- some code cleanup in imuxsock (consider this a release a major + modification, especially if problems show up) +- bugfix: /dev/log was unlinked even when passed in from systemd + in which case it should be preserved as systemd owns it +--------------------------------------------------------------------------- +Version 5.7.0 [V5-DEVEL] (rgerhards), 2010-09-16 +- added module impstat to emit periodic statistics on rsyslog counters +- support for systemd officially added + * acquire /dev/log socket optionally from systemd + thanks to Lennart Poettering for this patch + * sd-systemd API added as part of rsyslog runtime library +--------------------------------------------------------------------------- +Version 5.6.5 [V5-STABLE] (rgerhards), 2011-03-22 +- bugfix: failover did not work correctly if repeated msg reduction was on + affected directive was: $ActionExecOnlyWhenPreviousIsSuspended on + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=236 +- bugfix: omlibdbi did not use password from rsyslog.con + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=203 +- bugfix(kind of): tell users that config graph can currently not be + generated + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=232 +- bugfix: discard action did not work under some circumstances + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=217 + (backport from 5.7.8) +--------------------------------------------------------------------------- +Version 5.6.4 [V5-STABLE] (rgerhards), 2011-03-03 +- bugfix: potential abort condition when $RepeatedMsgReduction set to on + as well as potentially in a number of other places where MsgDup() was + used. This only happened when the imudp input module was used and it + depended on name resolution not yet had taken place. In other words, + this was a strange problem that could lead to hard to diagnose + instability. So if you experience instability, chances are good that + this fix will help. +- bugfix: fixed a memory leak and potential abort condition + this could happen if multiple rulesets were used and some output batches + contained messages belonging to more than one ruleset. + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=226 + fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=218 +- bugfix: memory leak when $RepeatedMsgReduction on was used + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225 +--------------------------------------------------------------------------- +Version 5.6.3 [V5-STABLE] (rgerhards), 2011-01-26 +- bugfix: action processor released memory too early, resulting in + potential issue in retry cases (but very unlikely due to another + bug, which I also fixed -- only after the fix this problem here + became actually visible). +- bugfix: batch processing flagged invalid message as "bad" under some + circumstances +- bugfix: uninitialized variable could cause issues under extreme conditions + plus some minor nits. This was found after a clang static code analyzer + analysis (great tool, and special thanks to Marcin for telling me about + it!) +- bugfix: batches which had actions in error were not properly retried in + all cases +- bugfix: imfile did duplicate messages under some circumstances +- bugfix: testbench was not activated if no Java was present on system + ... what actually was a left-over. Java is no longer required. +--------------------------------------------------------------------------- +Version 5.6.2 [V5-STABLE] (rgerhards), 2010-11-30 +- bugfix: compile failed on systems without epoll_create1() + Thanks to David Hill for providing a fix. +- bugfix: atomic increment for msg object may not work correct on all + platforms. Thanks to Chris Metcalf for the patch +- bugfix: replacements for atomic operations for non-int sized types had + problems. At least one instance of that problem could potentially lead + to abort (inside omfile). +- added the $InputFilePersistStateInterval config directive to imfile +- changed imfile so that the state file is never deleted (makes imfile + more robust in regard to fatal failures) +- bugfix: a slightly more informative error message when a TCP + connections is aborted +--------------------------------------------------------------------------- +Version 5.6.1 [V5-STABLE] (rgerhards), 2010-11-24 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +- permitted imptcp to work on systems which support epoll(), but not + epoll_create(). + Bug: http://bugzilla.adiscon.com/show_bug.cgi?id=204 + Thanks to Nicholas Brink for reporting this problem. +- bugfix: testbench failed if imptcp was not enabled +- bugfix: segfault when an *empty* template was used + Bug: http://bugzilla.adiscon.com/show_bug.cgi?id=206 + Thanks to David Hill for alerting us. +- bugfix: compile failed with --enable-unlimited-select + thanks varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 5.6.0 [V5-STABLE] (rgerhards), 2010-10-19 + +This release brings all changes and enhancements of the 5.5.x series +to the v5-stable branch. + +- bugfix: a couple of problems that imfile had on some platforms, namely + Ubuntu (not their fault, but occurred there) +- bugfix: imfile utilizes 32 bit to track offset. Most importantly, + this problem can not experienced on Fedora 64 bit OS (which has + 64 bit long's!) +--------------------------------------------------------------------------- +Version 5.5.7 [V5-BETA] (rgerhards), 2010-08-09 +- changed omudpspoof default spoof address to simplify typical use case + thanks to David Lang for suggesting this +- doc bugfix: pmlastmsg doc samples had errors +- bugfix[minor]: pmrfc3164sd had invalid name (resided in rsyslog name + space, what should not be the case for a contributed module) +- added omuxsock, which permits to write message to local Unix sockets + this is the counterpart to imuxsock, enabling fast local forwarding +--------------------------------------------------------------------------- +Version 5.5.6 [DEVEL] (rgerhards), 2010-07-21 +- added parser modules + * pmlastmsg, which supports the notoriously malformed "last message + repeated n times" messages from some syslogd's (namely sysklogd) + * pmrfc3164sd (contributed), supports RFC5424 structured data in + RFC3164 messages [untested] +- added new module type "string generator", used to speed up output + processing. Expected speedup for (typical) rsyslog processing is + roughly 5 to 6 percent compared to using string-based templates. + They may also be used to do more complex formatting with custom + C code, what provided greater flexibility and probably far higher + speed, for example if using multiple regular expressions within a + template. +- added 4 string generators for + * RSYSLOG_FileFormat + * RSYSLOG_TraditionalFileFormat + * RSYSLOG_ForwardFormat + * RSYSLOG_TraditionalForwardFormat +- bugfix: mutexes used to simulate atomic instructions were not destructed +- bugfix: regression caused more locking action in msg.c than necessary +- bugfix: "$ActionExecOnlyWhenPreviousIsSuspended on" was broken +- bugfix: segfault on HUP when "HUPIsRestart" was set to "on" + thanks varmojfekoj for the patch +- bugfix: default for $OMFileFlushOnTXEnd was wrong ("off"). + This, in default mode, caused buffered writing to be used, what + means that it looked like no output were written or partial + lines. Thanks to Michael Biebl for pointing out this bug. +- bugfix: programname filter in ! configuration can not be reset + Thanks to Kiss Gabor for the patch. +--------------------------------------------------------------------------- +Version 5.5.5 [DEVEL] (rgerhards), 2010-05-20 +- added new cancel-reduced action thread termination method + We now manage to cancel threads that block inside a retry loop to + terminate without the need to cancel the thread. Avoiding cancellation + helps keep the system complexity minimal and thus provides for better + stability. This also solves some issues with improper shutdown when + inside an action retry loop. +--------------------------------------------------------------------------- +Version 5.5.4 [DEVEL] (rgerhards), 2010-05-03 +- This version offers full support for Solaris on Intel and Sparc +- bugfix: problems with atomic operations emulation + replaced atomic operation emulation with new code. The previous code + seemed to have some issue and also limited concurrency severely. The + whole atomic operation emulation has been rewritten. +- bugfix: netstream ptcp support class was not correctly build on systems + without epoll() support +- bugfix: segfault on Solaris/Sparc +--------------------------------------------------------------------------- +Version 5.5.3 [DEVEL] (rgerhards), 2010-04-09 +- added basic but functional support for Solaris +- imported many bugfixes from 3.6.2/4.6.1 (see ChangeLog below!) +- added new property replacer option "date-rfc3164-buggyday" primarily + to ease migration from syslog-ng. See property replacer doc for + details. +- added capability to turn off standard LF delimiter in TCP server + via new directive "$InputTCPServerDisableLFDelimiter on" +- bugfix: failed to compile on systems without epoll support +- bugfix: comment char ('#') in literal terminated script parsing + and thus could not be used. + but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119 + [merged in from v3.22.2] +- imported patches from 4.6.0: + * improved testbench to contain samples for totally malformed messages + which miss parts of the message content + * bugfix: some malformed messages could lead to a missing LF inside files + or some other missing parts of the template content. + * bugfix: if a message ended immediately with a hostname, the hostname + was mistakenly interpreted as TAG, and localhost be used as hostname +--------------------------------------------------------------------------- +Version 5.5.2 [DEVEL] (rgerhards), 2010-02-05 +- applied patches that make rsyslog compile under Apple OS X. + Thanks to trey for providing these. +- replaced data type "bool" by "sbool" because this created some + portability issues. +- added $Escape8BitCharactersOnReceive directive + Thanks to David Lang for suggesting it. +- worked around an issue where omfile failed to compile on 32 bit platforms + under some circumstances (this smells like a gcc problem, but a simple + solution was available). Thanks to Kenneth Marshall for some advice. +- extended testbench +--------------------------------------------------------------------------- +Version 5.5.1 [DEVEL] (rgerhards), 2009-11-27 +- introduced the ability for netstream drivers to utilize an epoll interface + This offers increased performance and removes the select() FDSET size + limit from imtcp. Note that we fall back to select() if there is no + epoll netstream drivers. So far, an epoll driver has only been + implemented for plain tcp syslog, the rest will follow once the code + proves well in practice AND there is demand. +- re-implemented $EscapeControlCharacterTab config directive + Based on Jonathan Bond-Caron's patch for v4. This now also includes some + automated tests. +- bugfix: enabling GSSServer crashes rsyslog startup + Thanks to Tomas Kubina for the patch [imgssapi] +- bugfix (kind of): check if TCP connection is still alive if using TLS + Thanks to Jonathan Bond-Caron for the patch. +--------------------------------------------------------------------------- +Version 5.5.0 [DEVEL] (rgerhards), 2009-11-18 +- moved DNS resolution code out of imudp and into the backend processing + Most importantly, DNS resolution now never happens if the resolved name + is not required. Note that this applies to imudp - for the other inputs, + DNS resolution almost comes for free, so we do not do it there. However, + the new method has been implemented in a generic way and as such may + also be used by other modules in the future. +- added option to use unlimited-size select() calls + Thanks to varmojfekoj for the patch + This is not done in imudp, as it natively supports epoll(). +- doc: improved description of what loadable modules can do +--------------------------------------------------------------------------- +Version 5.4.2 [v5-stable] (rgerhards), 2010-03-?? +- bugfix(kind of): output plugin retry behavior could cause engine to loop + The rsyslog engine did not guard itself against output modules that do + not properly convey back the tryResume() behavior. This then leads to + what looks like an endless loop. I consider this to be a bug of the + engine not only because it should be hardened against plugin misbehavior, + but also because plugins may not be totally able to avoid this situation + (depending on the type of and processing done by the plugin). +- bugfix: testbench failed when not executed in UTC+1 timezone + accidentally, the time zone information was kept inside some + to-be-checked-for responses +- temporary bugfix replaced by permanent one for + message-induced off-by-one error (potential segfault) (see 4.6.2) + The analysis has been completed and a better fix been crafted and + integrated. +- bugfix(minor): status variable was uninitialized + However, this would have caused harm only if NO parser modules at + all were loaded, which would lead to a defunctional configuration + at all. And, even more important, this is impossible as two parser + modules are built-in and thus can not be "not loaded", so we always + have a minimum of two. +--------------------------------------------------------------------------- +Version 5.4.1 [v5-stable] (rgerhards), 2010-03-?? +- added new property replacer option "date-rfc3164-buggyday" primarily + to ease migration from syslog-ng. See property replacer doc for + details. [backport from 5.5.3 because urgently needed by some] +- imported all bugfixes vom 4.6.2 (see below) +--------------------------------------------------------------------------- +Version 5.4.0 [v5-stable] (rgerhards), 2010-03-08 +*************************************************************************** +* This is a new stable v5 version. It contains all fixes and enhancements * +* made during the 5.3.x phase as well as those listed below. * +* Note that the 5.2.x series was quite buggy and as such all users are * +* strongly advised to upgrade to 5.4.0. * +*************************************************************************** +- bugfix: omruleset failed to work in many cases + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=179 + Thanks to Ryan B. Lynch for reporting this issue. +- bugfix: comment char ('#') in literal terminated script parsing + and thus could not be used. + but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119 + [merged in from v3.22.2] +--------------------------------------------------------------------------- +Version 5.3.7 [BETA] (rgerhards), 2010-01-27 +- bugfix: queues in direct mode could case a segfault, especially if an + action failed for action queues. The issue was an invalid increment of + a stack-based pointer which lead to destruction of the stack frame and + thus a segfault on function return. + Thanks to Michael Biebl for alerting us on this problem. +- bugfix: hostname accidentally set to IP address for some message sources, + for example imudp. Thanks to Anton for reporting this bug. [imported v4] +- bugfix: ompgsql had problems with transaction support, what actually + rendered it unusable. Thanks to forum user "horhe" for alerting me + on this bug and helping to debug/fix it! [imported from 5.3.6] +- bugfix: $CreateDirs variable not properly initialized, default thus + was random (but most often "on") [imported from v3] +- bugfix: potential segfaults during queue shutdown + (bugs require certain non-standard settings to appear) + Thanks to varmojfekoj for the patch [imported from 4.5.8] + [backport from 5.5.2] +- bugfix: wrong memory assignment for a config variable (probably + without causing any harm) [backport from 5.2.2] +- bugfix: rsyslog hangs when writing to a named pipe which nobody was + reading. Thanks to Michael Biebl for reporting this bug. + Bugzilla entry: http://bugzilla.adiscon.com/show_bug.cgi?id=169 + [imported from 4.5.8] +--------------------------------------------------------------------------- +Version 5.3.6 [BETA] (rgerhards), 2010-01-13 +- bugfix: ompgsql did not properly check the server connection in + tryResume(), which could lead to rsyslog running in a tight loop +- bugfix: suspension during beginTransaction() was not properly handled + by rsyslog core +- bugfix: omfile output was only written when buffer was full, not at + end of transaction +- bugfix: commit transaction was not properly conveyed to message layer, + potentially resulting in non-message destruction and thus hangs +- bugfix: enabling GSSServer crashes rsyslog startup + Thanks to Tomas Kubina for the patch [imgssapi] +- bugfix (kind of): check if TCP connection is still alive if using TLS + Thanks to Jonathan Bond-Caron for the patch. +- bugfix: $CreateDirs variable not properly initialized, default thus + was random (but most often "on") [imported from v3] +- bugfix: ompgsql had problems with transaction support, what actually + rendered it unusable. Thanks to forum user "horhe" for alerting me + on this bug and helping to debug/fix it! +- bugfix: memory leak when sending messages in zip-compressed format + Thanks to Naoya Nakazawa for analyzing this issue and providing a patch. +- worked around an issue where omfile failed to compile on 32 bit platforms + under some circumstances (this smells like a gcc problem, but a simple + solution was available). Thanks to Kenneth Marshall for some advice. + [backported from 5.5.x branch] +--------------------------------------------------------------------------- +Version 5.3.5 [BETA] (rgerhards), 2009-11-13 +- some light performance enhancement by replacing time() call with much + faster (at least under linux) gettimeofday() calls. +- some improvement of omfile performance with dynafiles + saved costly time() calls by employing a logical clock, which is + sufficient for the use case +- bugfix: omudpspoof miscalculated source and destination ports + while this was probably not noticed for source ports, it resulted in + almost all destination ports being wrong, except for the default port + of 514, which by virtue of its binary representation was calculated + correct (and probably thus the bug not earlier detected). +- bugfixes imported from earlier releases + * bugfix: named pipes did no longer work (they always got an open error) + this was a regression from the omfile rewrite in 4.5.0 + * bugfix(testbench): sequence check was not always performed correctly, + that could result in tests reporting success when they actually failed +- improved testbench: added tests for UDP forwarding and omudpspoof +- doc bugfix: omudpspoof had wrong config command names ("om" missing) +- bugfix [imported from 4.4.3]: $ActionExecOnlyOnceEveryInterval did + not work. +- [inport v4] improved testbench, contains now tcp and gzip test cases +- [import v4] added a so-called "On Demand Debug" mode, in which debug + output can be generated only after the process has started, but not right + from the beginning. This is assumed to be useful for hard-to-find bugs. + Also improved the doc on the debug system. +- bugfix: segfault on startup when -q or -Q option was given + [imported from v3-stable] +--------------------------------------------------------------------------- +Version 5.3.4 [DEVEL] (rgerhards), 2009-11-04 +- added the ability to create custom message parsers +- added $RulesetParser config directive that permits to bind specific + parsers to specific rulesets +- added omruleset output module, which provides great flexibility in + action processing. THIS IS A VERY IMPORTANT ADDITION, see its doc + for why. +- added the capability to have ruleset-specific main message queues + This offers considerable additional flexibility AND superior performance + (in cases where multiple inputs now can avoid lock contention) +- bugfix: correct default for escape ('#') character restored + This was accidentally changed to '\\', thanks to David Lang for reporting +- bugfix(testbench): testcase did not properly wait for rsyslogd shutdown + thus some unpredictable behavior and a false negative test result + could occur. +--------------------------------------------------------------------------- +Version 5.3.3 [DEVEL] (rgerhards), 2009-10-27 +- simplified and thus speeded up the queue engine, also fixed some + potential race conditions (in very unusual shutdown conditions) + along the way. The threading model has seriously changes, so there may + be some regressions. +- enhanced test environment (including testbench): support for enhancing + probability of memory addressing failure by using non-NULL default + value for malloced memory (optional, only if requested by configure + option). This helps to track down some otherwise undetected issues + within the testbench. +- bugfix: potential abort if inputname property was not set + primarily a problem of imdiag +- bugfix: message processing states were not set correctly in all cases + however, this had no negative effect, as the message processing state + was not evaluated when a batch was deleted, and that was the only case + where the state could be wrong. +--------------------------------------------------------------------------- +Version 5.3.2 [DEVEL] (rgerhards), 2009-10-21 +- enhanced omfile to support transactional interface. This will increase + performance in many cases. +- added multi-ruleset support to imudp +- re-enabled input thread termination handling that does avoid thread + cancellation where possible. This provides a more reliable mode of + rsyslogd termination (canceling threads my result in not properly + freed resources and potential later hangs, even though we perform + proper cancel handling in our code). This is part of an effort to + reduce thread cancellation as much as possible in rsyslog. + NOTE: the code previously written code for this functionality had a + subtle race condition. The new code solves that. +- enhanced immark to support non-cancel input module termination +- improved imudp so that epoll can be used in more environments, + fixed potential compile time problem if EPOLL_CLOEXEC is not available. +- some cleanup/slight improvement: + * changed imuxsock to no longer use deprecated submitAndParseMsg() IF + * changed submitAndParseMsg() interface to be a wrapper around the new + way of message creation/submission. This enables older plugins to be + used together with the new interface. The removal also enables us to + drop a lot of duplicate code, reducing complexity and increasing + maintainability. +- bugfix: segfault when starting up with an invalid .qi file for a disk queue + Failed for both pure disk as well as DA queues. Now, we emit an error + message and disable disk queueing facility. +- bugfix: potential segfault on messages with empty MSG part. This was a + recently introduced regression. +- bugfix: debug string larger than 1K were improperly displayed. Max size + is now 32K, and if a string is even longer it is meaningfully truncated. +--------------------------------------------------------------------------- +Version 5.3.1 [DEVEL] (rgerhards), 2009-10-05 +- added $AbortOnUncleanConfig directive - permits to prevent startup when + there are problems with the configuration file. See it's doc for + details. +- included some important fixes from v4-stable: + * bugfix: invalid handling of zero-sized messages + * bugfix: zero-sized UDP messages are no longer processed + * bugfix: random data could be appended to message + * bugfix: reverse lookup reduction logic in imudp do DNS queries too often +- bugfixes imported from 4.5.4: + * bugfix: potential segfault in stream writer on destruction + * bugfix: potential race in object loader (obj.c) during use/release + * bugfixes: potential problems in out file zip writer +--------------------------------------------------------------------------- +Version 5.3.0 [DEVEL] (rgerhards), 2009-09-14 +- begun to add simple GUI programs to gain insight into running rsyslogd + instances and help setup and troubleshooting (active via the + --enable-gui ./configure switch) +- changed imudp to utilize epoll(), where available. This shall provide + slightly better performance (just slightly because we called select() + rather infrequently on a busy system) +--------------------------------------------------------------------------- +Version 5.2.2 [v5-stable] (rgerhards), 2009-11-?? +- bugfix: enabling GSSServer crashes rsyslog startup + Thanks to Tomas Kubina for the patch [imgssapi] +--------------------------------------------------------------------------- +Version 5.2.1 [v5-stable] (rgerhards), 2009-11-02 +- bugfix [imported from 4.4.3]: $ActionExecOnlyOnceEveryInterval did + not work. +- bugfix: segfault on startup when -q or -Q option was given + [imported from v3-stable] +--------------------------------------------------------------------------- +Version 5.2.0 [v5-stable] (rgerhards), 2009-11-02 +This is a re-release of version 5.1.6 as stable after we did not get any bug +reports during the whole beta phase. Still, this first v5-stable may not be +as stable as one hopes for, I am not sure if we did not get bug reports +just because nobody tried it. Anyhow, we need to go forward and so we +have the initial v5-stable. +--------------------------------------------------------------------------- +Version 5.1.6 [v5-beta] (rgerhards), 2009-10-15 +- feature imports from v4.5.6 +- bugfix: potential race condition when queue worker threads were + terminated +- bugfix: solved potential (temporary) stall of messages when the queue was + almost empty and few new data added (caused testbench to sometimes hang!) +- fixed some race condition in testbench +- added more elaborate diagnostics to parts of the testbench +- bugfixes imported from 4.5.4: + * bugfix: potential segfault in stream writer on destruction + * bugfix: potential race in object loader (obj.c) during use/release + * bugfixes: potential problems in out file zip writer +- included some important fixes from 4.4.2: + * bugfix: invalid handling of zero-sized messages + * bugfix: zero-sized UDP messages are no longer processed + * bugfix: random data could be appended to message + * bugfix: reverse lookup reduction logic in imudp do DNS queries too often +--------------------------------------------------------------------------- +Version 5.1.5 [v5-beta] (rgerhards), 2009-09-11 +- added new config option $ActionWriteAllMarkMessages + this option permits to process mark messages under all circumstances, + even if an action was recently called. This can be useful to use mark + messages as a kind of heartbeat. +- added new config option $InputUnixListenSocketCreatePath + to permit the auto-creation of paths to additional log sockets. This + turns out to be useful if they reside on temporary file systems and + rsyslogd starts up before the daemons that create these sockets + (rsyslogd always creates the socket itself if it does not exist). +- added $LogRSyslogStatusMessages configuration directive + permitting to turn off rsyslog start/stop/HUP messages. See Debian + ticket http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463793 +- bugfix: hostnames with dashes in them were incorrectly treated as + malformed, thus causing them to be treated as TAG (this was a regression + introduced from the "rfc3164 strict" change in 4.5.0). Testbench has been + updated to include a sample message with a hostname containing a dash. +- bugfix: strings improperly reused, resulting in some message properties + be populated with strings from previous messages. This was caused by + an improper predicate check. +- added new config directive $omfileForceChown [import from 4.7.0] +--------------------------------------------------------------------------- +Version 5.1.4 [DEVEL] (rgerhards), 2009-08-20 +- legacy syslog parser changed so that it now accepts date stamps in + wrong case. Some devices seem to create them and I do not see any harm + in supporting that. +- added $InputTCPMaxListeners directive - permits to specify how many + TCP servers shall be possible (default is 20). +- bugfix: memory leak with some input modules. Those inputs that + use parseAndSubmitMsg() leak two small memory blocks with every message. + Typically, those process only relatively few messages, so the issue + does most probably not have any effect in practice. +- bugfix: if tcp listen port could not be created, no error message was + emitted +- bugfix: discard action did not work (did not discard messages) +- bugfix: discard action caused segfault +- bugfix: potential segfault in output file writer (omfile) + In async write mode, we use modular arithmetic to index the output + buffer array. However, the counter variables accidentally were signed, + thus resulting in negative indices after integer overflow. That in turn + could lead to segfaults, but was depending on the memory layout of + the instance in question (which in turn depended on a number of + variables, like compile settings but also configuration). The counters + are now unsigned (as they always should have been) and so the dangling + mis-indexing does no longer happen. This bug potentially affected all + installations, even if only some may actually have seen a segfault. +--------------------------------------------------------------------------- +Version 5.1.3 [DEVEL] (rgerhards), 2009-07-28 +- architecture change: queue now always has at least one worker thread + if not running in direct mode. Previous versions could run without + any active workers. This simplifies the code at a very small expense. + See v5 compatibility note document for more in-depth discussion. +- enhance: UDP spoofing supported via new output module omudpspoof + See the omudpspoof documentation for details and samples +- bugfix: message could be truncated after TAG, often when forwarding + This was a result of an internal processing error if maximum field + sizes had been specified in the property replacer. +- bugfix: minor static memory leak while reading configuration + did NOT leak based on message volume +- internal: added ability to terminate input modules not via pthread_cancel + but an alternate approach via pthread_kill. This is somewhat safer as we + do not need to think about the cancel-safeness of all libraries we use. + However, not all inputs can easily supported, so this now is a feature + that can be requested by the input module (the most important ones + request it). +--------------------------------------------------------------------------- +Version 5.1.2 [DEVEL] (rgerhards), 2009-07-08 +- bugfix: properties inputname, fromhost, fromhost-ip, msg were lost when + working with disk queues +- some performance enhancements +- bugfix: abort condition when RecvFrom was not set and message reduction + was on. Happened e.g. with imuxsock. +- added $klogConsoleLogLevel directive which permits to set a new + console log level while rsyslog is active +- some internal code cleanup +--------------------------------------------------------------------------- +Version 5.1.1 [DEVEL] (rgerhards), 2009-07-03 +- bugfix: huge memory leak in queue engine (made rsyslogd unusable in + production). Occurred if at least one queue was in direct mode + (the default for action queues) +- imported many performance optimizations from v4-devel (4.5.0) +- bugfix: subtle (and usually irrelevant) issue in timeout processing + timeout could be one second too early if nanoseconds wrapped +- set a more sensible timeout for shutdown, now 1.5 seconds to complete + processing (this also removes those cases where the shutdown message + was not written because the termination happened before it) +--------------------------------------------------------------------------- +Version 5.1.0 [DEVEL] (rgerhards), 2009-05-29 + +*********************************** NOTE ********************************** +The v5 versions of rsyslog feature a greatly redesigned queue engine. The +major theme for the v5 release is twofold: + +a) greatly improved performance +b) enable audit-grade processing + +Here, audit-grade processing means that rsyslog, if used together with +audit-grade transports and configured correctly, will never lose messages +that already have been acknowledged, not even in fatal failure cases like +sudden loss of power. + +Note that large parts of rsyslog's important core components have been +restructured to support these design goals. As such, early versions of +the engine will probably be less stable than the v3/v4 engine. + +Also note that the initial versions do not cover all and everything. As +usual, the code will evolve toward the final goal as version numbers +increase. +*********************************** NOTE ********************************** + +- redesigned queue engine so that it supports ultra-reliable operations + This resulted in a rewrite of large parts. The new capability can be + used to build audit-grade systems on the basis of rsyslog. +- added $MainMsgQueueDequeueBatchSize and $ActionQueueDequeueBatchSize + configuration directives +- implemented a new transactional output module interface which provides + superior performance (for databases potentially far superior performance) +- increased ompgsql performance by adapting to new transactional + output module interface +--------------------------------------------------------------------------- +Version 4.8.1 [v4-stable], 2011-09-?? +- increased max config file line size to 64k + We now also emit an error message if even 64k is not enough (not + doing so previously may rightfully be considered as a bug) +- bugfix: omprog made rsyslog abort on startup if not binary to + execute was configured +- bugfix: $ActionExecOnlyOnce interval did not work properly + Thanks to Tomas Heinrich for the patch +- bugfix: potential abort if ultra-large file io buffers are used and + dynafile cache exhausts address space (primarily a problem on 32 bit + platforms) +- bugfix: potential abort after reading invalid X.509 certificate + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 + Thanks to Tomas Heinrich for the patch. +- bugfix: potential fatal abort in omgssapi + Thanks to Tomas Heinrich for the patch. +- added doc for omprog +- FQDN hostname for multihomed host was not always set to the correct name + if multiple aliases existed. Thanks to Tomas Heinreich for the patch. +- re-licensed larger parts of the codebase under the Apache license 2.0 +--------------------------------------------------------------------------- +Version 4.8.0 [v4-stable] (rgerhards), 2011-09-07 +*************************************************************************** +* This is a new stable v4 version. It contains all fixes and enhancements * +* made during the 4.7.x phase as well as those listed below. * +* Note: major new development to v4 is concluded and will only be done * +* for custom projects. * +*************************************************************************** +There are no changes compared to 4.7.5, just a re-release with the new +version number as new v4-stable. The most important new feature is Solaris +support. +--------------------------------------------------------------------------- +Version 4.7.5 [v4-beta], 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: potential misaddressing in property replacer +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +--------------------------------------------------------------------------- +Version 4.7.4 [v4-beta] (rgerhards), 2011-07-11 +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax in configuring user messages +- added $LocalHostName config directive +- bugfix: PRI was invalid on Solaris for message from local log socket +Version 4.7.3 [v4-devel] (rgerhards), 2010-11-25 +- added omuxsock, which permits to write message to local Unix sockets + this is the counterpart to imuxsock, enabling fast local forwarding +- added imptcp, a simplified, Linux-specific and potentially fast + syslog plain tcp input plugin (NOT supporting TLS!) +- bugfix: a couple of problems that imfile had on some platforms, namely + Ubuntu (not their fault, but occurred there) +- bugfix: imfile utilizes 32 bit to track offset. Most importantly, + this problem can not experienced on Fedora 64 bit OS (which has + 64 bit long's!) +- added the $InputFilePersistStateInterval config directive to imfile +- changed imfile so that the state file is never deleted (makes imfile + more robust in regard to fatal failures) +--------------------------------------------------------------------------- +Version 4.7.2 [v4-devel] (rgerhards), 2010-05-03 +- bugfix: problems with atomic operations emulation + replaced atomic operation emulation with new code. The previous code + seemed to have some issue and also limited concurrency severely. The + whole atomic operation emulation has been rewritten. +- added new $Sleep directive to hold processing for a couple of seconds + during startup +- bugfix: programname filter in ! configuration can not be reset + Thanks to Kiss Gabor for the patch. +--------------------------------------------------------------------------- +Version 4.7.1 [v4-devel] (rgerhards), 2010-04-22 +- Solaris support much improved -- was not truly usable in 4.7.0 + Solaris is no longer supported in imklog, but rather there is a new + plugin imsolaris, which is used to pull local log sources on a Solaris + machine. +- testbench improvement: Java is no longer needed for testing tool creation +--------------------------------------------------------------------------- +Version 4.7.0 [v4-devel] (rgerhards), 2010-04-14 +- new: support for Solaris added (but not yet the Solaris door API) +- added function getenv() to RainerScript +- added new config option $InputUnixListenSocketCreatePath + to permit the auto-creation of paths to additional log sockets. This + turns out to be useful if they reside on temporary file systems and + rsyslogd starts up before the daemons that create these sockets + (rsyslogd always creates the socket itself if it does not exist). +- added $LogRSyslogStatusMessages configuration directive + permitting to turn off rsyslog start/stop/HUP messages. See Debian + ticket http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463793 +- added new config directive $omfileForceChown to (try to) fix some broken + system configs. + See ticket for details: http://bugzilla.adiscon.com/show_bug.cgi?id=150 +- added $EscapeControlCharacterTab config directive + Thanks to Jonathan Bond-Caron for the patch. +- added option to use unlimited-size select() calls + Thanks to varmojfekoj for the patch +- debugondemand mode caused backgrounding to fail - close to a bug, but I'd + consider the ability to background in this mode a new feature... +- bugfix (kind of): check if TCP connection is still alive if using TLS + Thanks to Jonathan Bond-Caron for the patch. +- imported changes from 4.5.7 and below +- bugfix: potential segfault when -p command line option was used + Thanks for varmojfekoj for pointing me at this bug. +- imported changes from 4.5.6 and below +--------------------------------------------------------------------------- +Version 4.6.8 [v4-stable] (rgerhards), 2011-09-01 +- bugfix/security: off-by-two bug in legacy syslog parser, CVE-2011-3200 +- bugfix: potential misaddressing in property replacer +- bugfix: memcpy overflow can occur in allowed sender checking + if a name is resolved to IPv4-mapped-on-IPv6 address + Found by Ismail Dönmez at suse +- bugfix: The NUL-Byte for the syslogtag was not copied in MsgDup (msg.c) +--------------------------------------------------------------------------- +Version 4.6.7 [v4-stable] (rgerhards), 2011-07-11 +- added support for the ":omusrmsg:" syntax in configuring user messages +- added support for the ":omfile:" syntax for actions +--------------------------------------------------------------------------- +Version 4.6.6 [v4-stable] (rgerhards), 2011-06-24 +- bugfix: memory leak in imtcp & subsystems under some circumstances + This leak is tied to error conditions which lead to incorrect cleanup + of some data structures. [backport from v6, limited testing under v4] +- bugfix: invalid processing in QUEUE_FULL condition + If the the multi-submit interface was used and a QUEUE_FULL condition + occurred, the failed message was properly destructed. However, the + rest of the input batch, if it existed, was not processed. So this + lead to potential loss of messages and a memory leak. The potential + loss of messages was IMHO minor, because they would have been dropped + in most cases due to the queue remaining full, but very few lucky ones + from the batch may have made it. Anyhow, this has now been changed so + that the rest of the batch is properly tried to be enqueued and, if + not possible, destructed. +- bugfix: invalid storage type for config variables +- bugfix: stream driver mode was not correctly set on tcp output on big + endian systems. + thanks varmojfekoj for the patch +- bugfix: IPv6-address could not be specified in omrelp + this was due to improper parsing of ":" + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=250 +- bugfix: memory and file descriptor leak in stream processing + Leaks could occur under some circumstances if the file stream handler + errored out during the open call. Among others, this could cause very + big memory leaks if there were a problem with unreadable disk queue + files. In regard to the memory leak, this + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=256 +- bugfix: imfile potentially duplicates lines + This can happen when 0 bytes are read from the input file, and some + writer appends data to the file BEFORE we check if a rollover happens. + The check for rollover uses the inode and size as a criterion. So far, + we checked for equality of sizes, which is not given in this scenario, + but that does not indicate a rollover. From the source code comments: + Note that when we check the size, we MUST NOT check for equality. + The reason is that the file may have been written right after we + did try to read (so the file size has increased). That is NOT in + indicator of a rollover (this is an actual bug scenario we + experienced). So we need to check if the new size is smaller than + what we already have seen! + Also, under some circumstances an invalid truncation was detected. This + code has now been removed, a file change (and thus resent) is only + detected if the inode number changes. +- bugfix: a couple of problems that imfile had on some platforms, namely + Ubuntu (not their fault, but occurred there) +- bugfix: imfile utilizes 32 bit to track offset. Most importantly, + this problem can not experienced on Fedora 64 bit OS (which has + 64 bit long's!) +- bugfix: abort if imfile reads file line of more than 64KiB + Thanks to Peter Eisentraut for reporting and analyzing this problem. + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 +- bugfix: omlibdbi did not use password from rsyslog.con + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=203 +- bugfix: TCP connection invalidly aborted when messages needed to be + discarded (due to QUEUE_FULL or similar problem) +- bugfix: a slightly more informative error message when a TCP + connections is aborted +- bugfix: timestamp was incorrectly calculated for timezones with minute + offset + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 +- some improvements thanks to clang's static code analyzer + o overall cleanup (mostly unnecessary writes and otherwise unused stuff) + o bugfix: fixed a very remote problem in msg.c which could occur when + running under extremely low memory conditions +--------------------------------------------------------------------------- +Version 4.6.5 [v4-stable] (rgerhards), 2010-11-24 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +--------------------------------------------------------------------------- +Version 4.6.4 [v4-stable] (rgerhards), 2010-08-05 +- bugfix: zero-sized (empty) messages were processed by imtcp + they are now dropped as they always should have been +- bugfix: programname filter in ! configuration can not be reset + Thanks to Kiss Gabor for the patch. +--------------------------------------------------------------------------- +Version 4.6.3 [v4-stable] (rgerhards), 2010-07-07 +- improved testbench + - added test with truly random data received via syslog to test + robustness + - added new configure option that permits to disable and enable an + extended testbench +- bugfix: segfault on HUP when "HUPIsRestart" was set to "on" + thanks varmojfekoj for the patch +- bugfix: default for $OMFileFlushOnTXEnd was wrong ("off"). + This, in default mode, caused buffered writing to be used, what + means that it looked like no output were written or partial + lines. Thanks to Michael Biebl for pointing out this bug. +- bugfix: testbench failed when not executed in UTC+1 timezone + accidentally, the time zone information was kept inside some + to-be-checked-for responses +- temporary bugfix replaced by permanent one for + message-induced off-by-one error (potential segfault) (see 4.6.2) + The analysis has been completed and a better fix been crafted and + integrated. +- bugfix: the T/P/E config size specifiers did not work properly under + all 32-bit platforms +- bugfix: local unix system log socket was deleted even when it was + not configured +- some doc fixes; incorrect config samples could cause confusion + thanks to Anthony Edwards for pointing the problems out +--------------------------------------------------------------------------- +Version 4.6.2 [v4-stable] (rgerhards), 2010-03-26 +- new feature: "." action type added to support writing files to relative + paths (this is primarily meant as a debug aid) +- added replacements for atomic instructions on systems that do not + support them. [backport of Stefen Sledz' patch for v5) +- new feature: $OMFileAsyncWriting directive added + it permits to specify if asynchronous writing should be done or not +- bugfix(temporary): message-induced off-by-one error (potential segfault) + Some types of malformed messages could trigger an off-by-one error + (for example, \0 or \n as the last character, and generally control + character escaption is questionable). This is due to not strictly + following a the \0 or string counted string paradigm (during the last + optimization on the cstring class). As a temporary fix, we have + introduced a proper recalculation of the size. However, a final + patch is expected in the future. See bug tracker for further details + and when the final patch will be available: + http://bugzilla.adiscon.com/show_bug.cgi?id=184 + Note that the current patch is considered sufficient to solve the + situation, but it requires a bit more runtime than desirable. +- bugfix: potential segfault in dynafile cache + This bug was triggered by an open failure. The the cache was full and + a new entry needed to be placed inside it, a victim for eviction was + selected. That victim was freed, then the open of the new file tried. If + the open failed, the victim entry was still freed, and the function + exited. However, on next invocation and cache search, the victim entry + was used as if it were populated, most probably resulting in a segfault. +- bugfix: race condition during directory creation + If multiple files try to create a directory at (almost) the same time, + some of them may fail. This is a data race and also exists with other + processes that may create the same directory. We do now check for this + condition and gracefully handle it. +- bugfix: potential re-use of free()ed file stream object in omfile + when dynaCache is enabled, the cache is full, a new entry needs to + be allocated, thus the LRU discarded, then a new entry is opend and that + fails. In that case, it looks like the discarded stream may be reused + improperly (based on code analysis, test case and confirmation pending) +- added new property replacer option "date-rfc3164-buggyday" primarily + to ease migration from syslog-ng. See property replacer doc for + details. [backport from 5.5.3 because urgently needed by some] +- improved testbench +- bugfix: invalid buffer write in (file) stream class + currently being accessed buffer could be overwritten with new data. + While this probably did not cause access violations, it could case loss + and/or duplication of some data (definitely a race with no deterministic + outcome) +- bugfix: potential hang condition during filestream close + predicate was not properly checked when waiting for the background file + writer +- bugfix: improper synchronization when "$OMFileFlushOnTXEnd on" was used + Internal data structures were not properly protected due to missing + mutex calls. +- bugfix: potential data loss during file stream shutdown +- bugfix: potential problems during file stream shutdown + The shutdown/close sequence was not clean, what potentially (but + unlikely) could lead to some issues. We have not been able to describe + any fatal cases, but there was some bug potential. Sequence has now + been straighted out. +- bugfix: potential problem (loop, abort) when file write error occurred + When a write error occurred in stream.c, variable iWritten had the error + code but this was handled as if it were the actual number of bytes + written. That was used in pointer arithmetic later on, and thus could + lead to all sorts of problems. However, this could only happen if the + error was EINTR or the file in question was a tty. All other cases were + handled properly. Now, iWritten is reset to zero in such cases, resulting + in proper retries. +- bugfix: $omfileFlushOnTXEnd was turned on when set to off and vice + versa due to an invalid check +- bugfix: recent patch to fix small memory leak could cause invalid free. + This could only happen during config file parsing. +- bugfix(minor): handling of extremely large strings in dbgprintf() fixed + Previously, it could lead to garbage output and, in extreme cases, also + to segfaults. Note: this was a problem only when debug output was + actually enabled, so it caused no problem in production use. +- bugfix(minor): BSD_SO_COMPAT query function had some global vars not + properly initialized. However, in practice the loader initializes them + with zero, the desired value, so there were no actual issue in almost + all cases. +--------------------------------------------------------------------------- +Version 4.6.1 [v4-stable] (rgerhards), 2010-03-04 +- re-enabled old pipe output (using new module ompipe, built-in) after + some problems with pipes (and especially in regard to xconsole) were + discovered. Thanks to Michael Biebl for reporting the issues. +- bugfix: potential problems with large file support could cause segfault + ... and other weird problems. This seemed to affect 32bit-platforms + only, but I can not totally outrule there were issues on other + platforms as well. The previous code could cause system data types + to be defined inconsistently, and that could lead to various + troubles. Special thanks go to the Mandriva team for identifying + an initial problem, help discussing it and ultimately a fix they + contributed. +- bugfix: fixed problem that caused compilation on FreeBSD 9.0 to fail. + bugtracker: http://bugzilla.adiscon.com/show_bug.cgi?id=181 + Thanks to Christiano for reporting. +- bugfix: potential segfault in omfile when a dynafile open failed + In that case, a partial cache entry was written, and some internal + pointers (iCurrElt) not correctly updated. In the next iteration, that + could lead to a segfault, especially if iCurrElt then points to the + then-partial record. Not very likely, but could happen in practice. +- bugfix (theoretical): potential segfault in omfile under low memory + condition. This is only a theoretical bug, because it would only + happen when strdup() fails to allocate memory - which is highly + unlikely and will probably lead to all other sorts of errors. +- bugfix: comment char ('#') in literal terminated script parsing + and thus could not be used. + but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119 + [merged in from v3.22.2] +--------------------------------------------------------------------------- +Version 4.6.0 [v4-stable] (rgerhards), 2010-02-24 +*************************************************************************** +* This is a new stable v4 version. It contains all fixes and enhancements * +* made during the 4.5.x phase as well as those listed below. * +* Note: this version is scheduled to conclude the v4 development process. * +* Do not expect any more new developments in v4. The focus is now * +* on v5 (what also means we have a single devel branch again). * +* ("development" means new feature development, bug fixes are of * +* course provided for v4-stable) * +*************************************************************************** +- improved testbench to contain samples for totally malformed messages + which miss parts of the message content +- bugfix: some malformed messages could lead to a missing LF inside files + or some other missing parts of the template content. +- bugfix: if a message ended immediately with a hostname, the hostname + was mistakenly interpreted as TAG, and localhost be used as hostname +- bugfix: message without MSG part could case a segfault + [backported from v5 commit 98d1ed504ec001728955a5bcd7916f64cd85f39f] + This actually was a "recent" regression, but I did not realize that it + was introduced by the performance optimization in v4-devel. Shame on + me for having two devel versions at the same time... +--------------------------------------------------------------------------- +Version 4.5.8 [v4-beta] (rgerhards), 2010-02-10 +- enhanced doc for using PostgreSQL + Thanks to Marc Schiffbauer for the new/updated doc +- bugfix: property replacer returned invalid parameters under some (unusual) + conditions. In extreme cases, this could lead to garbled logs and/or + a system failure. +- bugfix: invalid length returned (often) when using regular expressions + inside the property replacer +- bugfix: submatch regex in property replacer did not honor "return 0 on + no match" config case +- bugfix: imuxsock incorrectly stated inputname "imudp" + Thanks to Ryan Lynch for reporting this. +- (slightly) enhanced support for FreeBSD by setting _PATH_MODDIR to + the correct value on FreeBSD. + Thanks to Cristiano for the patch. +- bugfix: -d did not enable display of debug messages + regression from introduction of "debug on demand" mode + Thanks to Michael Biebl for reporting this bug +- bugfix: blanks inside file names did not terminate file name parsing. + This could result in the whole rest of a line (including comments) + to be treated as file name in "write to file" actions. + Thanks to Jack for reporting this issue. +- bugfix: rsyslog hang when writing to a named pipe which nobody was + reading. Thanks to Michael Biebl for reporting this bug. + Bugzilla entry: http://bugzilla.adiscon.com/show_bug.cgi?id=169 +- bugfix: potential segfaults during queue shutdown + (bugs require certain non-standard settings to appear) + Thanks to varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 4.5.7 [v4-beta] (rgerhards), 2009-11-18 +- added a so-called "On Demand Debug" mode, in which debug output can + be generated only after the process has started, but not right from + the beginning. This is assumed to be useful for hard-to-find bugs. + Also improved the doc on the debug system. +- bugfix (kind of): check if TCP connection is still alive if using TLS + Thanks to Jonathan Bond-Caron for the patch. +- bugfix: hostname accidentally set to IP address for some message sources, + for example imudp. Thanks to Anton for reporting this bug. +- bugfix [imported from 4.4.3]: $ActionExecOnlyOnceEveryInterval did + not work. +--------------------------------------------------------------------------- +Version 4.5.6 [v4-beta] (rgerhards), 2009-11-05 +- bugfix: named pipes did no longer work (they always got an open error) + this was a regression from the omfile rewrite in 4.5.0 +- bugfix(minor): diag function returned wrong queue member count + for the main queue if an active DA queue existed. This had no relevance + to real deployments (assuming they are not running the debug/diagnostic + module...), but sometimes caused grief and false alerts in the + testbench. +- included some important fixes from v4-stable: + * bugfix: invalid handling of zero-sized messages + * bugfix: zero-sized UDP messages are no longer processed + * bugfix: random data could be appended to message + * bugfix: reverse lookup reduction logic in imudp do DNS queries too often +- bugfix(testbench): testcase did not properly wait for rsyslog shutdown + thus some unpredictable behavior and a false negative test result + could occur. [BACKPORTED from v5] +- bugfix(testbench): sequence check was not always performed correctly, + that could result in tests reporting success when they actually failed +--------------------------------------------------------------------------- +Version 4.5.5 [v4-beta] (rgerhards), 2009-10-21 +- added $InputTCPServerNotifyOnConnectionClose config directive + see doc for details +- bugfix: debug string larger than 1K were improperly displayed. Max size + is now 32K +- bugfix: invalid storage class selected for some size config parameters. + This resulted in wrong values. The most prominent victim was the + directory creation mode, which was set to zero in some cases. For + details, see related blog post: + http://blog.gerhards.net/2009/10/another-note-on-hard-to-find-bugs.html +--------------------------------------------------------------------------- +Version 4.5.4 [v4-beta] (rgerhards), 2009-09-29 +- bugfix: potential segfault in stream writer on destruction + Most severely affected omfile. The problem was that some buffers were + freed before the asynchronous writer thread was shut down. So the + writer thread accessed invalid data, which may even already be + overwritten. Symptoms (with omfile) were segfaults, garbled data + and files with random names placed around the file system (most + prominently into the root directory). Special thanks to Aaron for + helping to track this down. +- bugfix: potential race in object loader (obj.c) during use/release + of object interface +- bugfixes: potential problems in out file zip writer. Problems could + lead to abort and/or memory leak. The module is now hardened in a very + conservative way, which is sub-optimal from a performance point of view. + This should be improved if it has proven reliable in practice. +--------------------------------------------------------------------------- +Version 4.5.3 [v4-beta] (rgerhards), 2009-09-17 +- bugfix: repeated messages were incorrectly processed + this could lead to loss of the repeated message content. As a side- + effect, it could probably also be possible that some segfault occurs + (quite unlikely). The root cause was that some counters introduced + during the malloc optimizations were not properly duplicated in + MsgDup(). Note that repeated message processing is not enabled + by default. +- bugfix: message sanitation had some issues: + - control character DEL was not properly escaped + - NUL and LF characters were not properly stripped if no control + character replacement was to be done + - NUL characters in the message body were silently dropped (this was + a regression introduced by some of the recent optimizations) +- bugfix: strings improperly reused, resulting in some message properties + be populated with strings from previous messages. This was caused by + an improper predicate check. [backported from v5] +- fixed some minor portability issues +- bugfix: reverse lookup reduction logic in imudp do DNS queries too often + [imported from 4.4.2] +--------------------------------------------------------------------------- +Version 4.5.2 [v4-beta] (rgerhards), 2009-08-21 +- legacy syslog parser changed so that it now accepts date stamps in + wrong case. Some devices seem to create them and I do not see any harm + in supporting that. +- added $InputTCPMaxListeners directive - permits to specify how many + TCP servers shall be possible (default is 20). +- bugfix: memory leak with some input modules. Those inputs that + use parseAndSubmitMsg() leak two small memory blocks with every message. + Typically, those process only relatively few messages, so the issue + does most probably not have any effect in practice. +- bugfix: if tcp listen port could not be created, no error message was + emitted +- bugfix: potential segfault in output file writer (omfile) + In async write mode, we use modular arithmetic to index the output + buffer array. However, the counter variables accidentally were signed, + thus resulting in negative indices after integer overflow. That in turn + could lead to segfaults, but was depending on the memory layout of + the instance in question (which in turn depended on a number of + variables, like compile settings but also configuration). The counters + are now unsigned (as they always should have been) and so the dangling + mis-indexing does no longer happen. This bug potentially affected all + installations, even if only some may actually have seen a segfault. +- bugfix: hostnames with dashes in them were incorrectly treated as + malformed, thus causing them to be treated as TAG (this was a regression + introduced from the "rfc3164 strict" change in 4.5.0). +--------------------------------------------------------------------------- +Version 4.5.1 [DEVEL] (rgerhards), 2009-07-15 +- CONFIG CHANGE: $HUPisRestart default is now "off". We are doing this + to support removal of restart-type HUP in v5. +- bugfix: fromhost-ip was sometimes truncated +- bugfix: potential segfault when zip-compressed syslog records were + received (double free) +- bugfix: properties inputname, fromhost, fromhost-ip, msg were lost when + working with disk queues +- performance enhancement: much faster, up to twice as fast (depending + on configuration) +- bugfix: abort condition when RecvFrom was not set and message reduction + was on. Happened e.g. with imuxsock. +- added $klogConsoleLogLevel directive which permits to set a new + console log level while rsyslog is active +- bugfix: message could be truncated after TAG, often when forwarding + This was a result of an internal processing error if maximum field + sizes had been specified in the property replacer. +- added ability for the TCP output action to "rebind" its send socket after + sending n messages (actually, it re-opens the connection, the name is + used because this is a concept very similar to $ActionUDPRebindInterval). + New config directive $ActionSendTCPRebindInterval added for the purpose. + By default, rebinding is disabled. This is considered useful for load + balancers. +- testbench improvements +--------------------------------------------------------------------------- +Version 4.5.0 [DEVEL] (rgerhards), 2009-07-02 +- activation order of inputs changed, they are now activated only after + privileges are dropped. Thanks to Michael Terry for the patch. +- greatly improved performance +- greatly reduced memory requirements of msg object + to around half of the previous demand. This means that more messages can + be stored in core! Due to fewer cache misses, this also means some + performance improvement. +- improved config error messages: now contain a copy of the config line + that (most likely) caused the error +- reduced max value for $DynaFileCacheSize to 1,000 (the former maximum + of 10,000 really made no sense, even 1,000 is very high, but we like + to keep the user in control ;)). +- added capability to fsync() queue disk files for enhanced reliability + (also add's speed, because you do no longer need to run the whole file + system in sync mode) +- more strict parsing of the hostname in rfc3164 mode, hopefully + removes false positives (but may cause some trouble with hostname + parsing). For details, see this bug tracker: + http://bugzilla.adiscon.com/show_bug.cgi?id=126 +- omfile rewrite to natively support zip files (includes large extension + of the stream class) +- added configuration commands (see doc for explanations) + * $OMFileZipLevel + * $OMFileIOBufferSize + * $OMFileFlushOnTXEnd + * $MainMsgQueueSyncQueueFiles + * $ActionQueueSyncQueueFiles +- done some memory accesses explicitly atomic +- bugfix: subtle (and usually irrelevant) issue in timeout processing + timeout could be one second too early if nanoseconds wrapped +- set a more sensible timeout for shutdown, now 1.5 seconds to complete + processing (this also removes those cases where the shutdown message + was not written because the termination happened before it) +- internal bugfix: object pointer was only reset to NULL when an object + was actually destructed. This most likely had no effect to existing code, + but it may also have caused trouble in remote cases. Similarly, the fix + may also cause trouble... +- bugfix: missing initialization during timestamp creation + This could lead to timestamps written in the wrong format, but not to + an abort +--------------------------------------------------------------------------- +Version 4.4.3 [v4-stable] (rgerhards), 2009-10-?? +- bugfix: several smaller bugs resolved after flexelint review + Thanks to varmojfekoj for the patch. +- bugfix: $ActionExecOnlyOnceEveryInterval did not work. + This was a regression from the time() optimizations done in v4. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=143 + Thanks to Klaus Tachtler for reporting this bug. +- bugfix: potential segfault on queue shutdown + Thanks to varmojfekoj for the patch. +- bugfix: potential hang condition on queue shutdown + [imported from v3-stable] +- bugfix: segfault on startup when -q or -Q option was given + [imported from v3-stable] +--------------------------------------------------------------------------- +Version 4.4.2 [v4-stable] (rgerhards), 2009-10-09 +- bugfix: invalid handling of zero-sized messages, could lead to mis- + addressing and potential memory corruption/segfault +- bugfix: zero-sized UDP messages are no longer processed + until now, they were forwarded to processing, but this makes no sense + Also, it looks like the system seems to provide a zero return code + on a UDP recvfrom() from time to time for some internal reasons. These + "receives" are now silently ignored. +- bugfix: random data could be appended to message, possibly causing + segfaults +- bugfix: reverse lookup reduction logic in imudp do DNS queries too often + A comparison was done between the current and the former source address. + However, this was done on the full sockaddr_storage structure and not + on the host address only. This has now been changed for IPv4 and IPv6. + The end result of this bug could be a higher UDP message loss rate than + necessary (note that UDP message loss can not totally be avoided due + to the UDP spec) +--------------------------------------------------------------------------- +Version 4.4.1 [v4-stable] (rgerhards), 2009-09-02 +- features requiring Java are automatically disabled if Java is not + present (thanks to Michael Biebl for his help!) +- bugfix: invalid double-quoted PRI, among others in outgoing messages + This causes grief with all receivers. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=147 +- bugfix: Java testing tools were required, even if testbench was disabled + This resulted in build errors if no Java was present on the build system, + even though none of the selected option actually required Java. + (I forgot to backport a similar fix to newer releases). +- bugfix (backport): omfwd segfault + Note that the original (higher version) patch states this happens only + when debugging mode is turned on. That statement is wrong: if debug + mode is turned off, the message is not being emitted, but the division + by zero in the actual parameters still happens. +--------------------------------------------------------------------------- +Version 4.4.0 [v4-stable] (rgerhards), 2009-08-21 +- bugfix: stderr/stdout were not closed to be able to emit error messages, + but this caused ssh sessions to hang. Now we close them after the + initial initialization. See forum thread: + http://kb.monitorware.com/controlling-terminal-issues-t9875.html +- bugfix: sending syslog messages with zip compression did not work +--------------------------------------------------------------------------- +Version 4.3.2 [v4-beta] (rgerhards), 2009-06-24 +- removed long-obsoleted property UxTradMsg +- added a generic network stream server (in addition to rather specific + syslog tcp server) +- added ability for the UDP output action to rebind its send socket after + sending n messages. New config directive $ActionSendUDPRebindInterval + added for the purpose. By default, rebinding is disabled. This is + considered useful for load balancers. +- bugfix: imdiag/imtcp had a race condition +- improved testbench (now much better code design and reuse) +- added config switch --enable-testbench=no to turn off testbench +--------------------------------------------------------------------------- +Version 4.3.1 [DEVEL] (rgerhards), 2009-05-25 +- added capability to run multiple tcp listeners (on different ports) +- performance enhancement: imtcp calls parser no longer on input thread + but rather inside on of the potentially many main msg queue worker + threads (an enhancement scheduled for all input plugins where this is + possible) +- added $GenerateConfigGraph configuration command which can be used + to generate nice-looking (and very informative) rsyslog configuration + graphs. +- added $ActionName configuration directive (currently only used for + graph generation, but may find other uses) +- improved doc + * added (hopefully) easier to grasp queue explanation +- improved testbench + * added tests for queue disk-only mode (checks disk queue logic) +- bugfix: light and full delay watermarks had invalid values, badly + affecting performance for delayable inputs +- build system improvements - thanks to Michael Biebl +- added new testing module imdiag, which enables to talk to the + rsyslog core at runtime. The current implementation is only a + beginning, but can be expanded over time +--------------------------------------------------------------------------- +Version 4.3.0 [DEVEL] (rgerhards), 2009-04-17 +- new feature: new output plugin omprog, which permits to start program + and feed it (via its stdin) with syslog messages. If the program + terminates, it is restarted. +- improved internal handling of RainerScript functions, building the + necessary plumbing to support more functions with decent runtime + performance. This is also necessary towards the long-term goal + of loadable library modules. +- added new RainerScript function "tolower" +- improved testbench + * added tests for tcp-based reception + * added tcp-load test (1000 connections, 20,000 messages) +- added $MaxOpenFiles configuration directive +- bugfix: solved potential memory leak in msg processing, could manifest + itself in imtcp +- bugfix: ompgsql did not detect problems in sql command execution + this could cause loss of messages. The handling was correct if the + connection broke, but not if there was a problem with statement + execution. The most probable case for such a case would be invalid + sql inside the template, and this is now much easier to diagnose. +--------------------------------------------------------------------------- +Version 4.2.0 [v4-stable] (rgerhards), 2009-06-23 +- bugfix: light and full delay watermarks had invalid values, badly + affecting performance for delayable inputs +- imported all patches from 3.22.1 as of today (see below) +- bugfix: compile problems in im3195 +--------------------------------------------------------------------------- +Version 4.1.7 [BETA] (rgerhards), 2009-04-22 +- bugfix: $InputTCPMaxSessions config directive was accepted, but not + honored. This resulted in a fixed upper limit of 200 connections. +- bugfix: the default for $DirCreateMode was 0644, and as such wrong. + It has now been changed to 0700. For some background, please see + http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html +- bugfix: ompgsql did not detect problems in sql command execution + this could cause loss of messages. The handling was correct if the + connection broke, but not if there was a problem with statement + execution. The most probable case for such a case would be invalid + sql inside the template, and this is now much easier to diagnose. +--------------------------------------------------------------------------- +Version 4.1.6 [DEVEL] (rgerhards), 2009-04-07 +- added new "csv" property replacer options to enable simple creation + of CSV-formatted outputs (format from RFC4180 is used) +- implemented function support in RainerScript. That means the engine + parses and compile functions, as well as executes a few build-in + ones. Dynamic loading and registration of functions is not yet + supported - but we now have a good foundation to do that later on. +- implemented the strlen() RainerScript function +- added a template output module +- added -T rsyslogd command line option, enables to specify a directory + where to chroot() into on startup. This is NOT a security feature but + introduced to support testing. Thus, -T does not make sure chroot() + is used in a secure way. (may be removed later) +- added omstdout module for testing purposes. Spits out all messages to + stdout - no config option, no other features +- added a parser testing suite (still needs to be extended, but a good + start) +- modified $ModLoad statement so that for modules whom's name starts with + a dot, no path is prepended (this enables relative-paths and should + not break any valid current config) +- fixed a bug that caused action retries not to work correctly + situation was only cleared by a restart +- bugfix: closed dynafile was potentially never written until another + dynafile name was generated - potential loss of messages +- improved omfile so that it properly suspends itself if there is an + i/o or file name generation error. This enables it to be used with + the full high availability features of rsyslog's engine +- bugfix: fixed some segfaults on Solaris, where vsprintf() does not + check for NULL pointers +- improved performance of regexp-based filters + Thanks to Arnaud Cornet for providing the idea and initial patch. +- added a new way how output plugins may be passed parameters. This is + more efficient for some outputs. They new can receive fields not only + as a single string but rather in an array where each string is separated. +- added (some) developer documentation for output plugin interface +- bugfix: potential abort with DA queue after high watermark is reached + There exists a race condition that can lead to a segfault. Thanks + go to vbernetr, who performed the analysis and provided patch, which + I only tweaked a very little bit. +- bugfix: imtcp did incorrectly parse hostname/tag + Thanks to Luis Fernando Muñoz Mejías for the patch. +--------------------------------------------------------------------------- +Version 4.1.5 [DEVEL] (rgerhards), 2009-03-11 +- bugfix: parser did not correctly parse fields in UDP-received messages +- added ERE support in filter conditions + new comparison operation "ereregex" +- added new config directive $RepeatedMsgContainsOriginalMsg so that the + "last message repeated n times" messages, if generated, may + have an alternate format that contains the message that is being repeated +--------------------------------------------------------------------------- +Version 4.1.4 [DEVEL] (rgerhards), 2009-01-29 +- bugfix: inconsistent use of mutex/atomic operations could cause segfault + details are too many, for full analysis see blog post at: + http://blog.gerhards.net/2009/01/rsyslog-data-race-analysis.html +- bugfix: uninitialized mutex was used in msg.c:getPRI + This was subtle, because getPRI is called as part of the debugging code + (always executed) in syslogd.c:logmsg. +- bugfix: $PreserveFQDN was not properly handled for locally emitted + messages +--------------------------------------------------------------------------- +Version 4.1.3 [DEVEL] (rgerhards), 2008-12-17 +- added $InputTCPServerAddtlFrameDelimiter config directive, which + enables to specify an additional, non-standard message delimiter + for processing plain tcp syslog. This is primarily a fix for the invalid + framing used in Juniper's NetScreen products. Credit to forum user + Arv for suggesting this solution. +- added $InputTCPServerInputName property, which enables a name to be + specified that will be available during message processing in the + inputname property. This is considered useful for logic that treats + messages differently depending on which input received them. +- added $PreserveFQDN config file directive + Enables to use FQDNs in sender names where the legacy default + would have stripped the domain part. + Thanks to BlinkMind, Inc. http://www.blinkmind.com for sponsoring this + development. +- bugfix: imudp went into an endless loop under some circumstances + (but could also leave it under some other circumstances...) + Thanks to David Lang and speedfox for reporting this issue. +--------------------------------------------------------------------------- +Version 4.1.2 [DEVEL] (rgerhards), 2008-12-04 +- bugfix: code did not compile without zlib +- security bugfix: $AllowedSender was not honored, all senders were + permitted instead (see https://www.rsyslog.com/Article322.phtml) +- security fix: imudp emitted a message when a non-permitted sender + tried to send a message to it. This behavior is operator-configurable. + If enabled, a message was emitted each time. That way an attacker could + effectively fill the disk via this facility. The message is now + emitted only once in a minute (this currently is a hard-coded limit, + if someone comes up with a good reason to make it configurable, we + will probably do that). +- doc bugfix: typo in v3 compatibility document directive syntax + thanks to Andrej for reporting +- imported other changes from 3.21.8 and 3.20.1 (see there) +--------------------------------------------------------------------------- +Version 4.1.1 [DEVEL] (rgerhards), 2008-11-26 +- added $PrivDropToGroup, $PrivDropToUser, $PrivDropToGroupID, + $PrivDropToUserID config directives to enable dropping privileges. + This is an effort to provide a security enhancement. For the limits of this + approach, see http://wiki.rsyslog.com/index.php/Security +- re-enabled imklog to compile on FreeBSD (brought in from beta) +--------------------------------------------------------------------------- +Version 4.1.0 [DEVEL] (rgerhards), 2008-11-18 + +********************************* WARNING ********************************* +This version has a slightly different on-disk format for message entries. +As a consequence, old queue files being read by this version may have +an invalid output timestamp, which could result to some malfunction inside +the output driver. It is recommended to drain queues with the previous +version before switching to this one. +********************************* WARNING ********************************* + +- greatly enhanced performance when compared to v3. +- added configuration directive "HUPisRestart" which enables to configure + HUP to be either a full restart or "just" a lightweight way to + close open files. +- enhanced legacy syslog parser to detect year if part of the timestamp + the format is based on what Cisco devices seem to emit. +- added a setting "$OptimizeForUniprocessor" to enable users to turn off + pthread_yield calls which are counter-productive on multiprocessor + machines (but have been shown to be useful on uniprocessors) +- reordered imudp processing. Message parsing is now done as part of main + message queue worker processing (was part of the input thread) + This should also improve performance, as potentially more work is + done in parallel. +- bugfix: compressed syslog messages could be slightly mis-uncompressed + if the last byte of the compressed record was a NUL +- added $UDPServerTimeRequery option which enables to work with + less accurate timestamps in favor of performance. This enables querying + of the time only every n-th time if imudp is running in the tight + receive loop (aka receiving messages at a high rate) +- doc bugfix: queue doc had wrong parameter name for setting controlling + worker thread shutdown period +- restructured rsyslog.conf documentation +- bugfix: memory leak in ompgsql + Thanks to Ken for providing the patch +--------------------------------------------------------------------------- +Version 3.22.4 [v3-stable] (rgerhards), 2010-??-?? +- bugfix: action resume interval incorrectly handled, thus took longer to + resume +- bugfix: cosmetic: proper constant used instead of number in open call +- bugfix: timestamp was incorrectly calculated for timezones with minute + offset + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 +- improved some code based on clang static analyzer results +- bugfix: potential misaddressing in property replacer +- bugfix: improper handling of invalid PRI values + references: CVE-2014-3634 +--------------------------------------------------------------------------- +Version 3.22.3 [v3-stable] (rgerhards), 2010-11-24 +- bugfix(important): problem in TLS handling could cause rsyslog to loop + in a tight loop, effectively disabling functionality and bearing the + risk of unresponsiveness of the whole system. + Bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=194 +--------------------------------------------------------------------------- +Version 3.22.2 [v3-stable] (rgerhards), 2010-08-05 +- bugfix: comment char ('#') in literal terminated script parsing + and thus could not be used. + but tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119 +- enhance: imrelp now also provides remote peer's IP address + [if librelp != 1.0.0 is used] +- bugfix: sending syslog messages with zip compression did not work +- bugfix: potential hang condition on queue shutdown +- bugfix: segfault on startup when -q or -Q option was given + bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=157 + Thanks to Jonas Nogueira for reporting this bug. +- clarified use of $ActionsSendStreamDriver[AuthMode/PermittedPeers] + in doc set (require TLS drivers) +- bugfix: $CreateDirs variable not properly initialized, default thus + was random (but most often "on") +- bugfix: potential segfault when -p command line option was used + thanks to varmojfekoj for pointing me at this bug +- bugfix: programname filter in ! configuration can not be reset + Thanks to Kiss Gabor for the patch. +--------------------------------------------------------------------------- +Version 3.22.1 [v3-stable] (rgerhards), 2009-07-02 +- bugfix: invalid error message issued if $includeConfig was on an empty + set of files (e.g. *.conf, where none such files existed) + thanks to Michael Biebl for reporting this bug +- bugfix: when run in foreground (but not in debug mode), a + debug message ("DoDie called") was emitted at shutdown. Removed. + thanks to Michael Biebl for reporting this bug +- bugfix: some garbage was emitted to stderr on shutdown. This + garbage consisted of file names, which were written during + startup (key point: not a pointer error) + thanks to Michael Biebl for reporting this bug +- bugfix: startup and shutdown message were emitted to stdout + thanks to Michael Biebl for reporting this bug +- bugfix: error messages were not emitted to stderr in forked mode + (stderr and stdo are now kept open across forks) +- bugfix: internal messages were emitted to whatever file had fd2 when + rsyslogd ran in forked mode (as usual!) + Thanks to varmojfekoj for the patch +- small enhancement: config validation run now exits with code 1 if an + error is detected. This change is considered important but small enough + to apply it directly to the stable version. [But it is a border case, + the change requires more code than I had hoped. Thus I have NOT tried + to actually catch all cases, this is left for the current devel + releases, if necessary] +- bugfix: light and full delay watermarks had invalid values, badly + affecting performance for delayable inputs +- bugfix: potential segfault issue when multiple $UDPServerRun directives + are specified. Thanks to Michael Biebl for helping to debug this one. +- relaxed GnuTLS version requirement to 1.4.0 after confirmation from the + field that this version is sufficient +- bugfix: parser did not properly handle empty structured data +- bugfix: invalid mutex release in msg.c (detected under thread debugger, + seems not to have any impact on actual deployments) +--------------------------------------------------------------------------- +Version 3.22.0 [v3-stable] (rgerhards), 2009-04-21 +This is the first stable release that includes the full functionality +of the 3.21.x version tree. +- bugfix: $InputTCPMaxSessions config directive was accepted, but not + honored. This resulted in a fixed upper limit of 200 connections. +- bugfix: the default for $DirCreateMode was 0644, and as such wrong. + It has now been changed to 0700. For some background, please see + http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html +- bugfix: ompgsql did not detect problems in sql command execution + this could cause loss of messages. The handling was correct if the + connection broke, but not if there was a problem with statement + execution. The most probable case for such a case would be invalid + sql inside the template, and this is now much easier to diagnose. +--------------------------------------------------------------------------- +Version 3.21.11 [BETA] (rgerhards), 2009-04-03 +- build system improvements contributed by Michael Biebl - thx! +- all patches from 3.20.5 incorporated (see it's ChangeLog entry) +--------------------------------------------------------------------------- +Version 3.21.10 [BETA] (rgerhards), 2009-02-02 +- bugfix: inconsistent use of mutex/atomic operations could cause segfault + details are too many, for full analysis see blog post at: + http://blog.gerhards.net/2009/01/rsyslog-data-race-analysis.html +- the string "Do Die" was accidentally emited upon exit in non-debug mode + This has now been corrected. Thanks to varmojfekoj for the patch. +- some legacy options were not correctly processed. + Thanks to varmojfekoj for the patch. +- doc bugfix: v3-compatibility document had typo in config directive + thanks to Andrej for reporting this +--------------------------------------------------------------------------- +Version 3.21.9 [BETA] (rgerhards), 2008-12-04 +- re-release of 3.21.8 with an additional fix, that could also lead + to DoS; 3.21.8 has been removed from the official download archives +- security fix: imudp emitted a message when a non-permitted sender + tried to send a message to it. This behavior is operator-configurable. + If enabled, a message was emitted each time. That way an attacker could + effectively fill the disk via this facility. The message is now + emitted only once in a minute (this currently is a hard-coded limit, + if someone comes up with a good reason to make it configurable, we + will probably do that). +--------------------------------------------------------------------------- +Version 3.21.8 [BETA] (rgerhards), 2008-12-04 +- bugfix: imklog did not compile on FreeBSD +- security bugfix: $AllowedSender was not honored, all senders were + permitted instead (see https://www.rsyslog.com/Article322.phtml) +- merged in all other changes from 3.20.1 (see there) +--------------------------------------------------------------------------- +Version 3.21.7 [BETA] (rgerhards), 2008-11-11 +- this is the new beta branch, based on the former 3.21.6 devel +- new functionality: ZERO property replacer nomatch option (from v3-stable) +--------------------------------------------------------------------------- +Version 3.21.6 [DEVEL] (rgerhards), 2008-10-22 +- consolidated time calls during msg object creation, improves performance + and consistency +- bugfix: solved a segfault condition +- bugfix: subsecond time properties generated by imfile, imklog and + internal messages could be slightly inconsistent +- bugfix: (potentially big) memory leak on HUP if queues could not be + drained before timeout - thanks to David Lang for pointing this out +- added capability to support multiple module search paths. Thank + to Marius Tomaschewski for providing the patch. +- bugfix: im3195 did no longer compile +- improved "make distcheck" by ensuring everything relevant is recompiled +--------------------------------------------------------------------------- +Version 3.21.5 [DEVEL] (rgerhards), 2008-09-30 +- performance optimization: unnecessary time() calls during message + parsing removed - thanks to David Lang for his excellent performance + analysis +- added new capability to property replacer: multiple immediately + successive field delimiters are treated as a single one. + Thanks to Zhuang Yuyao for the patch. +- added message property "inputname", which contains the name of the + input (module) that generated it. Presence is depending on support in + each input module (else it is blank). +- added system property "$myhostname", which contains the name of the + local host as it knows itself. +- imported a number of fixes and enhancements from the stable and + devel branches, including a fix to a potential segfault on HUP + when using UDP listeners +- re-enabled gcc builtin atomic operations and added a proper + ./configure check +- bugfix: potential race condition when adding messages to queue + There was a wrong order of mutex lock operations. It is hard to + believe that really caused problems, but in theory it could and with + threading we often see that theory becomes practice if something is only + used long enough on a fast enough machine with enough CPUs ;) +- cleaned up internal debug system code and made it behave better + in regard to multi-threading +--------------------------------------------------------------------------- +Version 3.21.4 [DEVEL] (rgerhards), 2008-09-04 +- removed compile time fixed message size limit (was 2K), limit can now + be set via $MaxMessageSize global config directive (finally gotten rid + of MAXLINE ;)) +- enhanced doc for $ActionExecOnlyEveryNthTimeTimeout +- integrated a number of patches from 3.18.4, namely + - bugfix: order-of magnitude issue with base-10 size definitions + in config file parser. Could lead to invalid sizes, constraints + etc for e.g. queue files and any other object whose size was specified + in base-10 entities. Did not apply to binary entities. Thanks to + RB for finding this bug and providing a patch. + - bugfix: action was not called when system time was set backwards + (until the previous time was reached again). There are still some + side-effects when time is rolled back (A time rollback is really a bad + thing to do, ideally the OS should issue pseudo time (like NetWare did) + when the user tries to roll back time). Thanks to varmojfekoj for this + patch. + - doc bugfix: rsyslog.conf man page improved and minor nit fixed + thanks to Lukas Kuklinek for the patch. +--------------------------------------------------------------------------- +Version 3.21.3 [DEVEL] (rgerhards), 2008-08-13 +- added ability to specify flow control mode for imuxsock +- added ability to execute actions only after the n-th call of the action + This also lead to the addition of two new config directives: + $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout + This feature is useful, for example, for alerting: it permits you to + send an alert only after at least n occurrences of a specific message + have been seen by rsyslogd. This protects against false positives + due to waiting for additional confirmation. +- bugfix: IPv6 addresses could not be specified in forwarding actions + New syntax @[addr]:port introduced to enable that. Root problem was IPv6 + addresses contain colons. +- somewhat enhanced debugging messages +- imported from 3.18.3: + - enhanced ommysql to support custom port to connect to server + Port can be set via new $ActionOmmysqlServerPort config directive + Note: this was a very minor change and thus deemed appropriate to be + done in the stable release. + - bugfix: misspelled config directive, previously was + $MainMsgQueueWorkeTimeoutrThreadShutdown, is now + $MainMsgQueueWorkerTimeoutThreadShutdown. Note that the misspelled + directive is not preserved - if the misspelled directive was used + (which I consider highly unlikely), the config file must be changed. + Thanks to lperr for reporting the bug. +--------------------------------------------------------------------------- +Version 3.21.2 [DEVEL] (rgerhards), 2008-08-04 +- added $InputUnixListenSocketHostName config directive, which permits to + override the hostname being used on a local unix socket. This is useful + for differentiating "hosts" running in several jails. Feature was + suggested by David Darville, thanks for the suggestion. +- enhanced ommail to support multiple email recipients. This is done by + specifying $ActionMailTo multiple times. Note that this introduces a + small incompatibility to previous config file syntax: the recipient + list is now reset for each action (we honestly believe that will + not cause any problem - apologies if it does). +- enhanced troubleshooting documentation +--------------------------------------------------------------------------- +Version 3.21.1 [DEVEL] (rgerhards), 2008-07-30 +- bugfix: no error was reported if the target of a $IncludeConfig + could not be accessed. +- added testbed for common config errors +- added doc for -u option to rsyslogd man page +- enhanced config file checking - no active actions are detected +- added -N rsyslogd command line option for a config validation run + (which does not execute actual syslogd code and does not interfere + with a running instance) +- somewhat improved emergency configuration. It is now also selected + if the config contains no active actions +- rsyslogd error messages are now reported to stderr by default. can be + turned off by the new "$ErrorMessagesToStderr off" directive + Thanks to HKS for suggesting the new features. +--------------------------------------------------------------------------- +Version 3.21.0 [DEVEL] (rgerhards), 2008-07-18 +- starts a new devel branch +- added a generic test driver for RainerScript plus some test cases + to the testbench +- added a small diagnostic tool to obtain result of gethostname() API +- imported all changes from 3.18.1 until today (some quite important, + see below) +--------------------------------------------------------------------------- +Version 3.20.6 [v3-stable] (rgerhards), 2009-04-16 +- this is the last v3-stable for the 3.20.x series +- bugfix: $InputTCPMaxSessions config directive was accepted, but not + honored. This resulted in a fixed upper limit of 200 connections. +- bugfix: the default for $DirCreateMode was 0644, and as such wrong. + It has now been changed to 0700. For some background, please see + http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html +--------------------------------------------------------------------------- +Version 3.20.5 [v3-stable] (rgerhards), 2009-04-02 +- bugfix: potential abort with DA queue after high watermark is reached + There exists a race condition that can lead to a segfault. Thanks + go to vbernetr, who performed the analysis and provided patch, which + I only tweaked a very little bit. +- fixed bugs in RainerScript: + o when converting a number and a string to a common type, both were + actually converted to the other variable's type. + o the value of rsCStrConvertToNumber() was miscalculated. + Thanks to varmojfekoj for the patch +- fixed a bug in configure.ac which resulted in problems with + environment detection - thanks to Michael Biebl for the patch +- fixed a potential segfault problem in gssapi code + thanks to varmojfekoj for the patch +- doc enhance: provide standard template for MySQL module and instructions + on how to modify schema +--------------------------------------------------------------------------- +Version 3.20.4 [v3-stable] (rgerhards), 2009-02-09 +- bugfix: inconsistent use of mutex/atomic operations could cause segfault + details are too many, for full analysis see blog post at: + http://blog.gerhards.net/2009/01/rsyslog-data-race-analysis.html +- bugfix: invalid ./configure settings for RFC3195 + thanks to Michael Biebl for the patch +- bugfix: invalid mutex access in msg.c +- doc bugfix: dist tarball missed 2 files, had one extra file that no + longer belongs into it. Thanks to Michael Biebl for pointing this out. +--------------------------------------------------------------------------- +Version 3.20.3 [v3-stable] (rgerhards), 2009-01-19 +- doc bugfix: v3-compatibility document had typo in config directive + thanks to Andrej for reporting this +- fixed a potential segfault condition with $AllowedSender directive + On HUP, the root pointers were not properly cleaned up. Thanks to + Michael Biebl, olgoat, and Juha Koho for reporting and analyzing + the bug. +--------------------------------------------------------------------------- +Version 3.20.2 [v3-stable] (rgerhards), 2008-12-04 +- re-release of 3.20.1 with an additional fix, that could also lead + to DoS; 3.20.1 has been removed from the official download archives +- security fix: imudp emitted a message when a non-permitted sender + tried to send a message to it. This behavior is operator-configurable. + If enabled, a message was emitted each time. That way an attacker could + effectively fill the disk via this facility. The message is now + emitted only once in a minute (this currently is a hard-coded limit, + if someone comes up with a good reason to make it configurable, we + will probably do that). +--------------------------------------------------------------------------- +Version 3.20.1 [v3-stable] (rgerhards), 2008-12-04 +- security bugfix: $AllowedSender was not honored, all senders were + permitted instead +- enhance: regex nomatch option "ZERO" has been added + This allows one to return the string 0 if a regular expression is + not found. This is probably useful for storing numerical values into + database columns. +- bugfix: memory leak in gtls netstream driver fixed + memory was lost each time a TLS session was torn down. This could + result in a considerable memory leak if it happened quite frequently + (potential system crash condition) +- doc update: documented how to specify multiple property replacer + options + link to new online regex generator tool added +- minor bugfix: very small memory leak in gtls netstream driver + around a handful of bytes (< 20) for each HUP +- improved debug output for regular expressions inside property replacer + RE's seem to be a big trouble spot and I would like to have more + information inside the debug log. So I decided to add some additional + debug strings permanently. +--------------------------------------------------------------------------- +Version 3.20.0 [v3-stable] (rgerhards), 2008-11-05 +- this is the initial release of the 3.19.x branch as a stable release +- bugfix: double-free in pctp netstream driver. Thank to varmojfekoj + for the patch +--------------------------------------------------------------------------- +Version 3.19.12 [BETA] (rgerhards), 2008-10-16 +- bugfix: subseconds where not correctly extracted from a timestamp + if that timestamp did not contain any subsecond information (the + resulting string was garbage but should have been "0", what it + now is). +- increased maximum size of a configuration statement to 4K (was 1K) +- imported all fixes from the stable branch (quite a lot) +- bugfix: (potentially big) memory leak on HUP if queues could not be + drained before timeout - thanks to David Lang for pointing this out +--------------------------------------------------------------------------- +Version 3.19.11 [BETA] (rgerhards), 2008-08-25 +This is a refresh of the beta. No beta-specific fixes have been added. +- included fixes from v3-stable (most importantly 3.18.3) +--------------------------------------------------------------------------- +Version 3.19.10 [BETA] (rgerhards), 2008-07-15 +- start of a new beta branch based on former 3.19 devel branch +- bugfix: bad memory leak in disk-based queue modes +- bugfix: UDP syslog forwarding did not work on all platforms + the ai_socktype was incorrectly set to 1. On some platforms, this + lead to failing name resolution (e.g. FreeBSD 7). Thanks to HKS for + reporting the bug. +- bugfix: priority was incorrectly calculated on FreeBSD 7, + because the LOG_MAKEPRI() C macro has a different meaning there (it + is just a simple addition of facility and severity). I have changed + this to use own, consistent, code for PRI calculation. Thank to HKS + for reporting this bug. +- bugfix (cosmetical): authorization was not checked when gtls handshake + completed immediately. While this sounds scary, the situation can not + happen in practice. We use non-blocking IO only for server-based gtls + session setup. As TLS requires the exchange of multiple frames before + the handshake completes, it simply is impossible to do this in one + step. However, it is useful to have the code path correct even for + this case - otherwise, we may run into problems if the code is changed + some time later (e.g. to use blocking sockets). Thanks to varmojfekoj + for providing the patch. +- important queue bugfix from 3.18.1 imported (see below) +- cleanup of some debug messages +--------------------------------------------------------------------------- +Version 3.19.9 (rgerhards), 2008-07-07 +- added tutorial for creating a TLS-secured syslog infrastructure +- rewritten omusrmsg to no longer fork() a new process for sending messages + this caused some problems with the threading model, e.g. zombies. Also, + it was far less optimal than it is now. +- bugfix: machine certificate was required for client even in TLS anon mode + Reference: http://bugzilla.adiscon.com/show_bug.cgi?id=85 + The fix also slightly improves performance by not storing certificates in + client sessions when there is no need to do so. +- bugfix: RainerScript syntax error was not always detected +--------------------------------------------------------------------------- +Version 3.19.8 (rgerhards), 2008-07-01 +- bugfix: gtls module did not correctly handle EGAIN (and similar) recv() + states. This has been fixed by introducing a new abstraction layer inside + gtls. +- added (internal) error codes to error messages; added redirector to + web description of error codes + closes bug http://bugzilla.adiscon.com/show_bug.cgi?id=20 +- disabled compile warnings caused by third-party libraries +- reduced number of compile warnings in gcc's -pedantic mode +- some minor documentation improvements +- included all fixes from beta 3.17.5 +--------------------------------------------------------------------------- +Version 3.19.7 (rgerhards), 2008-06-11 +- added new property replacer option "date-subseconds" that enables + to query just the subsecond part of a high-precision timestamp +- somewhat improved plain tcp syslog reliability by doing a connection + check before sending. Credits to Martin Schuette for providing the + idea. Details are available at + http://blog.gerhards.net/2008/06/reliable-plain-tcp-syslog-once-again.html +- made rsyslog tickless in the (usual and default) case that repeated + message reduction is turned off. More info: + http://blog.gerhards.net/2008/06/coding-to-save-environment.html +- some build system cleanup, thanks to Michael Biebl +- bugfix: compile under (Free)BSD failed due to some invalid library + definitions - this is fixed now. Thanks to Michael Biebl for the patch. +--------------------------------------------------------------------------- +Version 3.19.6 (rgerhards), 2008-06-06 +- enhanced property replacer to support multiple regex matches +- bugfix: part of permittedPeer structure was not correctly initialized + thanks to varmojfekoj for spotting this +- bugfix: off-by-one bug during certificate check +- bugfix: removed some memory leaks in TLS code +--------------------------------------------------------------------------- +Version 3.19.5 (rgerhards), 2008-05-30 +- enabled Posix ERE expressions inside the property replacer + (previously BRE was permitted only) +- provided ability to specify that a regular expression submatch shall + be used inside the property replacer +- implemented in property replacer: if a regular expression does not match, + it can now either return "**NO MATCH** (default, as before), a blank + property or the full original property text +- enhanced property replacer to support multiple regex matches +--------------------------------------------------------------------------- +Version 3.19.4 (rgerhards), 2008-05-27 +- implemented x509/certvalid gtls auth mode +- implemented x509/name gtls auth mode (including wildcards) +- changed fingerprint gtls auth mode to new format fingerprint +- protected gtls error string function by a mutex. Without it, we + could have a race condition in extreme cases. This was very remote, + but now can no longer happen. +- changed config directive name to reflect different use + $ActionSendStreamDriverCertFingerprint is now + $ActionSendStreamDriverPermittedPeer and can be used both for + fingerprint and name authentication (similar to the input side) +- bugfix: sender information (fromhost et al) was missing in imudp + thanks to sandiso for reporting this bug +- this release fully implements IETF's syslog-transport-tls-12 plus + the latest text changes Joe Salowey provided via email. Not included + is ipAddress subjectAltName authentication, which I think will be + dropped from the draft. I don't think there is any real need for it. +This release also includes all bug fix up to today from the beta +and stable branches. Most importantly, this means the bugfix for +100% CPU utilization by imklog. +--------------------------------------------------------------------------- +Version 3.19.3 (rgerhards), 2008-05-21 +- added ability to authenticate the server against its certificate + fingerprint +- added ability for client to provide its fingerprint +- added ability for server to obtain client cert's fingerprint +- bugfix: small mem leak in omfwd on exit (strmdriver name was not freed) +- bugfix: $ActionSendStreamDriver had no effect +- bugfix: default syslog port was no longer used if none was + configured. Thanks to varmojfekoj for the patch +- bugfix: missing linker options caused build to fail on some + systems. Thanks to Tiziano Mueller for the patch. +--------------------------------------------------------------------------- +Version 3.19.2 (rgerhards), 2008-05-16 +- bugfix: TCP input modules did incorrectly set fromhost property + (always blank) +- bugfix: imklog did not set fromhost property +- added "fromhost-ip" property + Note that adding this property changes the on-disk format for messages. + However, that should not have any bad effect on existing spool files. + But you will run into trouble if you create a spool file with this + version and then try to process it with an older one (after a downgrade). + Don't do that ;) +- added "RSYSLOG_DebugFormat" canned template +- bugfix: hostname and fromhost were swapped when a persisted message + (in queued mode) was read in +- bugfix: lmtcpclt, lmtcpsrv and lmgssutil did all link to the static + runtime library, resulting in a large size increase (and potential + "interesting" effects). Thanks to Michael Biebl for reporting the size + issue. +- bugfix: TLS server went into an endless loop in some situations. + Thanks to Michael Biebl for reporting the problem. +- fixed potential segfault due to invalid call to cfsysline + thanks to varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 3.19.1 (rgerhards), 2008-05-07 +- configure help for --enable-gnutls wrong - said default is "yes" but + default actually is "no" - thanks to darix for pointing this out +- file dirty.h was missing - thanks to darix for pointing this out +- bugfix: man files were not properly distributed - thanks to + darix for reporting and to Michael Biebl for help with the fix +- some minor cleanup +--------------------------------------------------------------------------- +Version 3.19.0 (rgerhards), 2008-05-06 +- begins new devel branch version +- implemented TLS for plain tcp syslog (this is also the world's first + implementation of IETF's upcoming syslog-transport-tls draft) +- partly rewritten and improved omfwd among others, now loads TCP + code only if this is actually necessary +- split of a "runtime library" for rsyslog - this is not yet a clean + model, because some modularization is still outstanding. In theory, + this shall enable other utilities but rsyslogd to use the same + runtime +- implemented im3195, the RFC3195 input as a plugin +- changed directory structure, files are now better organized +- a lot of cleanup in regard to modularization +- -c option no longer must be the first option - thanks to varmojfekoj + for the patch +--------------------------------------------------------------------------- +Version 3.18.7 (rgerhards), 2008-12-?? +- bugfix: the default for $DirCreateMode was 0644, and as such wrong. + It has now been changed to 0700. For some background, please see + http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html +- fixed a potential segfault condition with $AllowedSender directive + On HUP, the root pointers were not properly cleaned up. Thanks to + Michael Biebl, olgoat, and Juha Koho for reporting and analyzing + the bug. +- some legacy options were not correctly processed. + Thanks to varmojfekoj for the patch. +- doc bugfix: some spelling errors in man pages corrected. Thanks to + Geoff Simmons for the patch. +--------------------------------------------------------------------------- +Version 3.18.6 (rgerhards), 2008-12-08 +- security bugfix: $AllowedSender was not honored, all senders were + permitted instead (see https://www.rsyslog.com/Article322.phtml) + (backport from v3-stable, v3.20.9) +- minor bugfix: dual close() call on tcp session closure +--------------------------------------------------------------------------- +Version 3.18.5 (rgerhards), 2008-10-09 +- bugfix: imudp input module could cause segfault on HUP + It did not properly de-init a variable acting as a linked list head. + That resulted in trying to access freed memory blocks after the HUP. +- bugfix: rsyslogd could hang on HUP + because getnameinfo() is not cancel-safe, but was not guarded against + being cancelled. pthread_cancel() is routinely being called during + HUP processing. +- bugfix[minor]: if queue size reached light_delay mark, enqueuing + could potentially be blocked for a longer period of time, which + was not the behavior desired. +- doc bugfix: $ActionExecOnlyWhenPreviousIsSuspended was still misspelled + as $...OnlyIfPrev... in some parts of the documentation. Thanks to + Lorenzo M. Catucci for reporting this bug. +- added doc on malformed messages, cause and how to work-around, to the + doc set +- added doc on how to build from source repository +--------------------------------------------------------------------------- +Version 3.18.4 (rgerhards), 2008-09-18 +- bugfix: order-of magnitude issue with base-10 size definitions + in config file parser. Could lead to invalid sizes, constraints + etc for e.g. queue files and any other object whose size was specified + in base-10 entities. Did not apply to binary entities. Thanks to + RB for finding this bug and providing a patch. +- bugfix: action was not called when system time was set backwards + (until the previous time was reached again). There are still some + side-effects when time is rolled back (A time rollback is really a bad + thing to do, ideally the OS should issue pseudo time (like NetWare did) + when the user tries to roll back time). Thanks to varmojfekoj for this + patch. +- doc bugfix: rsyslog.conf man page improved and minor nit fixed + thanks to Lukas Kuklinek for the patch. +- bugfix: error code -2025 was used for two different errors. queue full + is now -2074 and -2025 is unique again. (did cause no real problem + except for troubleshooting) +- bugfix: default discard severity was incorrectly set to 4, which lead + to discard-on-queue-full to be enabled by default. That could cause + message loss where non was expected. The default has now been changed + to the correct value of 8, which disables the functionality. This + problem applied both to the main message queue and the action queues. + Thanks to Raoul Bhatia for pointing out this problem. +- bugfix: option value for legacy -a option could not be specified, + resulting in strange operations. Thanks to Marius Tomaschewski + for the patch. +- bugfix: colon after date should be ignored, but was not. This has + now been corrected. Required change to the internal ParseTIMESTAMP3164() + interface. +--------------------------------------------------------------------------- +Version 3.18.3 (rgerhards), 2008-08-18 +- bugfix: imfile could cause a segfault upon rsyslogd HUP and termination + Thanks to lperr for an excellent bug report that helped detect this + problem. +- enhanced ommysql to support custom port to connect to server + Port can be set via new $ActionOmmysqlServerPort config directive + Note: this was a very minor change and thus deemed appropriate to be + done in the stable release. +- bugfix: misspelled config directive, previously was + $MainMsgQueueWorkeTimeoutrThreadShutdown, is now + $MainMsgQueueWorkerTimeoutThreadShutdown. Note that the misspelled + directive is not preserved - if the misspelled directive was used + (which I consider highly unlikely), the config file must be changed. + Thanks to lperr for reporting the bug. +- disabled flow control for imuxsock, as it could cause system hangs + under some circumstances. The devel (3.21.3 and above) will + re-enable it and provide enhanced configurability to overcome the + problems if they occur. +--------------------------------------------------------------------------- +Version 3.18.2 (rgerhards), 2008-08-08 +- merged in IPv6 forwarding address bugfix from v2-stable +--------------------------------------------------------------------------- +Version 3.18.1 (rgerhards), 2008-07-21 +- bugfix: potential segfault in creating message mutex in non-direct queue + mode. rsyslogd segfaults on freeBSD 7.0 (an potentially other platforms) + if an action queue is running in any other mode than non-direct. The + same problem can potentially be triggered by some main message queue + settings. In any case, it will manifest during rsyslog's startup. It is + unlikely to happen after a successful startup (the only window of + exposure may be a relatively seldom executed action running in queued + mode). This has been corrected. Thank to HKS for point out the problem. +- bugfix: priority was incorrectly calculated on FreeBSD 7, + because the LOG_MAKEPRI() C macro has a different meaning there (it + is just a simple addition of facility and severity). I have changed + this to use own, consistent, code for PRI calculation. [Backport from + 3.19.10] +- bugfix: remove PRI part from kernel message if it is present + Thanks to Michael Biebl for reporting this bug +- bugfix: mark messages were not correctly written to text log files + the markmessageinterval was not correctly propagated to all places + where it was needed. This resulted in rsyslog using the default + (20 minutes) in some code paths, what looked to the user like mark + messages were never written. +- added a new property replacer option "sp-if-no-1st-sp" to cover + a problem with RFC 3164 based interpretation of tag separation. While + it is a generic approach, it fixes a format problem introduced in + 3.18.0, where kernel messages no longer had a space after the tag. + This is done by a modification of the default templates. + Please note that this may affect some messages where there intentionally + is no space between the tag and the first character of the message + content. If so, this needs to be worked around via a specific + template. However, we consider this scenario to be quite remote and, + even if it exists, it is not expected that it will actually cause + problems with log parsers (instead, we assume the new default template + behavior may fix previous problems with log parsers due to the + missing space). +- bugfix: imklog module was not correctly compiled for GNU/kFreeBSD. + Thanks to Petr Salinger for the patch +- doc bugfix: property replacer options secpath-replace and + secpath-drop were not documented +- doc bugfix: fixed some typos in rsyslog.conf man page +- fixed typo in source comment - thanks to Rio Fujita +- some general cleanup (thanks to Michael Biebl) +--------------------------------------------------------------------------- +Version 3.18.0 (rgerhards), 2008-07-11 +- begun a new v3-stable based on former 3.17.4 beta plus patches to + previous v3-stable +- bugfix in RainerScript: syntax error was not always detected +--------------------------------------------------------------------------- +Version 3.17.5 (rgerhards), 2008-06-27 +- added doc: howto set up a reliable connection to remote server via + queued mode (and plain tcp protocol) +- bugfix: comments after actions were not properly treated. For some + actions (e.g. forwarding), this could also lead to invalid configuration +--------------------------------------------------------------------------- +Version 3.17.4 (rgerhards), 2008-06-16 +- changed default for $KlogSymbolLookup to "off". The directive is + also scheduled for removal in a later version. This was necessary + because on kernels >= 2.6, the kernel does the symbol lookup itself. The + imklog lookup logic then breaks the log message and makes it unusable. +--------------------------------------------------------------------------- +Version 3.17.3 (rgerhards), 2008-05-28 +- bugfix: imklog went into an endless loop if a PRI value was inside + a kernel log message (unusual case under Linux, frequent under BSD) +--------------------------------------------------------------------------- +Version 3.17.2 (rgerhards), 2008-05-04 +- this version is the new beta, based on 3.17.1 devel feature set +- merged in imklog bug fix from v3-stable (3.16.1) +--------------------------------------------------------------------------- +Version 3.17.1 (rgerhards), 2008-04-15 +- removed dependency on MAXHOSTNAMELEN as much as it made sense. + GNU/Hurd does not define it (because it has no limit), and we have taken + care for cases where it is undefined now. However, some very few places + remain where IMHO it currently is not worth fixing the code. If it is + not defined, we have used a generous value of 1K, which is above IETF + RFC's on hostname length at all. The memory consumption is no issue, as + there are only a handful of this buffers allocated *per run* -- that's + also the main reason why we consider it not worth to be fixed any further. +- enhanced legacy syslog parser to handle slightly malformed messages + (with a space in front of the timestamp) - at least HP procurve is + known to do that and I won't outrule that others also do it. The + change looks quite unintrusive and so we added it to the parser. +- implemented klogd functionality for BSD +- implemented high precision timestamps for the kernel log. Thanks to + Michael Biebl for pointing out that the kernel log did not have them. +- provided ability to discard non-kernel messages if they are present + in the kernel log (seems to happen on BSD) +- implemented $KLogInternalMsgFacility config directive +- implemented $KLogPermitNonKernelFacility config directive +Plus a number of bugfixes that were applied to v3-stable and beta +branches (not mentioned here in detail). +--------------------------------------------------------------------------- +Version 3.17.0 (rgerhards), 2008-04-08 +- added native ability to send mail messages +- removed no longer needed file relputil.c/.h +- added $ActionExecOnlyOnceEveryInterval config directive +- bugfix: memory leaks in script engine +- bugfix: zero-length strings were not supported in object + deserializer +- properties are now case-insensitive everywhere (script, filters, + templates) +- added the capability to specify a processing (actually dequeue) + timeframe with queues - so things can be configured to be done + at off-peak hours +- We have removed the 32 character size limit (from RFC3164) on the + tag. This had bad effects on existing environments, as sysklogd didn't + obey it either (probably another bug in RFC3164...). We now receive + the full size, but will modify the outputs so that only 32 characters + max are used by default. If you need large tags in the output, you need + to provide custom templates. +- changed command line processing. -v, -M, -c options are now parsed + and processed before all other options. Inter-option dependencies + have been relieved. Among others, permits to specify initial module + load path via -M only (not the environment) which makes it much + easier to work with non-standard module library locations. Thanks + to varmojfekoj for suggesting this change. Matches bugzilla bug 55. +- bugfix: some messages were emited without hostname +Plus a number of bugfixes that were applied to v3-stable and beta +branches (not mentioned here in detail). +--------------------------------------------------------------------------- +Version 3.16.3 (rgerhards), 2008-07-11 +- updated information on rsyslog packages +- bugfix: memory leak in disk-based queue modes +--------------------------------------------------------------------------- +Version 3.16.2 (rgerhards), 2008-06-25 +- fixed potential segfault due to invalid call to cfsysline + thanks to varmojfekoj for the patch +- bugfix: some whitespaces where incorrectly not ignored when parsing + the config file. This is now corrected. Thanks to Michael Biebl for + pointing out the problem. +--------------------------------------------------------------------------- +Version 3.16.1 (rgerhards), 2008-05-02 +- fixed a bug in imklog which lead to startup problems (including + segfault) on some platforms under some circumstances. Thanks to + Vieri for reporting this bug and helping to troubleshoot it. +--------------------------------------------------------------------------- +Version 3.16.0 (rgerhards), 2008-04-24 +- new v3-stable (3.16.x) based on beta 3.15.x (RELP support) +- bugfix: omsnmp had a too-small sized buffer for hostname+port. This + could not lead to a segfault, as snprintf() was used, but could cause + some trouble with extensively long hostnames. +- applied patch from Tiziano Müller to remove some compiler warnings +- added gssapi overview/howto thanks to Peter Vrabec +- changed some files to grant LGPLv3 extended permissions on top of GPLv3 + this also is the first sign of something that will evolve into a + well-defined "rsyslog runtime library" +--------------------------------------------------------------------------- +Version 3.15.1 (rgerhards), 2008-04-11 +- bugfix: some messages were emited without hostname +- disabled atomic operations for the time being because they introduce some + cross-platform trouble - need to see how to fix this in the best + possible way +- bugfix: zero-length strings were not supported in object + deserializer +- added librelp check via PKG_CHECK thanks to Michael Biebl's patch +- file relputil.c deleted, is not actually needed +- added more meaningful error messages to rsyslogd (when some errors + happens during startup) +- bugfix: memory leaks in script engine +- bugfix: $hostname and $fromhost in RainerScript did not work +This release also includes all changes applied to the stable versions +up to today. +--------------------------------------------------------------------------- +Version 3.15.0 (rgerhards), 2008-04-01 +- major new feature: imrelp/omrelp support reliable delivery of syslog + messages via the RELP protocol and librelp (http://www.librelp.com). + Plain tcp syslog, so far the best reliability solution, can lose + messages when something goes wrong or a peer goes down. With RELP, + this can no longer happen. See imrelp.html for more details. +- bugfix: rsyslogd was no longer build by default; man pages are + only installed if corresponding option is selected. Thanks to + Michael Biebl for pointing these problems out. +--------------------------------------------------------------------------- +Version 3.14.2 (rgerhards), 2008-04-09 +- bugfix: segfault with expression-based filters +- bugfix: omsnmp did not deref errmsg object on exit (no bad effects caused) +- some cleanup +- bugfix: imklog did not work well with kernel 2.6+. Thanks to Peter + Vrabec for patching it based on the development in sysklogd - and thanks + to the sysklogd project for upgrading klogd to support the new + functionality +- some cleanup in imklog +- bugfix: potential segfault in imklog when kernel is compiled without + /proc/kallsyms and the file System.map is missing. Thanks to + Andrea Morandi for pointing it out and suggesting a fix. +- bugfixes, credits to varmojfekoj: + * reset errno before printing a warning message + * misspelled directive name in code processing legacy options +- bugfix: some legacy options not correctly interpreted - thanks to + varmojfekoj for the patch +- improved detection of modules being loaded more than once + thanks to varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 3.14.1 (rgerhards), 2008-04-04 +- bugfix: some messages were emited without hostname +- bugfix: rsyslogd was no longer build by default; man pages are + only installed if corresponding option is selected. Thanks to + Michael Biebl for pointing these problems out. +- bugfix: zero-length strings were not supported in object + deserializer +- disabled atomic operations for this stable build as it caused + platform problems +- bugfix: memory leaks in script engine +- bugfix: $hostname and $fromhost in RainerScript did not work +- bugfix: some memory leak when queue is running in disk mode +- man pages improved thanks to varmojfekoj and Peter Vrabec +- We have removed the 32 character size limit (from RFC3164) on the + tag. This had bad effects on existing environments, as sysklogd didn't + obey it either (probably another bug in RFC3164...). We now receive + the full size, but will modify the outputs so that only 32 characters + max are used by default. If you need large tags in the output, you need + to provide custom templates. +- bugfix: some memory leak when queue is running in disk mode +--------------------------------------------------------------------------- +Version 3.14.0 (rgerhards), 2008-04-02 +An interim version was accidentally released to the web. It was named 3.14.0. +To avoid confusion, we have not assigned this version number to any +official release. If you happen to use 3.14.0, please update to 3.14.1. +--------------------------------------------------------------------------- +Version 3.13.0-dev0 (rgerhards), 2008-03-31 +- bugfix: accidentally set debug option in 3.12.5 reset to production + This option prevented dlclose() to be called. It had no real bad effects, + as the modules were otherwise correctly deinitialized and dlopen() + supports multiple opens of the same module without any memory footprint. +- removed --enable-mudflap, added --enable-valgrind ./configure setting +- bugfix: tcp receiver could segfault due to uninitialized variable +- docfix: queue doc had a wrong directive name that prevented max worker + threads to be correctly set +- worked a bit on atomic memory operations to support problem-free + threading (only at non-intrusive places) +- added a --enable/disable-rsyslogd configure option so that + source-based packaging systems can build plugins without the need + to compile rsyslogd +- some cleanup +- test of potential new version number scheme +--------------------------------------------------------------------------- +Version 3.12.5 (rgerhards), 2008-03-28 +- changed default for "last message repeated n times", which is now + off by default +- implemented backward compatibility commandline option parsing +- automatically generated compatibility config lines are now also + logged so that a user can diagnose problems with them +- added compatibility mode for -a, -o and -p options +- compatibility mode processing finished +- changed default file output format to include high-precision timestamps +- added a built-in template for previous syslogd file format +- added new $ActionFileDefaultTemplate directive +- added support for high-precision timestamps when receiving legacy + syslog messages +- added new $ActionForwardDefaultTemplate directive +- added new $ActionGSSForwardDefaultTemplate directive +- added build-in templates for easier configuration +- bugfix: fixed small memory leak in tcpclt.c +- bugfix: fixed small memory leak in template regular expressions +- bugfix: regular expressions inside property replacer did not work + properly +- bugfix: QHOUR and HHOUR properties were wrongly calculated +- bugfix: fixed memory leaks in stream class and imfile +- bugfix: $ModDir did invalid bounds checking, potential overflow in + dbgprintf() - thanks to varmojfekoj for the patch +- bugfix: -t and -g legacy options max number of sessions had a wrong + and much too high value +--------------------------------------------------------------------------- +Version 3.12.4 (rgerhards), 2008-03-25 +- Greatly enhanced rsyslogd's file write performance by disabling + file syncing capability of output modules by default. This + feature is usually not required, not useful and an extreme performance + hit (both to rsyslogd as well as the system at large). Unfortunately, + most users enable it by default, because it was most intuitive to enable + it in plain old sysklogd syslog.conf format. There is now the + $ActionFileEnableSync config setting which must be enabled in order to + support syncing. By default it is off. So even if the old-format config + lines request syncing, it is not done unless explicitly enabled. I am + sure this is a very useful change and not a risk at all. I need to think + if I undo it under compatibility mode, but currently this does not + happen (I fear a lot of lazy users will run rsyslogd in compatibility + mode, again bringing up this performance problem...). +- added flow control options to other input sources +- added $HHOUR and $QHOUR system properties - can be used for half- and + quarter-hour logfile rotation +- changed queue's discard severities default value to 8 (do not discard) + to prevent unintentional message loss +- removed a no-longer needed callback from the output module + interface. Results in reduced code complexity. +- bugfix/doc: removed no longer supported -h option from man page +- bugfix: imklog leaked several hundred KB on each HUP. Thanks to + varmojfekoj for the patch +- bugfix: potential segfault on module unload. Thanks to varmojfekoj for + the patch +- bugfix: fixed some minor memory leaks +- bugfix: fixed some slightly invalid memory accesses +- bugfix: internally generated messages had "FROMHOST" property not set +--------------------------------------------------------------------------- +Version 3.12.3 (rgerhards), 2008-03-18 +- added advanced flow control for congestion cases (mode depending on message + source and its capability to be delayed without bad side effects) +- bugfix: $ModDir should not be reset on $ResetConfig - this can cause a lot + of confusion and there is no real good reason to do so. Also conflicts with + the new -M option and environment setting. +- bugfix: TCP and GSSAPI framing mode variable was uninitialized, leading to + wrong framing (caused, among others, interop problems) +- bugfix: TCP (and GSSAPI) octet-counted frame did not work correctly in all + situations. If the header was split across two packet reads, it was invalidly + processed, causing loss or modification of messages. +- bugfix: memory leak in imfile +- bugfix: duplicate public symbol in omfwd and omgssapi could lead to + segfault. thanks to varmojfekoj for the patch. +- bugfix: rsyslogd aborted on sighup - thanks to varmojfekoj for the patch +- some more internal cleanup ;) +- begun relp modules, but these are not functional yet +- Greatly enhanced rsyslogd's file write performance by disabling + file syncing capability of output modules by default. This + feature is usually not required, not useful and an extreme performance + hit (both to rsyslogd as well as the system at large). Unfortunately, + most users enable it by default, because it was most intuitive to enable + it in plain old sysklogd syslog.conf format. There is now a new config + setting which must be enabled in order to support syncing. By default it + is off. So even if the old-format config lines request syncing, it is + not done unless explicitly enabled. I am sure this is a very useful + change and not a risk at all. I need to think if I undo it under + compatibility mode, but currently this does not happen (I fear a lot of + lazy users will run rsyslogd in compatibility mode, again bringing up + this performance problem...). +--------------------------------------------------------------------------- +Version 3.12.2 (rgerhards), 2008-03-13 +- added RSYSLOGD_MODDIR environment variable +- added -M rsyslogd option (allows one to specify module directory location) +- converted net.c into a loadable library plugin +- bugfix: debug module now survives unload of loadable module when + printing out function call data +- bugfix: not properly initialized data could cause several segfaults if + there were errors in the config file - thanks to varmojfekoj for the patch +- bugfix: rsyslogd segfaulted when imfile read an empty line - thanks + to Johnny Tan for an excellent bug report +- implemented dynamic module unload capability (not visible to end user) +- some more internal cleanup +- bugfix: imgssapi segfaulted under some conditions; this fix is actually + not just a fix but a change in the object model. Thanks to varmojfekoj + for providing the bug report, an initial fix and lots of good discussion + that lead to where we finally ended up. +- improved session recovery when outbound tcp connection breaks, reduces + probability of message loss at the price of a highly unlikely potential + (single) message duplication +--------------------------------------------------------------------------- +Version 3.12.1 (rgerhards), 2008-03-06 +- added library plugins, which can be automatically loaded +- bugfix: actions were not correctly retried; caused message loss +- changed module loader to automatically add ".so" suffix if not + specified (over time, this shall also ease portability of config + files) +- improved debugging support; debug runtime options can now be set via + an environment variable +- bugfix: removed debugging code that I forgot to remove before releasing + 3.12.0 (does not cause harm and happened only during startup) +- added support for the MonitorWare syslog MIB to omsnmp +- internal code improvements (more code converted into classes) +- internal code reworking of the imtcp/imgssapi module +- added capability to ignore client-provided timestamp on unix sockets and + made this mode the default; this was needed, as some programs (e.g. sshd) + log with inconsistent timezone information, what messes up the local + logs (which by default don't even contain time zone information). This + seems to be consistent with what sysklogd did for the past four years. + Alternate behavior may be desirable if gateway-like processes send + messages via the local log slot - in this case, it can be enabled + via the $InputUnixListenSocketIgnoreMsgTimestamp and + $SystemLogSocketIgnoreMsgTimestamp config directives +- added ability to compile on HP UX; verified that imudp worked on HP UX; + however, we are still in need of people trying out rsyslogd on HP UX, + so it can not yet be assumed it runs there +- improved session recovery when outbound tcp connection breaks, reduces + probability of message loss at the price of a highly unlikely potential + (single) message duplication +--------------------------------------------------------------------------- +Version 3.12.0 (rgerhards), 2008-02-28 +- added full expression support for filters; filters can now contain + arbitrary complex boolean, string and arithmetic expressions +--------------------------------------------------------------------------- +Version 3.11.6 (rgerhards), 2008-02-27 +- bugfix: gssapi libraries were still linked to rsyslog core, what should + no longer be necessary. Applied fix by Michael Biebl to solve this. +- enabled imgssapi to be loaded side-by-side with imtcp +- added InputGSSServerPermitPlainTCP config directive +- split imgssapi source code somewhat from imtcp +- bugfix: queue cancel cleanup handler could be called with + invalid pointer if dequeue failed +- bugfix: rsyslogd segfaulted on second SIGHUP + tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=38 +- improved stability of queue engine +- bugfix: queue disk file were not properly persisted when + immediately after closing an output file rsyslog was stopped + or huped (the new output file open must NOT have happened at + that point) - this lead to a sparse and invalid queue file + which could cause several problems to the engine (unpredictable + results). This situation should have happened only in very + rare cases. tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=40 +- bugfix: during queue shutdown, an assert invalidly triggered when + the primary queue's DA worker was terminated while the DA queue's + regular worker was still executing. This could result in a segfault + during shutdown. + tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=41 +- bugfix: queue properties sizeOnDisk, bytesRead were persisted to + disk with wrong data type (long instead of int64) - could cause + problems on 32 bit machines +- bugfix: queue aborted when it was shut down, DA-enabled, DA mode + was just initiated but not fully initialized (a race condition) +- bugfix: imfile could abort under extreme stress conditions + (when it was terminated before it could open all of its + to be monitored files) +- applied patch from varmojfekoj to fix an issue with compatibility + mode and default module directories (many thanks!): + I've also noticed a bug in the compatibility code; the problem is that + options are parsed before configuration file so options which need a + module to be loaded will currently ignore any $moddir directive. This + can be fixed by moving legacyOptsHook() after config file parsing. + (see the attached patch) This goes against the logical order of + processing, but the legacy options are only few and it doesn't seem to + be a problem. +- bugfix: object property deserializer did not handle negative numbers +--------------------------------------------------------------------------- +Version 3.11.5 (rgerhards), 2008-02-25 +- new imgssapi module, changed imtcp module - this enables to load/package + GSSAPI support separately - thanks to varmojfekoj for the patch +- compatibility mode (the -c option series) is now at least partly + completed - thanks to varmojfekoj for the patch +- documentation for imgssapi and imtcp added +- duplicate $ModLoad's for the same module are now detected and + rejected -- thanks to varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 3.11.4 (rgerhards), 2008-02-21 +- bugfix: debug.html was missing from release tarball - thanks to Michael + Biebl for bringing this to my attention +- some internal cleanup on the stringbuf object calling interface +- general code cleanup and further modularization +- $MainMessageQueueDiscardSeverity can now also handle textual severities + (previously only integers) +- bugfix: message object was not properly synchronized when the + main queue had a single thread and non-direct action queues were used +- some documentation improvements +--------------------------------------------------------------------------- +Version 3.11.3 (rgerhards), 2008-02-18 +- fixed a bug in imklog which lead to duplicate message content in + kernel logs +- added support for better plugin handling in libdbi (we contributed + a patch to do that, we just now need to wait for the next libdbi + version) +- bugfix: fixed abort when invalid template was provided to an action + bug http://bugzilla.adiscon.com/show_bug.cgi?id=4 +- re-instantiated SIGUSR1 function; added SIGUSR2 to generate debug + status output +- added some documentation on runtime-debug settings +- slightly improved man pages for novice users +--------------------------------------------------------------------------- +Version 3.11.2 (rgerhards), 2008-02-15 +- added the capability to monitor text files and process their content + as syslog messages (including forwarding) +- added support for libdbi, a database abstraction layer. rsyslog now + also supports the following databases via dbi drivers: + * Firebird/Interbase + * FreeTDS (access to MS SQL Server and Sybase) + * SQLite/SQLite3 + * Ingres (experimental) + * mSQL (experimental) + * Oracle (experimental) + Additional drivers may be provided by the libdbi-drivers project, which + can be used by rsyslog as soon as they become available. +- removed some left-over unnecessary dbgprintf's (cluttered screen, + cosmetic) +- doc bugfix: html documentation for omsnmp was missing +--------------------------------------------------------------------------- +Version 3.11.1 (rgerhards), 2008-02-12 +- SNMP trap sender added thanks to Andre Lorbach (omsnmp) +- added input-plugin interface specification in form of a (copy) template + input module +- applied documentation fix by Michael Biebl -- many thanks! +- bugfix: immark did not have MARK flags set... +- added x-info field to rsyslogd startup/shutdown message. Hopefully + points users to right location for further info (many don't even know + they run rsyslog ;)) +- bugfix: trailing ":" of tag was lost while parsing legacy syslog messages + without timestamp - thanks to Anders Blomdell for providing a patch! +- fixed a bug in stringbuf.c related to STRINGBUF_TRIM_ALLOCSIZE, which + wasn't supposed to be used with rsyslog. Put a warning message up that + tells this feature is not tested and probably not worth the effort. + Thanks to Anders Blomdell fro bringing this to our attention +- somewhat improved performance of string buffers +- fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf +- bugfix: setting for $EscapeControlCharactersOnReceive was not + properly initialized +- clarified usage of space-cc property replacer option +- improved abort diagnostic handler +- some initial effort for malloc/free runtime debugging support +- bugfix: using dynafile actions caused rsyslogd abort +- fixed minor man errors thanks to Michael Biebl +--------------------------------------------------------------------------- +Version 3.11.0 (rgerhards), 2008-01-31 +- implemented queued actions +- implemented simple rate limiting for actions +- implemented deliberate discarding of lower priority messages over higher + priority ones when a queue runs out of space +- implemented disk quotas for disk queues +- implemented the $ActionResumeRetryCount config directive +- added $ActionQueueFilename config directive +- added $ActionQueueSize config directive +- added $ActionQueueHighWaterMark config directive +- added $ActionQueueLowWaterMark config directive +- added $ActionQueueDiscardMark config directive +- added $ActionQueueDiscardSeverity config directive +- added $ActionQueueCheckpointInterval config directive +- added $ActionQueueType config directive +- added $ActionQueueWorkerThreads config directive +- added $ActionQueueTimeoutshutdown config directive +- added $ActionQueueTimeoutActionCompletion config directive +- added $ActionQueueTimeoutenQueue config directive +- added $ActionQueueTimeoutworkerThreadShutdown config directive +- added $ActionQueueWorkerThreadMinimumMessages config directive +- added $ActionQueueMaxFileSize config directive +- added $ActionQueueSaveonShutdown config directive +- addded $ActionQueueDequeueSlowdown config directive +- addded $MainMsgQueueDequeueSlowdown config directive +- bugfix: added forgotten docs to package +- improved debugging support +- fixed a bug that caused $MainMsgQueueCheckpointInterval to work incorrectly +- when a long-running action needs to be cancelled on shutdown, the message + that was processed by it is now preserved. This finishes support for + guaranteed delivery of messages (if the output supports it, of course) +- fixed bug in output module interface, see + http://sourceforge.net/tracker/index.php?func=detail&aid=1881008&group_id=123448&atid=696552 +- changed the ommysql output plugin so that the (lengthy) connection + initialization now takes place in message processing. This works much + better with the new queued action mode (fast startup) +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +- bugfixed stream class offset handling on 32bit platforms +--------------------------------------------------------------------------- +Version 3.10.3 (rgerhards), 2008-01-28 +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- run-time instrumentation added +- implemented disk-assisted queue mode, which enables on-demand disk + spooling if the queue's in-memory queue is exhausted +- implemented a dynamic worker thread pool for processing incoming + messages; workers are started and shut down as need arises +- implemented a run-time instrumentation debug package +- implemented the $MainMsgQueueSaveOnShutdown config directive +- implemented the $MainMsgQueueWorkerThreadMinimumMessages config directive +- implemented the $MainMsgQueueTimeoutWorkerThreadShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.2 (rgerhards), 2008-01-14 +- added the ability to keep stop rsyslogd without the need to drain + the main message queue. In disk queue mode, rsyslog continues to + run from the point where it stopped. In case of a system failure, it + continues to process messages from the last checkpoint. +- fixed a bug that caused a segfault on startup when no $WorkDir directive + was specified in rsyslog.conf +- provided more fine-grain control over shutdown timeouts and added a + way to specify the enqueue timeout when the main message queue is full +- implemented $MainMsgQueueCheckpointInterval config directive +- implemented $MainMsgQueueTimeoutActionCompletion config directive +- implemented $MainMsgQueueTimeoutEnqueue config directive +- implemented $MainMsgQueueTimeoutShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.1 (rgerhards), 2008-01-10 +- implemented the "disk" queue mode. However, it currently is of very + limited use, because it does not support persistence over rsyslogd + runs. So when rsyslogd is stopped, the queue is drained just as with + the in-memory queue modes. Persistent queues will be a feature of + the next release. +- performance-optimized string class, should bring an overall improvement +- fixed a memory leak in imudp -- thanks to varmojfekoj for the patch +- fixed a race condition that could lead to a rsyslogd hang when during + HUP or termination +- done some doc updates +- added $WorkDirectory config directive +- added $MainMsgQueueFileName config directive +- added $MainMsgQueueMaxFileSize config directive +--------------------------------------------------------------------------- +Version 3.10.0 (rgerhards), 2008-01-07 +- implemented input module interface and initial input modules +- enhanced threading for input modules (each on its own thread now) +- ability to bind UDP listeners to specific local interfaces/ports and + ability to run multiple of them concurrently +- added ability to specify listen IP address for UDP syslog server +- license changed to GPLv3 +- mark messages are now provided by loadble module immark +- rklogd is no longer provided. Its functionality has now been taken over + by imklog, a loadable input module. This offers a much better integration + into rsyslogd and makes sure that the kernel logger process is brought + up and down at the appropriate times +- enhanced $IncludeConfig directive to support wildcard characters + (thanks to Michael Biebl) +- all inputs are now implemented as loadable plugins +- enhanced threading model: each input module now runs on its own thread +- enhanced message queue which now supports different queueing methods + (among others, this can be used for performance fine-tuning) +- added a large number of new configuration directives for the new + input modules +- enhanced multi-threading utilizing a worker thread pool for the + main message queue +- compilation without pthreads is no longer supported +- much cleaner code due to new objects and removal of single-threading + mode +--------------------------------------------------------------------------- +Version 2.0.8 V2-STABLE (rgerhards), 2008-??-?? +- bugfix: ompgsql did not detect problems in sql command execution + this could cause loss of messages. The handling was correct if the + connection broke, but not if there was a problem with statement + execution. The most probable case for such a case would be invalid + sql inside the template, and this is now much easier to diagnose. +- doc bugfix: default for $DirCreateMode incorrectly stated +--------------------------------------------------------------------------- +Version 2.0.7 V2-STABLE (rgerhards), 2008-04-14 +- bugfix: the default for $DirCreateMode was 0644, and as such wrong. + It has now been changed to 0700. For some background, please see + http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html +- bugfix: "$CreateDirs off" also disabled file creation + Thanks to William Tisater for analyzing this bug and providing a patch. + The actual code change is heavily based on William's patch. +- bugfix: memory leak in ompgsql + Thanks to Ken for providing the patch +- bugfix: potential memory leak in msg.c + This one did not surface yet and the issue was actually found due to + a problem in v4 - but better fix it here, too +--------------------------------------------------------------------------- +Version 2.0.6 V2-STABLE (rgerhards), 2008-08-07 +- bugfix: memory leaks in rsyslogd, primarily in singlethread mode + Thanks to Frederico Nunez for providing the fix +- bugfix: copy&paste error lead to dangling if - this caused a very minor + issue with re-formatting a RFC3164 date when the message was invalidly + formatted and had a colon immediately after the date. This was in the + code for some years (even v1 had it) and I think it never had any + effect at all in practice. Though, it should be fixed - but definitely + nothing to worry about. +--------------------------------------------------------------------------- +Version 2.0.6 V2-STABLE (rgerhards), 2008-08-07 +- bugfix: IPv6 addresses could not be specified in forwarding actions + New syntax @[addr]:port introduced to enable that. Root problem was IPv6 + addresses contain colons. (backport from 3.21.3) +--------------------------------------------------------------------------- +Version 2.0.5 STABLE (rgerhards), 2008-05-15 +- bugfix: regular expressions inside property replacer did not work + properly +- adapted to liblogging 0.7.1+ +--------------------------------------------------------------------------- +Version 2.0.4 STABLE (rgerhards), 2008-03-27 +- bugfix: internally generated messages had "FROMHOST" property not set +- bugfix: continue parsing if tag is oversize (discard oversize part) - thanks + to mclaughlin77@gmail.com for the patch +- added $HHOUR and $QHOUR system properties - can be used for half- and + quarter-hour logfile rotation +--------------------------------------------------------------------------- +Version 2.0.3 STABLE (rgerhards), 2008-03-12 +- bugfix: setting for $EscapeControlCharactersOnReceive was not + properly initialized +- bugfix: resolved potential segfault condition on HUP (extremely + unlikely to happen in practice), for details see tracker: + http://bugzilla.adiscon.com/show_bug.cgi?id=38 +- improved the man pages a bit - thanks to Michael Biebl for the patch +- bugfix: not properly initialized data could cause several segfaults if + there were errors in the config file - thanks to varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 2.0.2 STABLE (rgerhards), 2008-02-12 +- fixed a bug that could cause invalid string handling via strerror_r + varmojfekoj provided the patch - many thanks! +- added x-info field to rsyslogd startup/shutdown message. Hopefully + points users to right location for further info (many don't even know + they run rsyslog ;)) +- bugfix: suspended actions were not always properly resumed + varmojfekoj provided the patch - many thanks! +- bugfix: errno could be changed during mark processing, leading to + invalid error messages when processing inputs. Thank to varmojfekoj for + pointing out this problem. +- bugfix: trailing ":" of tag was lost while parsing legacy syslog messages + without timestamp - thanks to Anders Blomdell for providing a patch! +- bugfix (doc): misspelled config directive, invalid signal info +- applied some doc fixes from Michel Biebl and cleaned up some no longer + needed files suggested by him +- cleaned up stringbuf.c to fix an annoyance reported by Anders Blomdell +- fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf +--------------------------------------------------------------------------- +Version 2.0.1 STABLE (rgerhards), 2008-01-24 +- fixed a bug in integer conversion - but this function was never called, + so it is not really a useful bug fix ;) +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +--------------------------------------------------------------------------- +Version 2.0.0 STABLE (rgerhards), 2008-01-02 +- re-release of 1.21.2 as STABLE with no modifications except some + doc updates +--------------------------------------------------------------------------- +Version 1.21.2 (rgerhards), 2007-12-28 +- created a gss-api output module. This keeps GSS-API code and + TCP/UDP code separated. It is also important for forward- + compatibility with v3. Please note that this change breaks compatibility + with config files created for 1.21.0 and 1.21.1 - this was considered + acceptable. +- fixed an error in forwarding retry code (could lead to message corruption + but surfaced very seldom) +- increased portability for older platforms (AI_NUMERICSERV moved) +- removed socket leak in omfwd.c +- cross-platform patch for GSS-API compile problem on some platforms + thanks to darix for the patch! +--------------------------------------------------------------------------- +Version 1.21.1 (rgerhards), 2007-12-23 +- small doc fix for $IncludeConfig +- fixed a bug in llDestroy() +- bugfix: fixing memory leak when message queue is full and during + parsing. Thanks to varmojfekoj for the patch. +- bugfix: when compiled without network support, unix sockets were + not properly closed +- bugfix: memory leak in cfsysline.c/doGetWord() fixed +--------------------------------------------------------------------------- +Version 1.21.0 (rgerhards), 2007-12-19 +- GSS-API support for syslog/TCP connections was added. Thanks to + varmojfekoj for providing the patch with this functionality +- code cleanup +- enhanced $IncludeConfig directive to support wildcard filenames +- changed some multithreading synchronization +--------------------------------------------------------------------------- +Version 1.20.1 (rgerhards), 2007-12-12 +- corrected a debug setting that survived release. Caused TCP connections + to be retried unnecessarily often. +- When a hostname ACL was provided and DNS resolution for that name failed, + ACL processing was stopped at that point. Thanks to mildew for the patch. + Fedora Bugzilla: http://bugzilla.redhat.com/show_bug.cgi?id=395911 +- fixed a potential race condition, see link for details: + http://rgerhards.blogspot.com/2007/12/rsyslog-race-condition.html + Note that the probability of problems from this bug was very remote +- fixed a memory leak that happened when PostgreSQL date formats were + used +--------------------------------------------------------------------------- +Version 1.20.0 (rgerhards), 2007-12-07 +- an output module for postgres databases has been added. Thanks to + sur5r for contributing this code +- unloading dynamic modules has been cleaned up, we now have a + real implementation and not just a dummy "good enough for the time + being". +- enhanced platform independence - thanks to Bartosz Kuzma and Michael + Biebl for their very useful contributions +- some general code cleanup (including warnings on 64 platforms, only) +--------------------------------------------------------------------------- +Version 1.19.12 (rgerhards), 2007-12-03 +- cleaned up the build system (thanks to Michael Biebl for the patch) +- fixed a bug where ommysql was still not compiled with -pthread option +--------------------------------------------------------------------------- +Version 1.19.11 (rgerhards), 2007-11-29 +- applied -pthread option to build when building for multi-threading mode + hopefully solves an issue with segfaulting +--------------------------------------------------------------------------- +Version 1.19.10 (rgerhards), 2007-10-19 +- introduced the new ":modulename:" syntax for calling module actions + in selector lines; modified ommysql to support it. This is primarily + an aid for further modules and a prerequisite to actually allow third + party modules to be created. +- minor fix in slackware startup script, "-r 0" is now "-r0" +- updated rsyslogd doc set man page; now in html format +- undid creation of a separate thread for the main loop -- this did not + turn out to be needed or useful, so reduce complexity once again. +- added doc fixes provided by Michael Biebl - thanks +--------------------------------------------------------------------------- +Version 1.19.9 (rgerhards), 2007-10-12 +- now packaging system which again contains all components in a single + tarball +- modularized main() a bit more, resulting in less complex code +- experimentally added an additional thread - will see if that affects + the segfault bug we experience on some platforms. Note that this change + is scheduled to be removed again later. +--------------------------------------------------------------------------- +Version 1.19.8 (rgerhards), 2007-09-27 +- improved repeated message processing +- applied patch provided by varmojfekoj to support building ommysql + in its own way (now also resides in a plugin subdirectory); + ommysql is now a separate package +- fixed a bug in cvthname() that lead to message loss if part + of the source hostname would have been dropped +- created some support for distributing ommysql together with the + main rsyslog package. I need to re-think it in the future, but + for the time being the current mode is best. I now simply include + one additional tarball for ommysql inside the main distribution. + I look forward to user feedback on how this should be done best. In the + long term, a separate project should be spawend for ommysql, but I'd + like to do that only after the plugin interface is fully stable (what + it is not yet). +--------------------------------------------------------------------------- +Version 1.19.7 (rgerhards), 2007-09-25 +- added code to handle situations where senders send us messages ending with + a NUL character. It is now simply removed. This also caused trailing LF + reduction to fail, when it was followed by such a NUL. This is now also + handled. +- replaced some non-thread-safe function calls by their thread-safe + counterparts +- fixed a minor memory leak that occurred when the %APPNAME% property was + used (I think nobody used that in practice) +- fixed a bug that caused signal handlers in cvthname() not to be restored when + a malicious pointer record was detected and processing of the message been + stopped for that reason (this should be really rare and can not be related + to the segfault bug we are hunting). +- fixed a bug in cvthname that lead to passing a wrong parameter - in + practice, this had no impact. +- general code cleanup (e.g. compiler warnings, comments) +--------------------------------------------------------------------------- +Version 1.19.6 (rgerhards), 2007-09-11 +- applied patch by varmojfekoj to change signal handling to the new + sigaction API set (replacing the depreciated signal() calls and its + friends. +- fixed a bug that in --enable-debug mode caused an assertion when the + discard action was used +- cleaned up compiler warnings +- applied patch by varmojfekoj to FIX a bug that could cause + segfaults if empty properties were processed using modifying + options (e.g. space-cc, drop-cc) +- fixed man bug: rsyslogd supports -l option +--------------------------------------------------------------------------- +Version 1.19.5 (rgerhards), 2007-09-07 +- changed part of the CStr interface so that better error tracking + is provided and the calling sequence is more intuitive (there were + invalid calls based on a too-weird interface) +- (hopefully) fixed some remaining bugs rooted in wrong use of + the CStr class. These could lead to program abort. +- applied patch by varmojfekoj two fix two potential segfault situations +- added $ModDir config directive +- modified $ModLoad so that an absolute path may be specified as + module name (e.g. /rsyslog/ommysql.so) +--------------------------------------------------------------------------- +Version 1.19.4 (rgerhards/varmojfekoj), 2007-09-04 +- fixed a number of small memory leaks - thanks varmojfekoj for patching +- fixed an issue with CString class that could lead to rsyslog abort + in tplToString() - thanks varmojfekoj for patching +- added a man-version of the config file documentation - thanks to Michel + Samia for providing the man file +- fixed bug: a template like this causes an infinite loop: + $template opts,"%programname:::a,b%" + thanks varmojfekoj for the patch +- fixed bug: case changing options crash freeing the string pointer + because they modify it: $template opts2,"%programname::1:lowercase%" + thanks varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 1.19.3 (mmeckelein/varmojfekoj), 2007-08-31 +- small mem leak fixed (after calling parseSelectorAct) - Thx varmojfekoj +- documentation section "Regular File" und "Blocks" updated +- solved an issue with dynamic file generation - Once again many thanks + to varmojfekoj +- the negative selector for program name filter (Blocks) does not work as + expected - Thanks varmojfekoj for patching +- added forwarding information to sysklogd (requires special template) + to config doc +--------------------------------------------------------------------------- +Version 1.19.2 (mmeckelein/varmojfekoj), 2007-08-28 +- a specifically formed message caused a segfault - Many thanks varmojfekoj + for providing a patch +- a typo and a weird condition are fixed in msg.c - Thanks again + varmojfekoj +- on file creation the file was always owned by root:root. This is fixed + now - Thanks ypsa for solving this issue +--------------------------------------------------------------------------- +Version 1.19.1 (mmeckelein), 2007-08-22 +- a bug that caused a high load when a TCP/UDP connection was closed is + fixed now - Thanks mildew for solving this issue +- fixed a bug which caused a segfault on reinit - Thx varmojfekoj for the + patch +- changed the hardcoded module path "/lib/rsyslog" to $(pkglibdir) in order + to avoid trouble e.g. on 64 bit platforms (/lib64) - many thanks Peter + Vrabec and darix, both provided a patch for solving this issue +- enhanced the unloading of modules - thanks again varmojfekoj +- applied a patch from varmojfekoj which fixes various little things in + MySQL output module +--------------------------------------------------------------------------- +Version 1.19.0 (varmojfekoj/rgerhards), 2007-08-16 +- integrated patch from varmojfekoj to make the mysql module a loadable one + many thanks for the patch, MUCH appreciated +--------------------------------------------------------------------------- +Version 1.18.2 (rgerhards), 2007-08-13 +- fixed a bug in outchannel code that caused templates to be incorrectly + parsed +- fixed a bug in ommysql that caused a wrong ";template" missing message +- added some code for unloading modules; not yet fully complete (and we do + not yet have loadable modules, so this is no problem) +- removed debian subdirectory by request of a debian packager (this is a special + subdir for debian and there is also no point in maintaining it when there + is a debian package available - so I gladly did this) in some cases +- improved overall doc quality (some pages were quite old) and linked to + more of the online resources. +- improved /contrib/delete_mysql script by adding a host option and some + other minor modifications +--------------------------------------------------------------------------- +Version 1.18.1 (rgerhards), 2007-08-08 +- applied a patch from varmojfekoj which solved a potential segfault + of rsyslogd on HUP +- applied patch from Michel Samia to fix compilation when the pthreads + feature is disabled +- some code cleanup (moved action object to its own file set) +- add config directive $MainMsgQueueSize, which now allows one to configure the + queue size dynamically +- all compile-time settings are now shown in rsyslogd -v, not just the + active ones +- enhanced performance a little bit more +- added config file directive $ActionResumeInterval +- fixed a bug that prevented compilation under debian sid +- added a contrib directory for user-contributed useful things +--------------------------------------------------------------------------- +Version 1.18.0 (rgerhards), 2007-08-03 +- rsyslog now supports fallback actions when an action did not work. This + is a great feature e.g. for backup database servers or backup syslog + servers +- modified rklogd to only change the console log level if -c is specified +- added feature to use multiple actions inside a single selector +- implemented $ActionExecOnlyWhenPreviousIsSuspended config directive +- error messages during startup are now spit out to the configured log + destinations +--------------------------------------------------------------------------- +Version 1.17.6 (rgerhards), 2007-08-01 +- continued to work on output module modularization - basic stage of + this work is now FINISHED +- fixed bug in OMSRcreate() - always returned SR_RET_OK +- fixed a bug that caused ommysql to always complain about missing + templates +- fixed a mem leak in OMSRdestruct - freeing the object itself was + forgotten - thanks to varmojfekoj for the patch +- fixed a memory leak in syslogd/init() that happened when the config + file could not be read - thanks to varmojfekoj for the patch +- fixed insufficient memory allocation in addAction() and its helpers. + The initial fix and idea was developed by mildew, I fine-tuned + it a bit. Thanks a lot for the fix, I'd probably had pulled out my + hair to find the bug... +- added output of config file line number when a parsing error occurred +- fixed bug in objomsr.c that caused program to abort in debug mode with + an invalid assertion (in some cases) +- fixed a typo that caused the default template for MySQL to be wrong. + thanks to mildew for catching this. +- added configuration file command $DebugPrintModuleList and + $DebugPrintCfSysLineHandlerList +- fixed an invalid value for the MARK timer - unfortunately, there was + a testing aid left in place. This resulted in quite frequent MARK messages +- added $IncludeConfig config directive +- applied a patch from mildew to prevent rsyslogd from freezing under heavy + load. This could happen when the queue was full. Now, we drop messages + but rsyslogd remains active. +--------------------------------------------------------------------------- +Version 1.17.5 (rgerhards), 2007-07-30 +- continued to work on output module modularization +- fixed a missing file bug - thanks to Andrea Montanari for reporting + this problem +- fixed a problem with shutting down the worker thread and freeing the + selector_t list - this caused messages to be lost, because the + message queue was not properly drained before the selectors got + destroyed. +--------------------------------------------------------------------------- +Version 1.17.4 (rgerhards), 2007-07-27 +- continued to work on output module modularization +- fixed a situation where rsyslogd could create zombie processes + thanks to mildew for the patch +- applied patch from Michel Samia to fix compilation when NOT + compiled for pthreads +--------------------------------------------------------------------------- +Version 1.17.3 (rgerhards), 2007-07-25 +- continued working on output module modularization +- fixed a bug that caused rsyslogd to segfault on exit (and + probably also on HUP), when there was an unsent message in a selector + that required forwarding and the dns lookup failed for that selector + (yes, it was pretty unlikely to happen;)) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed a memory leak in config file parsing and die() + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- rsyslogd now checks on startup if it is capable to perform any work + at all. If it cant, it complains and terminates + thanks to Michel Samia for providing the patch! +- fixed a small memory leak when HUPing syslogd. The allowed sender + list now gets freed. thanks to mildew for the patch. +- changed the way error messages in early startup are logged. They + now do no longer use the syslogd code directly but are rather + send to stderr. +--------------------------------------------------------------------------- +Version 1.17.2 (rgerhards), 2007-07-23 +- made the port part of the -r option optional. Needed for backward + compatibility with sysklogd +- replaced system() calls with something more reasonable. Please note that + this might break compatibility with some existing configuration files. + We accept this in favor of the gained security. +- removed a memory leak that could occur if timegenerated was used in + RFC 3164 format in templates +- did some preparation in msg.c for advanced multithreading - placed the + hooks, but not yet any active code +- worked further on modularization +- added $ModLoad MySQL (dummy) config directive +- added DropTrailingLFOnReception config directive +--------------------------------------------------------------------------- +Version 1.17.1 (rgerhards), 2007-07-20 +- fixed a bug that caused make install to install rsyslogd and rklogd under + the wrong names +- fixed bug that caused $AllowedSenders to handle IPv6 scopes incorrectly; + also fixed but that could garble $AllowedSender wildcards. Thanks to + mildew@gmail.com for the patch +- minor code cleanup - thanks to Peter Vrabec for the patch +- fixed minimal memory leak on HUP (caused by templates) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed another memory leak on HUPing and on exiting rsyslogd + again thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- code cleanup (removed compiler warnings) +- fixed portability bug in configure.ac - thanks to Bartosz Kuźma for patch +- moved msg object into its own file set +- added the capability to continue trying to write log files when the + file system is full. Functionality based on patch by Martin Schulze + to sysklogd package. +--------------------------------------------------------------------------- +Version 1.17.0 (RGer), 2007-07-17 +- added $RepeatedLineReduction config parameter +- added $EscapeControlCharactersOnReceive config parameter +- added $ControlCharacterEscapePrefix config parameter +- added $DirCreateMode config parameter +- added $CreateDirs config parameter +- added $DebugPrintTemplateList config parameter +- added $ResetConfigVariables config parameter +- added $FileOwner config parameter +- added $FileGroup config parameter +- added $DirOwner config parameter +- added $DirGroup config parameter +- added $FailOnChownFailure config parameter +- added regular expression support to the filter engine + thanks to Michel Samia for providing the patch! +- enhanced $AllowedSender functionality. Credits to mildew@gmail.com for + the patch doing that + - added IPv6 support + - allowed DNS hostnames + - allowed DNS wildcard names +- added new option $DropMsgsWithMaliciousDnsPTRRecords +- added autoconf so that rfc3195d, rsyslogd and klogd are stored to /sbin +- added capability to auto-create directories with dynaFiles +--------------------------------------------------------------------------- +Version 1.16.0 (RGer/Peter Vrabec), 2007-07-13 - The Friday, 13th Release ;) +- build system switched to autotools +- removed SYSV preprocessor macro use, replaced with autotools equivalents +- fixed a bug that caused rsyslogd to segfault when TCP listening was + disabled and it terminated +- added new properties "syslogfacility-text" and "syslogseverity-text" + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- added the -x option to disable hostname dns resolution + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- begun to better modularize syslogd.c - this is an ongoing project; moved + type definitions to a separate file +- removed some now-unused fields from struct filed +- move file size limit fields in struct field to the "right spot" (the file + writing part of the union - f_un.f_file) +- subdirectories linux and solaris are no longer part of the distribution + package. This is not because we cease support for them, but there are no + longer any files in them after the move to autotools +--------------------------------------------------------------------------- +Version 1.15.1 (RGer), 2007-07-10 +- fixed a bug that caused a dynaFile selector to stall when there was + an open error with one file +- improved template processing for dynaFiles; templates are now only + looked up during initialization - speeds up processing +- optimized memory layout in struct filed when compiled with MySQL + support +- fixed a bug that caused compilation without SYSLOG_INET to fail +- re-enabled the "last message repeated n times" feature. This + feature was not taken care of while rsyslogd evolved from sysklogd + and it was more or less defunct. Now it is fully functional again. +- added system properties: $NOW, $YEAR, $MONTH, $DAY, $HOUR, $MINUTE +- fixed a bug in iovAsString() that caused a memory leak under stress + conditions (most probably memory shortage). This was unlikely to + ever happen, but it doesn't hurt doing it right +- cosmetic: defined type "uchar", change all unsigned chars to uchar +--------------------------------------------------------------------------- +Version 1.15.0 (RGer), 2007-07-05 +- added ability to dynamically generate file names based on templates + and thus properties. This was a much-requested feature. It makes + life easy when it e.g. comes to splitting files based on the sender + address. +- added $umask and $FileCreateMode config file directives +- applied a patch from Bartosz Kuzma to compile cleanly under NetBSD +- checks for extra (unexpected) characters in system config file lines + have been added +- added IPv6 documentation - was accidentally missing from CVS +- begun to change char to unsigned char +--------------------------------------------------------------------------- +Version 1.14.2 (RGer), 2007-07-03 +** this release fixes all known nits with IPv6 ** +- restored capability to do /etc/service lookup for "syslog" + service when -r 0 was given +- documented IPv6 handling of syslog messages +- integrate patch from Bartosz Kuźma to make rsyslog compile under + Solaris again (the patch replaced a strndup() call, which is not + available under Solaris +- improved debug logging when waiting on select +- updated rsyslogd man page with new options (-46A) +--------------------------------------------------------------------------- +Version 1.14.1 (RGer/Peter Vrabec), 2007-06-29 +- added Peter Vrabec's patch for IPv6 TCP +- prefixed all messages send to stderr in rsyslogd with "rsyslogd: " +--------------------------------------------------------------------------- +Version 1.14.0 (RGer/Peter Vrabec), 2007-06-28 +- Peter Vrabec provided IPv6 for rsyslog, so we are now IPv6 enabled + IPv6 Support is currently for UDP only, TCP is to come soon. + AllowedSender configuration does not yet work for IPv6. +- fixed code in iovCreate() that broke C's strict aliasing rules +- fixed some char/unsigned char differences that forced the compiler + to spit out warning messages +- updated the Red Hat init script to fix a known issue (thanks to + Peter Vrabec) +--------------------------------------------------------------------------- +Version 1.13.5 (RGer), 2007-06-22 +- made the TCP session limit configurable via command line switch + now -t <port>,<max sessions> +- added man page for rklogd(8) (basically a copy from klogd, but now + there is one...) +- fixed a bug that caused internal messages (e.g. rsyslogd startup) to + appear without a tag. +- removed a minor memory leak that occurred when TAG processing requalified + a HOSTNAME to be a TAG (and a TAG already was set). +- removed potential small memory leaks in MsgSet***() functions. There + would be a leak if a property was re-set, something that happened + extremely seldom. +--------------------------------------------------------------------------- +Version 1.13.4 (RGer), 2007-06-18 +- added a new property "PRI-text", which holds the PRI field in + textual form (e.g. "syslog.info") +- added alias "syslogseverity" for "syslogpriority", which is a + misleading property name that needs to stay for historical + reasons (and backward-compatibility) +- added doc on how to record PRI value in log file +- enhanced signal handling in klogd, including removal of an unsafe + call to the logging system during signal handling +--------------------------------------------------------------------------- +Version 1.13.3 (RGer), 2007-06-15 +- create a version of syslog.c from scratch. This is now + - highly optimized for rsyslog + - removes an incompatible license problem as the original + version had a BSD license with advertising clause + - fixed in the regard that rklogd will continue to work when + rsyslogd has been restarted (the original version, as well + as sysklogd, will remain silent then) + - solved an issue with an extra NUL char at message end that the + original version had +- applied some changes to klogd to care for the new interface +- fixed a bug in syslogd.c which prevented compiling under debian +--------------------------------------------------------------------------- +Version 1.13.2 (RGer), 2007-06-13 +- lib order in makefile patched to facilitate static linking - thanks + to Bennett Todd for providing the patch +- Integrated a patch from Peter Vrabec (pvrabec@redhat.com): + - added klogd under the name of rklogd (remove dependency on + original sysklogd package + - createDB.sql now in UTF + - added additional config files for use on Red Hat +--------------------------------------------------------------------------- +Version 1.13.1 (RGer), 2007-02-05 +- changed the listen backlog limit to a more reasonable value based on + the maximum number of TCP connections configured (10% + 5) - thanks to Guy + Standen for the hint (actually, the limit was 5 and that was a + left-over from early testing). +- fixed a bug in makefile which caused DB-support to be disabled when + NETZIP support was enabled +- added the -e option to allow transmission of every message to remote + hosts (effectively turns off duplicate message suppression) +- (somewhat) improved memory consumption when compiled with MySQL support +- looks like we fixed an incompatibility with MySQL 5.x and above software + At least in one case, the remote server name was destroyed, leading to + a connection failure. The new, improved code does not have this issue and + so we see this as solved (the new code is generally somewhat better, so + there is a good chance we fixed this incompatibility). +--------------------------------------------------------------------------- +Version 1.13.0 (RGer), 2006-12-19 +- added '$' as ToPos property replacer specifier - means "up to the + end of the string" +- property replacer option "escape-cc", "drop-cc" and "space-cc" added +- changed the handling of \0 characters inside syslog messages. We now + consistently escape them to "#000". This is somewhat recommended in + the draft-ietf-syslog-protocol-19 draft. While the real recommendation + is to not escape any characters at all, we can not do this without + considerable modification of the code. So we escape it to "#000", which + is consistent with a sample found in the Internet-draft. +- removed message glue logic (see printchopped() comment for details) + Also caused removal of parts table and thus some improvements in + memory usage. +- changed the default MAXLINE to 2048 to take care of recent syslog + standardization efforts (can easily be changed in syslogd.c) +- added support for byte-counted TCP syslog messages (much like + syslog-transport-tls-05 Internet Draft). This was necessary to + support compression over TCP. +- added support for receiving compressed syslog messages +- added support for sending compressed syslog messages +- fixed a bug where the last message in a syslog/tcp stream was + lost if it was not properly terminated by a LF character +--------------------------------------------------------------------------- +Version 1.12.3 (RGer), 2006-10-04 +- implemented some changes to support Solaris (but support is not + yet complete) +- commented out (via #if 0) some methods that are currently not being use + but should be kept for further us +- added (interim) -u 1 option to turn off hostname and tag parsing +- done some modifications to better support Fedora +- made the field delimiter inside property replace configurable via + template +- fixed a bug in property replacer: if fields were used, the delimitor + became part of the field. Up until now, this was barely noticeable as + the delimiter as TAB only and thus invisible to a human. With other + delimiters available now, it quickly showed up. This bug fix might cause + some grief to existing installations if they used the extra TAB for + whatever reasons - sorry folks... Anyhow, a solution is easy: just add + a TAB character constant into your template. Thus, there has no attempt + been made to do this in a backwards-compatible way. +--------------------------------------------------------------------------- +Version 1.12.2 (RGer), 2006-02-15 +- fixed a bug in the RFC 3339 date formatter. An extra space was added + after the actual timestamp +- added support for providing high-precision RFC3339 timestamps for + (rsyslogd-)internally-generated messages +- very (!) experimental support for syslog-protocol internet draft + added (the draft is experimental, the code is solid ;)) +- added support for field-extracting in the property replacer +- enhanced the legacy-syslog parser so that it can interpret messages + that do not contain a TIMESTAMP +- fixed a bug that caused the default socket (usually /dev/log) to be + opened even when -o command line option was given +- fixed a bug in the Debian sample startup script - it caused rsyslogd + to listen to remote requests, which it shouldn't by default +--------------------------------------------------------------------------- +Version 1.12.1 (RGer), 2005-11-23 +- made multithreading work with BSD. Some signal-handling needed to be + restructured. Also, there might be a slight delay of up to 10 seconds + when huping and terminating rsyslogd under BSD +- fixed a bug where a NULL-pointer was passed to printf() in logmsg(). +- fixed a bug during "make install" where rc3195d was not installed + Thanks to Bennett Todd for spotting this. +- fixed a bug where rsyslogd dumped core when no TAG was found in the + received message +- enhanced message parser so that it can deal with missing hostnames + in many cases (may not be totally fail-safe) +- fixed a bug where internally-generated messages did not have the correct + TAG +--------------------------------------------------------------------------- +Version 1.12.0 (RGer), 2005-10-26 +- moved to a multi-threaded design. single-threading is still optionally + available. Multi-threading is experimental! +- fixed a potential race condition. In the original code, marking was done + by an alarm handler, which could lead to all sorts of bad things. This + has been changed now. See comments in syslogd.c/domark() for details. +- improved debug output for property-based filters +- not a code change, but: I have checked all exit()s to make sure that + none occurs once rsyslogd has started up. Even in unusual conditions + (like low-memory conditions) rsyslogd somehow remains active. Of course, + it might loose a message or two, but at least it does not abort and it + can also recover when the condition no longer persists. +- fixed a bug that could cause loss of the last message received + immediately before rsyslogd was terminated. +- added comments on thread-safety of global variables in syslogd.c +- fixed a small bug: spurios printf() when TCP syslog was used +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug with regular expression support (thanks to Andres Riancho) +- a little bit of code restructuring (especially main(), which was + horribly large) +--------------------------------------------------------------------------- +Version 1.11.1 (RGer), 2005-10-19 +- support for BSD-style program name and host blocks +- added a new property "programname" that can be used in templates +- added ability to specify listen port for rfc3195d +- fixed a bug that rendered the "startswith" comparison operation + unusable. +- changed more functions to "static" storage class to help compiler + optimize (should have been static in the first place...) +- fixed a potential memory leak in the string buffer class destructor. + As the destructor was previously never called, the leak did not actually + appear. +- some internal restructuring in anticipation/preparation of minimal + multi-threading support +- rsyslogd still shares some code with the sysklogd project. Some patches + for this shared code have been brought over from the sysklogd CVS. +--------------------------------------------------------------------------- +Version 1.11.0 (RGer), 2005-10-12 +- support for receiving messages via RFC 3195; added rfc3195d for that + purpose +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +--------------------------------------------------------------------------- +Version 1.10.2 (RGer), 2005-09-27 +- added comparison operations in property-based filters: + * isequal + * startswith +- added ability to negate all property-based filter comparison operations + by adding a !-sign right in front of the operation name +- added the ability to specify remote senders for UDP and TCP + received messages. Allows to block all but well-known hosts +- changed the $-config line directives to be case-INsensitive +- new command line option -w added: "do not display warnings if messages + from disallowed senders are received" +- fixed a bug that caused rsyslogd to dump core when the compare value + was not quoted in property-based filters +- fixed a bug in the new CStr compare function which lead to invalid + results (fortunately, this function was not yet used widely) +- added better support for "debugging" rsyslog.conf property filters + (only if -d switch is given) +- changed some function definitions to static, which eventually enables + some compiler optimizations +- fixed a bug in MySQL code; when a SQL error occurred, rsyslogd could + run in a tight loop. This was due to invalid sequence of error reporting + and is now fixed. +--------------------------------------------------------------------------- +Version 1.10.1 (RGer), 2005-09-23 +- added the ability to execute a shell script as an action. + Thanks to Bjoern Kalkbrenner for providing the code! +- fixed a bug in the MySQL code; due to the bug the automatic one-time + retry after an error did not happen - this lead to error message in + cases where none should be seen (e.g. after a MySQL restart) +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.10.0 (RGer), 2005-09-20 + REMINDER: 1.10 is the first unstable version if the 1.x series! +- added the capability to filter on any property in selector lines + (not just facility and priority) +- changed stringbuf into a new counted string class +- added support for a "discard" action. If a selector line with + discard (~ character) is found, no selector lines *after* that + line will be processed. +- thanks to Andres Riancho, regular expression support has been + added to the template engine +- added the FROMHOST property in the template processor, which could + previously not be obtained. Thanks to Cristian Testa for pointing + this out and even providing a fix. +- added display of compile-time options to -v output +- performance improvement for production build - made some checks + to happen only during debug mode +- fixed a problem with compiling on SUSE and - while doing so - removed + the socket call to set SO_BSDCOMPAT in cases where it is obsolete. +--------------------------------------------------------------------------- +Version 1.0.4 (RGer), 2006-02-01 +- a small but important fix: the tcp receiver had two forgotten printf's + in it that caused a lot of unnecessary output to stdout. This was + important enough to justify a new release +--------------------------------------------------------------------------- +Version 1.0.3 (RGer), 2005-11-14 +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +- applied some patches available from the sysklogd project to code + shared from there +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug in the TCP sender that caused the retry logic to fail + after an error or receiver overrun +- fixed a bug in init() that could lead to dumping core +- fixed a bug that could lead to dumping core when no HOSTNAME or no TAG + was present in the syslog message +--------------------------------------------------------------------------- +Version 1.0.2 (RGer), 2005-10-05 +- fixed an issue with MySQL error reporting. When an error occurred, + the MySQL driver went into an endless loop (at least in most cases). +--------------------------------------------------------------------------- +Version 1.0.1 (RGer), 2005-09-23 +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.0.0 (RGer), 2005-09-12 +- changed install doc to cover daily cron scripts - a trouble source +- added rc script for slackware (provided by Chris Elvidge - thanks!) +- fixed a really minor bug in usage() - the -r option was still + reported as without the port parameter +--------------------------------------------------------------------------- +Version 0.9.8 (RGer), 2005-09-05 +- made startup and shutdown message more consistent and included the + pid, so that they can be easier correlated. Used syslog-protocol + structured data format for this purpose. +- improved config info in startup message, now tells not only + if it is listening remote on udp, but also for tcp. Also includes + the port numbers. The previous startup message was misleading, because + it did not say "remote reception" if rsyslogd was only listening via + tcp (but not via udp). +- added a "how can you help" document to the doc set +--------------------------------------------------------------------------- +Version 0.9.7 (RGer), 2005-08-15 +- some of the previous doc files (like INSTALL) did not properly + reflect the changes to the build process and the new doc. Fixed + that. +- changed syslogd.c so that when compiled without database support, + an error message is displayed when a database action is detected + in the config file (previously this was used as an user rule ;)) +- fixed a bug in the os-specific Makefiles which caused MySQL + support to not be compiled, even if selected +--------------------------------------------------------------------------- +Version 0.9.6 (RGer), 2005-08-09 +- greatly enhanced documentation. Now available in html format in + the "doc" folder and FreeBSD. Finally includes an install howto. +- improved MySQL error messages a little - they now show up as log + messages, too (formerly only in debug mode) +- added the ability to specify the listen port for udp syslog. + WARNING: This introduces an incompatibility. Formerly, udp + syslog was enabled by the -r command line option. Now, it is + "-r [port]", which is consistent with the tcp listener. However, + just -r will now return an error message. +- added sample startup scripts for Debian and FreeBSD +- added support for easy feature selection in the makefile. Un- + fortunately, this also means I needed to spilt the make file + for different OS and distros. There are some really bad syntax + differences between FreeBSD and Linux make. +--------------------------------------------------------------------------- +Version 0.9.5 (RGer), 2005-08-01 +- the "semicolon bug" was actually not (fully) solved in 0.9.4. One + part of the bug was solved, but another still existed. This one + is fixed now, too. +- the "semicolon bug" actually turned out to be a more generic bug. + It appeared whenever an invalid template name was given. With some + selector actions, rsyslogd dumped core, with other it "just" had + a small resource leak with others all worked well. These anomalies + are now fixed. Note that they only appeared during system initialization + once the system was running, nothing bad happened. +- improved error reporting for template errors on startup. They are now + shown on the console and the start-up tty. Formerly, they were only + visible in debug mode. +- support for multiple instances of rsyslogd on a single machine added +- added new option "-o" --> omit local unix domain socket. This option + enables rsyslogd NOT to listen to the local socket. This is most + helpful when multiple instances of rsyslogd (or rsyslogd and another + syslogd) shall run on a single system. +- added new option "-i <pidfile>" which allows one to specify the pidfile. + This is needed when multiple instances of rsyslogd are to be run. +- the new project home page is now online at www.rsyslog.com +--------------------------------------------------------------------------- +Version 0.9.4 (RGer), 2005-07-25 +- finally added the TCP sender. It now supports non-blocking mode, no + longer disabling message reception during connect. As it is now, it + is usable in production. The code could be more sophisticated, but + I've kept it short in anticipation of the move to liblogging, which + will lead to the removal of the code just written ;) +- the "exiting on signal..." message still had the "syslogd" name in + it. Changed this to "rsyslogd", as we do not have a large user base + yet, this should pose no problem. +- fixed "the semicolon" bug. rsyslogd dumped core if a write-db action + was specified but no semicolon was given after the password (an empty + template was ok, but the semicolon needed to be present). +- changed a default for traditional output format. During testing, it + was seen that the timestamp written to file in default format was + the time of message reception, not the time specified in the TIMESTAMP + field of the message itself. Traditionally, the message TIMESTAMP is + used and this has been changed now. +--------------------------------------------------------------------------- +Version 0.9.3 (RGer), 2005-07-19 +- fixed a bug in the message parser. In June, the RFC 3164 timestamp + was not correctly parsed (yes, only in June and some other months, + see the code comment to learn why...) +- added the ability to specify the destination port when forwarding + syslog messages (both for TCP and UDP) +- added an very experimental TCP sender (activated by + @@machine:port in config). This is not yet for production use. If + the receiver is not alive, rsyslogd will wait quite some time until + the connection request times out, which most probably leads to + loss of incoming messages. + +--------------------------------------------------------------------------- +Version 0.9.2 (RGer), around 2005-07-06 +- I intended to change the maxsupported message size to 32k to + support IHE - but given the memory inefficiency in the usual use + cases, I have not done this. I have, however, included very + specific instructions on how to do this in the source code. I have + also done some testing with 32k messages, so you can change the + max size without taking too much risk. +- added a syslog/tcp receiver; we now can receive messages via + plain tcp, but we can still send only via UDP. The syslog/tcp + receiver is the primary enhancement of this release. +- slightly changed some error messages that contained a spurios \n at + the end of the line (which gives empty lines in your log...) + +--------------------------------------------------------------------------- +Version 0.9.1 (RGer) +- fixed code so that it compiles without errors under FreeBSD +- removed now unused function "allocate_log()" from syslogd.c +- changed the make file so that it contains more defines for + different environments (in the long term, we need a better + system for disabling/enabling features...) +- changed some printf's printing off_t types to %lld and + explicit (long long) casts. I tried to figure out the exact type, + but did not succeed in this. In the worst case, ultra-large peta- + byte files will now display funny informational messages on rollover, + something I think we can live with for the neersion 3.11.2 (rgerhards), 2008-02-?? +--------------------------------------------------------------------------- +Version 3.11.1 (rgerhards), 2008-02-12 +- SNMP trap sender added thanks to Andre Lorbach (omsnmp) +- added input-plugin interface specification in form of a (copy) template + input module +- applied documentation fix by Michael Biebl -- many thanks! +- bugfix: immark did not have MARK flags set... +- added x-info field to rsyslogd startup/shutdown message. Hopefully + points users to right location for further info (many don't even know + they run rsyslog ;)) +- bugfix: trailing ":" of tag was lost while parsing legacy syslog messages + without timestamp - thanks to Anders Blomdell for providing a patch! +- fixed a bug in stringbuf.c related to STRINGBUF_TRIM_ALLOCSIZE, which + wasn't supposed to be used with rsyslog. Put a warning message up that + tells this feature is not tested and probably not worth the effort. + Thanks to Anders Blomdell fro bringing this to our attention +- somewhat improved performance of string buffers +- fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf +- bugfix: setting for $EscapeControlCharactersOnReceive was not + properly initialized +- clarified usage of space-cc property replacer option +- improved abort diagnostic handler +- some initial effort for malloc/free runtime debugging support +- bugfix: using dynafile actions caused rsyslogd abort +- fixed minor man errors thanks to Michael Biebl +--------------------------------------------------------------------------- +Version 3.11.0 (rgerhards), 2008-01-31 +- implemented queued actions +- implemented simple rate limiting for actions +- implemented deliberate discarding of lower priority messages over higher + priority ones when a queue runs out of space +- implemented disk quotas for disk queues +- implemented the $ActionResumeRetryCount config directive +- added $ActionQueueFilename config directive +- added $ActionQueueSize config directive +- added $ActionQueueHighWaterMark config directive +- added $ActionQueueLowWaterMark config directive +- added $ActionQueueDiscardMark config directive +- added $ActionQueueDiscardSeverity config directive +- added $ActionQueueCheckpointInterval config directive +- added $ActionQueueType config directive +- added $ActionQueueWorkerThreads config directive +- added $ActionQueueTimeoutshutdown config directive +- added $ActionQueueTimeoutActionCompletion config directive +- added $ActionQueueTimeoutenQueue config directive +- added $ActionQueueTimeoutworkerThreadShutdown config directive +- added $ActionQueueWorkerThreadMinimumMessages config directive +- added $ActionQueueMaxFileSize config directive +- added $ActionQueueSaveonShutdown config directive +- addded $ActionQueueDequeueSlowdown config directive +- addded $MainMsgQueueDequeueSlowdown config directive +- bugfix: added forgotten docs to package +- improved debugging support +- fixed a bug that caused $MainMsgQueueCheckpointInterval to work incorrectly +- when a long-running action needs to be cancelled on shutdown, the message + that was processed by it is now preserved. This finishes support for + guaranteed delivery of messages (if the output supports it, of course) +- fixed bug in output module interface, see + http://sourceforge.net/tracker/index.php?func=detail&aid=1881008&group_id=123448&atid=696552 +- changed the ommysql output plugin so that the (lengthy) connection + initialization now takes place in message processing. This works much + better with the new queued action mode (fast startup) +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +- bugfixed stream class offset handling on 32bit platforms +--------------------------------------------------------------------------- +Version 3.10.3 (rgerhards), 2008-01-28 +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- run-time instrumentation added +- implemented disk-assisted queue mode, which enables on-demand disk + spooling if the queue's in-memory queue is exhausted +- implemented a dynamic worker thread pool for processing incoming + messages; workers are started and shut down as need arises +- implemented a run-time instrumentation debug package +- implemented the $MainMsgQueueSaveOnShutdown config directive +- implemented the $MainMsgQueueWorkerThreadMinimumMessages config directive +- implemented the $MainMsgQueueTimeoutWorkerThreadShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.2 (rgerhards), 2008-01-14 +- added the ability to keep stop rsyslogd without the need to drain + the main message queue. In disk queue mode, rsyslog continues to + run from the point where it stopped. In case of a system failure, it + continues to process messages from the last checkpoint. +- fixed a bug that caused a segfault on startup when no $WorkDir directive + was specified in rsyslog.conf +- provided more fine-grain control over shutdown timeouts and added a + way to specify the enqueue timeout when the main message queue is full +- implemented $MainMsgQueueCheckpointInterval config directive +- implemented $MainMsgQueueTimeoutActionCompletion config directive +- implemented $MainMsgQueueTimeoutEnqueue config directive +- implemented $MainMsgQueueTimeoutShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.1 (rgerhards), 2008-01-10 +- implemented the "disk" queue mode. However, it currently is of very + limited use, because it does not support persistence over rsyslogd + runs. So when rsyslogd is stopped, the queue is drained just as with + the in-memory queue modes. Persistent queues will be a feature of + the next release. +- performance-optimized string class, should bring an overall improvement +- fixed a memory leak in imudp -- thanks to varmojfekoj for the patch +- fixed a race condition that could lead to a rsyslogd hang when during + HUP or termination +- done some doc updates +- added $WorkDirectory config directive +- added $MainMsgQueueFileName config directive +- added $MainMsgQueueMaxFileSize config directive +--------------------------------------------------------------------------- +Version 3.10.0 (rgerhards), 2008-01-07 +- implemented input module interface and initial input modules +- enhanced threading for input modules (each on its own thread now) +- ability to bind UDP listeners to specific local interfaces/ports and + ability to run multiple of them concurrently +- added ability to specify listen IP address for UDP syslog server +- license changed to GPLv3 +- mark messages are now provided by loadble module immark +- rklogd is no longer provided. Its functionality has now been taken over + by imklog, a loadable input module. This offers a much better integration + into rsyslogd and makes sure that the kernel logger process is brought + up and down at the appropriate times +- enhanced $IncludeConfig directive to support wildcard characters + (thanks to Michael Biebl) +- all inputs are now implemented as loadable plugins +- enhanced threading model: each input module now runs on its own thread +- enhanced message queue which now supports different queueing methods + (among others, this can be used for performance fine-tuning) +- added a large number of new configuration directives for the new + input modules +- enhanced multi-threading utilizing a worker thread pool for the + main message queue +- compilation without pthreads is no longer supported +- much cleaner code due to new objects and removal of single-threading + mode +--------------------------------------------------------------------------- +Version 2.0.1 STABLE (rgerhards), 2008-01-24 +- fixed a bug in integer conversion - but this function was never called, + so it is not really a useful bug fix ;) +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +--------------------------------------------------------------------------- +Version 2.0.0 STABLE (rgerhards), 2008-01-02 +- re-release of 1.21.2 as STABLE with no modifications except some + doc updates +--------------------------------------------------------------------------- +Version 1.21.2 (rgerhards), 2007-12-28 +- created a gss-api output module. This keeps GSS-API code and + TCP/UDP code separated. It is also important for forward- + compatibility with v3. Please note that this change breaks compatibility + with config files created for 1.21.0 and 1.21.1 - this was considered + acceptable. +- fixed an error in forwarding retry code (could lead to message corruption + but surfaced very seldom) +- increased portability for older platforms (AI_NUMERICSERV moved) +- removed socket leak in omfwd.c +- cross-platform patch for GSS-API compile problem on some platforms + thanks to darix for the patch! +--------------------------------------------------------------------------- +Version 1.21.1 (rgerhards), 2007-12-23 +- small doc fix for $IncludeConfig +- fixed a bug in llDestroy() +- bugfix: fixing memory leak when message queue is full and during + parsing. Thanks to varmojfekoj for the patch. +- bugfix: when compiled without network support, unix sockets were + not properly closed +- bugfix: memory leak in cfsysline.c/doGetWord() fixed +--------------------------------------------------------------------------- +Version 1.21.0 (rgerhards), 2007-12-19 +- GSS-API support for syslog/TCP connections was added. Thanks to + varmojfekoj for providing the patch with this functionality +- code cleanup +- enhanced $IncludeConfig directive to support wildcard filenames +- changed some multithreading synchronization +--------------------------------------------------------------------------- +Version 1.20.1 (rgerhards), 2007-12-12 +- corrected a debug setting that survived release. Caused TCP connections + to be retried unnecessarily often. +- When a hostname ACL was provided and DNS resolution for that name failed, + ACL processing was stopped at that point. Thanks to mildew for the patch. + Fedora Bugzilla: http://bugzilla.redhat.com/show_bug.cgi?id=395911 +- fixed a potential race condition, see link for details: + http://rgerhards.blogspot.com/2007/12/rsyslog-race-condition.html + Note that the probability of problems from this bug was very remote +- fixed a memory leak that happened when PostgreSQL date formats were + used +--------------------------------------------------------------------------- +Version 1.20.0 (rgerhards), 2007-12-07 +- an output module for postgres databases has been added. Thanks to + sur5r for contributing this code +- unloading dynamic modules has been cleaned up, we now have a + real implementation and not just a dummy "good enough for the time + being". +- enhanced platform independence - thanks to Bartosz Kuzma and Michael + Biebl for their very useful contributions +- some general code cleanup (including warnings on 64 platforms, only) +--------------------------------------------------------------------------- +Version 1.19.12 (rgerhards), 2007-12-03 +- cleaned up the build system (thanks to Michael Biebl for the patch) +- fixed a bug where ommysql was still not compiled with -pthread option +--------------------------------------------------------------------------- +Version 1.19.11 (rgerhards), 2007-11-29 +- applied -pthread option to build when building for multi-threading mode + hopefully solves an issue with segfaulting +--------------------------------------------------------------------------- +Version 1.19.10 (rgerhards), 2007-10-19 +- introduced the new ":modulename:" syntax for calling module actions + in selector lines; modified ommysql to support it. This is primarily + an aid for further modules and a prerequisite to actually allow third + party modules to be created. +- minor fix in slackware startup script, "-r 0" is now "-r0" +- updated rsyslogd doc set man page; now in html format +- undid creation of a separate thread for the main loop -- this did not + turn out to be needed or useful, so reduce complexity once again. +- added doc fixes provided by Michael Biebl - thanks +--------------------------------------------------------------------------- +Version 1.19.9 (rgerhards), 2007-10-12 +- now packaging system which again contains all components in a single + tarball +- modularized main() a bit more, resulting in less complex code +- experimentally added an additional thread - will see if that affects + the segfault bug we experience on some platforms. Note that this change + is scheduled to be removed again later. +--------------------------------------------------------------------------- +Version 1.19.8 (rgerhards), 2007-09-27 +- improved repeated message processing +- applied patch provided by varmojfekoj to support building ommysql + in its own way (now also resides in a plugin subdirectory); + ommysql is now a separate package +- fixed a bug in cvthname() that lead to message loss if part + of the source hostname would have been dropped +- created some support for distributing ommysql together with the + main rsyslog package. I need to re-think it in the future, but + for the time being the current mode is best. I now simply include + one additional tarball for ommysql inside the main distribution. + I look forward to user feedback on how this should be done best. In the + long term, a separate project should be spawend for ommysql, but I'd + like to do that only after the plugin interface is fully stable (what + it is not yet). +--------------------------------------------------------------------------- +Version 1.19.7 (rgerhards), 2007-09-25 +- added code to handle situations where senders send us messages ending with + a NUL character. It is now simply removed. This also caused trailing LF + reduction to fail, when it was followed by such a NUL. This is now also + handled. +- replaced some non-thread-safe function calls by their thread-safe + counterparts +- fixed a minor memory leak that occurred when the %APPNAME% property was + used (I think nobody used that in practice) +- fixed a bug that caused signal handlers in cvthname() not to be restored when + a malicious pointer record was detected and processing of the message been + stopped for that reason (this should be really rare and can not be related + to the segfault bug we are hunting). +- fixed a bug in cvthname that lead to passing a wrong parameter - in + practice, this had no impact. +- general code cleanup (e.g. compiler warnings, comments) +--------------------------------------------------------------------------- +Version 1.19.6 (rgerhards), 2007-09-11 +- applied patch by varmojfekoj to change signal handling to the new + sigaction API set (replacing the depreciated signal() calls and its + friends. +- fixed a bug that in --enable-debug mode caused an assertion when the + discard action was used +- cleaned up compiler warnings +- applied patch by varmojfekoj to FIX a bug that could cause + segfaults if empty properties were processed using modifying + options (e.g. space-cc, drop-cc) +- fixed man bug: rsyslogd supports -l option +--------------------------------------------------------------------------- +Version 1.19.5 (rgerhards), 2007-09-07 +- changed part of the CStr interface so that better error tracking + is provided and the calling sequence is more intuitive (there were + invalid calls based on a too-weird interface) +- (hopefully) fixed some remaining bugs rooted in wrong use of + the CStr class. These could lead to program abort. +- applied patch by varmojfekoj two fix two potential segfault situations +- added $ModDir config directive +- modified $ModLoad so that an absolute path may be specified as + module name (e.g. /rsyslog/ommysql.so) +--------------------------------------------------------------------------- +Version 1.19.4 (rgerhards/varmojfekoj), 2007-09-04 +- fixed a number of small memory leaks - thanks varmojfekoj for patching +- fixed an issue with CString class that could lead to rsyslog abort + in tplToString() - thanks varmojfekoj for patching +- added a man-version of the config file documentation - thanks to Michel + Samia for providing the man file +- fixed bug: a template like this causes an infinite loop: + $template opts,"%programname:::a,b%" + thanks varmojfekoj for the patch +- fixed bug: case changing options crash freeing the string pointer + because they modify it: $template opts2,"%programname::1:lowercase%" + thanks varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 1.19.3 (mmeckelein/varmojfekoj), 2007-08-31 +- small mem leak fixed (after calling parseSelectorAct) - Thx varmojfekoj +- documentation section "Regular File" und "Blocks" updated +- solved an issue with dynamic file generation - Once again many thanks + to varmojfekoj +- the negative selector for program name filter (Blocks) does not work as + expected - Thanks varmojfekoj for patching +- added forwarding information to sysklogd (requires special template) + to config doc +--------------------------------------------------------------------------- +Version 1.19.2 (mmeckelein/varmojfekoj), 2007-08-28 +- a specifically formed message caused a segfault - Many thanks varmojfekoj + for providing a patch +- a typo and a weird condition are fixed in msg.c - Thanks again + varmojfekoj +- on file creation the file was always owned by root:root. This is fixed + now - Thanks ypsa for solving this issue +--------------------------------------------------------------------------- +Version 1.19.1 (mmeckelein), 2007-08-22 +- a bug that caused a high load when a TCP/UDP connection was closed is + fixed now - Thanks mildew for solving this issue +- fixed a bug which caused a segfault on reinit - Thx varmojfekoj for the + patch +- changed the hardcoded module path "/lib/rsyslog" to $(pkglibdir) in order + to avoid trouble e.g. on 64 bit platforms (/lib64) - many thanks Peter + Vrabec and darix, both provided a patch for solving this issue +- enhanced the unloading of modules - thanks again varmojfekoj +- applied a patch from varmojfekoj which fixes various little things in + MySQL output module +--------------------------------------------------------------------------- +Version 1.19.0 (varmojfekoj/rgerhards), 2007-08-16 +- integrated patch from varmojfekoj to make the mysql module a loadable one + many thanks for the patch, MUCH appreciated +--------------------------------------------------------------------------- +Version 1.18.2 (rgerhards), 2007-08-13 +- fixed a bug in outchannel code that caused templates to be incorrectly + parsed +- fixed a bug in ommysql that caused a wrong ";template" missing message +- added some code for unloading modules; not yet fully complete (and we do + not yet have loadable modules, so this is no problem) +- removed debian subdirectory by request of a debian packager (this is a special + subdir for debian and there is also no point in maintaining it when there + is a debian package available - so I gladly did this) in some cases +- improved overall doc quality (some pages were quite old) and linked to + more of the online resources. +- improved /contrib/delete_mysql script by adding a host option and some + other minor modifications +--------------------------------------------------------------------------- +Version 1.18.1 (rgerhards), 2007-08-08 +- applied a patch from varmojfekoj which solved a potential segfault + of rsyslogd on HUP +- applied patch from Michel Samia to fix compilation when the pthreads + feature is disabled +- some code cleanup (moved action object to its own file set) +- add config directive $MainMsgQueueSize, which now allows one to configure the + queue size dynamically +- all compile-time settings are now shown in rsyslogd -v, not just the + active ones +- enhanced performance a little bit more +- added config file directive $ActionResumeInterval +- fixed a bug that prevented compilation under debian sid +- added a contrib directory for user-contributed useful things +--------------------------------------------------------------------------- +Version 1.18.0 (rgerhards), 2007-08-03 +- rsyslog now supports fallback actions when an action did not work. This + is a great feature e.g. for backup database servers or backup syslog + servers +- modified rklogd to only change the console log level if -c is specified +- added feature to use multiple actions inside a single selector +- implemented $ActionExecOnlyWhenPreviousIsSuspended config directive +- error messages during startup are now spit out to the configured log + destinations +--------------------------------------------------------------------------- +Version 1.17.6 (rgerhards), 2007-08-01 +- continued to work on output module modularization - basic stage of + this work is now FINISHED +- fixed bug in OMSRcreate() - always returned SR_RET_OK +- fixed a bug that caused ommysql to always complain about missing + templates +- fixed a mem leak in OMSRdestruct - freeing the object itself was + forgotten - thanks to varmojfekoj for the patch +- fixed a memory leak in syslogd/init() that happened when the config + file could not be read - thanks to varmojfekoj for the patch +- fixed insufficient memory allocation in addAction() and its helpers. + The initial fix and idea was developed by mildew, I fine-tuned + it a bit. Thanks a lot for the fix, I'd probably had pulled out my + hair to find the bug... +- added output of config file line number when a parsing error occurred +- fixed bug in objomsr.c that caused program to abort in debug mode with + an invalid assertion (in some cases) +- fixed a typo that caused the default template for MySQL to be wrong. + thanks to mildew for catching this. +- added configuration file command $DebugPrintModuleList and + $DebugPrintCfSysLineHandlerList +- fixed an invalid value for the MARK timer - unfortunately, there was + a testing aid left in place. This resulted in quite frequent MARK messages +- added $IncludeConfig config directive +- applied a patch from mildew to prevent rsyslogd from freezing under heavy + load. This could happen when the queue was full. Now, we drop messages + but rsyslogd remains active. +--------------------------------------------------------------------------- +Version 1.17.5 (rgerhards), 2007-07-30 +- continued to work on output module modularization +- fixed a missing file bug - thanks to Andrea Montanari for reporting + this problem +- fixed a problem with shutting down the worker thread and freeing the + selector_t list - this caused messages to be lost, because the + message queue was not properly drained before the selectors got + destroyed. +--------------------------------------------------------------------------- +Version 1.17.4 (rgerhards), 2007-07-27 +- continued to work on output module modularization +- fixed a situation where rsyslogd could create zombie processes + thanks to mildew for the patch +- applied patch from Michel Samia to fix compilation when NOT + compiled for pthreads +--------------------------------------------------------------------------- +Version 1.17.3 (rgerhards), 2007-07-25 +- continued working on output module modularization +- fixed a bug that caused rsyslogd to segfault on exit (and + probably also on HUP), when there was an unsent message in a selector + that required forwarding and the dns lookup failed for that selector + (yes, it was pretty unlikely to happen;)) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed a memory leak in config file parsing and die() + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- rsyslogd now checks on startup if it is capable to perform any work + at all. If it cant, it complains and terminates + thanks to Michel Samia for providing the patch! +- fixed a small memory leak when HUPing syslogd. The allowed sender + list now gets freed. thanks to mildew for the patch. +- changed the way error messages in early startup are logged. They + now do no longer use the syslogd code directly but are rather + send to stderr. +--------------------------------------------------------------------------- +Version 1.17.2 (rgerhards), 2007-07-23 +- made the port part of the -r option optional. Needed for backward + compatibility with sysklogd +- replaced system() calls with something more reasonable. Please note that + this might break compatibility with some existing configuration files. + We accept this in favor of the gained security. +- removed a memory leak that could occur if timegenerated was used in + RFC 3164 format in templates +- did some preparation in msg.c for advanced multithreading - placed the + hooks, but not yet any active code +- worked further on modularization +- added $ModLoad MySQL (dummy) config directive +- added DropTrailingLFOnReception config directive +--------------------------------------------------------------------------- +Version 1.17.1 (rgerhards), 2007-07-20 +- fixed a bug that caused make install to install rsyslogd and rklogd under + the wrong names +- fixed bug that caused $AllowedSenders to handle IPv6 scopes incorrectly; + also fixed but that could garble $AllowedSender wildcards. Thanks to + mildew@gmail.com for the patch +- minor code cleanup - thanks to Peter Vrabec for the patch +- fixed minimal memory leak on HUP (caused by templates) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed another memory leak on HUPing and on exiting rsyslogd + again thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- code cleanup (removed compiler warnings) +- fixed portability bug in configure.ac - thanks to Bartosz Kuźma for patch +- moved msg object into its own file set +- added the capability to continue trying to write log files when the + file system is full. Functionality based on patch by Martin Schulze + to sysklogd package. +--------------------------------------------------------------------------- +Version 1.17.0 (RGer), 2007-07-17 +- added $RepeatedLineReduction config parameter +- added $EscapeControlCharactersOnReceive config parameter +- added $ControlCharacterEscapePrefix config parameter +- added $DirCreateMode config parameter +- added $CreateDirs config parameter +- added $DebugPrintTemplateList config parameter +- added $ResetConfigVariables config parameter +- added $FileOwner config parameter +- added $FileGroup config parameter +- added $DirOwner config parameter +- added $DirGroup config parameter +- added $FailOnChownFailure config parameter +- added regular expression support to the filter engine + thanks to Michel Samia for providing the patch! +- enhanced $AllowedSender functionality. Credits to mildew@gmail.com for + the patch doing that + - added IPv6 support + - allowed DNS hostnames + - allowed DNS wildcard names +- added new option $DropMsgsWithMaliciousDnsPTRRecords +- added autoconf so that rfc3195d, rsyslogd and klogd are stored to /sbin +- added capability to auto-create directories with dynaFiles +--------------------------------------------------------------------------- +Version 1.16.0 (RGer/Peter Vrabec), 2007-07-13 - The Friday, 13th Release ;) +- build system switched to autotools +- removed SYSV preprocessor macro use, replaced with autotools equivalents +- fixed a bug that caused rsyslogd to segfault when TCP listening was + disabled and it terminated +- added new properties "syslogfacility-text" and "syslogseverity-text" + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- added the -x option to disable hostname dns resolution + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- begun to better modularize syslogd.c - this is an ongoing project; moved + type definitions to a separate file +- removed some now-unused fields from struct filed +- move file size limit fields in struct field to the "right spot" (the file + writing part of the union - f_un.f_file) +- subdirectories linux and solaris are no longer part of the distribution + package. This is not because we cease support for them, but there are no + longer any files in them after the move to autotools +--------------------------------------------------------------------------- +Version 1.15.1 (RGer), 2007-07-10 +- fixed a bug that caused a dynaFile selector to stall when there was + an open error with one file +- improved template processing for dynaFiles; templates are now only + looked up during initialization - speeds up processing +- optimized memory layout in struct filed when compiled with MySQL + support +- fixed a bug that caused compilation without SYSLOG_INET to fail +- re-enabled the "last message repeated n times" feature. This + feature was not taken care of while rsyslogd evolved from sysklogd + and it was more or less defunct. Now it is fully functional again. +- added system properties: $NOW, $YEAR, $MONTH, $DAY, $HOUR, $MINUTE +- fixed a bug in iovAsString() that caused a memory leak under stress + conditions (most probably memory shortage). This was unlikely to + ever happen, but it doesn't hurt doing it right +- cosmetic: defined type "uchar", change all unsigned chars to uchar +--------------------------------------------------------------------------- +Version 1.15.0 (RGer), 2007-07-05 +- added ability to dynamically generate file names based on templates + and thus properties. This was a much-requested feature. It makes + life easy when it e.g. comes to splitting files based on the sender + address. +- added $umask and $FileCreateMode config file directives +- applied a patch from Bartosz Kuzma to compile cleanly under NetBSD +- checks for extra (unexpected) characters in system config file lines + have been added +- added IPv6 documentation - was accidentally missing from CVS +- begun to change char to unsigned char +--------------------------------------------------------------------------- +Version 1.14.2 (RGer), 2007-07-03 +** this release fixes all known nits with IPv6 ** +- restored capability to do /etc/service lookup for "syslog" + service when -r 0 was given +- documented IPv6 handling of syslog messages +- integrate patch from Bartosz Kuźma to make rsyslog compile under + Solaris again (the patch replaced a strndup() call, which is not + available under Solaris +- improved debug logging when waiting on select +- updated rsyslogd man page with new options (-46A) +--------------------------------------------------------------------------- +Version 1.14.1 (RGer/Peter Vrabec), 2007-06-29 +- added Peter Vrabec's patch for IPv6 TCP +- prefixed all messages send to stderr in rsyslogd with "rsyslogd: " +--------------------------------------------------------------------------- +Version 1.14.0 (RGer/Peter Vrabec), 2007-06-28 +- Peter Vrabec provided IPv6 for rsyslog, so we are now IPv6 enabled + IPv6 Support is currently for UDP only, TCP is to come soon. + AllowedSender configuration does not yet work for IPv6. +- fixed code in iovCreate() that broke C's strict aliasing rules +- fixed some char/unsigned char differences that forced the compiler + to spit out warning messages +- updated the Red Hat init script to fix a known issue (thanks to + Peter Vrabec) +--------------------------------------------------------------------------- +Version 1.13.5 (RGer), 2007-06-22 +- made the TCP session limit configurable via command line switch + now -t <port>,<max sessions> +- added man page for rklogd(8) (basically a copy from klogd, but now + there is one...) +- fixed a bug that caused internal messages (e.g. rsyslogd startup) to + appear without a tag. +- removed a minor memory leak that occurred when TAG processing requalified + a HOSTNAME to be a TAG (and a TAG already was set). +- removed potential small memory leaks in MsgSet***() functions. There + would be a leak if a property was re-set, something that happened + extremely seldom. +--------------------------------------------------------------------------- +Version 1.13.4 (RGer), 2007-06-18 +- added a new property "PRI-text", which holds the PRI field in + textual form (e.g. "syslog.info") +- added alias "syslogseverity" for "syslogpriority", which is a + misleading property name that needs to stay for historical + reasons (and backward-compatibility) +- added doc on how to record PRI value in log file +- enhanced signal handling in klogd, including removal of an unsafe + call to the logging system during signal handling +--------------------------------------------------------------------------- +Version 1.13.3 (RGer), 2007-06-15 +- create a version of syslog.c from scratch. This is now + - highly optimized for rsyslog + - removes an incompatible license problem as the original + version had a BSD license with advertising clause + - fixed in the regard that rklogd will continue to work when + rsyslogd has been restarted (the original version, as well + as sysklogd, will remain silent then) + - solved an issue with an extra NUL char at message end that the + original version had +- applied some changes to klogd to care for the new interface +- fixed a bug in syslogd.c which prevented compiling under debian +--------------------------------------------------------------------------- +Version 1.13.2 (RGer), 2007-06-13 +- lib order in makefile patched to facilitate static linking - thanks + to Bennett Todd for providing the patch +- Integrated a patch from Peter Vrabec (pvrabec@redhat.com): + - added klogd under the name of rklogd (remove dependency on + original sysklogd package + - createDB.sql now in UTF + - added additional config files for use on Red Hat +--------------------------------------------------------------------------- +Version 1.13.1 (RGer), 2007-02-05 +- changed the listen backlog limit to a more reasonable value based on + the maximum number of TCP connections configured (10% + 5) - thanks to Guy + Standen for the hint (actually, the limit was 5 and that was a + left-over from early testing). +- fixed a bug in makefile which caused DB-support to be disabled when + NETZIP support was enabled +- added the -e option to allow transmission of every message to remote + hosts (effectively turns off duplicate message suppression) +- (somewhat) improved memory consumption when compiled with MySQL support +- looks like we fixed an incompatibility with MySQL 5.x and above software + At least in one case, the remote server name was destroyed, leading to + a connection failure. The new, improved code does not have this issue and + so we see this as solved (the new code is generally somewhat better, so + there is a good chance we fixed this incompatibility). +--------------------------------------------------------------------------- +Version 1.13.0 (RGer), 2006-12-19 +- added '$' as ToPos property replacer specifier - means "up to the + end of the string" +- property replacer option "escape-cc", "drop-cc" and "space-cc" added +- changed the handling of \0 characters inside syslog messages. We now + consistently escape them to "#000". This is somewhat recommended in + the draft-ietf-syslog-protocol-19 draft. While the real recommendation + is to not escape any characters at all, we can not do this without + considerable modification of the code. So we escape it to "#000", which + is consistent with a sample found in the Internet-draft. +- removed message glue logic (see printchopped() comment for details) + Also caused removal of parts table and thus some improvements in + memory usage. +- changed the default MAXLINE to 2048 to take care of recent syslog + standardization efforts (can easily be changed in syslogd.c) +- added support for byte-counted TCP syslog messages (much like + syslog-transport-tls-05 Internet Draft). This was necessary to + support compression over TCP. +- added support for receiving compressed syslog messages +- added support for sending compressed syslog messages +- fixed a bug where the last message in a syslog/tcp stream was + lost if it was not properly terminated by a LF character +--------------------------------------------------------------------------- +Version 1.12.3 (RGer), 2006-10-04 +- implemented some changes to support Solaris (but support is not + yet complete) +- commented out (via #if 0) some methods that are currently not being use + but should be kept for further us +- added (interim) -u 1 option to turn off hostname and tag parsing +- done some modifications to better support Fedora +- made the field delimiter inside property replace configurable via + template +- fixed a bug in property replacer: if fields were used, the delimitor + became part of the field. Up until now, this was barely noticeable as + the delimiter as TAB only and thus invisible to a human. With other + delimiters available now, it quickly showed up. This bug fix might cause + some grief to existing installations if they used the extra TAB for + whatever reasons - sorry folks... Anyhow, a solution is easy: just add + a TAB character constant into your template. Thus, there has no attempt + been made to do this in a backwards-compatible way. +--------------------------------------------------------------------------- +Version 1.12.2 (RGer), 2006-02-15 +- fixed a bug in the RFC 3339 date formatter. An extra space was added + after the actual timestamp +- added support for providing high-precision RFC3339 timestamps for + (rsyslogd-)internally-generated messages +- very (!) experimental support for syslog-protocol internet draft + added (the draft is experimental, the code is solid ;)) +- added support for field-extracting in the property replacer +- enhanced the legacy-syslog parser so that it can interpret messages + that do not contain a TIMESTAMP +- fixed a bug that caused the default socket (usually /dev/log) to be + opened even when -o command line option was given +- fixed a bug in the Debian sample startup script - it caused rsyslogd + to listen to remote requests, which it shouldn't by default +--------------------------------------------------------------------------- +Version 1.12.1 (RGer), 2005-11-23 +- made multithreading work with BSD. Some signal-handling needed to be + restructured. Also, there might be a slight delay of up to 10 seconds + when huping and terminating rsyslogd under BSD +- fixed a bug where a NULL-pointer was passed to printf() in logmsg(). +- fixed a bug during "make install" where rc3195d was not installed + Thanks to Bennett Todd for spotting this. +- fixed a bug where rsyslogd dumped core when no TAG was found in the + received message +- enhanced message parser so that it can deal with missing hostnames + in many cases (may not be totally fail-safe) +- fixed a bug where internally-generated messages did not have the correct + TAG +--------------------------------------------------------------------------- +Version 1.12.0 (RGer), 2005-10-26 +- moved to a multi-threaded design. single-threading is still optionally + available. Multi-threading is experimental! +- fixed a potential race condition. In the original code, marking was done + by an alarm handler, which could lead to all sorts of bad things. This + has been changed now. See comments in syslogd.c/domark() for details. +- improved debug output for property-based filters +- not a code change, but: I have checked all exit()s to make sure that + none occurs once rsyslogd has started up. Even in unusual conditions + (like low-memory conditions) rsyslogd somehow remains active. Of course, + it might loose a message or two, but at least it does not abort and it + can also recover when the condition no longer persists. +- fixed a bug that could cause loss of the last message received + immediately before rsyslogd was terminated. +- added comments on thread-safety of global variables in syslogd.c +- fixed a small bug: spurios printf() when TCP syslog was used +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug with regular expression support (thanks to Andres Riancho) +- a little bit of code restructuring (especially main(), which was + horribly large) +--------------------------------------------------------------------------- +Version 1.11.1 (RGer), 2005-10-19 +- support for BSD-style program name and host blocks +- added a new property "programname" that can be used in templates +- added ability to specify listen port for rfc3195d +- fixed a bug that rendered the "startswith" comparison operation + unusable. +- changed more functions to "static" storage class to help compiler + optimize (should have been static in the first place...) +- fixed a potential memory leak in the string buffer class destructor. + As the destructor was previously never called, the leak did not actually + appear. +- some internal restructuring in anticipation/preparation of minimal + multi-threading support +- rsyslogd still shares some code with the sysklogd project. Some patches + for this shared code have been brought over from the sysklogd CVS. +--------------------------------------------------------------------------- +Version 1.11.0 (RGer), 2005-10-12 +- support for receiving messages via RFC 3195; added rfc3195d for that + purpose +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +--------------------------------------------------------------------------- +Version 1.10.2 (RGer), 2005-09-27 +- added comparison operations in property-based filters: + * isequal + * startswith +- added ability to negate all property-based filter comparison operations + by adding a !-sign right in front of the operation name +- added the ability to specify remote senders for UDP and TCP + received messages. Allows to block all but well-known hosts +- changed the $-config line directives to be case-INsensitive +- new command line option -w added: "do not display warnings if messages + from disallowed senders are received" +- fixed a bug that caused rsyslogd to dump core when the compare value + was not quoted in property-based filters +- fixed a bug in the new CStr compare function which lead to invalid + results (fortunately, this function was not yet used widely) +- added better support for "debugging" rsyslog.conf property filters + (only if -d switch is given) +- changed some function definitions to static, which eventually enables + some compiler optimizations +- fixed a bug in MySQL code; when a SQL error occurred, rsyslogd could + run in a tight loop. This was due to invalid sequence of error reporting + and is now fixed. +--------------------------------------------------------------------------- +Version 1.10.1 (RGer), 2005-09-23 +- added the ability to execute a shell script as an action. + Thanks to Bjoern Kalkbrenner for providing the code! +- fixed a bug in the MySQL code; due to the bug the automatic one-time + retry after an error did not happen - this lead to error message in + cases where none should be seen (e.g. after a MySQL restart) +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.10.0 (RGer), 2005-09-20 + REMINDER: 1.10 is the first unstable version if the 1.x series! +- added the capability to filter on any property in selector lines + (not just facility and priority) +- changed stringbuf into a new counted string class +- added support for a "discard" action. If a selector line with + discard (~ character) is found, no selector lines *after* that + line will be processed. +- thanks to Andres Riancho, regular expression support has been + added to the template engine +- added the FROMHOST property in the template processor, which could + previously not be obtained. Thanks to Cristian Testa for pointing + this out and even providing a fix. +- added display of compile-time options to -v output +- performance improvement for production build - made some checks + to happen only during debug mode +- fixed a problem with compiling on SUSE and - while doing so - removed + the socket call to set SO_BSDCOMPAT in cases where it is obsolete. +--------------------------------------------------------------------------- +Version 1.0.4 (RGer), 2006-02-01 +- a small but important fix: the tcp receiver had two forgotten printf's + in it that caused a lot of unnecessary output to stdout. This was + important enough to justify a new release +--------------------------------------------------------------------------- +Version 1.0.3 (RGer), 2005-11-14 +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +- applied some patches available from the sysklogd project to code + shared from there +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug in the TCP sender that caused the retry logic to fail + after an error or receiver overrun +- fixed a bug in init() that could lead to dumping core +- fixed a bug that could lead to dumping core when no HOSTNAME or no TAG + was present in the syslog message +--------------------------------------------------------------------------- +Version 1.0.2 (RGer), 2005-10-05 +- fixed an issue with MySQL error reporting. When an error occurred, + the MySQL driver went into an endless loop (at least in most cases). +--------------------------------------------------------------------------- +Version 1.0.1 (RGer), 2005-09-23 +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.0.0 (RGer), 2005-09-12 +- changed install doc to cover daily cron scripts - a trouble source +- added rc script for slackware (provided by Chris Elvidge - thanks!) +- fixed a really minor bug in usage() - the -r option was still + reported as without the port parameter +--------------------------------------------------------------------------- +Version 0.9.8 (RGer), 2005-09-05 +- made startup and shutdown message more consistent and included the + pid, so that they can be easier correlated. Used syslog-protocol + structured data format for this purpose. +- improved config info in startup message, now tells not only + if it is listening remote on udp, but also for tcp. Also includes + the port numbers. The previous startup message was misleading, because + it did not say "remote reception" if rsyslogd was only listening via + tcp (but not via udp). +- added a "how can you help" document to the doc set +--------------------------------------------------------------------------- +Version 0.9.7 (RGer), 2005-08-15 +- some of the previous doc files (like INSTALL) did not properly + reflect the changes to the build process and the new doc. Fixed + that. +- changed syslogd.c so that when compiled without database support, + an error message is displayed when a database action is detected + in the config file (previously this was used as an user rule ;)) +- fixed a bug in the os-specific Makefiles which caused MySQL + support to not be compiled, even if selected +--------------------------------------------------------------------------- +Version 0.9.6 (RGer), 2005-08-09 +- greatly enhanced documentation. Now available in html format in + the "doc" folder and FreeBSD. Finally includes an install howto. +- improved MySQL error messages a little - they now show up as log + messages, too (formerly only in debug mode) +- added the ability to specify the listen port for udp syslog. + WARNING: This introduces an incompatibility. Formerly, udp + syslog was enabled by the -r command line option. Now, it is + "-r [port]", which is consistent with the tcp listener. However, + just -r will now return an error message. +- added sample startup scripts for Debian and FreeBSD +- added support for easy feature selection in the makefile. Un- + fortunately, this also means I needed to spilt the make file + for different OS and distros. There are some really bad syntax + differences between FreeBSD and Linux make. +--------------------------------------------------------------------------- +Version 0.9.5 (RGer), 2005-08-01 +- the "semicolon bug" was actually not (fully) solved in 0.9.4. One + part of the bug was solved, but another still existed. This one + is fixed now, too. +- the "semicolon bug" actually turned out to be a more generic bug. + It appeared whenever an invalid template name was given. With some + selector actions, rsyslogd dumped core, with other it "just" had + a small resource leak with others all worked well. These anomalies + are now fixed. Note that they only appeared during system initialization + once the system was running, nothing bad happened. +- improved error reporting for template errors on startup. They are now + shown on the console and the start-up tty. Formerly, they were only + visible in debug mode. +- support for multiple instances of rsyslogd on a single machine added +- added new option "-o" --> omit local unix domain socket. This option + enables rsyslogd NOT to listen to the local socket. This is most + helpful when multiple instances of rsyslogd (or rsyslogd and another + syslogd) shall run on a single system. +- added new option "-i <pidfile>" which allows one to specify the pidfile. + This is needed when multiple instances of rsyslogd are to be run. +- the new project home page is now online at www.rsyslog.com +--------------------------------------------------------------------------- +Version 0.9.4 (RGer), 2005-07-25 +- finally added the TCP sender. It now supports non-blocking mode, no + longer disabling message reception during connect. As it is now, it + is usable in production. The code could be more sophisticated, but + I've kept it short in anticipation of the move to liblogging, which + will lead to the removal of the code just written ;) +- the "exiting on signal..." message still had the "syslogd" name in + it. Changed this to "rsyslogd", as we do not have a large user base + yet, this should pose no problem. +- fixed "the semicolon" bug. rsyslogd dumped core if a write-db action + was specified but no semicolon was given after the password (an empty + template was ok, but the semicolon needed to be present). +- changed a default for traditional output format. During testing, it + was seen that the timestamp written to file in default format was + the time of message reception, not the time specified in the TIMESTAMP + field of the message itself. Traditionally, the message TIMESTAMP is + used and this has been changed now. +--------------------------------------------------------------------------- +Version 0.9.3 (RGer), 2005-07-19 +- fixed a bug in the message parser. In June, the RFC 3164 timestamp + was not correctly parsed (yes, only in June and some other months, + see the code comment to learn why...) +- added the ability to specify the destination port when forwarding + syslog messages (both for TCP and UDP) +- added an very experimental TCP sender (activated by + @@machine:port in config). This is not yet for production use. If + the receiver is not alive, rsyslogd will wait quite some time until + the connection request times out, which most probably leads to + loss of incoming messages. + +--------------------------------------------------------------------------- +Version 0.9.2 (RGer), around 2005-07-06 +- I intended to change the maxsupported message size to 32k to + support IHE - but given the memory inefficiency in the usual use + cases, I have not done this. I have, however, included very + specific instructions on how to do this in the source code. I have + also done some testing with 32k messages, so you can change the + max size without taking too much risk. +- added a syslog/tcp receiver; we now can receive messages via + plain tcp, but we can still send only via UDP. The syslog/tcp + receiver is the primary enhancement of this release. +- slightly changed some error messages that contained a spurios \n at + the end of the line (which gives empty lines in your log...) + +--------------------------------------------------------------------------- +Version 0.9.1 (RGer) +- fixed code so that it compiles without errors under FreeBSD +- removed now unused function "allocate_log()" from syslogd.c +- changed the make file so that it contains more defines for + different environments (in the long term, we need a better + system for disabling/enabling features...) +- changed some printf's printing off_t types to %lld and + explicit (long long) casts. I tried to figure out the exact type, + but did not succeed in this. In the worst case, ultra-large peta- + byte files will now display funny informational messages on rollover, + something I think we can live with for the neersion 3.11.2 (rgerhards), 2008-02-?? +--------------------------------------------------------------------------- +Version 3.11.1 (rgerhards), 2008-02-12 +- SNMP trap sender added thanks to Andre Lorbach (omsnmp) +- added input-plugin interface specification in form of a (copy) template + input module +- applied documentation fix by Michael Biebl -- many thanks! +- bugfix: immark did not have MARK flags set... +- added x-info field to rsyslogd startup/shutdown message. Hopefully + points users to right location for further info (many don't even know + they run rsyslog ;)) +- bugfix: trailing ":" of tag was lost while parsing legacy syslog messages + without timestamp - thanks to Anders Blomdell for providing a patch! +- fixed a bug in stringbuf.c related to STRINGBUF_TRIM_ALLOCSIZE, which + wasn't supposed to be used with rsyslog. Put a warning message up that + tells this feature is not tested and probably not worth the effort. + Thanks to Anders Blomdell fro bringing this to our attention +- somewhat improved performance of string buffers +- fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf +- bugfix: setting for $EscapeControlCharactersOnReceive was not + properly initialized +- clarified usage of space-cc property replacer option +- improved abort diagnostic handler +- some initial effort for malloc/free runtime debugging support +- bugfix: using dynafile actions caused rsyslogd abort +- fixed minor man errors thanks to Michael Biebl +--------------------------------------------------------------------------- +Version 3.11.0 (rgerhards), 2008-01-31 +- implemented queued actions +- implemented simple rate limiting for actions +- implemented deliberate discarding of lower priority messages over higher + priority ones when a queue runs out of space +- implemented disk quotas for disk queues +- implemented the $ActionResumeRetryCount config directive +- added $ActionQueueFilename config directive +- added $ActionQueueSize config directive +- added $ActionQueueHighWaterMark config directive +- added $ActionQueueLowWaterMark config directive +- added $ActionQueueDiscardMark config directive +- added $ActionQueueDiscardSeverity config directive +- added $ActionQueueCheckpointInterval config directive +- added $ActionQueueType config directive +- added $ActionQueueWorkerThreads config directive +- added $ActionQueueTimeoutshutdown config directive +- added $ActionQueueTimeoutActionCompletion config directive +- added $ActionQueueTimeoutenQueue config directive +- added $ActionQueueTimeoutworkerThreadShutdown config directive +- added $ActionQueueWorkerThreadMinimumMessages config directive +- added $ActionQueueMaxFileSize config directive +- added $ActionQueueSaveonShutdown config directive +- addded $ActionQueueDequeueSlowdown config directive +- addded $MainMsgQueueDequeueSlowdown config directive +- bugfix: added forgotten docs to package +- improved debugging support +- fixed a bug that caused $MainMsgQueueCheckpointInterval to work incorrectly +- when a long-running action needs to be cancelled on shutdown, the message + that was processed by it is now preserved. This finishes support for + guaranteed delivery of messages (if the output supports it, of course) +- fixed bug in output module interface, see + http://sourceforge.net/tracker/index.php?func=detail&aid=1881008&group_id=123448&atid=696552 +- changed the ommysql output plugin so that the (lengthy) connection + initialization now takes place in message processing. This works much + better with the new queued action mode (fast startup) +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +- bugfixed stream class offset handling on 32bit platforms +--------------------------------------------------------------------------- +Version 3.10.3 (rgerhards), 2008-01-28 +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- run-time instrumentation added +- implemented disk-assisted queue mode, which enables on-demand disk + spooling if the queue's in-memory queue is exhausted +- implemented a dynamic worker thread pool for processing incoming + messages; workers are started and shut down as need arises +- implemented a run-time instrumentation debug package +- implemented the $MainMsgQueueSaveOnShutdown config directive +- implemented the $MainMsgQueueWorkerThreadMinimumMessages config directive +- implemented the $MainMsgQueueTimeoutWorkerThreadShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.2 (rgerhards), 2008-01-14 +- added the ability to keep stop rsyslogd without the need to drain + the main message queue. In disk queue mode, rsyslog continues to + run from the point where it stopped. In case of a system failure, it + continues to process messages from the last checkpoint. +- fixed a bug that caused a segfault on startup when no $WorkDir directive + was specified in rsyslog.conf +- provided more fine-grain control over shutdown timeouts and added a + way to specify the enqueue timeout when the main message queue is full +- implemented $MainMsgQueueCheckpointInterval config directive +- implemented $MainMsgQueueTimeoutActionCompletion config directive +- implemented $MainMsgQueueTimeoutEnqueue config directive +- implemented $MainMsgQueueTimeoutShutdown config directive +--------------------------------------------------------------------------- +Version 3.10.1 (rgerhards), 2008-01-10 +- implemented the "disk" queue mode. However, it currently is of very + limited use, because it does not support persistence over rsyslogd + runs. So when rsyslogd is stopped, the queue is drained just as with + the in-memory queue modes. Persistent queues will be a feature of + the next release. +- performance-optimized string class, should bring an overall improvement +- fixed a memory leak in imudp -- thanks to varmojfekoj for the patch +- fixed a race condition that could lead to a rsyslogd hang when during + HUP or termination +- done some doc updates +- added $WorkDirectory config directive +- added $MainMsgQueueFileName config directive +- added $MainMsgQueueMaxFileSize config directive +--------------------------------------------------------------------------- +Version 3.10.0 (rgerhards), 2008-01-07 +- implemented input module interface and initial input modules +- enhanced threading for input modules (each on its own thread now) +- ability to bind UDP listeners to specific local interfaces/ports and + ability to run multiple of them concurrently +- added ability to specify listen IP address for UDP syslog server +- license changed to GPLv3 +- mark messages are now provided by loadble module immark +- rklogd is no longer provided. Its functionality has now been taken over + by imklog, a loadable input module. This offers a much better integration + into rsyslogd and makes sure that the kernel logger process is brought + up and down at the appropriate times +- enhanced $IncludeConfig directive to support wildcard characters + (thanks to Michael Biebl) +- all inputs are now implemented as loadable plugins +- enhanced threading model: each input module now runs on its own thread +- enhanced message queue which now supports different queueing methods + (among others, this can be used for performance fine-tuning) +- added a large number of new configuration directives for the new + input modules +- enhanced multi-threading utilizing a worker thread pool for the + main message queue +- compilation without pthreads is no longer supported +- much cleaner code due to new objects and removal of single-threading + mode +--------------------------------------------------------------------------- +Version 2.0.1 STABLE (rgerhards), 2008-01-24 +- fixed a bug in integer conversion - but this function was never called, + so it is not really a useful bug fix ;) +- fixed a bug with standard template definitions (not a big deal) - thanks + to varmojfekoj for spotting it +- fixed a bug that caused a potential hang in file and fwd output module + varmojfekoj provided the patch - many thanks! +--------------------------------------------------------------------------- +Version 2.0.0 STABLE (rgerhards), 2008-01-02 +- re-release of 1.21.2 as STABLE with no modifications except some + doc updates +--------------------------------------------------------------------------- +Version 1.21.2 (rgerhards), 2007-12-28 +- created a gss-api output module. This keeps GSS-API code and + TCP/UDP code separated. It is also important for forward- + compatibility with v3. Please note that this change breaks compatibility + with config files created for 1.21.0 and 1.21.1 - this was considered + acceptable. +- fixed an error in forwarding retry code (could lead to message corruption + but surfaced very seldom) +- increased portability for older platforms (AI_NUMERICSERV moved) +- removed socket leak in omfwd.c +- cross-platform patch for GSS-API compile problem on some platforms + thanks to darix for the patch! +--------------------------------------------------------------------------- +Version 1.21.1 (rgerhards), 2007-12-23 +- small doc fix for $IncludeConfig +- fixed a bug in llDestroy() +- bugfix: fixing memory leak when message queue is full and during + parsing. Thanks to varmojfekoj for the patch. +- bugfix: when compiled without network support, unix sockets were + not properly closed +- bugfix: memory leak in cfsysline.c/doGetWord() fixed +--------------------------------------------------------------------------- +Version 1.21.0 (rgerhards), 2007-12-19 +- GSS-API support for syslog/TCP connections was added. Thanks to + varmojfekoj for providing the patch with this functionality +- code cleanup +- enhanced $IncludeConfig directive to support wildcard filenames +- changed some multithreading synchronization +--------------------------------------------------------------------------- +Version 1.20.1 (rgerhards), 2007-12-12 +- corrected a debug setting that survived release. Caused TCP connections + to be retried unnecessarily often. +- When a hostname ACL was provided and DNS resolution for that name failed, + ACL processing was stopped at that point. Thanks to mildew for the patch. + Fedora Bugzilla: http://bugzilla.redhat.com/show_bug.cgi?id=395911 +- fixed a potential race condition, see link for details: + http://rgerhards.blogspot.com/2007/12/rsyslog-race-condition.html + Note that the probability of problems from this bug was very remote +- fixed a memory leak that happened when PostgreSQL date formats were + used +--------------------------------------------------------------------------- +Version 1.20.0 (rgerhards), 2007-12-07 +- an output module for postgres databases has been added. Thanks to + sur5r for contributing this code +- unloading dynamic modules has been cleaned up, we now have a + real implementation and not just a dummy "good enough for the time + being". +- enhanced platform independence - thanks to Bartosz Kuzma and Michael + Biebl for their very useful contributions +- some general code cleanup (including warnings on 64 platforms, only) +--------------------------------------------------------------------------- +Version 1.19.12 (rgerhards), 2007-12-03 +- cleaned up the build system (thanks to Michael Biebl for the patch) +- fixed a bug where ommysql was still not compiled with -pthread option +--------------------------------------------------------------------------- +Version 1.19.11 (rgerhards), 2007-11-29 +- applied -pthread option to build when building for multi-threading mode + hopefully solves an issue with segfaulting +--------------------------------------------------------------------------- +Version 1.19.10 (rgerhards), 2007-10-19 +- introduced the new ":modulename:" syntax for calling module actions + in selector lines; modified ommysql to support it. This is primarily + an aid for further modules and a prerequisite to actually allow third + party modules to be created. +- minor fix in slackware startup script, "-r 0" is now "-r0" +- updated rsyslogd doc set man page; now in html format +- undid creation of a separate thread for the main loop -- this did not + turn out to be needed or useful, so reduce complexity once again. +- added doc fixes provided by Michael Biebl - thanks +--------------------------------------------------------------------------- +Version 1.19.9 (rgerhards), 2007-10-12 +- now packaging system which again contains all components in a single + tarball +- modularized main() a bit more, resulting in less complex code +- experimentally added an additional thread - will see if that affects + the segfault bug we experience on some platforms. Note that this change + is scheduled to be removed again later. +--------------------------------------------------------------------------- +Version 1.19.8 (rgerhards), 2007-09-27 +- improved repeated message processing +- applied patch provided by varmojfekoj to support building ommysql + in its own way (now also resides in a plugin subdirectory); + ommysql is now a separate package +- fixed a bug in cvthname() that lead to message loss if part + of the source hostname would have been dropped +- created some support for distributing ommysql together with the + main rsyslog package. I need to re-think it in the future, but + for the time being the current mode is best. I now simply include + one additional tarball for ommysql inside the main distribution. + I look forward to user feedback on how this should be done best. In the + long term, a separate project should be spawend for ommysql, but I'd + like to do that only after the plugin interface is fully stable (what + it is not yet). +--------------------------------------------------------------------------- +Version 1.19.7 (rgerhards), 2007-09-25 +- added code to handle situations where senders send us messages ending with + a NUL character. It is now simply removed. This also caused trailing LF + reduction to fail, when it was followed by such a NUL. This is now also + handled. +- replaced some non-thread-safe function calls by their thread-safe + counterparts +- fixed a minor memory leak that occurred when the %APPNAME% property was + used (I think nobody used that in practice) +- fixed a bug that caused signal handlers in cvthname() not to be restored when + a malicious pointer record was detected and processing of the message been + stopped for that reason (this should be really rare and can not be related + to the segfault bug we are hunting). +- fixed a bug in cvthname that lead to passing a wrong parameter - in + practice, this had no impact. +- general code cleanup (e.g. compiler warnings, comments) +--------------------------------------------------------------------------- +Version 1.19.6 (rgerhards), 2007-09-11 +- applied patch by varmojfekoj to change signal handling to the new + sigaction API set (replacing the depreciated signal() calls and its + friends. +- fixed a bug that in --enable-debug mode caused an assertion when the + discard action was used +- cleaned up compiler warnings +- applied patch by varmojfekoj to FIX a bug that could cause + segfaults if empty properties were processed using modifying + options (e.g. space-cc, drop-cc) +- fixed man bug: rsyslogd supports -l option +--------------------------------------------------------------------------- +Version 1.19.5 (rgerhards), 2007-09-07 +- changed part of the CStr interface so that better error tracking + is provided and the calling sequence is more intuitive (there were + invalid calls based on a too-weird interface) +- (hopefully) fixed some remaining bugs rooted in wrong use of + the CStr class. These could lead to program abort. +- applied patch by varmojfekoj two fix two potential segfault situations +- added $ModDir config directive +- modified $ModLoad so that an absolute path may be specified as + module name (e.g. /rsyslog/ommysql.so) +--------------------------------------------------------------------------- +Version 1.19.4 (rgerhards/varmojfekoj), 2007-09-04 +- fixed a number of small memory leaks - thanks varmojfekoj for patching +- fixed an issue with CString class that could lead to rsyslog abort + in tplToString() - thanks varmojfekoj for patching +- added a man-version of the config file documentation - thanks to Michel + Samia for providing the man file +- fixed bug: a template like this causes an infinite loop: + $template opts,"%programname:::a,b%" + thanks varmojfekoj for the patch +- fixed bug: case changing options crash freeing the string pointer + because they modify it: $template opts2,"%programname::1:lowercase%" + thanks varmojfekoj for the patch +--------------------------------------------------------------------------- +Version 1.19.3 (mmeckelein/varmojfekoj), 2007-08-31 +- small mem leak fixed (after calling parseSelectorAct) - Thx varmojfekoj +- documentation section "Regular File" und "Blocks" updated +- solved an issue with dynamic file generation - Once again many thanks + to varmojfekoj +- the negative selector for program name filter (Blocks) does not work as + expected - Thanks varmojfekoj for patching +- added forwarding information to sysklogd (requires special template) + to config doc +--------------------------------------------------------------------------- +Version 1.19.2 (mmeckelein/varmojfekoj), 2007-08-28 +- a specifically formed message caused a segfault - Many thanks varmojfekoj + for providing a patch +- a typo and a weird condition are fixed in msg.c - Thanks again + varmojfekoj +- on file creation the file was always owned by root:root. This is fixed + now - Thanks ypsa for solving this issue +--------------------------------------------------------------------------- +Version 1.19.1 (mmeckelein), 2007-08-22 +- a bug that caused a high load when a TCP/UDP connection was closed is + fixed now - Thanks mildew for solving this issue +- fixed a bug which caused a segfault on reinit - Thx varmojfekoj for the + patch +- changed the hardcoded module path "/lib/rsyslog" to $(pkglibdir) in order + to avoid trouble e.g. on 64 bit platforms (/lib64) - many thanks Peter + Vrabec and darix, both provided a patch for solving this issue +- enhanced the unloading of modules - thanks again varmojfekoj +- applied a patch from varmojfekoj which fixes various little things in + MySQL output module +--------------------------------------------------------------------------- +Version 1.19.0 (varmojfekoj/rgerhards), 2007-08-16 +- integrated patch from varmojfekoj to make the mysql module a loadable one + many thanks for the patch, MUCH appreciated +--------------------------------------------------------------------------- +Version 1.18.2 (rgerhards), 2007-08-13 +- fixed a bug in outchannel code that caused templates to be incorrectly + parsed +- fixed a bug in ommysql that caused a wrong ";template" missing message +- added some code for unloading modules; not yet fully complete (and we do + not yet have loadable modules, so this is no problem) +- removed debian subdirectory by request of a debian packager (this is a special + subdir for debian and there is also no point in maintaining it when there + is a debian package available - so I gladly did this) in some cases +- improved overall doc quality (some pages were quite old) and linked to + more of the online resources. +- improved /contrib/delete_mysql script by adding a host option and some + other minor modifications +--------------------------------------------------------------------------- +Version 1.18.1 (rgerhards), 2007-08-08 +- applied a patch from varmojfekoj which solved a potential segfault + of rsyslogd on HUP +- applied patch from Michel Samia to fix compilation when the pthreads + feature is disabled +- some code cleanup (moved action object to its own file set) +- add config directive $MainMsgQueueSize, which now allows one to configure the + queue size dynamically +- all compile-time settings are now shown in rsyslogd -v, not just the + active ones +- enhanced performance a little bit more +- added config file directive $ActionResumeInterval +- fixed a bug that prevented compilation under debian sid +- added a contrib directory for user-contributed useful things +--------------------------------------------------------------------------- +Version 1.18.0 (rgerhards), 2007-08-03 +- rsyslog now supports fallback actions when an action did not work. This + is a great feature e.g. for backup database servers or backup syslog + servers +- modified rklogd to only change the console log level if -c is specified +- added feature to use multiple actions inside a single selector +- implemented $ActionExecOnlyWhenPreviousIsSuspended config directive +- error messages during startup are now spit out to the configured log + destinations +--------------------------------------------------------------------------- +Version 1.17.6 (rgerhards), 2007-08-01 +- continued to work on output module modularization - basic stage of + this work is now FINISHED +- fixed bug in OMSRcreate() - always returned SR_RET_OK +- fixed a bug that caused ommysql to always complain about missing + templates +- fixed a mem leak in OMSRdestruct - freeing the object itself was + forgotten - thanks to varmojfekoj for the patch +- fixed a memory leak in syslogd/init() that happened when the config + file could not be read - thanks to varmojfekoj for the patch +- fixed insufficient memory allocation in addAction() and its helpers. + The initial fix and idea was developed by mildew, I fine-tuned + it a bit. Thanks a lot for the fix, I'd probably had pulled out my + hair to find the bug... +- added output of config file line number when a parsing error occurred +- fixed bug in objomsr.c that caused program to abort in debug mode with + an invalid assertion (in some cases) +- fixed a typo that caused the default template for MySQL to be wrong. + thanks to mildew for catching this. +- added configuration file command $DebugPrintModuleList and + $DebugPrintCfSysLineHandlerList +- fixed an invalid value for the MARK timer - unfortunately, there was + a testing aid left in place. This resulted in quite frequent MARK messages +- added $IncludeConfig config directive +- applied a patch from mildew to prevent rsyslogd from freezing under heavy + load. This could happen when the queue was full. Now, we drop messages + but rsyslogd remains active. +--------------------------------------------------------------------------- +Version 1.17.5 (rgerhards), 2007-07-30 +- continued to work on output module modularization +- fixed a missing file bug - thanks to Andrea Montanari for reporting + this problem +- fixed a problem with shutting down the worker thread and freeing the + selector_t list - this caused messages to be lost, because the + message queue was not properly drained before the selectors got + destroyed. +--------------------------------------------------------------------------- +Version 1.17.4 (rgerhards), 2007-07-27 +- continued to work on output module modularization +- fixed a situation where rsyslogd could create zombie processes + thanks to mildew for the patch +- applied patch from Michel Samia to fix compilation when NOT + compiled for pthreads +--------------------------------------------------------------------------- +Version 1.17.3 (rgerhards), 2007-07-25 +- continued working on output module modularization +- fixed a bug that caused rsyslogd to segfault on exit (and + probably also on HUP), when there was an unsent message in a selector + that required forwarding and the dns lookup failed for that selector + (yes, it was pretty unlikely to happen;)) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed a memory leak in config file parsing and die() + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- rsyslogd now checks on startup if it is capable to perform any work + at all. If it cant, it complains and terminates + thanks to Michel Samia for providing the patch! +- fixed a small memory leak when HUPing syslogd. The allowed sender + list now gets freed. thanks to mildew for the patch. +- changed the way error messages in early startup are logged. They + now do no longer use the syslogd code directly but are rather + send to stderr. +--------------------------------------------------------------------------- +Version 1.17.2 (rgerhards), 2007-07-23 +- made the port part of the -r option optional. Needed for backward + compatibility with sysklogd +- replaced system() calls with something more reasonable. Please note that + this might break compatibility with some existing configuration files. + We accept this in favor of the gained security. +- removed a memory leak that could occur if timegenerated was used in + RFC 3164 format in templates +- did some preparation in msg.c for advanced multithreading - placed the + hooks, but not yet any active code +- worked further on modularization +- added $ModLoad MySQL (dummy) config directive +- added DropTrailingLFOnReception config directive +--------------------------------------------------------------------------- +Version 1.17.1 (rgerhards), 2007-07-20 +- fixed a bug that caused make install to install rsyslogd and rklogd under + the wrong names +- fixed bug that caused $AllowedSenders to handle IPv6 scopes incorrectly; + also fixed but that could garble $AllowedSender wildcards. Thanks to + mildew@gmail.com for the patch +- minor code cleanup - thanks to Peter Vrabec for the patch +- fixed minimal memory leak on HUP (caused by templates) + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- fixed another memory leak on HUPing and on exiting rsyslogd + again thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- code cleanup (removed compiler warnings) +- fixed portability bug in configure.ac - thanks to Bartosz Kuźma for patch +- moved msg object into its own file set +- added the capability to continue trying to write log files when the + file system is full. Functionality based on patch by Martin Schulze + to sysklogd package. +--------------------------------------------------------------------------- +Version 1.17.0 (RGer), 2007-07-17 +- added $RepeatedLineReduction config parameter +- added $EscapeControlCharactersOnReceive config parameter +- added $ControlCharacterEscapePrefix config parameter +- added $DirCreateMode config parameter +- added $CreateDirs config parameter +- added $DebugPrintTemplateList config parameter +- added $ResetConfigVariables config parameter +- added $FileOwner config parameter +- added $FileGroup config parameter +- added $DirOwner config parameter +- added $DirGroup config parameter +- added $FailOnChownFailure config parameter +- added regular expression support to the filter engine + thanks to Michel Samia for providing the patch! +- enhanced $AllowedSender functionality. Credits to mildew@gmail.com for + the patch doing that + - added IPv6 support + - allowed DNS hostnames + - allowed DNS wildcard names +- added new option $DropMsgsWithMaliciousDnsPTRRecords +- added autoconf so that rfc3195d, rsyslogd and klogd are stored to /sbin +- added capability to auto-create directories with dynaFiles +--------------------------------------------------------------------------- +Version 1.16.0 (RGer/Peter Vrabec), 2007-07-13 - The Friday, 13th Release ;) +- build system switched to autotools +- removed SYSV preprocessor macro use, replaced with autotools equivalents +- fixed a bug that caused rsyslogd to segfault when TCP listening was + disabled and it terminated +- added new properties "syslogfacility-text" and "syslogseverity-text" + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- added the -x option to disable hostname dns resolution + thanks to varmojfekoj <varmojfekoj@gmail.com> for the patch +- begun to better modularize syslogd.c - this is an ongoing project; moved + type definitions to a separate file +- removed some now-unused fields from struct filed +- move file size limit fields in struct field to the "right spot" (the file + writing part of the union - f_un.f_file) +- subdirectories linux and solaris are no longer part of the distribution + package. This is not because we cease support for them, but there are no + longer any files in them after the move to autotools +--------------------------------------------------------------------------- +Version 1.15.1 (RGer), 2007-07-10 +- fixed a bug that caused a dynaFile selector to stall when there was + an open error with one file +- improved template processing for dynaFiles; templates are now only + looked up during initialization - speeds up processing +- optimized memory layout in struct filed when compiled with MySQL + support +- fixed a bug that caused compilation without SYSLOG_INET to fail +- re-enabled the "last message repeated n times" feature. This + feature was not taken care of while rsyslogd evolved from sysklogd + and it was more or less defunct. Now it is fully functional again. +- added system properties: $NOW, $YEAR, $MONTH, $DAY, $HOUR, $MINUTE +- fixed a bug in iovAsString() that caused a memory leak under stress + conditions (most probably memory shortage). This was unlikely to + ever happen, but it doesn't hurt doing it right +- cosmetic: defined type "uchar", change all unsigned chars to uchar +--------------------------------------------------------------------------- +Version 1.15.0 (RGer), 2007-07-05 +- added ability to dynamically generate file names based on templates + and thus properties. This was a much-requested feature. It makes + life easy when it e.g. comes to splitting files based on the sender + address. +- added $umask and $FileCreateMode config file directives +- applied a patch from Bartosz Kuzma to compile cleanly under NetBSD +- checks for extra (unexpected) characters in system config file lines + have been added +- added IPv6 documentation - was accidentally missing from CVS +- begun to change char to unsigned char +--------------------------------------------------------------------------- +Version 1.14.2 (RGer), 2007-07-03 +** this release fixes all known nits with IPv6 ** +- restored capability to do /etc/service lookup for "syslog" + service when -r 0 was given +- documented IPv6 handling of syslog messages +- integrate patch from Bartosz Kuźma to make rsyslog compile under + Solaris again (the patch replaced a strndup() call, which is not + available under Solaris +- improved debug logging when waiting on select +- updated rsyslogd man page with new options (-46A) +--------------------------------------------------------------------------- +Version 1.14.1 (RGer/Peter Vrabec), 2007-06-29 +- added Peter Vrabec's patch for IPv6 TCP +- prefixed all messages send to stderr in rsyslogd with "rsyslogd: " +--------------------------------------------------------------------------- +Version 1.14.0 (RGer/Peter Vrabec), 2007-06-28 +- Peter Vrabec provided IPv6 for rsyslog, so we are now IPv6 enabled + IPv6 Support is currently for UDP only, TCP is to come soon. + AllowedSender configuration does not yet work for IPv6. +- fixed code in iovCreate() that broke C's strict aliasing rules +- fixed some char/unsigned char differences that forced the compiler + to spit out warning messages +- updated the Red Hat init script to fix a known issue (thanks to + Peter Vrabec) +--------------------------------------------------------------------------- +Version 1.13.5 (RGer), 2007-06-22 +- made the TCP session limit configurable via command line switch + now -t <port>,<max sessions> +- added man page for rklogd(8) (basically a copy from klogd, but now + there is one...) +- fixed a bug that caused internal messages (e.g. rsyslogd startup) to + appear without a tag. +- removed a minor memory leak that occurred when TAG processing requalified + a HOSTNAME to be a TAG (and a TAG already was set). +- removed potential small memory leaks in MsgSet***() functions. There + would be a leak if a property was re-set, something that happened + extremely seldom. +--------------------------------------------------------------------------- +Version 1.13.4 (RGer), 2007-06-18 +- added a new property "PRI-text", which holds the PRI field in + textual form (e.g. "syslog.info") +- added alias "syslogseverity" for "syslogpriority", which is a + misleading property name that needs to stay for historical + reasons (and backward-compatibility) +- added doc on how to record PRI value in log file +- enhanced signal handling in klogd, including removal of an unsafe + call to the logging system during signal handling +--------------------------------------------------------------------------- +Version 1.13.3 (RGer), 2007-06-15 +- create a version of syslog.c from scratch. This is now + - highly optimized for rsyslog + - removes an incompatible license problem as the original + version had a BSD license with advertising clause + - fixed in the regard that rklogd will continue to work when + rsyslogd has been restarted (the original version, as well + as sysklogd, will remain silent then) + - solved an issue with an extra NUL char at message end that the + original version had +- applied some changes to klogd to care for the new interface +- fixed a bug in syslogd.c which prevented compiling under debian +--------------------------------------------------------------------------- +Version 1.13.2 (RGer), 2007-06-13 +- lib order in makefile patched to facilitate static linking - thanks + to Bennett Todd for providing the patch +- Integrated a patch from Peter Vrabec (pvrabec@redhat.com): + - added klogd under the name of rklogd (remove dependency on + original sysklogd package + - createDB.sql now in UTF + - added additional config files for use on Red Hat +--------------------------------------------------------------------------- +Version 1.13.1 (RGer), 2007-02-05 +- changed the listen backlog limit to a more reasonable value based on + the maximum number of TCP connections configured (10% + 5) - thanks to Guy + Standen for the hint (actually, the limit was 5 and that was a + left-over from early testing). +- fixed a bug in makefile which caused DB-support to be disabled when + NETZIP support was enabled +- added the -e option to allow transmission of every message to remote + hosts (effectively turns off duplicate message suppression) +- (somewhat) improved memory consumption when compiled with MySQL support +- looks like we fixed an incompatibility with MySQL 5.x and above software + At least in one case, the remote server name was destroyed, leading to + a connection failure. The new, improved code does not have this issue and + so we see this as solved (the new code is generally somewhat better, so + there is a good chance we fixed this incompatibility). +--------------------------------------------------------------------------- +Version 1.13.0 (RGer), 2006-12-19 +- added '$' as ToPos property replacer specifier - means "up to the + end of the string" +- property replacer option "escape-cc", "drop-cc" and "space-cc" added +- changed the handling of \0 characters inside syslog messages. We now + consistently escape them to "#000". This is somewhat recommended in + the draft-ietf-syslog-protocol-19 draft. While the real recommendation + is to not escape any characters at all, we can not do this without + considerable modification of the code. So we escape it to "#000", which + is consistent with a sample found in the Internet-draft. +- removed message glue logic (see printchopped() comment for details) + Also caused removal of parts table and thus some improvements in + memory usage. +- changed the default MAXLINE to 2048 to take care of recent syslog + standardization efforts (can easily be changed in syslogd.c) +- added support for byte-counted TCP syslog messages (much like + syslog-transport-tls-05 Internet Draft). This was necessary to + support compression over TCP. +- added support for receiving compressed syslog messages +- added support for sending compressed syslog messages +- fixed a bug where the last message in a syslog/tcp stream was + lost if it was not properly terminated by a LF character +--------------------------------------------------------------------------- +Version 1.12.3 (RGer), 2006-10-04 +- implemented some changes to support Solaris (but support is not + yet complete) +- commented out (via #if 0) some methods that are currently not being use + but should be kept for further us +- added (interim) -u 1 option to turn off hostname and tag parsing +- done some modifications to better support Fedora +- made the field delimiter inside property replace configurable via + template +- fixed a bug in property replacer: if fields were used, the delimitor + became part of the field. Up until now, this was barely noticeable as + the delimiter as TAB only and thus invisible to a human. With other + delimiters available now, it quickly showed up. This bug fix might cause + some grief to existing installations if they used the extra TAB for + whatever reasons - sorry folks... Anyhow, a solution is easy: just add + a TAB character constant into your template. Thus, there has no attempt + been made to do this in a backwards-compatible way. +--------------------------------------------------------------------------- +Version 1.12.2 (RGer), 2006-02-15 +- fixed a bug in the RFC 3339 date formatter. An extra space was added + after the actual timestamp +- added support for providing high-precision RFC3339 timestamps for + (rsyslogd-)internally-generated messages +- very (!) experimental support for syslog-protocol internet draft + added (the draft is experimental, the code is solid ;)) +- added support for field-extracting in the property replacer +- enhanced the legacy-syslog parser so that it can interpret messages + that do not contain a TIMESTAMP +- fixed a bug that caused the default socket (usually /dev/log) to be + opened even when -o command line option was given +- fixed a bug in the Debian sample startup script - it caused rsyslogd + to listen to remote requests, which it shouldn't by default +--------------------------------------------------------------------------- +Version 1.12.1 (RGer), 2005-11-23 +- made multithreading work with BSD. Some signal-handling needed to be + restructured. Also, there might be a slight delay of up to 10 seconds + when huping and terminating rsyslogd under BSD +- fixed a bug where a NULL-pointer was passed to printf() in logmsg(). +- fixed a bug during "make install" where rc3195d was not installed + Thanks to Bennett Todd for spotting this. +- fixed a bug where rsyslogd dumped core when no TAG was found in the + received message +- enhanced message parser so that it can deal with missing hostnames + in many cases (may not be totally fail-safe) +- fixed a bug where internally-generated messages did not have the correct + TAG +--------------------------------------------------------------------------- +Version 1.12.0 (RGer), 2005-10-26 +- moved to a multi-threaded design. single-threading is still optionally + available. Multi-threading is experimental! +- fixed a potential race condition. In the original code, marking was done + by an alarm handler, which could lead to all sorts of bad things. This + has been changed now. See comments in syslogd.c/domark() for details. +- improved debug output for property-based filters +- not a code change, but: I have checked all exit()s to make sure that + none occurs once rsyslogd has started up. Even in unusual conditions + (like low-memory conditions) rsyslogd somehow remains active. Of course, + it might loose a message or two, but at least it does not abort and it + can also recover when the condition no longer persists. +- fixed a bug that could cause loss of the last message received + immediately before rsyslogd was terminated. +- added comments on thread-safety of global variables in syslogd.c +- fixed a small bug: spurios printf() when TCP syslog was used +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug with regular expression support (thanks to Andres Riancho) +- a little bit of code restructuring (especially main(), which was + horribly large) +--------------------------------------------------------------------------- +Version 1.11.1 (RGer), 2005-10-19 +- support for BSD-style program name and host blocks +- added a new property "programname" that can be used in templates +- added ability to specify listen port for rfc3195d +- fixed a bug that rendered the "startswith" comparison operation + unusable. +- changed more functions to "static" storage class to help compiler + optimize (should have been static in the first place...) +- fixed a potential memory leak in the string buffer class destructor. + As the destructor was previously never called, the leak did not actually + appear. +- some internal restructuring in anticipation/preparation of minimal + multi-threading support +- rsyslogd still shares some code with the sysklogd project. Some patches + for this shared code have been brought over from the sysklogd CVS. +--------------------------------------------------------------------------- +Version 1.11.0 (RGer), 2005-10-12 +- support for receiving messages via RFC 3195; added rfc3195d for that + purpose +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +--------------------------------------------------------------------------- +Version 1.10.2 (RGer), 2005-09-27 +- added comparison operations in property-based filters: + * isequal + * startswith +- added ability to negate all property-based filter comparison operations + by adding a !-sign right in front of the operation name +- added the ability to specify remote senders for UDP and TCP + received messages. Allows to block all but well-known hosts +- changed the $-config line directives to be case-INsensitive +- new command line option -w added: "do not display warnings if messages + from disallowed senders are received" +- fixed a bug that caused rsyslogd to dump core when the compare value + was not quoted in property-based filters +- fixed a bug in the new CStr compare function which lead to invalid + results (fortunately, this function was not yet used widely) +- added better support for "debugging" rsyslog.conf property filters + (only if -d switch is given) +- changed some function definitions to static, which eventually enables + some compiler optimizations +- fixed a bug in MySQL code; when a SQL error occurred, rsyslogd could + run in a tight loop. This was due to invalid sequence of error reporting + and is now fixed. +--------------------------------------------------------------------------- +Version 1.10.1 (RGer), 2005-09-23 +- added the ability to execute a shell script as an action. + Thanks to Bjoern Kalkbrenner for providing the code! +- fixed a bug in the MySQL code; due to the bug the automatic one-time + retry after an error did not happen - this lead to error message in + cases where none should be seen (e.g. after a MySQL restart) +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.10.0 (RGer), 2005-09-20 + REMINDER: 1.10 is the first unstable version if the 1.x series! +- added the capability to filter on any property in selector lines + (not just facility and priority) +- changed stringbuf into a new counted string class +- added support for a "discard" action. If a selector line with + discard (~ character) is found, no selector lines *after* that + line will be processed. +- thanks to Andres Riancho, regular expression support has been + added to the template engine +- added the FROMHOST property in the template processor, which could + previously not be obtained. Thanks to Cristian Testa for pointing + this out and even providing a fix. +- added display of compile-time options to -v output +- performance improvement for production build - made some checks + to happen only during debug mode +- fixed a problem with compiling on SUSE and - while doing so - removed + the socket call to set SO_BSDCOMPAT in cases where it is obsolete. +--------------------------------------------------------------------------- +Version 1.0.4 (RGer), 2006-02-01 +- a small but important fix: the tcp receiver had two forgotten printf's + in it that caused a lot of unnecessary output to stdout. This was + important enough to justify a new release +--------------------------------------------------------------------------- +Version 1.0.3 (RGer), 2005-11-14 +- added an additional guard to prevent rsyslogd from aborting when the + 2gb file size limit is hit. While a user can configure rsyslogd to + handle such situations, it would abort if that was not done AND large + file support was not enabled (ok, this is hopefully an unlikely scenario) +- fixed a bug that caused additional Unix domain sockets to be incorrectly + processed - could lead to message loss in extreme cases +- applied some patches available from the sysklogd project to code + shared from there +- fixed a bug that causes rsyslogd to dump core on termination when one + of the selector lines did not receive a message during the run (very + unlikely) +- fixed an one-too-low memory allocation in the TCP sender. Could result + in rsyslogd dumping core. +- fixed a bug in the TCP sender that caused the retry logic to fail + after an error or receiver overrun +- fixed a bug in init() that could lead to dumping core +- fixed a bug that could lead to dumping core when no HOSTNAME or no TAG + was present in the syslog message +--------------------------------------------------------------------------- +Version 1.0.2 (RGer), 2005-10-05 +- fixed an issue with MySQL error reporting. When an error occurred, + the MySQL driver went into an endless loop (at least in most cases). +--------------------------------------------------------------------------- +Version 1.0.1 (RGer), 2005-09-23 +- fixed a security issue with SQL-escaping in conjunction with + non-(SQL-)standard MySQL features. +--------------------------------------------------------------------------- +Version 1.0.0 (RGer), 2005-09-12 +- changed install doc to cover daily cron scripts - a trouble source +- added rc script for slackware (provided by Chris Elvidge - thanks!) +- fixed a really minor bug in usage() - the -r option was still + reported as without the port parameter +--------------------------------------------------------------------------- +Version 0.9.8 (RGer), 2005-09-05 +- made startup and shutdown message more consistent and included the + pid, so that they can be easier correlated. Used syslog-protocol + structured data format for this purpose. +- improved config info in startup message, now tells not only + if it is listening remote on udp, but also for tcp. Also includes + the port numbers. The previous startup message was misleading, because + it did not say "remote reception" if rsyslogd was only listening via + tcp (but not via udp). +- added a "how can you help" document to the doc set +--------------------------------------------------------------------------- +Version 0.9.7 (RGer), 2005-08-15 +- some of the previous doc files (like INSTALL) did not properly + reflect the changes to the build process and the new doc. Fixed + that. +- changed syslogd.c so that when compiled without database support, + an error message is displayed when a database action is detected + in the config file (previously this was used as an user rule ;)) +- fixed a bug in the os-specific Makefiles which caused MySQL + support to not be compiled, even if selected +--------------------------------------------------------------------------- +Version 0.9.6 (RGer), 2005-08-09 +- greatly enhanced documentation. Now available in html format in + the "doc" folder and FreeBSD. Finally includes an install howto. +- improved MySQL error messages a little - they now show up as log + messages, too (formerly only in debug mode) +- added the ability to specify the listen port for udp syslog. + WARNING: This introduces an incompatibility. Formerly, udp + syslog was enabled by the -r command line option. Now, it is + "-r [port]", which is consistent with the tcp listener. However, + just -r will now return an error message. +- added sample startup scripts for Debian and FreeBSD +- added support for easy feature selection in the makefile. Un- + fortunately, this also means I needed to spilt the make file + for different OS and distros. There are some really bad syntax + differences between FreeBSD and Linux make. +--------------------------------------------------------------------------- +Version 0.9.5 (RGer), 2005-08-01 +- the "semicolon bug" was actually not (fully) solved in 0.9.4. One + part of the bug was solved, but another still existed. This one + is fixed now, too. +- the "semicolon bug" actually turned out to be a more generic bug. + It appeared whenever an invalid template name was given. With some + selector actions, rsyslogd dumped core, with other it "just" had + a small resource leak with others all worked well. These anomalies + are now fixed. Note that they only appeared during system initialization + once the system was running, nothing bad happened. +- improved error reporting for template errors on startup. They are now + shown on the console and the start-up tty. Formerly, they were only + visible in debug mode. +- support for multiple instances of rsyslogd on a single machine added +- added new option "-o" --> omit local unix domain socket. This option + enables rsyslogd NOT to listen to the local socket. This is most + helpful when multiple instances of rsyslogd (or rsyslogd and another + syslogd) shall run on a single system. +- added new option "-i <pidfile>" which allows one to specify the pidfile. + This is needed when multiple instances of rsyslogd are to be run. +- the new project home page is now online at www.rsyslog.com +--------------------------------------------------------------------------- +Version 0.9.4 (RGer), 2005-07-25 +- finally added the TCP sender. It now supports non-blocking mode, no + longer disabling message reception during connect. As it is now, it + is usable in production. The code could be more sophisticated, but + I've kept it short in anticipation of the move to liblogging, which + will lead to the removal of the code just written ;) +- the "exiting on signal..." message still had the "syslogd" name in + it. Changed this to "rsyslogd", as we do not have a large user base + yet, this should pose no problem. +- fixed "the semicolon" bug. rsyslogd dumped core if a write-db action + was specified but no semicolon was given after the password (an empty + template was ok, but the semicolon needed to be present). +- changed a default for traditional output format. During testing, it + was seen that the timestamp written to file in default format was + the time of message reception, not the time specified in the TIMESTAMP + field of the message itself. Traditionally, the message TIMESTAMP is + used and this has been changed now. +--------------------------------------------------------------------------- +Version 0.9.3 (RGer), 2005-07-19 +- fixed a bug in the message parser. In June, the RFC 3164 timestamp + was not correctly parsed (yes, only in June and some other months, + see the code comment to learn why...) +- added the ability to specify the destination port when forwarding + syslog messages (both for TCP and UDP) +- added an very experimental TCP sender (activated by + @@machine:port in config). This is not yet for production use. If + the receiver is not alive, rsyslogd will wait quite some time until + the connection request times out, which most probably leads to + loss of incoming messages. + +--------------------------------------------------------------------------- +Version 0.9.2 (RGer), around 2005-07-06 +- I intended to change the maxsupported message size to 32k to + support IHE - but given the memory inefficiency in the usual use + cases, I have not done this. I have, however, included very + specific instructions on how to do this in the source code. I have + also done some testing with 32k messages, so you can change the + max size without taking too much risk. +- added a syslog/tcp receiver; we now can receive messages via + plain tcp, but we can still send only via UDP. The syslog/tcp + receiver is the primary enhancement of this release. +- slightly changed some error messages that contained a spurios \n at + the end of the line (which gives empty lines in your log...) + +--------------------------------------------------------------------------- +Version 0.9.1 (RGer) +- fixed code so that it compiles without errors under FreeBSD +- removed now unused function "allocate_log()" from syslogd.c +- changed the make file so that it contains more defines for + different environments (in the long term, we need a better + system for disabling/enabling features...) +- changed some printf's printing off_t types to %lld and + explicit (long long) casts. I tried to figure out the exact type, + but did not succeed in this. In the worst case, ultra-large peta- + byte files will now display funny informational messages on rollover, + something I think we can live with for the next 10 years or so... + +--------------------------------------------------------------------------- +Version 0.9.0 (RGer) +- changed the filed structure to be a linked list. Previously, it + was a table - well, for non-SYSV it was defined as linked list, + but from what I see that code did no longer work after my + modifications. I am now using a linked list in general because + that is needed for other upcoming modifications. +- fixed a bug that caused rsyslogd not to listen to anything if + the configuration file could not be read +- previous versions disabled network logging (send/receive) if + syslog/udp port was not in /etc/services. Now defaulting to + port 514 in this case. +- internal error messages are now supported up to 256 bytes +- error message seen during config file read are now also displayed + to the attached tty and not only the console +- changed some error messages during init to be sent to the console + and/or emergency log. Previously, they were only seen if the + -d (debug) option was present on the command line. +- fixed the "2gb file issue on 32bit systems". If a file grew to + more than 2gb, the syslogd was aborted with "file size exceeded". + Now, defines have been added according to + http://www.daimi.au.dk/~kasperd/comp.os.linux.development.faq.html#LARGEFILE + Testing revealed that they work ;) + HOWEVER, if your file system, glibc, kernel, whatever does not + support files larger 2gb, you need to set a file size limit with + the new output channel mechanism. +- updated man pages to reflect the changes + +--------------------------------------------------------------------------- +Version 0.8.4 + +- improved -d debug output (removed developer-only content) +- now compiles under FreeBSD and NetBSD (only quick testing done on NetBSD) +--------------------------------------------------------------------------- +Version 0.8.3 + +- security model in "make install" changed +- minor doc updates +--------------------------------------------------------------------------- +Version 0.8.2 + +- added man page for rsyslog.conf and rsyslogd +- gave up on the concept of rsyslog being a "drop in" replacement + for syslogd. Now, the user installs rsyslogd and also needs to + adjust his system settings to this specifically. This also lead + to these changes: + * changed Makefile so that install now installs rsyslogd instead + of dealing with syslogd + * changed the default config file name to rsyslog.conf +--------------------------------------------------------------------------- +Version 0.8.1 + +- fixed a nasty memory leak (probably not the last one with this release) +- some enhancements to Makefile as suggested by Bennett Todd +- syslogd-internal messages (like restart) were missing the hostname + this has been corrected +--------------------------------------------------------------------------- +Version 0.8.0 + +Initial testing release. Based on the sysklogd package. Thanks to the +sysklogd maintainers for all their good work! +--------------------------------------------------------------------------- + +---------------------------------------------------------------------- +The following comments were left in the syslogd source. While they provide +not too much detail, the help to date when Rainer started work on the +project (which was 2003, now even surprising for Rainer himself ;)). + * \author Rainer Gerhards <rgerhards@adiscon.com> + * \date 2003-10-17 + * Some initial modifications on the sysklogd package to support + * liblogging. These have actually not yet been merged to the + * source you see currently (but they hopefully will) + * + * \date 2004-10-28 + * Restarted the modifications of sysklogd. This time, we + * focus on a simpler approach first. The initial goal is to + * provide MySQL database support (so that syslogd can log + * to the database). +---------------------------------------------------------------------- +The following comments are from the stock syslogd.c source. They provide +some insight into what happened to the source before we forked +rsyslogd. However, much of the code already has been replaced and more +is to be replaced. So over time, these comments become less valuable. +I have moved them out of the syslogd.c file to shrink it, especially +as a lot of them do no longer apply. For historical reasons and +understanding of how the daemon evolved, they are probably still +helpful. + * Author: Eric Allman + * extensive changes by Ralph Campbell + * more extensive changes by Eric Allman (again) + * + * Steve Lord: Fix UNIX domain socket code, added linux kernel logging + * change defines to + * SYSLOG_INET - listen on a UDP socket + * SYSLOG_UNIXAF - listen on unix domain socket + * SYSLOG_KERNEL - listen to linux kernel + * + * Mon Feb 22 09:55:42 CST 1993: Dr. Wettstein + * Additional modifications to the source. Changed priority scheme + * to increase the level of configurability. In its stock configuration + * syslogd no longer logs all messages of a certain priority and above + * to a log file. The * wildcard is supported to specify all priorities. + * Note that this is a departure from the BSD standard. + * + * Syslogd will now listen to both the inetd and the unixd socket. The + * strategy is to allow all local programs to direct their output to + * syslogd through the unixd socket while the program listens to the + * inetd socket to get messages forwarded from other hosts. + * + * Fri Mar 12 16:55:33 CST 1993: Dr. Wettstein + * Thanks to Stephen Tweedie (dcs.ed.ac.uk!sct) for helpful bug-fixes + * and an enlightened commentary on the prioritization problem. + * + * Changed the priority scheme so that the default behavior mimics the + * standard BSD. In this scenario all messages of a specified priority + * and above are logged. + * + * Add the ability to specify a wildcard (=) as the first character + * of the priority name. Doing this specifies that ONLY messages with + * this level of priority are to be logged. For example: + * + * *.=debug /usr/adm/debug + * + * Would log only messages with a priority of debug to the /usr/adm/debug + * file. + * + * Providing an * as the priority specifies that all messages are to be + * logged. Note that this case is degenerate with specifying a priority + * level of debug. The wildcard * was retained because I believe that + * this is more intuitive. + * + * Thu Jun 24 11:34:13 CDT 1993: Dr. Wettstein + * Modified sources to incorporate changes in libc4.4. Messages from + * syslog are now null-terminated, syslogd code now parses messages + * based on this termination scheme. Linux as of libc4.4 supports the + * fsync system call. Modified code to fsync after all writes to + * log files. + * + * Sat Dec 11 11:59:43 CST 1993: Dr. Wettstein + * Extensive changes to the source code to allow compilation with no + * complaints with -Wall. + * + * Reorganized the facility and priority name arrays so that they + * compatible with the syslog.h source found in /usr/include/syslog.h. + * NOTE that this should really be changed. The reason I do not + * allow the use of the values defined in syslog.h is on account of + * the extensions made to allow the wildcard character in the + * priority field. To fix this properly one should malloc an array, + * copy the contents of the array defined by syslog.h and then + * make whatever modifications that are desired. Next round. + * + * Thu Jan 6 12:07:36 CST 1994: Dr. Wettstein + * Added support for proper decomposition and re-assembly of + * fragment messages on UNIX domain sockets. Lack of this capability + * was causing 'partial' messages to be output. Since facility and + * priority information is encoded as a leader on the messages this + * was causing lines to be placed in erroneous files. + * + * Also added a patch from Shane Alderton (shane@ion.apana.org.au) to + * correct a problem with syslogd dumping core when an attempt was made + * to write log messages to a logged-on user. Thank you. + * + * Many thanks to Juha Virtanen (jiivee@hut.fi) for a series of + * interchanges which lead to the fixing of problems with messages set + * to priorities of none and emerg. Also thanks to Juha for a patch + * to exclude users with a class of LOGIN from receiving messages. + * + * Shane Alderton provided an additional patch to fix zombies which + * were conceived when messages were written to multiple users. + * + * Mon Feb 6 09:57:10 CST 1995: Dr. Wettstein + * Patch to properly reset the single priority message flag. Thanks + * to Christopher Gori for spotting this bug and forwarding a patch. + * + * Wed Feb 22 15:38:31 CST 1995: Dr. Wettstein + * Added version information to startup messages. + * + * Added defines so that paths to important files are taken from + * the definitions in paths.h. Hopefully this will insure that + * everything follows the FSSTND standards. Thanks to Chris Metcalf + * for a set of patches to provide this functionality. Also thanks + * Elias Levy for prompting me to get these into the sources. + * + * Wed Jul 26 18:57:23 MET DST 1995: Martin Schulze + * Linux' gethostname only returns the hostname and not the fqdn as + * expected in the code. But if you call hostname with an fqdn then + * gethostname will return an fqdn, so we have to mention that. This + * has been changed. + * + * The 'LocalDomain' and the hostname of a remote machine is + * converted to lower case, because the original caused some + * inconsistency, because the (at least my) nameserver did respond an + * fqdn containing of upper- _and_ lowercase letters while + * 'LocalDomain' consisted only of lowercase letters and that didn't + * match. + * + * Sat Aug 5 18:59:15 MET DST 1995: Martin Schulze + * Now no messages that were received from any remote host are sent + * out to another. At my domain this missing feature caused ugly + * syslog-loops, sometimes. + * + * Remember that no message is sent out. I can't figure out any + * scenario where it might be useful to change this behavior and to + * send out messages to other hosts than the one from which we + * received the message, but I might be shortsighted. :-/ + * + * Thu Aug 10 19:01:08 MET DST 1995: Martin Schulze + * Added my pidfile.[ch] to it to perform a better handling with + * pidfiles. Now both, syslogd and klogd, can only be started + * once. They check the pidfile. + * + * Sun Aug 13 19:01:41 MET DST 1995: Martin Schulze + * Add an addition to syslog.conf's interpretation. If a priority + * begins with an exclamation mark ('!') the normal interpretation + * of the priority is inverted: ".!*" is the same as ".none", ".!=info" + * don't logs the info priority, ".!crit" won't log any message with + * the priority crit or higher. For example: + * + * mail.*;mail.!=info /usr/adm/mail + * + * Would log all messages of the facility mail except those with + * the priority info to /usr/adm/mail. This makes the syslogd + * much more flexible. + * + * Defined TABLE_ALLPRI=255 and changed some occurrences. + * + * Sat Aug 19 21:40:13 MET DST 1995: Martin Schulze + * Making the table of facilities and priorities while in debug + * mode more readable. + * + * If debugging is turned on, printing the whole table of + * facilities and priorities every hexadecimal or 'X' entry is + * now 2 characters wide. + * + * The number of the entry is prepended to each line of + * facilities and priorities, and F_UNUSED lines are not shown + * anymore. + * + * Corrected some #ifdef SYSV's. + * + * Mon Aug 21 22:10:35 MET DST 1995: Martin Schulze + * Corrected a strange behavior during parsing of configuration + * file. The original BSD syslogd doesn't understand spaces as + * separators between specifier and action. This syslogd now + * understands them. The old behavior caused some confusion over + * the Linux community. + * + * Thu Oct 19 00:02:07 MET 1995: Martin Schulze + * The default behavior has changed for security reasons. The + * syslogd will not receive any remote message unless you turn + * reception on with the "-r" option. + * + * Not defining SYSLOG_INET will result in not doing any network + * activity, i.e. not sending or receiving messages. I changed + * this because the old idea is implemented with the "-r" option + * and the old thing didn't work anyway. + * + * Thu Oct 26 13:14:06 MET 1995: Martin Schulze + * Added another logfile type F_FORW_UNKN. The problem I ran into + * was a name server that runs on my machine and a forwarder of + * kern.crit to another host. The hosts address can only be + * fetched using the nameserver. But named is started after + * syslogd, so syslogd complained. + * + * This logfile type will retry to get the address of the + * hostname ten times and then complain. This should be enough to + * get the named up and running during boot sequence. + * + * Fri Oct 27 14:08:15 1995: Dr. Wettstein + * Changed static array of logfiles to a dynamic array. This + * can grow during process. + * + * Fri Nov 10 23:08:18 1995: Martin Schulze + * Inserted a new tabular sys_h_errlist that contains plain text + * for error codes that are returned from the net subsystem and + * stored in h_errno. I have also changed some wrong lookups to + * sys_errlist. + * + * Wed Nov 22 22:32:55 1995: Martin Schulze + * Added the fabulous strip-domain feature that allows us to + * strip off (several) domain names from the fqdn and only log + * the simple hostname. This is useful if you're in a LAN that + * has a central log server and also different domains. + * + * I have also also added the -l switch do define hosts as + * local. These will get logged with their simple hostname, too. + * + * Thu Nov 23 19:02:56 MET DST 1995: Martin Schulze + * Added the possibility to omit fsyncing of logfiles after every + * write. This will give some performance back if you have + * programs that log in a very verbose manner (like innd or + * smartlist). Thanks to Stephen R. van den Berg <srb@cuci.nl> + * for the idea. + * + * Thu Jan 18 11:14:36 CST 1996: Dr. Wettstein + * Added patch from beta-testers to stop compile error. Also + * added removal of pid file as part of termination cleanup. + * + * Wed Feb 14 12:42:09 CST 1996: Dr. Wettstein + * Allowed forwarding of messages received from remote hosts to + * be controlled by a command-line switch. Specifying -h allows + * forwarding. The default behavior is to disable forwarding of + * messages which were received from a remote host. + * + * Parent process of syslogd does not exit until child process has + * finished initialization process. This allows rc.* startup to + * pause until syslogd facility is up and operating. + * + * Re-arranged the select code to move UNIX domain socket accepts + * to be processed later. This was a contributed change which + * has been proposed to correct the delays sometimes encountered + * when syslogd starts up. + * + * Minor code cleanups. + * + * Thu May 2 15:15:33 CDT 1996: Dr. Wettstein + * Fixed bug in init function which resulted in file descriptors + * being orphaned when syslogd process was re-initialized with SIGHUP + * signal. Thanks to Edvard Tuinder + * (Edvard.Tuinder@praseodymium.cistron.nl) for putting me on the + * trail of this bug. I am amazed that we didn't catch this one + * before now. + * + * Tue May 14 00:03:35 MET DST 1996: Martin Schulze + * Corrected a mistake that causes the syslogd to stop logging at + * some virtual consoles under Linux. This was caused by checking + * the wrong error code. Thanks to Michael Nonweiler + * <mrn20@hermes.cam.ac.uk> for sending me a patch. + * + * Mon May 20 13:29:32 MET DST 1996: Miquel van Smoorenburg <miquels@cistron.nl> + * Added continuation line supported and fixed a bug in + * the init() code. + * + * Tue May 28 00:58:45 MET DST 1996: Martin Schulze + * Corrected behavior of blocking pipes - i.e. the whole system + * hung. Michael Nonweiler <mrn20@hermes.cam.ac.uk> has sent us + * a patch to correct this. A new logfile type F_PIPE has been + * introduced. + * + * Mon Feb 3 10:12:15 MET DST 1997: Martin Schulze + * Corrected behavior of logfiles if the file can't be opened. + * There was a bug that causes syslogd to try to log into non + * existing files which ate cpu power. + * + * Sun Feb 9 03:22:12 MET DST 1997: Martin Schulze + * Modified syslogd.c to not kill itself which confuses bash 2.0. + * + * Mon Feb 10 00:09:11 MET DST 1997: Martin Schulze + * Improved debug code to decode the numeric facility/priority + * pair into textual information. + * + * Tue Jun 10 12:35:10 MET DST 1997: Martin Schulze + * Corrected freeing of logfiles. Thanks to Jos Vos <jos@xos.nl> + * for reporting the bug and sending an idea to fix the problem. + * + * Tue Jun 10 12:51:41 MET DST 1997: Martin Schulze + * Removed sleep(10) from parent process. This has caused a slow + * startup in former times - and I don't see any reason for this. + * + * Sun Jun 15 16:23:29 MET DST 1997: Michael Alan Dorman + * Some more glibc patches made by <mdorman@debian.org>. + * + * Thu Jan 1 16:04:52 CET 1998: Martin Schulze <joey@infodrom.north.de + * Applied patch from Herbert Thielen <Herbert.Thielen@lpr.e-technik.tu-muenchen.de>. + * This included some balance parentheses for emacs and a bug in + * the exclamation mark handling. + * + * Fixed small bug which caused syslogd to write messages to the + * wrong logfile under some very rare conditions. Thanks to + * Herbert Xu <herbert@gondor.apana.org.au> for fiddling this out. + * + * Thu Jan 8 22:46:35 CET 1998: Martin Schulze <joey@infodrom.north.de> + * Reworked one line of the above patch as it prevented syslogd + * from binding the socket with the result that no messages were + * forwarded to other hosts. + * + * Sat Jan 10 01:33:06 CET 1998: Martin Schulze <joey@infodrom.north.de> + * Fixed small bugs in F_FORW_UNKN mechanism. Thanks to Torsten + * Neumann <torsten@londo.rhein-main.de> for pointing me to it. + * + * Mon Jan 12 19:50:58 CET 1998: Martin Schulze <joey@infodrom.north.de> + * Modified debug output concerning remote reception. + * + * Mon Feb 23 23:32:35 CET 1998: Topi Miettinen <Topi.Miettinen@ml.tele.fi> + * Re-worked handling of Unix and UDP sockets to support closing / + * opening of them in order to have it open only if it is needed + * either for forwarding to a remote host or by reception from + * the network. + * + * Wed Feb 25 10:54:09 CET 1998: Martin Schulze <joey@infodrom.north.de> + * Fixed little comparison mistake that prevented the MARK + * feature to work properly. + * + * Wed Feb 25 13:21:44 CET 1998: Martin Schulze <joey@infodrom.north.de> + * Corrected Topi's patch as it prevented forwarding during + * startup due to an unknown LogPort. + * + * Sat Oct 10 20:01:48 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Added support for TESTING define which will turn syslogd into + * stdio-mode used for debugging. + * + * Sun Oct 11 20:16:59 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Reworked the initialization/fork code. Now the parent + * process activates a signal handler which the daughter process + * will raise if it is initialized. Only after that one the + * parent process may exit. Otherwise klogd might try to flush + * its log cache while syslogd can't receive the messages yet. + * + * Mon Oct 12 13:30:35 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Redirected some error output with regard to argument parsing to + * stderr. + * + * Mon Oct 12 14:02:51 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Applied patch provided vom Topi Miettinen with regard to the + * people from OpenBSD. This provides the additional '-a' + * argument used for specifying additional UNIX domain sockets to + * listen to. This is been used with chroot()'ed named's for + * example. See for http://www.psionic.com/papers/dns.html + * + * Mon Oct 12 18:29:44 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Added `ftp' facility which was introduced in glibc version 2. + * It's #ifdef'ed so won't harm with older libraries. + * + * Mon Oct 12 19:59:21 MET DST 1998: Martin Schulze <joey@infodrom.north.de> + * Code cleanups with regard to bsd -> posix transition and + * stronger security (buffer length checking). Thanks to Topi + * Miettinen <tom@medialab.sonera.net> + * . index() --> strchr() + * . sprintf() --> snprintf() + * . bcopy() --> memcpy() + * . bzero() --> memset() + * . UNAMESZ --> UT_NAMESIZE + * . sys_errlist --> strerror() + * + * Mon Oct 12 20:22:59 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Added support for setutent()/getutent()/endutent() instead of + * binary reading the UTMP file. This is the the most portable + * way. This allows /var/run/utmp format to change, even to a + * real database or utmp daemon. Also if utmp file locking is + * implemented in libc, syslog will use it immediately. Thanks + * to Topi Miettinen <tom@medialab.sonera.net>. + * + * Mon Oct 12 20:49:18 MET DST 1998: Martin Schulze <joey@infodrom.north.de> + * Avoid logging of SIGCHLD when syslogd is in the process of + * exiting and closing its files. Again thanks to Topi. + * + * Mon Oct 12 22:18:34 CEST 1998: Martin Schulze <joey@infodrom.north.de> + * Modified printline() to support 8bit characters - such as + * russian letters. Thanks to Vladas Lapinskas <lapinskas@mail.iae.lt>. + * + * Sat Nov 14 02:29:37 CET 1998: Martin Schulze <joey@infodrom.north.de> + * ``-m 0'' now turns of MARK logging entirely. + * + * Tue Jan 19 01:04:18 MET 1999: Martin Schulze <joey@infodrom.north.de> + * Finally fixed an error with `-a' processing, thanks to Topi + * Miettinen <tom@medialab.sonera.net>. + * + * Sun May 23 10:08:53 CEST 1999: Martin Schulze <joey@infodrom.north.de> + * Removed superfluous call to utmpname(). The path to the utmp + * file is defined in the used libc and should not be hardcoded + * into the syslogd binary referring the system it was compiled on. + * + * Sun Sep 17 20:45:33 CEST 2000: Martin Schulze <joey@infodrom.ffis.de> + * Fixed some bugs in printline() code that did not escape + * control characters '\177' through '\237' and contained a + * single-byte buffer overflow. Thanks to Solar Designer + * <solar@false.com>. + * + * Sun Sep 17 21:26:16 CEST 2000: Martin Schulze <joey@infodrom.ffis.de> + * Don't close open sockets upon reload. Thanks to Bill + * Nottingham. + * + * Mon Sep 18 09:10:47 CEST 2000: Martin Schulze <joey@infodrom.ffis.de> + * Fixed bug in printchopped() that caused syslogd to emit + * kern.emerg messages when splitting long lines. Thanks to + * Daniel Jacobowitz <dan@debian.org> for the fix. + * + * Mon Sep 18 15:33:26 CEST 2000: Martin Schulze <joey@infodrom.ffis.de> + * Removed unixm/unix domain sockets and switch to Datagram Unix + * Sockets. This should remove one possibility to play DoS with + * syslogd. Thanks to Olaf Kirch <okir@caldera.de> for the patch. + * + * Sun Mar 11 20:23:44 CET 2001: Martin Schulze <joey@infodrom.ffis.de> + * Don't return a closed fd if `-a' is called with a wrong path. + * Thanks to Bill Nottingham <notting@redhat.com> for providing + * a patch. |