From b5896ba9f6047e7031e2bdee0622d543e11a6734 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Mon, 6 May 2024 03:46:30 +0200 Subject: Adding upstream version 3.4.23. Signed-off-by: Daniel Baumann --- proto/STRESS_README.html | 566 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 566 insertions(+) create mode 100644 proto/STRESS_README.html (limited to 'proto/STRESS_README.html') diff --git a/proto/STRESS_README.html b/proto/STRESS_README.html new file mode 100644 index 0000000..b5e3ccb --- /dev/null +++ b/proto/STRESS_README.html @@ -0,0 +1,566 @@ + + + + + + +Postfix Stress-Dependent Configuration + + + + + + + +

Postfix +Stress-Dependent Configuration

+ +

Overview

+ +

This document describes the symptoms of Postfix SMTP server +overload. It presents permanent main.cf changes to avoid overload +during normal operation, and temporary main.cf changes to cope with +an unexpected burst of mail. This document makes specific suggestions +for Postfix 2.5 and later which support stress-adaptive behavior, +and for earlier Postfix versions that don't.

+ +

Topics covered in this document:

+ +

Symptoms of Postfix SMTP server overload + +
Automatic stress-adaptive behavior + +
Service more SMTP clients at the same time + +
Spend less time per SMTP client + +
Disconnect suspicious SMTP clients + +
Temporary measures for older Postfix releases + +
Detecting support for stress-adaptive behavior + +
Forcing stress-adaptive behavior on or off + +
Other measures to off-load zombies + +
Credits + +

+ +

Symptoms of Postfix SMTP server overload

+ +

Under normal conditions, the Postfix SMTP server responds +immediately when an SMTP client connects to it; the time to deliver +mail is noticeable only with large messages. Performance degrades +dramatically when the number of SMTP clients exceeds the number of +Postfix SMTP server processes. When an SMTP client connects while +all Postfix SMTP server processes are busy, the client must wait +until a server process becomes available.

+ +

SMTP server overload may be caused by a surge of legitimate +mail (example: a DNS registrar opens a new zone for registrations), +by mistake (mail explosion caused by a forwarding loop) or by malice +(worm outbreak, botnet, or other illegitimate activity).

+ +

Symptoms of Postfix SMTP server overload are:

+ +

Remote SMTP clients experience a long delay before Postfix +sends the "220 hostname.example.com ESMTP Postfix" greeting.
+ +
- NOTE: Broken DNS configurations can also cause lengthy +delays before Postfix sends "220 hostname.example.com ...". These +delays also exist when Postfix is NOT overloaded.
  + +
- NOTE: To avoid "overload" delays for end-user mail +clients, enable the "submission" service entry in master.cf (present +since Postfix 2.1), and tell users to connect to this instead of +the public SMTP service.
  + +
+ +
The Postfix SMTP server logs an increased number of "lost +connection after CONNECT" events. This happens because remote SMTP +clients disconnect before Postfix answers the connection.
+ +
- NOTE: A portscan for open SMTP ports can also result in +"lost connection ..." logfile messages.
  + +
+ +

Postfix 2.3 and later logs a warning that all server ports +are busy:

+ +

+Oct  3 20:39:27 spike postfix/master[28905]: warning: service "smtp"
+ (25) has reached its process limit "30": new clients may experience
+ noticeable delays
+Oct  3 20:39:27 spike postfix/master[28905]: warning: to avoid this
+ condition, increase the process count in master.cf or reduce the
+ service time per client
+Oct  3 20:39:27 spike postfix/master[28905]: warning: see
+  http://www.postfix.org/STRESS_README.html for examples of
+  stress-adapting configuration settings
+

+ +

Legitimate mail that doesn't get through during an episode of +Postfix SMTP server overload is not necessarily lost. It should +still arrive once the situation returns to normal, as long as the +overload condition is temporary.

+ +

Automatic stress-adaptive behavior

+ +

Postfix version 2.5 introduces automatic stress-adaptive behavior. +It works as follows. When a "public" network service such as the +SMTP server runs into an "all server ports are busy" condition, the +Postfix master(8) daemon logs a warning, restarts the service +(without interrupting existing network sessions), and runs the +service with "-o stress=yes" on the server process command line: +

+ +

+80821  ??  S      0:00.24 smtpd -n smtp -t inet -u -c -o stress=yes
+

+ +

Normally, the Postfix master(8) daemon runs such a service with +"-o stress=" on the command line (i.e. with an empty parameter +value):

+ +

+83326  ??  S      0:00.28 smtpd -n smtp -t inet -u -c -o stress=
+

+ +

You won't see "-o stress" command-line parameters with services +that have local clients only. These include services internal to +Postfix such as the queue manager, and services that listen on a +loopback interface only, such as after-filter SMTP services.

+ +

The "stress" parameter value is the key to making main.cf +parameter settings stress adaptive. The following settings are the +default with Postfix 2.6 and later.

+ +

+1 smtpd_timeout = ${stress?{10}:{300}}s
+2 smtpd_hard_error_limit = ${stress?{1}:{20}}
+3 smtpd_junk_command_limit = ${stress?{1}:{100}}
+4 # Parameters added after Postfix 2.6:
+5 smtpd_per_record_deadline = ${stress?{yes}:{no}}
+6 smtpd_starttls_timeout = ${stress?{10}:{300}}s
+7 address_verify_poll_count = ${stress?{1}:{3}}
+

+ +

Postfix versions before 3.0 use the older form ${stress?x}${stress:y} +instead of the newer form ${stress?{x}:{y}}.

+ +

The syntax of ${name?{value}:{value}}, ${name?value} and +${name:value} is explained at the beginning of the postconf(5) +manual page.

+ +

Translation:

+ +

Line 1: under conditions of stress, use an smtpd_timeout +value of 10 seconds instead of the default 300 seconds. Experience +on the postfix-users list from a variety of sysadmins shows that +reducing the "normal" smtpd_timeout to 60s is unlikely to affect +legitimate clients. However, it is unlikely to become the Postfix +default because it's not RFC compliant. Setting smtpd_timeout to +10s or even 5s under stress will still allow most +legitimate clients to connect and send mail, but may delay mail +from some clients. No mail should be lost, as long as this measure +is used only temporarily.
+ +
Line 2: under conditions of stress, use an smtpd_hard_error_limit +of 1 instead of the default 20. This disconnects clients +after a single error, giving other clients a chance to connect. +However, this may cause significant delays with legitimate mail, +such as a mailing list that contains a few no-longer-active user +names that didn't bother to unsubscribe. No mail should be lost, +as long as this measure is used only temporarily.
+ +
Line 3: under conditions of stress, use an +smtpd_junk_command_limit of 1 instead of the default 100. This +prevents clients from keeping connections open by repeatedly +sending HELO, EHLO, NOOP, RSET, VRFY or ETRN commands.
+ +
Line 5: under conditions of stress, change the behavior +of smtpd_timeout and smtpd_starttls_timeout, from a time limit per +read or write system call, to a time limit to send or receive a +complete record (an SMTP command line, SMTP response line, SMTP +message content line, or TLS protocol message).
+ +
Line 6: under conditions of stress, reduce the time limit +for TLS protocol handshake messages to 10 seconds, from the default +value of 300 seconds. See also the smtpd_timeout discussion above. +
+ +
Line 7: under conditions of stress, do not wait up to 6 +seconds for the completion of an address verification probe. If the +result is not already in the address verification cache, reply +immediately with $unverified_recipient_tempfail_action or +$unverified_sender_tempfail_action. No mail should be lost, as long +as this measure is used only temporarily.
+ +

+ +

NOTE: Please keep in mind that the stress-adaptive feature is +a fairly desperate measure to keep some legitimate mail +flowing under overload conditions. If a site is reaching the SMTP +server process limit when there isn't an attack or bot flood +occurring, then either the process limit needs to be raised or more +hardware needs to be added.

+ +

Service more SMTP clients at the same time

+ +

This section and the ones that follow discuss permanent measures +against mail server overload.

+ +

One measure to avoid the "all server processes busy" condition +is to service more SMTP clients simultaneously. For this you need +to increase the number of Postfix SMTP server processes. This will +improve the +responsiveness for remote SMTP clients, as long as the server machine +has enough hardware and software resources to run the additional +processes, and as long as the file system can keep up with the +additional load.

+ +

You increase the number of SMTP server processes either +by increasing the default_process_limit in main.cf (line 3 below), +or by increasing the SMTP server's "maxproc" field in master.cf +(line 10 below). Either way, you need to issue a "postfix reload" +command to make the change effective.
+ +
Process limits above 1000 require Postfix version 2.4 or +later, and an operating system that supports kernel-based event +filters (BSD kqueue(2), Linux epoll(4), or Solaris /dev/poll). +
+ +

More processes use more memory. You can reduce the Postfix +memory footprint by using cdb: +lookup tables instead of Berkeley DB's hash: or btree: tables.

+ +

+ 1 /etc/postfix/main.cf:
+ 2     # Raise the global process limit, 100 since Postfix 2.0.
+ 3     default_process_limit = 200
+ 4
+ 5 /etc/postfix/master.cf:
+ 6     # =============================================================
+ 7     # service type  private unpriv  chroot  wakeup  maxproc command
+ 8     # =============================================================
+ 9     # Raise the SMTP service process limit only.
+10     smtp      inet  n       -       n       -       200     smtpd
+

+ +

NOTE: older versions of the SMTPD_POLICY_README document +contain a mistake: they configure a fixed number of policy daemon +processes. When you raise the SMTP server's "maxproc" field in +master.cf, SMTP server processes will report problems when connecting +to policy server processes, because there aren't enough of them. +Examples of errors are "connection refused" or "operation timed +out".
+ +
To fix, edit master.cf and specify a zero "maxproc" field +in all policy server entries; see line 6 in the example below. +Issue a "postfix reload" command to make the change effective.
+ +
```
+1 /etc/postfix/master.cf:
+2     # =============================================================
+3     # service type  private unpriv  chroot  wakeup  maxproc command
+4     # =============================================================
+5     # Disable the policy service process limit.
+6     policy    unix  -       n       n       -       0       spawn
+7         user=nobody argv=/some/where/policy-server
+
```
+ +

+ +

Spend less time per SMTP client

+ +

When increasing the number of SMTP server processes is not +practical, you can improve Postfix server responsiveness by eliminating +delays. When Postfix spends less time per SMTP session, the same +number of SMTP server processes can service more clients in a given +amount of time.

+ +

Eliminate non-functional RBL lookups (blocklists that are +no longer in operation). These lookups can degrade performance. +Postfix logs a warning when an RBL server does not respond.
+ +
Eliminate redundant RBL lookups (people often use multiple +Spamhaus RBLs that include each other). To find out whether RBLs +include other RBLs, look up the websites that document the RBL's +policies.
+ +
Eliminate header_checks and body_checks, and keep just a few +emergency patterns to block the latest worm explosion or backscatter +mail. See BACKSCATTER_README for examples of the latter. + +

Group your header_checks and body_checks patterns to avoid +unnecessary pattern matching operations: + +

+ 1  /etc/postfix/header_checks:
+ 2      if /^Subject:/
+ 3      /^Subject: virus found in mail from you/ reject
+ 4      /^Subject: ..other../ reject
+ 5      endif
+ 6  
+ 7      if /^Received:/
+ 8      /^Received: from (postfix\.org) / reject forged client name in received header: $1
+ 9      /^Received: from ..other../ reject ....
+10      endif
+

+ +

Disconnect suspicious SMTP clients

+ +

Under conditions of overload you can improve Postfix SMTP server +responsiveness by hanging up on suspicious clients, so that other +clients get a chance to talk to Postfix.

+ +

Use "521" SMTP reply codes (Postfix 2.6 and later) or "421" +(Postfix 2.3-2.5) to hang up on clients that that match botnet-related +RBLs (see next bullet) or that match selected non-RBL restrictions +such as SMTP access maps. The Postfix SMTP server will reject mail +and disconnect without waiting for the remote SMTP client to send +a QUIT command.
+ +

To hang up connections from blacklisted zombies, you can +set specific Postfix SMTP server reject codes for specific RBLs, +and for individual responses from specific RBLs. We'll use +zen.spamhaus.org as an example; by the time you read this document, +details may have changed. Right now, their documents say that a +response of 127.0.0.10 or 127.0.0.11 indicates a dynamic client IP +address, which means that the machine is probably running a bot of +some kind. To give a 521 response instead of the default 554 +response, use something like:

+ +

+ 1  /etc/postfix/main.cf:
+ 2      smtpd_client_restrictions =
+ 3         permit_mynetworks
+ 4         reject_rbl_client zen.spamhaus.org=127.0.0.10
+ 5         reject_rbl_client zen.spamhaus.org=127.0.0.11
+ 6         reject_rbl_client zen.spamhaus.org
+ 7  
+ 8      rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
+ 9  
+10  /etc/postfix/rbl_reply_maps:
+11      # With Postfix 2.3-2.5 use "421" to hang up connections.
+12      zen.spamhaus.org=127.0.0.10 521 4.7.1 Service unavailable;
+13       $rbl_class [$rbl_what] blocked using
+14       $rbl_domain${rbl_reason?; $rbl_reason}
+15  
+16      zen.spamhaus.org=127.0.0.11 521 4.7.1 Service unavailable;
+17       $rbl_class [$rbl_what] blocked using
+18       $rbl_domain${rbl_reason?; $rbl_reason}
+

+ +

Although the above example shows three RBL lookups (lines 4-6), +Postfix will only do a single DNS query, so it does not affect the +performance.

+ +

With Postfix 2.3-2.5, use reply code 421 (521 will not +cause Postfix to disconnect). The down-side of replying with 421 +is that it works only for zombies and other malware. If the client +is running a real MTA, then it may connect again several times until +the mail expires in its queue. When this is a problem, stick with +the default 554 reply, and use "smtpd_hard_error_limit = 1" as +described below.
+ +
You can automatically turn on the above overload measure +with Postfix 2.5 and later, or with earlier releases that contain +the stress-adaptive behavior source code patch from the mirrors +listed at http://www.postfix.org/download.html. Simply replace line +above 8 with:
+ +
```
+ 8      rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps}
+
```
+ +

+ +

More information about automatic stress-adaptive behavior is +in section "Automatic stress-adaptive behavior". +

+ +

Temporary measures for older Postfix releases

+ +

See the section "Automatic stress-adaptive +behavior" if you are running Postfix version 2.5 or later, or +if you have applied the source code patch for stress-adaptive +behavior from the mirrors listed at http://www.postfix.org/download.html. +

+ +

The following measures can be applied temporarily during overload. +They still allow most legitimate clients to connect and send +mail, but may affect some legitimate clients.

+ +

Reduce smtpd_timeout (default: 300s). Experience on the +postfix-users list from a variety of sysadmins shows that reducing +the "normal" smtpd_timeout to 60s is unlikely to affect legitimate +clients. However, it is unlikely to become the Postfix default +because it's not RFC compliant. Setting smtpd_timeout to 10s (line +2 below) or even 5s under stress will still allow most +legitimate clients to connect and send mail, but may delay mail +from some clients. No mail should be lost, as long as this measure +is used only temporarily.
+ +
Reduce smtpd_hard_error_limit (default: 20). Setting this +to 1 under stress (line 3 below) helps by disconnecting clients +after a single error, giving other clients a chance to connect. +However, this may cause significant delays with legitimate mail, +such as a mailing list that contains a few no-longer-active user +names that didn't bother to unsubscribe. No mail should be lost, +as long as this measure is used only temporarily.
+ +
Use an smtpd_junk_command_limit of 1 instead of the default +100. This prevents clients from keeping idle connections open by +repeatedly sending NOOP or RSET commands.
+ +

+ +

+1  /etc/postfix/main.cf:
+2      smtpd_timeout = 10
+3      smtpd_hard_error_limit = 1
+4      smtpd_junk_command_limit = 1
+

+ +

With these measures, no mail should be lost, as long +as these measures are used only temporarily. The next section of +this document introduces a way to automate this process.

+ +

Detecting support for stress-adaptive behavior

+ +

To find out if your Postfix installation supports stress-adaptive +behavior, use the "ps" command, and look for the smtpd processes. +Postfix has stress-adaptive support when you see "-o stress=" or +"-o stress=yes" command-line options. Remember that Postfix never +enables stress-adaptive behavior on servers that listen on local +addresses only.

+ +

The following example is for FreeBSD or Linux. On Solaris, HP-UX +and other System-V flavors, use "ps -ef" instead of "ps ax".

+ +

+$ ps ax|grep smtpd
+83326  ??  S      0:00.28 smtpd -n smtp -t inet -u -c -o stress=
+84345  ??  Ss     0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd-policy.pl
+

+ +

You can't use postconf(1) to detect stress-adaptive support. +The postconf(1) command ignores the existence of the stress parameter +in main.cf, because the parameter has no effect there. Command-line +"-o parameter" settings always take precedence over main.cf parameter +settings.

+ +

If you configure stress-adaptive behavior in main.cf when it +isn't supported, nothing bad will happen. The processes will run +as if the stress parameter always has an empty value.

+ +

Forcing stress-adaptive behavior on or off

+ +

You can manually force stress-adaptive behavior on, by adding +a "-o stress=yes" command-line option in master.cf. This can be +useful for testing overrides on the SMTP service. Issue "postfix +reload" to make the change effective.

+ +

Note: setting the stress parameter in main.cf has no effect for +services that accept remote connections.

+ +

+1 /etc/postfix/master.cf:
+2     # =============================================================
+3     # service type  private unpriv  chroot  wakeup  maxproc command
+4     # =============================================================
+5     # 
+6     smtp      inet  n       -       n       -       -       smtpd
+7         -o stress=yes
+8         -o . . .
+

+ +

To permanently force stress-adaptive behavior off with a specific +service, specify "-o stress=" on its master.cf command line. This +may be desirable for the "submission" service. Issue "postfix reload" +to make the change effective.

+ +

Note: setting the stress parameter in main.cf has no effect for +services that accept remote connections.

+ +

+1 /etc/postfix/master.cf:
+2     # =============================================================
+3     # service type  private unpriv  chroot  wakeup  maxproc command
+4     # =============================================================
+5     # 
+6     submission inet n       -       n       -       -       smtpd
+7         -o stress=
+8         -o . . .
+

+ +

Other measures to off-load zombies

+ +

The postscreen(8) daemon, introduced with Postfix 2.8, provides +additional protection against mail server overload. One postscreen(8) +process handles multiple inbound SMTP connections, and decides which +clients may to talk to a Postfix SMTP server process. By keeping +spambots away, postscreen(8) leaves more SMTP server processes +available for legitimate clients, and delays the onset of server +overload conditions.

+ +

Credits

+ +

Thanks to the postfix-users mailing list members for sharing +early experiences with the stress-adaptive feature. + +
The RBL example and several other paragraphs of text were +adapted from postfix-users postings by Noel Jones. + +
Wietse implemented stress-adaptive behavior as the smallest +possible patch while he should be working on other things. + +

+ + -- cgit v1.2.3