summaryrefslogtreecommitdiffstats
path: root/README_FILES/TUNING_README
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 16:18:56 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 16:18:56 +0000
commitb7c15c31519dc44c1f691e0466badd556ffe9423 (patch)
treef944572f288bab482a615e09af627d9a2b6727d8 /README_FILES/TUNING_README
parentInitial commit. (diff)
downloadpostfix-b7c15c31519dc44c1f691e0466badd556ffe9423.tar.xz
postfix-b7c15c31519dc44c1f691e0466badd556ffe9423.zip
Adding upstream version 3.7.10.upstream/3.7.10upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'README_FILES/TUNING_README')
-rw-r--r--README_FILES/TUNING_README494
1 files changed, 494 insertions, 0 deletions
diff --git a/README_FILES/TUNING_README b/README_FILES/TUNING_README
new file mode 100644
index 0000000..1080164
--- /dev/null
+++ b/README_FILES/TUNING_README
@@ -0,0 +1,494 @@
+ PPoossttffiixx PPeerrffoorrmmaannccee TTuunniinngg
+
+-------------------------------------------------------------------------------
+
+PPuurrppoossee ooff PPoossttffiixx ppeerrffoorrmmaannccee ttuunniinngg
+
+The hints and tips in this document help you improve the performance of Postfix
+systems that already work. If your Postfix system is unable to receive or
+deliver mail, then you need to solve those problems first, using the
+DEBUG_README document as guidance.
+
+For tuning external content filter performance, first read the respective
+information in the FILTER_README and SMTPD_PROXY_README documents. Then make
+sure to avoid latency in the content filter code. As much as possible avoid
+performing queries against external data sources with a high or highly variable
+delay. Your content filter will run with a small concurrency to avoid CPU/
+memory starvation, and if any latency creeps in, content filter throughput will
+suffer. High volume environments should avoid RBL lookups, complex database
+queries and so on.
+
+Topics on mail receiving performance:
+
+ * General mail receiving performance tips
+ * Doing more work with your SMTP server processes
+ * Slowing down SMTP clients that make many errors
+ * Measures against clients that make too many connections
+
+Topics on mail delivery performance:
+
+ * General mail delivery performance tips
+ * Tuning the frequency of deferred mail delivery attempts
+ * Tuning the number of simultaneous deliveries
+ * Tuning the number of recipients per delivery
+
+Other Postfix performance tuning topics:
+
+ * Tuning the number of Postfix processes
+ * Tuning the number of processes on the system
+ * Tuning the number of open files or sockets
+
+The following tools can be used to measure mail system performance under
+artificial loads. They are normally not installed with Postfix.
+
+ * smtp-source, SMTP/LMTP message generator
+ * smtp-sink, SMTP/LMTP message dump
+ * qmqp-source, QMQP message generator
+ * qmqp-sink, QMQP message dump
+
+GGeenneerraall mmaaiill rreecceeiivviinngg ppeerrffoorrmmaannccee ttiippss
+
+ * Read and understand the maildrop queue, incoming queue, and active queue
+ discussions in the QSHAPE_README document.
+
+ * Run a local name server to reduce slow-down due to DNS lookups. If you run
+ multiple Postfix systems, point each local name server to a shared
+ forwarding server to reduce the number of lookups across the upstream
+ network link.
+
+ * Eliminate unnecessary LDAP lookups, by specifying a domain filter. This
+ eliminates lookups for addresses in remote domains, and eliminates lookups
+ of partial addresses. See ldap_table(5) for details.
+
+When Postfix responds slowly to SMTP clients:
+
+ * Look for obvious signs of trouble as described in the DEBUG_README
+ document, and eliminate those problems first.
+
+ * Turn off your header_checks and body_checks patterns and see if the problem
+ goes away.
+
+ * Turn off chroot operation as described in the DEBUG_README document and see
+ if the problem goes away.
+
+ * If Postfix logs the SMTP client as "unknown" then you have a name service
+ problem: the name server is bad, or the resolv.conf file contains bad
+ information, or some packet filter is blocking the DNS requests or replies.
+
+ * If the number of smtpd(8) processes has reached the process limit as
+ specified in master.cf, new SMTP clients must wait until a process becomes
+ available. See the STRESS_README and POSTSCREEN_README documents for
+ measures that help to prevent SMTP server overload.
+
+DDooiinngg mmoorree wwoorrkk wwiitthh yyoouurr SSMMTTPP sseerrvveerr pprroocceesssseess
+
+With Postfix versions 2.0 and earlier, the smtpd(8) server pauses before
+reporting an error to an SMTP client. The idea is called tar pitting. However,
+these delays also slow down Postfix. When the smtpd(8) server replies slowly,
+sessions take more time, so that more smtpd(8) server processes are needed to
+handle the load. When your Postfix smtpd(8) server process limit is reached,
+new clients must wait until a server process becomes available. This means that
+all clients experience poor performance.
+
+You can speed up the handling of smtpd(8) server error replies by turning off
+the delay:
+
+ /etc/postfix/main.cf:
+ # Not needed with Postfix 2.1
+ smtpd_error_sleep_time = 0
+
+With the above setting, Postfix 2.0 and earlier can serve more SMTP clients
+with the same number SMTP server processes. The next section describes how
+Postfix deals with clients that make a large number of errors.
+
+SSlloowwiinngg ddoowwnn SSMMTTPP cclliieennttss tthhaatt mmaakkee mmaannyy eerrrroorrss
+
+The Postfix smtpd(8) server maintains a per-session error count. The error
+count is reset when a message is transferred successfully, and is incremented
+when a client request is unrecognized or unimplemented, when a client request
+violates access restrictions, or when some other error happens.
+
+As the per-session error count increases, the smtpd(8) server changes behavior
+and begins to insert delays into the responses. The idea is to slow down a run-
+away client in order to limit resource usage. The behavior is Postfix version
+dependent.
+
+IMPORTANT: These delays slow down Postfix, too. When too much delay is
+configured, the number of simultaneous SMTP sessions will increase until it
+reaches the smtpd(8) server process limit, and new SMTP clients must wait until
+an smtpd(8) server process becomes available.
+
+Postfix version 2.1 and later:
+
+ * When the error count reaches $smtpd_soft_error_limit (default: 10), the
+ Postfix smtpd(8) server delays all non-error and error responses by
+ $smtpd_error_sleep_time seconds (default: 1 second).
+
+ * When the error count reaches $smtpd_hard_error_limit (default: 20) the
+ Postfix smtpd(8) server breaks the connection.
+
+Postfix version 2.0 and earlier:
+
+ * When the error count is less than $smtpd_soft_error_limit (default: 10) the
+ Postfix smtpd(8) server delays all error replies by $smtpd_error_sleep_time
+ (1 second with Postfix 2.0, 5 seconds with Postfix 1.1 and earlier).
+
+ * When the error count reaches $smtpd_soft_error_limit, the Postfix smtpd(8)
+ server delays all responses by "error count" seconds or
+ $smtpd_error_sleep_time, whichever is more.
+
+ * When the error count reaches $smtpd_hard_error_limit (default: 20) the
+ Postfix smtpd(8) server breaks the connection.
+
+MMeeaassuurreess aaggaaiinnsstt cclliieennttss tthhaatt mmaakkee ttoooo mmaannyy ccoonnnneeccttiioonnss
+
+Note: these features use the Postfix anvil(8) service, introduced with Postfix
+version 2.2.
+
+The Postfix smtpd(8) server can limit the number of simultaneous connections
+from the same SMTP client, as well as the connection rate and the rate of
+certain SMTP commands from the same client. These statistics are maintained by
+the anvil(8) server (translation: if anvil(8) breaks, then connection limits
+stop working).
+
+IMPORTANT: These limits must not be used to regulate legitimate traffic: mail
+will suffer grotesque delays if you do so. The limits are designed to protect
+the smtpd(8) server against abuse by out-of-control clients.
+
+ smtpd_client_connection_count_limit (default: 50)
+ The maximum number of connections that an SMTP client may make
+ simultaneously.
+ smtpd_client_connection_rate_limit (default: no limit)
+ The maximum number of connections that an SMTP client may make in the
+ time interval specified with anvil_rate_time_unit (default: 60s).
+ smtpd_client_message_rate_limit (default: no limit)
+ The maximum number of message delivery requests that an SMTP client may
+ make in the time interval specified with anvil_rate_time_unit (default:
+ 60s).
+ smtpd_client_recipient_rate_limit (default: no limit)
+ The maximum number of recipient addresses that an SMTP client may
+ specify in the time interval specified with anvil_rate_time_unit
+ (default: 60s).
+ smtpd_client_new_tls_session_rate_limit (default: no limit)
+ The maximum number of new TLS sessions (without using the TLS session
+ cache) that an SMTP client may negotiate in the time interval specified
+ with anvil_rate_time_unit (default: 60s).
+ smtpd_client_auth_rate_limit (default: no limit)
+ The maximum number of AUTH commands that an SMTP client may send in the
+ time interval specified with anvil_rate_time_unit (default: 60s).
+ Available in Postfix 3.1 and later.
+ smtpd_client_event_limit_exceptions (default: $mynetworks)
+ SMTP clients that are excluded from connection and rate limits
+ specified above.
+
+GGeenneerraall mmaaiill ddeelliivveerryy ppeerrffoorrmmaannccee ttiippss
+
+ * Read and understand the maildrop queue, incoming queue, active queue and
+ deferred queue discussions in the QSHAPE_README document.
+
+ * In case of slow delivery, run the qshape tool as described in the
+ QSHAPE_README document.
+
+ * Submit multiple recipients per message instead of submitting messages with
+ only a few recipients.
+
+ * Submit mail via SMTP instead of /usr/sbin/sendmail. You may have to adjust
+ the smtpd_recipient_limit parameter setting.
+
+ * Don't overwhelm the disk with mail submissions. Optimize the mail
+ submission rate by tuning the number of parallel submissions and/or by
+ tuning the Postfix in_flow_delay parameter setting.
+
+ * Run a local name server to reduce slow-down due to DNS lookups. If you run
+ multiple Postfix systems, point each local name server to a shared
+ forwarding server to reduce the number of lookups across the upstream
+ network link.
+
+ * Reduce the smtp_connect_timeout and smtp_helo_timeout values so that
+ Postfix does not waste lots of time connecting to non-responding remote
+ SMTP servers.
+
+ * Use a dedicated mail delivery transport for problematic destinations, with
+ reduced timeouts and with adjusted concurrency. See "Tuning the number of
+ simultaneous deliveries" below.
+
+ * Use a fallback_relay host for mail that cannot be delivered upon the first
+ attempt. This "graveyard" machine can use shorter retry times for difficult
+ to reach destinations. See "Tuning the frequency of deferred mail delivery
+ attempts" below.
+
+ * Speed up disk updates with a large (64MB) persistent write cache. This
+ allows disk updates to be sorted for optimal access speed without
+ compromising file system integrity when the system crashes.
+
+ * Use a solid-state disk (a persistent RAM disk). This is an expensive
+ solution that should be used in combination with short SMTP timeouts and a
+ fallback_relay "graveyard" machine that delivers mail for problem
+ destinations.
+
+TTuunniinngg tthhee nnuummbbeerr ooff ssiimmuullttaanneeoouuss ddeelliivveerriieess
+
+Although Postfix can be configured to run 1000 SMTP client processes at the
+same time, it is rarely desirable that it makes 1000 simultaneous connections
+to the same remote system. For this reason, Postfix has safety mechanisms in
+place to avoid this so-called "thundering herd" problem.
+
+The Postfix queue manager implements the analog of the TCP slow start flow
+control strategy: when delivering to a site, send a small number of messages
+first, then increase the concurrency as long as all goes well; reduce
+concurrency in the face of congestion.
+
+ * The initial_destination_concurrency parameter (default: 5) controls how
+ many messages are initially sent to the same destination before adapting
+ delivery concurrency. Of course, this setting is effective only as long as
+ it does not exceed the process limit and the destination concurrency limit
+ for the specific mail transport channel.
+
+ * The default_destination_concurrency_limit parameter (default: 20) controls
+ how many messages may be sent to the same destination simultaneously. You
+ can override this setting for specific message delivery transports by
+ taking the name of the master.cf entry and appending
+ "_destination_concurrency_limit".
+
+Examples of transport specific concurrency limits are:
+
+ * The local_destination_concurrency_limit parameter (default: 2) controls how
+ many messages are delivered simultaneously to the same local recipient. The
+ recommended limit is low because delivery to the same mailbox must happen
+ sequentially, so massive parallelism is not useful. Another good reason to
+ limit delivery concurrency to the same recipient: if the recipient has an
+ expensive shell command in her .forward file, or if the recipient is a
+ mailing list manager, you don't want to run too many instances of those
+ processes at the same time.
+
+ * The default smtp_destination_concurrency_limit of 20 seems enough to
+ noticeably load a system without bringing it to its knees. Be careful when
+ changing this to a much larger number.
+
+The above default values of the concurrency limits work well in a broad range
+of situations. Knee-jerk changes to these parameters in the face of congestion
+can actually make problems worse. Specifically, large destination concurrencies
+should never be the default. They should be used only for transports that
+deliver mail to a small number of high volume domains.
+
+A common situation where high concurrency is called for is on gateways relaying
+a high volume of mail between the Internet and an intranet mail environment.
+Approximately half the mail (assuming equal volumes inbound and outbound) will
+be destined for the internal mail hubs. Since the internal mail hubs will be
+receiving all external mail exclusively from the gateway, it is reasonable to
+configure the gateway to make greater demands on the capacity of the internal
+SMTP servers.
+
+The tuning of the inbound concurrency limits need not be trial and error. A
+high volume capable mailhub should be able to easily handle 50 or 100 (rather
+than the default 20) simultaneous connections, especially if the gateway
+forwards to multiple MX hosts. When all MX hosts are up and accepting
+connections in a timely fashion, throughput will be high. If any MX host is
+down and completely unresponsive, the average connection latency rises to at
+least 1/N * $smtp_connect_timeout, if there are N MX hosts. This limits
+throughput to at most the destination concurrency * N / $smtp_connect_timeout.
+
+For example, with a destination concurrency of 100 and 2 MX hosts, each host
+will handle up to 50 simultaneous connections. If one MX host is down and the
+default SMTP connection timeout is 30s, the throughput limit is 100 * 2 / 30 ~=
+6 messages per second. This suggests that high volume destinations with good
+connectivity and multiple MX hosts need a lower connection timeout, values as
+low as 5s or even 1s can be used to prevent congestion when one or more, but
+not all MX hosts are down.
+
+If necessary, set a higher transport_destination_concurrency_limit (in main.cf
+since this is a queue manager parameter) and a lower smtp_connect_timeout (with
+a "-o" override in master.cf since this parameter has no per-transport name)
+for the relay transport and any transports dedicated for specific high volume
+destinations.
+
+TTuunniinngg tthhee nnuummbbeerr ooff rreecciippiieennttss ppeerr ddeelliivveerryy
+
+The default_destination_recipient_limit parameter (default: 50) controls how
+many recipients a Postfix delivery agent will send with each copy of an email
+message. You can override this setting for specific Postfix delivery agents.
+For example, "uucp_destination_recipient_limit = 100" would limit the number of
+recipients per UUCP delivery to 100.
+
+If an email message exceeds the recipient limit for some destination, the
+Postfix queue manager breaks up the list of recipients into smaller lists.
+Postfix will attempt to send multiple copies of the message in parallel.
+
+IMPORTANT: Be careful when increasing the recipient limit per message delivery;
+some SMTP servers abort the connection when they run out of memory or when a
+hard recipient limit is reached, so that the message will never be delivered.
+
+The smtpd_recipient_limit parameter (default: 1000) controls how many
+recipients the Postfix smtpd(8) server will take per delivery. The default
+limit is more than any reasonable SMTP client would send. The limit exists to
+protect the local mail system against a run-away client.
+
+TTuunniinngg tthhee ffrreeqquueennccyy ooff ddeeffeerrrreedd mmaaiill ddeelliivveerryy aatttteemmppttss
+
+When a Postfix delivery agent (smtp(8), local(8), etc.) is unable to deliver a
+message it may blame the message itself, or it may blame the receiving party.
+
+ * When the delivery agent blames the message, the queue manager gives the
+ queue file a time stamp into the future, so it won't be looked at for a
+ while. By default, the amount of time to cool down is the amount of time
+ that has passed since the message arrived. This results in so-called
+ exponential backoff behavior.
+
+ * When the delivery agent blames the receiving party (for example a local
+ recipient user, or a remote host), the queue manager not only advances the
+ queue file time stamp, but also puts the receiving party on a "dead" list
+ so that it will be skipped for some amount of time.
+
+This process is governed by a bunch of little parameters.
+
+ queue_run_delay (default: 300 seconds; before Postfix 2.4: 1000s)
+ How often the queue manager scans the queue for deferred mail.
+ minimal_backoff_time (default: 300 seconds; before Postfix 2.4: 1000s)
+ The minimal amount of time a message won't be looked at, and the
+ minimal amount of time to stay away from a "dead" destination.
+ maximal_backoff_time (default: 4000 seconds)
+ The maximal amount of time a message won't be looked at after a
+ delivery failure.
+ maximal_queue_lifetime (default: 5 days)
+ How long a message stays in the queue before it is sent back as
+ undeliverable. Specify 0 for mail that should be returned immediately
+ after the first unsuccessful delivery attempt.
+ bounce_queue_lifetime (default: 5 days, available with Postfix version 2.1
+ and later)
+ How long a MAILER-DAEMON message stays in the queue before it is
+ considered undeliverable. Specify 0 for mail that should be tried only
+ once.
+ qmgr_message_recipient_limit (default: 20000)
+ The size of many in-memory queue manager data structures. Among others,
+ this parameter limits the size of the short-term, in-memory list of
+ "dead" destinations. Destinations that don't fit the list are not
+ added.
+ transport_destination_concurrency_failed_cohort_limit
+ Controls when a destination is considered "dead". This parameter is
+ critical with a non-zero transport_destination_rate_delay, with a
+ reduced transport_destination_concurrency_limit, or with a reduced
+ initial_destination_concurrency.
+
+IMPORTANT: If you increase the frequency of deferred mail delivery attempts, or
+if you flush the deferred mail queue frequently, then you may find that Postfix
+mail delivery performance actually becomes worse. The symptoms are as follows:
+
+ * The active queue becomes saturated with mail that has delivery problems.
+ New mail enters the active queue only when an old message is deferred. This
+ is a slow process that usually requires timing out one or more SMTP
+ connections.
+
+ * All available Postfix delivery agents become occupied trying to connect to
+ unreachable sites etc. New mail has to wait until a delivery agent becomes
+ available. This is a slow process that usually requires timing out one or
+ more SMTP connections.
+
+When mail is being deferred frequently, fixing the problem is always better
+than increasing the frequency of delivery attempts. However, if you can control
+only the delivery attempt frequency, consider using a dedicated fallback_relay
+"graveyard" machine for bad destinations, so that these destinations do not
+ruin the performance of normal mail deliveries.
+
+TTuunniinngg tthhee nnuummbbeerr ooff PPoossttffiixx pprroocceesssseess
+
+The default_process_limit configuration parameter gives direct control over how
+many daemon processes Postfix will run. As of Postfix 2.0 the default limit is
+100 SMTP client processes, 100 SMTP server processes, and so on. This may
+overwhelm systems with little memory, as well as networks with low bandwidth.
+
+You can change the global process limit by specifying a non-default
+default_process_limit in the main.cf file. For example, to run up to 10 SMTP
+client processes, 10 SMTP server processes, and so on:
+
+ /etc/postfix/main.cf:
+ default_process_limit = 10
+
+You need to execute "postfix reload" to make the change effective. This limit
+is enforced by the Postfix master(8) daemon which does not automatically read
+main.cf when it changes.
+
+You can override the process limit for specific Postfix daemons by editing the
+master.cf file. For example, if you do not wish to receive 100 SMTP messages at
+the same time, but do not want to change the process limits for other Postfix
+daemons, you could specify:
+
+ /etc/postfix/master.cf:
+ # ====================================================================
+ # service type private unpriv chroot wakeup maxproc command + args
+ # (yes) (yes) (yes) (never) (100)
+ # ====================================================================
+ . . .
+ smtp inet n - - - 10 smtpd
+ . . .
+
+TTuunniinngg tthhee nnuummbbeerr ooff pprroocceesssseess oonn tthhee ssyysstteemm
+
+ * MacOS X will run out of process slots when you increase Postfix process
+ limits. The following works with OSX 10.4 and OSX 10.5.
+
+ MacOS X kernel parameters can be specified in /etc/sysctl.conf.
+
+ /etc/sysctl.conf:
+ kern.maxproc=2048
+ kern.maxprocperuid=2048
+
+ Unfortunately these can't simply be set on the fly with "sysctl -w". You
+ also have to set the following in /etc/launchd.conf so that the root user
+ after boot will have the right process limit (2048). Otherwise you have to
+ always run ulimit -u 2048 as root, then start a user shell, and then start
+ processes for things to take effect.
+
+ /etc/launchd.conf:
+ limit maxproc 2048
+
+ Once these are in place, reboot the system. After that, the limits will
+ stay in place.
+
+TTuunniinngg tthhee nnuummbbeerr ooff ooppeenn ffiilleess oorr ssoocckkeettss
+
+When Postfix opens too many files or sockets, processes will abort with fatal
+errors, and the system may log "file table full" errors.
+
+ * Depending on your Postfix and operating system versions you may need to
+ recompile Postfix if you need more than 1024 file descriptors per process:
+
+ o No recompilation is needed for Postfix version 2.4 and later, when it
+ was compiled for systems that support BSD kqueue(2) (FreeBSD 4.1,
+ NetBSD 2.0, OpenBSD 2.9), Solaris 8 /dev/poll, or Linux 2.6 epoll(4).
+
+ o Otherwise, Postfix needs to be recompiled to override the default
+ FD_SETSIZE value.
+
+ * Reduce the number of processes as described under "Tuning the number of
+ Postfix processes" above. Fewer processes need fewer open files and
+ sockets.
+
+ * Configure the kernel for more open files and sockets. The details are
+ extremely system dependent and change with the operating system version. Be
+ sure to verify the following information with your system tuning guide:
+
+ o Some FreeBSD kernel parameters can be specified in /boot/loader.conf,
+ and some can be specified in /etc/sysctl.conf or changed with sysctl
+ commands. Which is which depends on the version.
+
+ kern.ipc.maxsockets="5000"
+ kern.ipc.nmbclusters="65536"
+ kern.maxproc="2048"
+ kern.maxfiles="16384"
+ kern.maxfilesperproc="16384"
+
+ o Linux kernel parameters can be specified in /etc/sysctl.conf or changed
+ with sysctl commands:
+
+ fs.file-max=16384
+ kernel.threads-max=2048
+
+ o Solaris kernel parameters can be specified in /etc/system, as described
+ in the Solaris FAQ entry titled "How can I increase the number of file
+ descriptors per process?"
+
+ * set hard limit on file descriptors
+ set rlim_fd_max = 4096
+ * set soft limit on file descriptors
+ set rlim_fd_cur = 1024
+