diff options
Diffstat (limited to 'README_FILES')
61 files changed, 24645 insertions, 0 deletions
diff --git a/README_FILES/AAAREADME b/README_FILES/AAAREADME new file mode 100644 index 0000000..9afa3b7 --- /dev/null +++ b/README_FILES/AAAREADME @@ -0,0 +1,86 @@ + PPoossttffiixx DDooccuummeennttaattiioonn + +------------------------------------------------------------------------------- +GGeenneerraall ccoonnffiigguurraattiioonn + + * BASIC_CONFIGURATION_README: Basic configuration + * SOHO_README: Small/home office hints and tips + * STANDARD_CONFIGURATION_README: Standard configuration examples + * ADDRESS_REWRITING_README: Address rewriting + * VIRTUAL_README: Virtual domain hosting + * SASL_README: SASL Authentication + * TLS_README: TLS Encryption and authentication + * FORWARD_SECRECY_README: TLS Forward Secrecy + * IPV6_README: IP Version 6 Support + * SMTPUTF8_README: SMTPUTF8 Support + * MAILLOG_README: Postfix logging to file or stdout + * COMPATIBILITY_README: Backwards-Compatibility Safety Net + * INSTALL: Installation from source code + +PPrroobblleemm ssoollvviinngg + + * QSHAPE_README: Bottleneck analysis + * STRESS_README: Stress-dependent configuration + * TUNING_README: Performance tuning + * DEBUG_README: Debugging strategies + +CCoonntteenntt iinnssppeeccttiioonn + + * CONTENT_INSPECTION_README: Content inspection overview + * BACKSCATTER_README: Stopping backscatter mail + * BUILTIN_FILTER_README: Built-in content inspection + + * FILTER_README: After-queue content filter + * SMTPD_PROXY_README: Before-queue content filter + * MILTER_README: Before-queue Milter applications + +SSMMTTPP RReellaayy aanndd aacccceessss ccoonnttrrooll + + * SMTPD_ACCESS_README: Relay/access control overview + * SMTPD_POLICY_README: Access policy delegation + * ADDRESS_VERIFICATION_README: Address verification + * RESTRICTION_CLASS_README: Per-client/user/etc. access + * POSTSCREEN_README: SMTP connection triage + * ETRN_README: ETRN Support + * UUCP_README: LAN connected via UUCP + +LLooookkuupp ttaabblleess ((ddaattaabbaasseess)) + + * DATABASE_README: Lookup table overview + * DB_README: Berkeley DB Howto + * CDB_README: CDB Howto + * LDAP_README: LDAP Howto + * LMDB_README: LMDB Howto + * MEMCACHE_README: Memcache Howto + * MYSQL_README: MySQL Howto + * PCRE_README: PCRE Howto + * PGSQL_README: PostgreSQL Howto + * SQLITE_README: SQLite Howto + +MMaaiilliinngg lliisstt ssuuppppoorrtt + + * VERP_README: VERP Support + +SSppeecciiffiicc eennvviirroonnmmeennttss + + * LINUX_README: Linux issues + * NFS_README: NFS issues + +OOtthheerr mmaaiill ddeelliivveerryy aaggeennttss + + * MAILDROP_README: Maildrop + +OOtthheerr ttooppiiccss + + * OVERVIEW: Architecture overview + * postconf(5): All main.cf parameters + * LOCAL_RECIPIENT_README: Rejecting Unknown Local Recipients + * ADDRESS_CLASS_README: Address Classes + * CONNECTION_CACHE_README: Connection cache howto + * DSN_README: Postfix DSN support + * BDAT_README: Postfix BDAT (CHUNKING) support + * PACKAGE_README: Guidelines for Package Builders + * SCHEDULER_README: Queue Scheduler + * XCLIENT_README: XCLIENT Command + * XFORWARD_README: XFORWARD Command + diff --git a/README_FILES/ADDRESS_CLASS_README b/README_FILES/ADDRESS_CLASS_README new file mode 100644 index 0000000..7605aa2 --- /dev/null +++ b/README_FILES/ADDRESS_CLASS_README @@ -0,0 +1,202 @@ +PPoossttffiixx AAddddrreessss CCllaasssseess + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix version 2.0 introduces the concept of address classes. This is a way of +grouping recipient addresses by their delivery method. The idea comes from +discussions with Victor Duchovni. Although address classes introduced a few +incompatibilities they also made it possible to improve the handling of hosted +domains and of unknown recipients. + +This document provides information on the following topics: + + * What are address classes good for? + * What address classes does Postfix implement? + * Improvements compared to Postfix 1.1 + * Incompatibilities with Postfix 1.1 + +WWhhaatt aarree aaddddrreessss ccllaasssseess ggoooodd ffoorr?? + +Why should you care about address classes? This is how Postfix decides what +mail to accept, and how to deliver it. In other words, address classes are very +important for the operation of Postfix. + +An address class is defined by three items. + + * The list of domains that are a member of that address class: for example, + all local domains, or all relay domains. + + * The default delivery transport for that address class. For example, the + local, virtual or relay delivery transport (delivery transports are defined + in master.cf). This helps to keep Postfix configurations simple, by + avoiding the need for explicit routing information in transport maps. + + * The list of valid recipient addresses for that address class. The Postfix + SMTP server rejects invalid recipients with "User unknown in <name of + address class here> table". This helps to keep the Postfix queue free of + undeliverable MAILER-DAEMON messages. + +WWhhaatt aaddddrreessss ccllaasssseess ddooeess PPoossttffiixx iimmpplleemmeenntt?? + +Initially the list of address classes is hard coded, but this is meant to +become extensible. The summary below describes the main purpose of each class, +and what the relevant configuration parameters are. + +The local domain class. + + * Purpose: final delivery for traditional UNIX system accounts and + traditional Sendmail-style aliases. This is typically used for the + canonical domains of the machine (for example, $myhostname, $mydomain). For + a discussion of the difference between canonical domains, hosted domains + and other domains, see the VIRTUAL_README file. + + * Domain names are listed with the mydestination parameter. This domain class + also includes mail for user@[ipaddress] when the IP address is listed with + the inet_interfaces or proxy_interfaces parameters. + + * Valid recipient addresses are listed with the local_recipient_maps + parameter, as described in LOCAL_RECIPIENT_README. The Postfix SMTP server + rejects invalid recipients with "User unknown in local recipient table". If + the local_recipient_maps parameter value is empty, then the Postfix SMTP + server accepts any address in the local domain class. + + * The mail delivery transport is specified with the local_transport + parameter. The default value is llooccaall::$$mmyyhhoossttnnaammee for delivery with the + local(8) delivery agent. + +The virtual alias domain class. + + * Purpose: hosted domains where each recipient address is aliased to an + address in a different domain, for example, a local UNIX system account or + a remote address. A virtual alias example is given in the VIRTUAL_README + file. + + * Domain names are listed in virtual_alias_domains. The default value is + $virtual_alias_maps for Postfix 1.1 compatibility. + + * Valid recipient addresses are listed with the virtual_alias_maps parameter. + The Postfix SMTP server rejects invalid recipients with "User unknown in + virtual alias table". The default value is $virtual_maps for Postfix 1.1 + compatibility. + + * There is no mail delivery transport parameter. Every address must be + aliased to an address in some other domain. + +The virtual mailbox domain class. + + * Purpose: final delivery for hosted domains where each recipient address can + have its own mailbox, and where users do not need to have a UNIX system + account. A virtual mailbox example is given in the VIRTUAL_README file. + + * Domain names are listed with the virtual_mailbox_domains parameter. The + default value is $virtual_mailbox_maps for Postfix 1.1 compatibility. + + * Valid recipient addresses are listed with the virtual_mailbox_maps + parameter. The Postfix SMTP server rejects invalid recipients with "User + unknown in virtual mailbox table". If this parameter value is empty, the + Postfix SMTP server accepts all recipients for domains listed in + $virtual_mailbox_domains. + + * The mail delivery transport is specified with the virtual_transport + parameter. The default value is vviirrttuuaall for delivery with the virtual(8) + delivery agent. + +The relay domain class. + + * Purpose: mail forwarding to remote destinations that list your system as + primary or backup MX host. For a discussion of the basic configuration + details, see the BASIC_CONFIGURATION_README document. For a discussion of + the difference between canonical domains, hosted domains and other domains, + see the VIRTUAL_README file. + + * Domain names are listed with the relay_domains parameter. + + * Valid recipient addresses are listed with the relay_recipient_maps + parameter. The Postfix SMTP server rejects invalid recipients with "User + unknown in relay recipient table". If this parameter value is empty, the + Postfix SMTP server accepts all recipients for domains listed with the + relay_domains parameter. + + * The mail delivery transport is specified with the relay_transport + parameter. The default value is rreellaayy which is a clone of the smtp(8) + delivery agent. + +The default domain class. + + * Purpose: mail forwarding to the Internet on behalf of authorized clients. + For a discussion of the basic configuration details, see the + BASIC_CONFIGURATION_README file. For a discussion of the difference between + canonical domains, hosted domains and other domains, see the VIRTUAL_README + file. + + * This class has no destination domain table. + + * This class has no valid recipient address table. + + * The mail delivery transport is specified with the default_transport + parameter. The default value is ssmmttpp for delivery with the smtp(8) delivery + agent. + +IImmpprroovveemmeennttss ccoommppaarreedd ttoo PPoossttffiixx 11..11 + +Postfix 2.0 address classes made the following improvements possible over +earlier Postfix versions: + + * You no longer need to specify all the virtual(8) mailbox domains in the + Postfix transport map. The virtual(8) delivery agent has become a first- + class citizen just like local(8) or smtp(8). + + * On mail gateway systems, address classes provide separation of inbound mail + relay traffic ($relay_transport) from outbound traffic + ($default_transport). This eliminates a problem where inbound mail + deliveries could become resource starved in the presence of a high volume + of outbound mail. + + * The SMTP server rejects unknown recipients in a more consistent manner than + was possible with Postfix version 1. This is needed to keep undeliverable + mail (and bounced undeliverable mail) out of the mail queue. This is + controlled by the smtpd_reject_unlisted_recipient configuration parameter. + + * As of Postfix version 2.1, the SMTP server can also reject unknown sender + addresses (i.e. addresses that it would reject as an unknown recipient + addresses). Sender "egress filtering" can help to slow down an email worm + explosion. This is controlled by the smtpd_reject_unlisted_sender + configuration parameter. + +IInnccoommppaattiibbiilliittiieess wwiitthh PPoossttffiixx 11..11 + +Postfix 2.0 address classes introduce a few incompatible changes in documented +behavior. In order to ease the transitions, new parameters have default values +that are backwards compatible. + + * The virtual_maps parameter is replaced by virtual_alias_maps (for address + lookups) and by virtual_alias_domains (for the names of what were formerly + called "Postfix-style virtual domains"). + + For backwards compatibility with Postfix version 1.1, the new + virtual_alias_maps parameter defaults to $virtual_maps, and the new + virtual_alias_domains parameter defaults to $virtual_alias_maps. + + * The virtual_mailbox_maps parameter now has a companion parameter called + virtual_mailbox_domains (for the names of domains served by the virtual + delivery agent). The virtual_mailbox_maps parameter is now used for address + lookups only. + + For backwards compatibility with Postfix version 1.1, the new + virtual_mailbox_domains parameter defaults to $virtual_mailbox_maps. + + * Introduction of the relay_recipient_maps parameter. The Postfix SMTP server + can use this to block mail for relay recipients that don't exist. This list + is empty by default, which means accept any recipient. + + * The local_recipient_maps feature is now turned on by default. The Postfix + SMTP server uses this to reject mail for unknown local recipients. See the + LOCAL_RECIPIENT_README file hints and tips. + + * Introduction of the relay delivery transport in master.cf. This helps to + avoid mail delivery scheduling problems on inbound mail relays when there + is a lot of outbound mail, but may require that you update your + "defer_transports" setting. + diff --git a/README_FILES/ADDRESS_REWRITING_README b/README_FILES/ADDRESS_REWRITING_README new file mode 100644 index 0000000..78237b9 --- /dev/null +++ b/README_FILES/ADDRESS_REWRITING_README @@ -0,0 +1,840 @@ +PPoossttffiixx AAddddrreessss RReewwrriittiinngg + +------------------------------------------------------------------------------- + +PPoossttffiixx aaddddrreessss rreewwrriittiinngg ppuurrppoossee + +Address rewriting is at the heart of the Postfix mail system. Postfix rewrites +addresses for many different purposes. Some are merely cosmetic, and some are +necessary to deliver correctly formatted mail to the correct destination. +Examples of address rewriting in Postfix are: + + * Transform an incomplete address into a complete address. For example, + transform "username" into "username@example.com", or transform + "username@hostname" into "username@hostname.example.com". + + * Replace an address by an equivalent address. For example, replace + "username@example.com" by "firstname.lastname@example.com" when sending + mail, and do the reverse transformation when receiving mail. + + * Replace an internal address by an external address. For example, replace + "username@localdomain.local" by "isp-account@isp.example" when sending mail + from a home computer to the Internet. + + * Replace an address by multiple addresses. For example, replace the address + of an alias by the addresses listed under that alias. + + * Determine how and where to deliver mail for a specific address. For + example, deliver mail for "username@example.com" with the smtp(8) delivery + agent, to the hosts that are listed in the DNS as the mail servers for the + domain "example.com". + +Although Postfix currently has no address rewriting language, it can do +surprisingly powerful address manipulation via table lookup. Postfix typically +uses lookup tables with fixed strings to map one address to one or multiple +addresses, and typically uses regular expressions to map multiple addresses to +one or multiple addresses. Fixed-string lookup tables may be in the form of +local files, or in the form of NIS, LDAP or SQL databases. The DATABASE_README +document gives an introduction to Postfix lookup tables. + +Topics covered in this document: + + * To rewrite message headers or not, or to label as invalid + * Postfix address rewriting overview + * Address rewriting when mail is received + + o Rewrite addresses to standard form + o Canonical address mapping + o Address masquerading + o Automatic BCC recipients + o Virtual aliasing + + * Address rewriting when mail is delivered + + o Resolve address to destination + o Mail transport switch + o Relocated users table + + * Address rewriting with remote delivery + + o Generic mapping for outgoing SMTP mail + + * Address rewriting with local delivery + + o Local alias database + o Local per-user .forward files + o Local catch-all address + + * Debugging your address manipulations + +TToo rreewwrriittee mmeessssaaggee hheeaaddeerrss oorr nnoott,, oorr ttoo llaabbeell aass iinnvvaalliidd + +Postfix versions 2.1 and earlier always rewrite message header addresses, and +append Postfix's own domain information to addresses that Postfix considers +incomplete. While rewriting message header addresses is OK for mail with a +local origin, it is undesirable for remote mail: + + * Message header address rewriting is frowned upon by mail standards, + * Appending Postfix's own domain produces incorrect results with some + incomplete addresses, + * Appending Postfix's own domain sometimes creates the appearance that spam + is sent by local users. + +Postfix versions 2.2 give you the option to either not rewrite message headers +from remote SMTP clients at all, or to label incomplete addresses in such +message headers as invalid. Here is how it works: + + * Postfix always rewrites message headers from local SMTP clients and from + the Postfix sendmail command, and appends its own domain to incomplete + addresses. The local_header_rewrite_clients parameter controls what SMTP + clients Postfix considers local (by default, only local network interface + addresses). + * Postfix never rewrites message header addresses from remote SMTP clients + when the remote_header_rewrite_domain parameter value is empty (the default + setting). + * Otherwise, Postfix rewrites message headers from remote SMTP clients, and + appends the remote_header_rewrite_domain value to incomplete addresses. + This feature can be used to append a reserved domain such as + "domain.invalid", so that incomplete addresses cannot be mistaken for local + addresses. + +PPoossttffiixx aaddddrreessss rreewwrriittiinngg oovveerrvviieeww + +The figure below zooms in on those parts of Postfix that are most involved with +address rewriting activity. See the OVERVIEW document for an overview of the +complete Postfix architecture. Names followed by a number are Postfix daemon +programs, while unnumbered names represent Postfix queues or internal sources +of mail messages. + + trivial- trivial- + rewrite(8) rewrite(8) + (std form) (resolve) + + ^ | ^ | + | v | v + + smtpd(8) smtp(8) + + qmqpd(8) >- cleanup(8) -> incoming -> active -> qmgr(8) -< lmtp(8) + + pickup(8) local(8) + + ^ ^ | + | | v + + bounces + forwarding deferred + notices + +The table below summarizes all Postfix address manipulations. If you're reading +this document for the first time, skip forward to "Address rewriting when mail +is received". Once you've finished reading the remainder of this document, the +table will help you to quickly find what you need. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |AAddddrreessss |SSccooppee |DDaaeemmoonn |GGlloobbaall ttuurrnn--oonn |SSeelleeccttiivvee ttuurrnn--ooffff ccoonnttrrooll | + |mmaanniippuullaattiioonn| | |ccoonnttrrooll | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Rewrite | |trivial-|append_at_myorigin, | | + |addresses to|all mail|rewrite |append_dot_mydomain,|local_header_rewrite_clients,| + |standard | |(8) |swap_bangpath, |remote_header_rewrite_domain | + |form | | |allow_percent_hack | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Canonical | |cleanup | |receive_override_options, | + |address |all mail|(8) |canonical_maps |local_header_rewrite_clients,| + |mapping | | | |remote_header_rewrite_domain | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Address | |cleanup | |receive_override_options, | + |masquerading|all mail|(8) |masquerade_domains |local_header_rewrite_clients,| + | | | | |remote_header_rewrite_domain | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Automatic | |cleanup |always_bcc, | | + |BCC |new mail|(8) |sender_bcc_maps, |receive_override_options | + |recipients | | |recipient_bcc_maps | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Virtual |all mail|cleanup |virtual_alias_maps |receive_override_options | + |aliasing | |(8) | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Resolve | |trivial-| | | + |address to |all mail|rewrite |none |none | + |destination | |(8) | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Mail | |trivial-| | | + |transport |all mail|rewrite |transport_maps |none | + |switch | |(8) | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Relocated | |trivial-| | | + |users table |all mail|rewrite |relocated_maps |none | + | | |(8) | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Generic |outgoing| | | | + |mapping |SMTP |smtp(8) |smtp_generic_maps |none | + |table |mail | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local alias |local | | | | + |database |mail |local(8)|alias_maps |none | + | |only | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local per- |local | | | | + |user |mail |local(8)|forward_path |none | + |.forward |only | | | | + |files | | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local catch-|local | | | | + |all address |mail |local(8)|luser_relay |none | + | |only | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +AAddddrreessss rreewwrriittiinngg wwhheenn mmaaiill iiss rreecceeiivveedd + +The cleanup(8) server receives mail from outside of Postfix as well as mail +from internal sources such as forwarded mail, undeliverable mail that is +bounced to the sender, and postmaster notifications about problems with the +mail system. + +The cleanup(8) server transforms the sender, recipients and message content +into a standard form before writing it to an incoming queue file. The server +cleans up sender and recipient addresses in message headers and in the +envelope, adds missing message headers such as From: or Date: that are required +by mail standards, and removes message headers such as Bcc: that should not be +present. The cleanup(8) server delegates the more complex address manipulations +to the trivial-rewrite(8) server as described later in this document. + +Address manipulations at this stage are: + + * Rewrite addresses to standard form + * Canonical address mapping + * Address masquerading + * Automatic BCC recipients + * Virtual aliasing + +RReewwrriittee aaddddrreesssseess ttoo ssttaannddaarrdd ffoorrmm + +Before the cleanup(8) daemon runs an address through any address mapping lookup +table, it first rewrites the address to the standard +"user@fully.qualified.domain" form, by sending the address to the trivial- +rewrite(8) daemon. The purpose of rewriting to standard form is to reduce the +number of entries needed in lookup tables. + +The Postfix trivial-rewrite(8) daemon implements the following hard-coded +address manipulations: + + Rewrite "@hosta,@hostb:user@site" to "user@site" + In case you wonder what this is, the address form above is called a + route address, and specifies that mail for "user@site" be delivered via + "hosta" and "hostb". Usage of this form has been deprecated for a long + time. Postfix has no ability to handle route addresses, other than to + strip off the route part. + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter, or if the + remote_header_rewrite_domain configuration parameter specifies a non- + empty value. To get the behavior before Postfix 2.2, specify + "local_header_rewrite_clients = static:all". + + Rewrite "site!user" to "user@site" + This feature is controlled by the boolean swap_bangpath parameter + (default: yes). The purpose is to rewrite UUCP-style addresses to + domain style. This is useful only when you receive mail via UUCP, but + it probably does not hurt otherwise. + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter, or if the + remote_header_rewrite_domain configuration parameter specifies a non- + empty value. To get the behavior before Postfix 2.2, specify + "local_header_rewrite_clients = static:all". + + Rewrite "user%domain" to "user@domain" + This feature is controlled by the boolean allow_percent_hack parameter + (default: yes). Typically, this is used in order to deal with + monstrosities such as "user%domain@otherdomain". + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter, or if the + remote_header_rewrite_domain configuration parameter specifies a non- + empty value. To get the behavior before Postfix 2.2, specify + "local_header_rewrite_clients = static:all". + + Rewrite "user" to "user@$myorigin" + This feature is controlled by the boolean append_at_myorigin parameter + (default: yes). You should never turn off this feature, because a lot + of Postfix components expect that all addresses have the form + "user@domain". + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter; otherwise they append the + domain name specified with the remote_header_rewrite_domain + configuration parameter, if one is specified. To get the behavior + before Postfix 2.2, specify "local_header_rewrite_clients = static: + all". + + If your machine is not the main machine for $myorigin and you wish to + have some users delivered locally without going via that main machine, + make an entry in the virtual alias table that redirects + "user@$myorigin" to "user@$myhostname". See also the "delivering some + users locally" section in the STANDARD_CONFIGURATION_README document. + + Rewrite "user@host" to "user@host.$mydomain" + This feature is controlled by the boolean append_dot_mydomain parameter + (default: Postfix ≥ 3.0: no, Postfix < 3.0: yes). The purpose is to + get consistent treatment of different forms of the same hostname. + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter; otherwise they append the + domain name specified with the remote_header_rewrite_domain + configuration parameter, if one is specified. To get the behavior + before Postfix 2.2, specify "local_header_rewrite_clients = static: + all". + + Some will argue that rewriting "host" to "host.domain" is bad. That is + why it can be turned off. Others like the convenience of having + Postfix's own domain appended automatically. + + Rewrite "user@site." to "user@site" (without the trailing dot). + A single trailing dot is silently removed. However, an address that + ends in multiple dots will be rejected as an invalid address. + + NOTE: Postfix versions 2.2 and later rewrite message headers from + remote SMTP clients only if the client matches the + local_header_rewrite_clients parameter, or if the + remote_header_rewrite_domain configuration parameter specifies a non- + empty value. To get the behavior before Postfix 2.2, specify + "local_header_rewrite_clients = static:all". + +CCaannoonniiccaall aaddddrreessss mmaappppiinngg + +The cleanup(8) daemon uses the canonical(5) tables to rewrite addresses in +message envelopes and in message headers. By default all header and envelope +addresses are rewritten; this is controlled with the canonical_classes +configuration parameter. + +NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP +clients only if the client matches the local_header_rewrite_clients parameter, +or if the remote_header_rewrite_domain configuration parameter specifies a non- +empty value. To get the behavior before Postfix 2.2, specify +"local_header_rewrite_clients = static:all". + +Address rewriting is done for local and remote addresses. The mapping is useful +to replace login names by "Firstname.Lastname" style addresses, or to clean up +invalid domains in mail addresses produced by legacy mail systems. + +Canonical mapping is disabled by default. To enable, edit the canonical_maps +parameter in the main.cf file and specify one or more lookup tables, separated +by whitespace or commas. + +Example: + + /etc/postfix/main.cf: + canonical_maps = hash:/etc/postfix/canonical + + /etc/postfix/canonical: + wietse Wietse.Venema + +For static mappings as shown above, lookup tables such as hash:, ldap:, mysql: +or pgsql: are sufficient. For dynamic mappings you can use regular expression +tables. This requires that you become intimately familiar with the ideas +expressed in regexp_table(5), pcre_table(5) and canonical(5). + +In addition to the canonical maps which are applied to both sender and +recipient addresses, you can specify canonical maps that are applied only to +sender addresses or to recipient addresses. + +Example: + + /etc/postfix/main.cf: + sender_canonical_maps = hash:/etc/postfix/sender_canonical + recipient_canonical_maps = hash:/etc/postfix/recipient_canonical + +The sender and recipient canonical maps are applied before the common canonical +maps. The sender_canonical_classes and recipient_canonical_classes parameters +control what addresses are subject to sender_canonical_maps and +recipient_canonical_maps mappings, respectively. + +Sender-specific rewriting is useful when you want to rewrite ugly sender +addresses to pretty ones, and still want to be able to send mail to the those +ugly address without creating a mailer loop. + +Canonical mapping can be turned off selectively for mail received by smtpd(8), +qmqpd(8), or pickup(8), by overriding main.cf settings in the master.cf file. +This feature is available in Postfix version 2.1 and later. + +Example: + + /etc/postfix/master.cf: + 127.0.0.1:10026 inet n - n - - smtpd + -o receive_override_options=no_address_mappings + +Note: do not specify whitespace around the "=" here. + +AAddddrreessss mmaassqquueerraaddiinngg + +Address masquerading is a method to hide hosts inside a domain behind their +mail gateway, and to make it appear as if the mail comes from the gateway +itself, instead of from individual machines. + +NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP +clients only if the client matches the local_header_rewrite_clients parameter, +or if the remote_header_rewrite_domain configuration parameter specifies a non- +empty value. To get the behavior before Postfix 2.2, specify +"local_header_rewrite_clients = static:all". + +Address masquerading is disabled by default, and is implemented by the cleanup +(8) server. To enable, edit the masquerade_domains parameter in the main.cf +file and specify one or more domain names separated by whitespace or commas. +When Postfix tries to masquerade a domain, it processes the list from left to +right, and processing stops at the first match. + +Example: + + /etc/postfix/main.cf: + masquerade_domains = foo.example.com example.com + +strips "any.thing.foo.example.com" to "foo.example.com", but strips +"any.thing.else.example.com" to "example.com". + +A domain name prefixed with "!" means do not masquerade this domain or its +subdomains: + + /etc/postfix/main.cf: + masquerade_domains = !foo.example.com example.com + +does not change "any.thing.foo.example.com" and "foo.example.com", but strips +"any.thing.else.example.com" to "example.com". + +The masquerade_exceptions configuration parameter specifies what user names +should not be subjected to address masquerading. Specify one or more user names +separated by whitespace or commas. + +Example: + + /etc/postfix/main.cf: + masquerade_exceptions = root + +By default, Postfix makes no exceptions. + +Subtle point: by default, address masquerading is applied only to message +headers and to envelope sender addresses, but not to envelope recipients. This +allows you to use address masquerading on a mail gateway machine, while still +being able to forward mail from outside to users on individual machines. + +In order to subject envelope recipient addresses to masquerading, too, specify +(Postfix version 1.1 and later): + + /etc/postfix/main.cf: + masquerade_classes = envelope_sender, envelope_recipient, + header_sender, header_recipient + +If you rewrite the envelope recipient like this, Postfix will no longer be able +to send mail to individual machines. + +Address masquerading can be turned off selectively for mail received by smtpd +(8), qmqpd(8), or pickup(8), by overriding main.cf settings in the master.cf +file. This feature is available in Postfix version 2.1 and later. + +Example: + + /etc/postfix/master.cf: + 127.0.0.1:10026 inet n - n - - smtpd + -o receive_override_options=no_address_mappings + +Note: do not specify whitespace around the "=" here. + +AAuuttoommaattiicc BBCCCC rreecciippiieennttss + +After applying the canonical and masquerade mappings, the cleanup(8) daemon can +generate optional BCC (blind carbon-copy) recipients. Postfix provides three +mechanisms: + + always_bcc = address + Deliver a copy of all mail to the specified address. In Postfix + versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), + or pickup(8). + sender_bcc_maps = type:table + Search the specified "type:table" lookup table with the envelope sender + address for an automatic BCC address. This feature is available in + Postfix 2.1 and later. + recipient_bcc_maps = type:table + Search the specified "type:table" lookup table with the envelope + recipient address for an automatic BCC address. This feature is + available in Postfix 2.1 and later. + +Note: automatic BCC recipients are produced only for new mail. To avoid mailer +loops, automatic BCC recipients are not generated for mail that Postfix +forwards internally, nor for mail that Postfix generates itself. + +Automatic BCC recipients (including always_bcc) can be turned off selectively +for mail received by smtpd(8), qmqpd(8), or pickup(8), by overriding main.cf +settings in the master.cf file. This feature is available in Postfix version +2.1 and later. + +Example: + + /etc/postfix/master.cf: + 127.0.0.1:10026 inet n - n - - smtpd + -o receive_override_options=no_address_mappings + +Note: do not specify whitespace around the "=" here. + +VViirrttuuaall aalliiaassiinngg + +Before writing the recipients to the queue file, the cleanup(8) daemon uses the +optional virtual(5) alias tables to redirect mail for recipients. The mapping +affects only envelope recipient addresses; it has no effect on message headers +or envelope sender addresses. Virtual alias lookups are useful to redirect mail +for virtual alias domains to real user mailboxes, and to redirect mail for +domains that no longer exist. Virtual alias lookups can also be used to +transform " Firstname.Lastname " back into UNIX login names, although it seems +that local aliases may be a more appropriate vehicle. See the VIRTUAL_README +document for an overview of methods to host virtual domains with Postfix. + +Virtual aliasing is disabled by default. To enable, edit the virtual_alias_maps +parameter in the main.cf file and specify one or more lookup tables, separated +by whitespace or commas. + +Example: + + /etc/postfix/main.cf: + virtual_alias_maps = hash:/etc/postfix/virtual + + /etc/postfix/virtual: + Wietse.Venema wietse + +Addresses found in virtual alias maps are subjected to another iteration of +virtual aliasing, but are not subjected to canonical mapping, in order to avoid +loops. + +For static mappings as shown above, lookup tables such as hash:, ldap:, mysql: +or pgsql: are sufficient. For dynamic mappings you can use regular expression +tables. This requires that you become intimately familiar with the ideas +expressed in regexp_table(5), pcre_table(5) and virtual(5). + +Virtual aliasing can be turned off selectively for mail received by smtpd(8), +qmqpd(8), or pickup(8), by overriding main.cf settings in the master.cf file. +This feature is available in Postfix version 2.1 and later. + +Example: + + /etc/postfix/master.cf: + 127.0.0.1:10026 inet n - n - - smtpd + -o receive_override_options=no_address_mappings + +Note: do not specify whitespace around the "=" here. + +At this point the message is ready to be stored into the Postfix incoming +queue. + +AAddddrreessss rreewwrriittiinngg wwhheenn mmaaiill iiss ddeelliivveerreedd + +The Postfix queue manager sorts mail according to its destination and gives it +to Postfix delivery agents such as local(8), smtp(8), or lmtp(8). Just like the +cleanup(8) server, the Postfix queue manager delegates the more complex address +manipulations to the trivial-rewrite(8) server. + +Address manipulations at this stage are: + + * Resolve address to destination + * Mail transport switch + * Relocated users table + +Each Postfix delivery agent tries to deliver the mail to its destination, while +encapsulating the sender, recipients, and message content according to the +rules of the SMTP, LMTP, etc. protocol. When mail cannot be delivered, it is +either returned to the sender or moved to the deferred queue and tried again +later. + +Address manipulations when mail is delivered via the smtp(8) delivery agent: + + * Generic mapping for outgoing SMTP mail + +Address manipulations when mail is delivered via the local(8) delivery agent: + + * Local alias database + * Local per-user .forward files + * Local catch-all address + +The remainder of this document presents each address manipulation step in more +detail, with specific examples or with pointers to documentation with examples. + +RReessoollvvee aaddddrreessss ttoo ddeessttiinnaattiioonn + +The Postfix qmgr(8) queue manager selects new mail from the incoming queue or +old mail from the deferred queue, and asks the trivial-rewrite(8) address +rewriting and resolving daemon where it should be delivered. + +As of version 2.0, Postfix distinguishes four major address classes. Each class +has its own list of domain names, and each class has its own default delivery +method, as shown in the table below. See the ADDRESS_CLASS_README document for +the fine details. Postfix versions before 2.0 only distinguish between local +delivery and everything else. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |DDeessttiinnaattiioonn ddoommaaiinn lliisstt |DDeeffaauulltt ddeelliivveerryy mmeetthhoodd|AAvvaaiillaabbiilliittyy| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |$mydestination, $inet_interfaces,|$local_transport |Postfix 1.0 | + |$proxy_interfaces | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |$virtual_mailbox_domains |$virtual_transport |Postfix 2.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |$relay_domains |$relay_transport |Postfix 2.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |none |$default_transport |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + +MMaaiill ttrraannssppoorrtt sswwiittcchh + +Once the trivial-rewrite(8) daemon has determined a default delivery method it +searches the optional transport(5) table for information that overrides the +message destination and/or delivery method. Typical use of the transport(5) +table is to send mail to a system that is not connected to the Internet, or to +use a special SMTP client configuration for destinations that have special +requirements. See, for example, the STANDARD_CONFIGURATION_README and +UUCP_README documents, and the examples in the transport(5) manual page. + +Transport table lookups are disabled by default. To enable, edit the +transport_maps parameter in the main.cf file and specify one or more lookup +tables, separated by whitespace or commas. + +Example: + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + +RReellooccaatteedd uusseerrss ttaabbllee + +Next, the trivial-rewrite(8) address rewriting and resolving daemon runs each +recipient through the relocated(5) database. This table provides information on +how to reach users that no longer have an account, or what to do with mail for +entire domains that no longer exist. When mail is sent to an address that is +listed in this table, the message is returned to the sender with an informative +message. + +The relocated(5) database is searched after transport(5) table lookups, in +anticipation of transport(5) tables that can replace one recipient address by a +different one. + +Lookups of relocated users are disabled by default. To enable, edit the +relocated_maps parameter in the main.cf file and specify one or more lookup +tables, separated by whitespace or commas. + +Example: + + /etc/postfix/main.cf: + relocated_maps = hash:/etc/postfix/relocated + + /etc/postfix/relocated: + username@example.com otheruser@elsewhere.tld + +As of Postfix version 2, mail for a relocated user will be rejected by the SMTP +server with the reason "user has moved to otheruser@elsewhere.tld". Older +Postfix versions will receive the mail first, and then return it to the sender +as undeliverable, with the same reason. + +GGeenneerriicc mmaappppiinngg ffoorr oouuttggooiinngg SSMMTTPP mmaaiill + +Some hosts have no valid Internet domain name, and instead use a name such as +localdomain.local. This can be a problem when you want to send mail over the +Internet, because many mail servers reject mail addresses with invalid domain +names. + +With the smtp_generic_maps parameter you can specify generic(5) lookup tables +that replace local mail addresses by valid Internet addresses when mail leaves +the machine via SMTP. The generic(5) mapping replaces envelope and header +addresses, and is non-recursive. It does not happen when you send mail between +addresses on the local machine. + +This feature is available in Postfix version 2.2 and later. + +Example: + + /etc/postfix/main.cf: + smtp_generic_maps = hash:/etc/postfix/generic + + /etc/postfix/generic: + his@localdomain.local hisaccount@hisisp.example + her@localdomain.local heraccount@herisp.example + @localdomain.local hisaccount+local@hisisp.example + +When mail is sent to a remote host via SMTP, this replaces +his@localdomain.local by his ISP mail address, replaces her@localdomain.local +by her ISP mail address, and replaces other local addresses by his ISP account, +with an address extension of +local (this example assumes that the ISP supports +"+" style address extensions). + +LLooccaall aalliiaass ddaattaabbaassee + +When mail is to be delivered locally, the local(8) delivery agent runs each +local recipient name through the aliases(5) database. The mapping does not +affect addresses in message headers. Local aliases are typically used to +implement distribution lists, or to direct mail for standard aliases such as +postmaster to real people. The table can also be used to map +"Firstname.Lastname" addresses to login names. + +Alias lookups are enabled by default. The default configuration depends on the +operating system environment, but it is typically one of the following: + + /etc/postfix/main.cf: + alias_maps = hash:/etc/aliases + alias_maps = dbm:/etc/aliases, nis:mail.aliases + +The pathname of the alias database file is controlled with the alias_database +configuration parameter. The value is system dependent. Usually it is one of +the following: + + /etc/postfix/main.cf: + alias_database = hash:/etc/aliases (4.4BSD, LINUX) + alias_database = dbm:/etc/aliases (4.3BSD, SYSV<4) + alias_database = dbm:/etc/mail/aliases (SYSV4) + +An aliases(5) file can specify that mail should be delivered to a local file, +or to a command that receives the message in the standard input stream. For +security reasons, deliveries to command and file destinations are performed +with the rights of the alias database owner. A default userid, default_privs, +is used for deliveries to commands or files in "root"-owned aliases. + +LLooccaall ppeerr--uusseerr ..ffoorrwwaarrdd ffiilleess + +With delivery via the local(8) delivery agent, users can control their own mail +delivery by specifying destinations in a file called .forward in their home +directories. The syntax of these files is the same as with the local aliases(5) +file, except that the left-hand side of the alias (lookup key and colon) are +not present. + +LLooccaall ccaattcchh--aallll aaddddrreessss + +When the local(8) delivery agent finds that a message recipient does not exist, +the message is normally returned to the sender ("user unknown"). Sometimes it +is desirable to forward mail for non-existing recipients to another machine. +For this purpose you can specify an alternative destination with the +luser_relay configuration parameter. + +Alternatively, mail for non-existent recipients can be delegated to an entirely +different message transport, as specified with the fallback_transport +configuration parameter. For details, see the local(8) delivery agent +documentation. + +Note: if you use the luser_relay feature in order to receive mail for non-UNIX +accounts, then you must specify: + + /etc/postfix/main.cf: + local_recipient_maps = + +(i.e. empty) in the main.cf file, otherwise the Postfix SMTP server will reject +mail for non-UNIX accounts with "User unknown in local recipient table". See +the LOCAL_RECIPIENT_README file for more information on this. + +luser_relay can specify one address. It is subjected to "$name" expansions. +Examples: + + $user@other.host + The bare username, without address extension, is prepended to + "@other.host". For example, mail for "username+foo" is sent to + "username@other.host". + + $local@other.host + The entire original recipient localpart, including address extension, + is prepended to "@other.host". For example, mail for "username+foo" is + sent to "username+foo@other.host". + + sysadmin+$user + The bare username, without address extension, is appended to + "sysadmin". For example, mail for "username+foo" is sent to + "sysadmin+username". + + sysadmin+$local + The entire original recipient localpart, including address extension, + is appended to "sysadmin". For example, mail for "username+foo" is sent + to "sysadmin+username+foo". + +DDeebbuuggggiinngg yyoouurr aaddddrreessss mmaanniippuullaattiioonnss + +Postfix version 2.1 and later can produce mail delivery reports for debugging +purposes. These reports not only show sender/recipient addresses after address +rewriting and alias expansion or forwarding, they also show information about +delivery to mailbox, delivery to non-Postfix command, responses from remote +SMTP servers, and so on. + +Postfix can produce two types of mail delivery reports for debugging: + + * What-if: report what would happen, but do not actually deliver mail. This + mode of operation is requested with: + + $ //uussrr//ssbbiinn//sseennddmmaaiill --bbvv aaddddrreessss...... + Mail Delivery Status Report will be mailed to <your login name>. + + * What happened: deliver mail and report successes and/or failures, including + replies from remote SMTP servers. This mode of operation is requested with: + + $ //uussrr//ssbbiinn//sseennddmmaaiill --vv aaddddrreessss...... + Mail Delivery Status Report will be mailed to <your login name>. + +These reports contain information that is generated by Postfix delivery agents. +Since these run as daemon processes and do not interact with users directly, +the result is sent as mail to the sender of the test message. The format of +these reports is practically identical to that of ordinary non-delivery +notifications. + +As an example, below is the delivery report that is produced with the command +"sendmail -bv postfix-users@postfix.org". The first part of the report contains +human-readable text. In this case, mail would be delivered via mail.cloud9.net, +and the SMTP server replies with "250 Ok". Other reports may show delivery to +mailbox, or delivery to non-Postfix command. + + Content-Description: Notification + Content-Type: text/plain + + This is the mail system at host spike.porcupine.org. + + Enclosed is the mail delivery report that you requested. + + The mail system + + <postfix-users@postfix.org>: delivery via mail.cloud9.net[168.100.1.4]: 250 + 2.1.5 Ok + +The second part of the report is in machine-readable form, and includes the +following information: + + * The envelope sender address (wietse@porcupine.org). + * The envelope recipient address (postfix-users@postfix.org). If the + recipient address was changed by Postfix then Postfix also includes the + original recipient address. + * The delivery status. + +Some details depend on Postfix version. The example below is for Postfix +version 2.3 and later. + + Content-Description: Delivery report + Content-Type: message/delivery-status + + Reporting-MTA: dns; spike.porcupine.org + X-Postfix-Queue-ID: 84863BC0E5 + X-Postfix-Sender: rfc822; wietse@porcupine.org + Arrival-Date: Sun, 26 Nov 2006 17:01:01 -0500 (EST) + + Final-Recipient: rfc822; postfix-users@postfix.org + Action: deliverable + Status: 2.1.5 + Remote-MTA: dns; mail.cloud9.net + Diagnostic-Code: smtp; 250 2.1.5 Ok + +The third part of the report contains the message that Postfix would have +delivered, including From: and To: message headers, so that you can see any +effects of address rewriting on those. Mail submitted with "sendmail -bv" has +no body content so none is shown in the example below. + + Content-Description: Message + Content-Type: message/rfc822 + + Received: by spike.porcupine.org (Postfix, from userid 1001) + id 84863BC0E5; Sun, 26 Nov 2006 17:01:01 -0500 (EST) + Subject: probe + To: postfix-users@postfix.org + Message-Id: <20061126220101.84863BC0E5@spike.porcupine.org> + Date: Sun, 26 Nov 2006 17:01:01 -0500 (EST) + From: wietse@porcupine.org (Wietse Venema) + diff --git a/README_FILES/ADDRESS_VERIFICATION_README b/README_FILES/ADDRESS_VERIFICATION_README new file mode 100644 index 0000000..3a7e51a --- /dev/null +++ b/README_FILES/ADDRESS_VERIFICATION_README @@ -0,0 +1,451 @@ +PPoossttffiixx AAddddrreessss VVeerriiffiiccaattiioonn HHoowwttoo + +------------------------------------------------------------------------------- + +WWAARRNNIINNGG + +Recipient address verification may cause an increased load on down-stream +servers in the case of a dictionary attack or a flood of backscatter bounces. +Sender address verification may cause your site to be denylisted by some +providers. See also the "Limitations" section below for more. + +WWhhaatt PPoossttffiixx aaddddrreessss vveerriiffiiccaattiioonn ccaann ddoo ffoorr yyoouu + +Address verification is a feature that allows the Postfix SMTP server to block +a sender (MAIL FROM) or recipient (RCPT TO) address until the address has been +verified to be deliverable. + +The technique has obvious uses to reject junk mail with an unreplyable sender +address. + +The technique is also useful to block mail for undeliverable recipients, for +example on a mail relay host that does not have a list of all the valid +recipient addresses. This prevents undeliverable junk mail from entering the +queue, so that Postfix doesn't have to waste resources trying to send MAILER- +DAEMON messages back. + +This feature is available in Postfix version 2.1 and later. + +Topics covered in this document: + + * How address verification works + * Limitations of address verification + * Recipient address verification + * Sender address verification for mail from frequently forged domains + * Sender address verification for all email + * Address verification database + * Managing the address verification database + * Controlling the routing of address verification probes + * Forced probe routing examples + * Limitations of forced probe routing + +HHooww aaddddrreessss vveerriiffiiccaattiioonn wwoorrkkss + +A Postfix MTA verifies a sender or recipient address by probing the preferred +MTAs for that address, without actually delivering mail. The preferred MTAs +could include the Postfix MTA itself, or some remote MTAs (SMTP interruptus). +Probe messages are like normal mail, except that they are never delivered, +deferred or bounced; probe messages are always discarded. + + probe Postfix + message -> mail + Postfix Postfix -> queue + Internet -> SMTP <-> verify + server server | + v + + <- Postfix + probe <- delivery -> Local + status agents -> Remote + ^ + | + v + + Address + verification + database + +With Postfix address verification turned on, normal mail will suffer only a +short delay of up to 6 seconds while an address is being verified for the first +time. Once an address status is known, the status is cached and Postfix replies +immediately. + +When verification takes too long the Postfix SMTP server defers the sender or +recipient address with a 450 reply. Normal mail clients will connect again +after some delay. The address verification delay is configurable with the +main.cf address_verify_poll_count and address_verify_poll_delay parameters. See +postconf(5) for details. + +LLiimmiittaattiioonnss ooff aaddddrreessss vveerriiffiiccaattiioonn + + * Postfix assumes that a remote SMTP server will reject unknown addresses in + reply to the RCPT TO command. However, some sites report this in reply to + the DATA command. For such sites you may configure a workaround with the + smtp_address_verify_target parameter (Postfix 3.0 and later). + + * When verifying a remote address, Postfix probes the preferred MTAs for that + address, without actually delivering mail. If a preferred MTA accepts the + address, then Postfix assumes that the address is deliverable. In reality, + mail for a remote address can bounce AFTER a preferred MTA accepts the + recipient address, or AFTER a preferred MTA accepts the message content. + + * Some sites may denylist you when you are probing them too often (a probe is + an SMTP session that does not deliver mail), or when you are probing them + too often for a non-existent address. This is one reason why you should use + sender address verification sparingly, if at all, when your site receives + lots of email. + + * Normally, address verification probe messages follow the same path as + regular mail. However, some sites send mail to the Internet via an + intermediate relayhost; this breaks address verification. See below, + section "Controlling the routing of address verification probes", for how + to override mail routing and for possible limitations when you have to do + this. + + * Postfix assumes that an address is undeliverable when a preferred MTA for + the address rejects the probe, regardless of the reason for rejection + (client rejected, HELO rejected, MAIL FROM rejected, etc.). Thus, Postfix + rejects an address when a preferred MTA for that address rejects mail from + your machine for any reason. This is not a limitation, but it is mentioned + here just in case people believe that it is a limitation. + + * Unfortunately, some sites do not reject unknown addresses in reply to the + RCPT TO or DATA command, but instead report a delivery failure in response + to end of DATA after a message is transferred. Postfix address verification + does not work with such sites. + + * By default, Postfix probe messages have a sender address "double- + bounce@$myorigin" (with Postfix versions before 2.5, the default is + "postmaster@$myorigin"). This is SAFE because the Postfix SMTP server does + not reject mail for this address. + + You can change the probe sender address into the null address + ("address_verify_sender ="). This is UNSAFE because address probes will + fail with mis-configured sites that reject MAIL FROM: <>, while probes from + "double-bounce@$myorigin" would succeed. + + * The downside of using a non-empty sender address is that the address may + end up on spammer mailing lists. Although Postfix always discards mail to + the double-bounce address, this still results in wasted network bandwidth + and server capacity. To defeat address harvesting, Postfix 2.9 and later + support time-dependent sender addresses when you specify a non-zero + address_verify_sender_ttl value. + +RReecciippiieenntt aaddddrreessss vveerriiffiiccaattiioonn + +As mentioned earlier, recipient address verification is useful to block mail +for undeliverable recipients on a mail relay host that does not have a list of +all valid recipient addresses. This can help to prevent the mail queue from +filling up with MAILER-DAEMON messages. + +Recipient address verification is relatively straightforward and there are no +surprises. If a recipient probe fails, then Postfix rejects mail for the +recipient address. If a recipient probe succeeds, then Postfix accepts mail for +the recipient address. However, recipient address verification probes can +increase the load on down-stream MTAs when you're being flooded by backscatter +bounces, or when some spammer is mounting a dictionary attack. + +By default, address verification results are saved in a persistent database +(Postfix version 2.7 and later; with earlier versions, specify the database in +main.cf as described later). The persistent database helps to avoid probing the +same address repeatedly. + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + permit_mynetworks + # reject_unauth_destination is not needed here if the mail + # relay policy is specified under smtpd_relay_restrictions + # (available with Postfix 2.10 and later). + reject_unauth_destination + ... + reject_unknown_recipient_domain + reject_unverified_recipient + ... + # Postfix 2.6 and later privacy feature. + # unverified_recipient_reject_reason = Address lookup failed + + # Postfix 3.2 and earlier workaround. + # Do not set enable_original_recipient=no. This prevents Postfix + # from saving the recipient address verification result under + # the original address, when the address verification probe + # message goes through address aliasing or canonical mapping. + +The "reject_unknown_recipient_domain" restriction blocks mail for non-existent +domains. Putting this before "reject_unverified_recipient" avoids the overhead +of generating unnecessary probe messages. + +The unverified_recipient_reject_code parameter (default 450) specifies the +numerical Postfix SMTP server reply code when a recipient address is known to +bounce. Change this setting into 550 when you trust Postfix's judgments. + +The following features are available in Postfix 2.6 and later. + +The unverified_recipient_defer_code parameter (default 450) specifies the +numerical Postfix SMTP server reply code when a recipient address probe fails +with some temporary error. Some sites insist on changing this into 250. NOTE: +This change turns MX servers into backscatter sources when the load is high. + +The unverified_recipient_reject_reason parameter (default: empty) specifies +fixed text that Postfix will send to remote SMTP clients, instead of sending +actual address verification details. Do not specify the SMTP status code or +enhanced status code. + +The unverified_recipient_tempfail_action parameter (default: defer_if_permit) +specifies the Postfix SMTP server action when a recipient address verification +probe fails with some temporary error. + +SSeennddeerr aaddddrreessss vveerriiffiiccaattiioonn ffoorr mmaaiill ffrroomm ffrreeqquueennttllyy ffoorrggeedd ddoommaaiinnss + +Only for very small sites, it is relatively safe to turn on sender address +verification for specific domains that often appear in forged email. + + /etc/postfix/main.cf: + smtpd_sender_restrictions = hash:/etc/postfix/sender_access + unverified_sender_reject_code = 550 + # Postfix 2.6 and later. + # unverified_sender_defer_code = 250 + + # Default setting for Postfix 2.7 and later. + # Note 1: Be sure to read the "Caching" section below! + # Note 2: Avoid hash files here. Use btree or lmdb instead. + address_verify_map = btree:/var/lib/postfix/verify + + # Postfix 3.2 and earlier workaround. + # Do not set enable_original_recipient=no. This prevents Postfix + # from saving the sender address verification result under the + # original address, when the address verification probe message + # goes through address aliasing or canonical mapping. + + /etc/postfix/sender_access: + # Don't do this when you handle lots of email. + aol.com reject_unverified_sender + hotmail.com reject_unverified_sender + bigfoot.com reject_unverified_sender + ... etcetera ... + +At some point in cyberspace/time, a list of frequently forged MAIL FROM domains +could be found at http://www.monkeys.com/anti-spam/filtering/sender-domain- +validate.in. + +NOTE: One of the first things you might want to do is to turn on sender address +verification for all your own domains. + +SSeennddeerr aaddddrreessss vveerriiffiiccaattiioonn ffoorr aallll eemmaaiill + +Unfortunately, sender address verification cannot simply be turned on for all +email - you are likely to lose legitimate mail from mis-configured systems. You +almost certainly will have to set up allow lists for specific addresses, or +even for entire domains. + +To find out how sender address verification would affect your mail, specify +"warn_if_reject reject_unverified_sender" so that you can see what mail would +be blocked: + + /etc/postfix/main.cf: + smtpd_sender_restrictions = + permit_mynetworks + ... + check_sender_access hash:/etc/postfix/sender_access + reject_unknown_sender_domain + warn_if_reject reject_unverified_sender + ... + # Postfix 2.6 and later. + # unverified_sender_reject_reason = Address verification failed + + # Default setting for Postfix 2.7 and later. + # Note 1: Be sure to read the "Caching" section below! + # Note 2: Avoid hash files here. Use btree or lmdb instead. + address_verify_map = btree:/var/lib/postfix/verify + +This is also a good way to populate your cache with address verification +results before you start to actually reject mail. + +The sender_access restriction is needed to allowlist domains or addresses that +are known to be OK. Although Postfix will not mark a known-to-be-good address +as bad after a probe fails, it is better to be safe than sorry. + +NOTE: You will have to allowlist sites such as securityfocus.com and other +sites that operate mailing lists that use a different sender address for each +posting (VERP). Such addresses pollute the address verification cache quickly, +and generate unnecessary sender verification probes. + + /etc/postfix/sender_access + securityfocus.com OK + ... + +The "reject_unknown_sender_domain" restriction blocks mail from non-existent +domains. Putting this before "reject_unverified_sender" avoids the overhead of +generating unnecessary probe messages. + +The unverified_sender_reject_code parameter (default 450) specifies the +numerical Postfix server reply code when a sender address is known to bounce. +Change this setting into 550 when you trust Postfix's judgments. + +The following features are available in Postfix 2.6 and later. + +The unverified_sender_defer_code parameter (default 450) specifies the +numerical Postfix SMTP server reply code when a sender address verification +probe fails with some temporary error. Specify a valid 2xx or 4xx code. + +The unverified_sender_reject_reason parameter (default: empty) specifies fixed +text that Postfix will send to remote SMTP clients, instead of sending actual +address verification details. Do not specify the SMTP status code or enhanced +status code. + +The unverified_sender_tempfail_action parameter (default: defer_if_permit) +specifies the Postfix SMTP server action when a sender address verification +probe fails with some temporary error. + +AAddddrreessss vveerriiffiiccaattiioonn ddaattaabbaassee + +To improve performance, the Postfix verify(8) daemon can save address +verification results to a persistent database. This is enabled by default with +Postfix 2.7 and later. The address_verify_map (NOTE: singular) configuration +parameter specifies persistent storage for sender or recipient address +verification results. If you specify an empty value, all address verification +results are lost after "postfix reload" or "postfix stop". + + # Example 1: Default setting for Postfix 2.7 and later. + # Note: avoid hash files here. Use btree or lmdb instead. + /etc/postfix/main.cf: + address_verify_map = btree:$data_directory/verify_cache + + # Example 2: Shared persistent lmdb: cache (Postfix 2.11 or later). + # Disable automatic cache cleanup in all Postfix instances except + # for one instance that will be responsible for cache cleanup. + /etc/postfix/main.cf: + address_verify_map = lmdb:$data_directory/verify_cache + # address_verify_cache_cleanup_interval = 0 + + # Example 3: Shared persistent btree: cache (Postfix 2.9 or later). + # Disable automatic cache cleanup in all Postfix instances except + # for one instance that will be responsible for cache cleanup. + /etc/postfix/main.cf: + address_verify_map = proxy:btree:$data_directory/verify_cache + # address_verify_cache_cleanup_interval = 0 + + # Example 4: Shared memory cache (requires Postfix 2.9 or later). + # Disable automatic cache cleanup in all Postfix instances. + # See memcache_table(5) for details. + /etc/postfix/main.cf: + address_verify_map = memcache:/etc/postfix/verify-memcache.cf + address_verify_cache_cleanup_interval = 0 + + # Example 5: Default setting for Postfix 2.6 and earlier. + # This uses non-persistent storage only. + /etc/postfix/main.cf: + address_verify_map = + +NOTE 1: The database file should be stored under a Postfix-owned directory, +such as $data_directory. + + As of version 2.5, Postfix no longer uses root privileges when opening this + file. To maintain backwards compatibility, an attempt to open the file + under a non-Postfix directory is redirected to the Postfix-owned + data_directory, and a warning is logged. If you wish to continue using a + pre-existing database file, change its file ownership to the account + specified with the mail_owner parameter, and either move the file to the + data_directory, or move it to some other Postfix-owned directory. + +NOTE 2: Do not put this file in a file system that may run out of space. When +the address verification table gets corrupted the world comes to an end and YOU +will have to MANUALLY fix things as described in the next section. Meanwhile, +you will not receive mail via SMTP. + +NOTE 3: The verify(8) daemon will create a new database when none exists. It +will open or create the file before entering the chroot jail. + +MMaannaaggiinngg tthhee aaddddrreessss vveerriiffiiccaattiioonn ddaattaabbaassee + +The verify(8) manual page describes parameters that control how long address +verification results are cached before they need to be refreshed, and how long +results can remain "unrefreshed" before they expire. Postfix uses different +controls for positive results (address was accepted) and for negative results +(address was rejected, or address verification failed for some other reason). + +The verify(8) daemon will periodically remove expired entries from the address +verification database, and log the number of entries retained and dropped +(Postfix versions 2.7 and later). A cleanup run is logged as "partial" when the +daemon terminates early because of "postfix reload, "postfix stop", or because +the daemon received no requests for $max_idle seconds. Postfix versions 2.6 and +earlier do not implement automatic address verification database cleanup. +There, the database is managed manually as described next. + +When the address verification database file becomes too big, or when it becomes +corrupted, the solution is to manually rename or delete (NOT: truncate) the +file and run "postfix reload". The verify(8) daemon will then create a new +database file. + +CCoonnttrroolllliinngg tthhee rroouuttiinngg ooff aaddddrreessss vveerriiffiiccaattiioonn pprroobbeess + +By default, Postfix sends address verification probe messages via the same +route as regular mail, because that normally produces the most accurate result. +It's no good to verify a local address by connecting to your own SMTP port; +that just triggers all kinds of mailer loop alarms. The same is true for any +destination that your machine is best MX host for: hidden domains, virtual +domains, etc. + +However, some sites have a complex infrastructure where mail is not sent +directly to the Internet, but is instead given to an intermediate relayhost. +This is a problem for address verification, because remote Internet addresses +can be verified only when Postfix can access remote destinations directly. + +For this reason, Postfix allows you to override the routing parameters when it +delivers an address verification probe message. + +First, the address_verify_relayhost parameter allows you to override the +relayhost setting, and the address_verify_transport_maps parameter allows you +to override the transport_maps setting. The +address_verify_sender_dependent_relayhost_maps parameter does the same for +sender-dependent relayhost selection. + +Second, each address class is given its own address verification version of the +message delivery transport, as shown in the table below. Address classes are +defined in the ADDRESS_CLASS_README file. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |DDoommaaiinn lliisstt |RReegguullaarr ttrraannssppoorrtt|VVeerriiffyy ttrraannssppoorrtt | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mydestination |local_transport |address_verify_local_transport | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |virtual_alias_domains |(not applicable) |(not applicable) | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |virtual_mailbox_domains|virtual_transport|address_verify_virtual_transport| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |relay_domains |relay_transport |address_verify_relay_transport | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |(not applicable) |default_transport|address_verify_default_transport| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +By default, the parameters that control delivery of address probes have the +same value as the parameters that control normal mail delivery. + +FFoorrcceedd pprroobbee rroouuttiinngg eexxaammpplleess + +In a typical scenario one would override the relayhost setting for address +verification probes and leave everything else alone: + + /etc/postfix/main.cf: + relayhost = $mydomain + address_verify_relayhost = + ... + +Sites behind a network address translation box might have to use a different +SMTP client that sends the correct hostname information: + + /etc/postfix/main.cf: + relayhost = $mydomain + address_verify_relayhost = + address_verify_default_transport = direct_smtp + + /etc/postfix/master.cf: + direct_smtp .. .. .. .. .. .. .. .. .. smtp + -o smtp_helo_name=nat.box.tld + +LLiimmiittaattiioonnss ooff ffoorrcceedd pprroobbee rroouuttiinngg + +Inconsistencies can happen when probe messages don't follow the same path as +regular mail. For example, a message can be accepted when it follows the +regular route while an otherwise identical probe message is rejected when it +follows the forced route. The opposite can happen, too, but is less likely. + diff --git a/README_FILES/BACKSCATTER_README b/README_FILES/BACKSCATTER_README new file mode 100644 index 0000000..8095df2 --- /dev/null +++ b/README_FILES/BACKSCATTER_README @@ -0,0 +1,281 @@ +PPoossttffiixx BBaacckkssccaatttteerr HHoowwttoo + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document describes features that require Postfix version 2.0 or later. + +Topics covered in this document: + + * What is backscatter mail? + * How do I block backscatter mail to random recipient addresses? + * How do I block backscatter mail to real recipient addresses? + + o Blocking backscatter mail with forged mail server information + o Blocking backscatter mail with forged sender information + o Blocking backscatter mail with other forged information + o Blocking backscatter mail from virus scanners + +The examples use Perl Compatible Regular Expressions (Postfix pcre: tables), +but also provide a translation to POSIX regular expressions (Postfix regexp: +tables). PCRE is preferred primarily because the implementation is often +faster. + +WWhhaatt iiss bbaacckkssccaatttteerr mmaaiill?? + +When a spammer or worm sends mail with forged sender addresses, innocent sites +are flooded with undeliverable mail notifications. This is called backscatter +mail. With Postfix, you know that you're a backscatter victim when your logfile +goes on and on like this: + + Dec 4 04:30:09 hostname postfix/smtpd[58549]: NOQUEUE: reject: + RCPT from xxxxxxx[x.x.x.x]: 550 5.1.1 <yyyyyy@your.domain.here>: + Recipient address rejected: User unknown; from=<> + to=<yyyyyy@your.domain.here> proto=ESMTP helo=<zzzzzz> + +What you see are lots of "user unknown" errors with "from=<>". These are error +reports from MAILER-DAEMONs elsewhere on the Internet, about email that was +sent with a false sender address in your domain. + +HHooww ddoo II bblloocckk bbaacckkssccaatttteerr mmaaiill ttoo rraannddoomm rreecciippiieenntt aaddddrreesssseess?? + +If your machine receives backscatter mail to random addresses, configure +Postfix to reject all mail for non-existent recipients as described in the +LOCAL_RECIPIENT_README and STANDARD_CONFIGURATION_README documentation. + +If your machine runs Postfix 2.0 and earlier, disable the "pause before reject" +feature in the SMTP server. If your system is under stress then it should not +waste time. + + /etc/postfix/main.cf: + # Not needed with Postfix 2.1 and later. + smtpd_error_sleep_time = 0 + + # Not needed with Postfix 2.4 and later. + unknown_local_recipient_reject_code = 550 + +HHooww ddoo II bblloocckk bbaacckkssccaatttteerr mmaaiill ttoo rreeaall rreecciippiieenntt aaddddrreesssseess?? + +When backscatter mail passes the "unknown recipient" barrier, there still is no +need to despair. Many mail systems are kind enough to attach the message +headers of the undeliverable mail in the non-delivery notification. These +message headers contain information that you can use to recognize and block +forged mail. + +BBlloocckkiinngg bbaacckkssccaatttteerr mmaaiill wwiitthh ffoorrggeedd mmaaiill sseerrvveerr iinnffoorrmmaattiioonn + +Although my email address is "wietse@porcupine.org", all my mail systems +announce themselves with the SMTP HELO command as "hostname.porcupine.org". +Thus, if returned mail has a Received: message header like this: + + Received: from porcupine.org ... + +Then I know that this is almost certainly forged mail (almost; see next section +for the fly in the ointment). Mail that is really sent by my systems looks like +this: + + Received: from hostname.porcupine.org ... + +For the same reason the following message headers are very likely to be the +result of forgery: + + Received: from host.example.com ([1.2.3.4] helo=porcupine.org) ... + Received: from [1.2.3.4] (port=12345 helo=porcupine.org) ... + Received: from host.example.com (HELO porcupine.org) ... + Received: from host.example.com (EHLO porcupine.org) ... + +Some forgeries show up in the way that a mail server reports itself in +Received: message headers. Keeping in mind that all my systems have a mail +server name of hostname.porcupine.org, the following is definitely a forgery: + + Received: by porcupine.org ... + Received: from host.example.com ( ... ) by porcupine.org ... + +Another frequent sign of forgery is the Message-ID: header. My systems produce +a Message-ID: of <stuff@hostname.porcupine.org>. The following are forgeries, +especially the first one: + + Message-ID: <1cb479435d8eb9.2beb1.qmail@porcupine.org> + Message-ID: <yulszqocfzsficvzzju@porcupine.org> + +To block such backscatter I use header_checks and body_checks patterns like +this: + + /etc/postfix/main.cf: + header_checks = pcre:/etc/postfix/header_checks + body_checks = pcre:/etc/postfix/body_checks + + /etc/postfix/header_checks: + # Do not indent the patterns between "if" and "endif". + if /^Received:/ + /^Received: +from +(porcupine\.org) +/ + reject forged client name in Received: header: $1 + /^Received: +from +[^ ]+ +\(([^ ]+ +[he]+lo=|[he]+lo +) + (porcupine\.org)\)/ + reject forged client name in Received: header: $2 + /^Received:.* +by +(porcupine\.org)\b/ + reject forged mail server name in Received: header: $1 + endif + /^Message-ID:.* <!&!/ DUNNO + /^Message-ID:.*@(porcupine\.org)/ + reject forged domain name in Message-ID: header: $1 + + /etc/postfix/body_checks: + # Do not indent the patterns between "if" and "endif". + if /^[> ]*Received:/ + /^[> ]*Received: +from +(porcupine\.org) / + reject forged client name in Received: header: $1 + /^[> ]*Received: +from +[^ ]+ +\(([^ ]+ +[he]+lo=|[he]+lo +) + (porcupine\.org)\)/ + reject forged client name in Received: header: $2 + /^[> ]*Received:.* +by +(porcupine\.org)\b/ + reject forged mail server name in Received: header: $1 + endif + /^[> ]*Message-ID:.* <!&!/ DUNNO + /^[> ]*Message-ID:.*@(porcupine\.org)/ + reject forged domain name in Message-ID: header: $1 + +Notes: + + * The example uses pcre: tables mainly for speed; with minor modifications, + you can use regexp: tables as explained below. + + * The example is simplified for educational purposes. In reality my patterns + list multiple domain names, as "(domain|domain|...)". + + * The "\." matches "." literally. Without the "\", the "." would match any + character. + + * The "\(" and "\)" match "(" and ")" literally. Without the "\", the "(" and + ")" would be grouping operators. + + * The "\b" is used here to match the end of a word. If you use regexp: + tables, specify "[[:>:]]" (on some systems you should specify "\>" instead; + for details see your system documentation). + + * The "if /pattern/" and "endif" eliminate unnecessary matching attempts. DO + NOT indent lines starting with /pattern/ between the "if" and "endif"! + + * The two "Message-ID:.* <!&!" rules are workarounds for some versions of + Outlook express, as described in the caveats section below. + +CCaavveeaattss + + * Netscape Messenger (and reportedly, Mozilla) sends a HELO name that is + identical to the sender address domain part. If you have such clients then + the above patterns would block legitimate email. + + My network has only one such machine, and to prevent its mail from being + blocked I have configured it to send mail as user@hostname.porcupine.org. + On the Postfix server, a canonical mapping translates this temporary + address into user@porcupine.org. + + /etc/postfix/main.cf: + canonical_maps = hash:/etc/postfix/canonical + + /etc/postfix/canonical: + @hostname.porcupine.org @porcupine.org + + This is of course practical only when you have very few systems that send + HELO commands like this, and when you never have to send mail to a user on + such a host. + + An alternative would be to remove the hostname from + "hostname.porcupine.org" with address masquerading, as described in the + ADDRESS_REWRITING_README document. + + * Reportedly, Outlook 2003 (perhaps Outlook Express, and other versions as + well) present substantially different Message-ID headers depending upon + whether or not a DSN is requested (via Options "Request a delivery receipt + for this message"). + + When a DSN is requested, Outlook 2003 uses a Message-ID string that ends in + the sender's domain name: + + Message-ID: <!&! ...very long string... ==@example.com> + + where example.com is the domain name part of the email address specified in + Outlook's account settings for the user. Since many users configure their + email addresses as username@example.com, messages with DSN turned on will + trigger the REJECT action in the previous section. + + If you have such clients then you can exclude their Message-ID strings with + the two "Message-ID:.* <!&!" patterns that are shown in the previous + section. Otherwise you will not be able to use the two backscatter rules to + stop forged Message ID strings. Of course this workaround may break the + next time Outlook is changed. + +BBlloocckkiinngg bbaacckkssccaatttteerr mmaaiill wwiitthh ffoorrggeedd sseennddeerr iinnffoorrmmaattiioonn + +Like many people I still have a few email addresses in domains that I used in +the past. Mail for those addresses is forwarded to my current address. Most of +the backscatter mail that I get claims to be sent from these addresses. Such +mail is obviously forged and is very easy to stop. + + /etc/postfix/main.cf: + header_checks = pcre:/etc/postfix/header_checks + body_checks = pcre:/etc/postfix/body_checks + + /etc/postfix/header_checks: + /^(From|Return-Path):.*\b(user@domain\.tld)\b/ + reject forged sender address in $1: header: $2 + + /etc/postfix/body_checks: + /^[> ]*(From|Return-Path):.*\b(user@domain\.tld)\b/ + reject forged sender address in $1: header: $2 + +Notes: + + * The example uses pcre: tables mainly for speed; with minor modifications, + you can use regexp: tables as explained below. + + * The example is simplified for educational purposes. In reality, my patterns + list multiple email addresses as "(user1@domain1\.tld|user2@domain2\.tld)". + + * The two "\b" as used in "\b(user@domain\.tld)\b" match the beginning and + end of a word, respectively. If you use regexp: tables, specify "[[:<:]] + and [[:>:]]" (on some systems you should specify "\< and \>" instead; for + details see your system documentation). + + * The "\." matches "." literally. Without the "\", the "." would match any + character. + +BBlloocckkiinngg bbaacckkssccaatttteerr mmaaiill wwiitthh ootthheerr ffoorrggeedd iinnffoorrmmaattiioonn + +Another sign of forgery can be found in the IP address that is recorded in +Received: headers next to your HELO host or domain name. This information must +be used with care, though. Some mail servers are behind a network address +translator and never see the true client IP address. + +BBlloocckkiinngg bbaacckkssccaatttteerr mmaaiill ffrroomm vviirruuss ssccaannnneerrss + +With all the easily recognizable forgeries eliminated, there is one category of +backscatter mail that remains, and that is notifications from virus scanner +software. Unfortunately, some virus scanning software doesn't know that viruses +forge sender addresses. To make matters worse, the software also doesn't know +how to report a mail delivery problem, so that we cannot use the above +techniques to recognize forgeries. + +Recognizing virus scanner mail is an error prone process, because there is a +lot of variation in report formats. The following is only a small example of +message header patterns. For a large collection of header and body patterns +that recognize virus notification email, see https://web.archive.org/web/ +20100317123907/http://std.dkuug.dk/keld/virus/ or http://www.t29.dk/ +antiantivirus.txt. + + /etc/postfix/header_checks: + /^Subject: *Your email contains VIRUSES/ DISCARD virus notification + /^Content-Disposition:.*VIRUS1_DETECTED_AND_REMOVED/ + DISCARD virus notification + /^Content-Disposition:.*VirusWarning.txt/ DISCARD virus notification + +Note: these documents haven't been updated since 2004, so they are useful only +as a starting point. + +A plea to virus or spam scanner operators: please do not make the problem worse +by sending return mail to forged sender addresses. You're only harassing +innocent people. If you must return mail to the purported sender, please return +the full message headers, so that the sender can filter out the obvious +forgeries. + diff --git a/README_FILES/BASIC_CONFIGURATION_README b/README_FILES/BASIC_CONFIGURATION_README new file mode 100644 index 0000000..6f3606d --- /dev/null +++ b/README_FILES/BASIC_CONFIGURATION_README @@ -0,0 +1,489 @@ +PPoossttffiixx BBaassiicc CCoonnffiigguurraattiioonn + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix has several hundred configuration parameters that are controlled via +the main.cf file. Fortunately, all parameters have sensible default values. In +many cases, you need to configure only two or three parameters before you can +start to play with the mail system. Here's a quick introduction to the syntax: + + * Postfix configuration files + +The text below assumes that you already have Postfix installed on the system, +either by compiling the source code yourself (as described in the INSTALL file) +or by installing an already compiled version. + +This document covers basic Postfix configuration. Information about how to +configure Postfix for specific applications such as mailhub, firewall or dial- +up client can be found in the STANDARD_CONFIGURATION_README file. But don't go +there until you already have covered the material presented below. + +The first parameters of interest specify the machine's identity and role in the +network. + + * What domain name to use in outbound mail + + * What domains to receive mail for + + * What clients to relay mail from + + * What destinations to relay mail to + + * What delivery method: direct or indirect + +The default values for many other configuration parameters are derived from +just these. + +The next parameter of interest controls the amount of mail sent to the local +postmaster: + + * What trouble to report to the postmaster + +Be sure to set the following correctly if you're behind a proxy or network +address translator, and you are running a backup MX host for some other domain: + + * Proxy/NAT external network addresses + +Postfix daemon processes run in the background, and log problems and normal +activity to the syslog daemon. Here are a few things that you need to be aware +of: + + * What you need to know about Postfix logging + +If your machine has unusual security requirements you may want to run Postfix +daemon processes inside a chroot environment. + + * Running Postfix daemon processes chrooted + +If you run Postfix on a virtual network interface, or if your machine runs +other mailers on virtual interfaces, you'll have to look at the other +parameters listed here as well: + + * My own hostname + + * My own domain name + + * My own network addresses + +PPoossttffiixx ccoonnffiigguurraattiioonn ffiilleess + +By default, Postfix configuration files are in /etc/postfix. The two most +important files are main.cf and master.cf; these files must be owned by root. +Giving someone else write permission to main.cf or master.cf (or to their +parent directories) means giving root privileges to that person. + +In /etc/postfix/main.cf you will have to set up a minimal number of +configuration parameters. Postfix configuration parameters resemble shell +variables, with two important differences: the first one is that Postfix does +not know about quotes like the UNIX shell does. + +You specify a configuration parameter as: + + /etc/postfix/main.cf: + parameter = value + +and you use it by putting a "$" character in front of its name: + + /etc/postfix/main.cf: + other_parameter = $parameter + +You can use $parameter before it is given a value (that is the second main +difference with UNIX shell variables). The Postfix configuration language uses +lazy evaluation, and does not look at a parameter value until it is needed at +runtime. + +Postfix uses database files for access control, address rewriting and other +purposes. The DATABASE_README file gives an introduction to how Postfix works +with Berkeley DB, LDAP or SQL and other types. Here is a common example of how +Postfix invokes a database: + + /etc/postfix/main.cf: + virtual_alias_maps = hash:/etc/postfix/virtual + +Whenever you make a change to the main.cf or master.cf file, execute the +following command as root in order to refresh a running mail system: + + # postfix reload + +WWhhaatt ddoommaaiinn nnaammee ttoo uussee iinn oouuttbboouunndd mmaaiill + +The myorigin parameter specifies the domain that appears in mail that is posted +on this machine. The default is to use the local machine name, $myhostname, +which defaults to the name of the machine. Unless you are running a really +small site, you probably want to change that into $mydomain, which defaults to +the parent domain of the machine name. + +For the sake of consistency between sender and recipient addresses, myorigin +also specifies the domain name that is appended to an unqualified recipient +address. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + myorigin = $myhostname (default: send mail as "user@$myhostname") + myorigin = $mydomain (probably desirable: "user@$mydomain") + +WWhhaatt ddoommaaiinnss ttoo rreecceeiivvee mmaaiill ffoorr + +The mydestination parameter specifies what domains this machine will deliver +locally, instead of forwarding to another machine. The default is to receive +mail for the machine itself. See the VIRTUAL_README file for how to configure +Postfix for hosted domains. + +You can specify zero or more domain names, "/file/name" patterns and/or "type: +table" lookup tables (such as hash:, btree:, nis:, ldap:, or mysql:), separated +by whitespace and/or commas. A "/file/name" pattern is replaced by its +contents; "type:table" requests that a table lookup is done and merely tests +for existence: the lookup result is ignored. + +IMPORTANT: If your machine is a mail server for its entire domain, you must +list $mydomain as well. + +Example 1: default setting. + + /etc/postfix/main.cf: + mydestination = $myhostname localhost.$mydomain localhost + +Example 2: domain-wide mail server. + + /etc/postfix/main.cf: + mydestination = $myhostname localhost.$mydomain localhost $mydomain + +Example 3: host with multiple DNS A records. + + /etc/postfix/main.cf: + mydestination = $myhostname localhost.$mydomain localhost + www.$mydomain ftp.$mydomain + +Caution: in order to avoid mail delivery loops, you must list all hostnames of +the machine, including $myhostname, and localhost.$mydomain. + +WWhhaatt cclliieennttss ttoo rreellaayy mmaaiill ffrroomm + +By default, Postfix will forward mail from clients in authorized network blocks +to any destination. Authorized networks are defined with the mynetworks +configuration parameter. The current default is to authorize the local machine +only. Prior to Postfix 3.0, the default was to authorize all clients in the IP +subnetworks that the local machine is attached to. + +Postfix can also be configured to relay mail from "mobile" clients that send +mail from outside an authorized network block. This is explained in the +SASL_README and TLS_README documents. + +IMPORTANT: If your machine is connected to a wide area network then the +"mynetworks_style = subnet" setting may be too friendly. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + mynetworks_style = subnet (not safe on a wide area network) + mynetworks_style = host (authorize local machine only) + mynetworks = 127.0.0.0/8 (authorize local machine only) + mynetworks = 127.0.0.0/8 168.100.189.2/32 (authorize local machine) + mynetworks = 127.0.0.0/8 168.100.189.2/28 (authorize local networks) + +You can specify the trusted networks in the main.cf file, or you can let +Postfix do the work for you. The default is to let Postfix do the work. The +result depends on the mynetworks_style parameter value. + + * Specify "mynetworks_style = host" (the default when compatibility_level >= + 2) when Postfix should forward mail from only the local machine. + + * Specify "mynetworks_style = subnet" (the default when compatibility_level < + 2) when Postfix should forward mail from SMTP clients in the same IP + subnetworks as the local machine. On Linux, this works correctly only with + interfaces specified with the "ifconfig" or "ip" command. + + * Specify "mynetworks_style = class" when Postfix should forward mail from + SMTP clients in the same IP class A/B/C networks as the local machine. + Don't do this with a dialup site - it would cause Postfix to "trust" your + entire provider's network. Instead, specify an explicit mynetworks list by + hand, as described below. + +Alternatively, you can specify the mynetworks list by hand, in which case +Postfix ignores the mynetworks_style setting. To specify the list of trusted +networks by hand, specify network blocks in CIDR (network/mask) notation, for +example: + + /etc/postfix/main.cf: + mynetworks = 168.100.189.0/28, 127.0.0.0/8 + +You can also specify the absolute pathname of a pattern file instead of listing +the patterns in the main.cf file. + +WWhhaatt ddeessttiinnaattiioonnss ttoo rreellaayy mmaaiill ttoo + +By default, Postfix will forward mail from strangers (clients outside +authorized networks) to authorized remote destinations only. Authorized remote +destinations are defined with the relay_domains configuration parameter. The +default is to authorize all domains (and subdomains) of the domains listed with +the mydestination parameter. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + relay_domains = $mydestination (default) + relay_domains = (safe: never forward mail from strangers) + relay_domains = $mydomain (forward mail to my domain and subdomains) + +WWhhaatt ddeelliivveerryy mmeetthhoodd:: ddiirreecctt oorr iinnddiirreecctt + +By default, Postfix tries to deliver mail directly to the Internet. Depending +on your local conditions this may not be possible or desirable. For example, +your system may be turned off outside office hours, it may be behind a +firewall, or it may be connected via a provider who does not allow direct mail +to the Internet. In those cases you need to configure Postfix to deliver mail +indirectly via a relay host. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + relayhost = (default: direct delivery to Internet) + relayhost = $mydomain (deliver via local mailhub) + relayhost = [mail.$mydomain] (deliver via local mailhub) + relayhost = [mail.isp.tld] (deliver via provider mailhub) + +The form enclosed with [] eliminates DNS MX lookups. Don't worry if you don't +know what that means. Just be sure to specify the [] around the mailhub +hostname that your ISP gave to you, otherwise mail may be mis-delivered. + +The STANDARD_CONFIGURATION_README file has more hints and tips for firewalled +and/or dial-up networks. + +WWhhaatt ttrroouubbllee ttoo rreeppoorrtt ttoo tthhee ppoossttmmaasstteerr + +You should set up a postmaster alias in the aliases(5) table that directs mail +to a human person. The postmaster address is required to exist, so that people +can report mail delivery problems. While you're updating the aliases(5) table, +be sure to direct mail for the super-user to a human person too. + + /etc/aliases: + postmaster: you + root: you + +Execute the command "newaliases" after changing the aliases file. Instead of / +etc/aliases, your alias file may be located elsewhere. Use the command +"postconf alias_maps" to find out. + +The Postfix system reports problems to the postmaster alias. You may not be +interested in all types of trouble reports, so this reporting mechanism is +configurable. The default is to report only serious problems (resource, +software) to postmaster: + +Default setting: + + /etc/postfix/main.cf: + notify_classes = resource, software + +The meaning of the classes is as follows: + + bounce + Inform the postmaster of undeliverable mail. Either send the postmaster + a copy of undeliverable mail that is returned to the sender, or send a + transcript of the SMTP session when Postfix rejected mail. For privacy + reasons, the postmaster copy of undeliverable mail is truncated after + the original message headers. This implies "2bounce" (see below). See + also the luser_relay feature. The notification is sent to the address + specified with the bounce_notice_recipient configuration parameter + (default: postmaster). + 2bounce + When Postfix is unable to return undeliverable mail to the sender, send + it to the postmaster instead (without truncating the message after the + primary headers). The notification is sent to the address specified + with the 2bounce_notice_recipient configuration parameter (default: + postmaster). + delay + Inform the postmaster of delayed mail. In this case, the postmaster + receives message headers only. The notification is sent to the address + specified with the delay_notice_recipient configuration parameter + (default: postmaster). + policy + Inform the postmaster of client requests that were rejected because of + (UCE) policy restrictions. The postmaster receives a transcript of the + SMTP session. The notification is sent to the address specified with + the error_notice_recipient configuration parameter (default: + postmaster). + protocol + Inform the postmaster of protocol errors (client or server side) or + attempts by a client to execute unimplemented commands. The postmaster + receives a transcript of the SMTP session. The notification is sent to + the address specified with the error_notice_recipient configuration + parameter (default: postmaster). + resource + Inform the postmaster of mail not delivered due to resource problems + (for example, queue file write errors). The notification is sent to the + address specified with the error_notice_recipient configuration + parameter (default: postmaster). + software + Inform the postmaster of mail not delivered due to software problems. + The notification is sent to the address specified with the + error_notice_recipient configuration parameter (default: postmaster). + +PPrrooxxyy//NNAATT eexxtteerrnnaall nneettwwoorrkk aaddddrreesssseess + +Some mail servers are connected to the Internet via a network address +translator (NAT) or proxy. This means that systems on the Internet connect to +the address of the NAT or proxy, instead of connecting to the network address +of the mail server. The NAT or proxy forwards the connection to the network +address of the mail server, but Postfix does not know this. + +If you run a Postfix server behind a proxy or NAT, you need to configure the +proxy_interfaces parameter and specify all the external proxy or NAT addresses +that Postfix receives mail on. You may specify symbolic hostnames instead of +network addresses. + +IMPORTANT: You must specify your proxy/NAT external addresses when your system +is a backup MX host for other domains, otherwise mail delivery loops will +happen when the primary MX host is down. + +Example: host behind NAT box running a backup MX host. + + /etc/postfix/main.cf: + proxy_interfaces = 1.2.3.4 (the proxy/NAT external network address) + +WWhhaatt yyoouu nneeeedd ttoo kknnooww aabboouutt PPoossttffiixx llooggggiinngg + +Postfix daemon processes run in the background, and log problems and normal +activity to the syslog daemon. The syslogd process sorts events by class and +severity, and appends them to logfiles. The logging classes, levels and logfile +names are usually specified in /etc/syslog.conf. At the very least you need +something like: + + /etc/syslog.conf: + mail.err /dev/console + mail.debug /var/log/maillog + +After changing the syslog.conf file, send a "HUP" signal to the syslogd +process. + +IMPORTANT: many syslogd implementations will not create files. You must create +files before (re)starting syslogd. + +IMPORTANT: on Linux you need to put a "-" character before the pathname, e.g., +-/var/log/maillog, otherwise the syslogd process will use more system resources +than Postfix. + +Hopefully, the number of problems will be small, but it is a good idea to run +every night before the syslog files are rotated: + + # postfix check + # grep -E '(reject|warning|error|fatal|panic):' /some/log/file + + * The first line (postfix check) causes Postfix to report file permission/ + ownership discrepancies. + + * The second line looks for problem reports from the mail software, and + reports how effective the relay and junk mail access blocks are. This may + produce a lot of output. You will want to apply some postprocessing to + eliminate uninteresting information. + +The DEBUG_README document describes the meaning of the "warning" etc. labels in +Postfix logging. + +RRuunnnniinngg PPoossttffiixx ddaaeemmoonn pprroocceesssseess cchhrrooootteedd + +Postfix daemon processes can be configured (via the master.cf file) to run in a +chroot jail. The processes run at a fixed low privilege and with file system +access limited to the Postfix queue directories (/var/spool/postfix). This +provides a significant barrier against intrusion. The barrier is not +impenetrable (chroot limits file system access only), but every little bit +helps. + +With the exception of Postfix daemons that deliver mail locally and/or that +execute non-Postfix commands, every Postfix daemon can run chrooted. + +Sites with high security requirements should consider to chroot all daemons +that talk to the network: the smtp(8) and smtpd(8) processes, and perhaps also +the lmtp(8) client. The author's own porcupine.org mail server runs all daemons +chrooted that can be chrooted. + +The default /etc/postfix/master.cf file specifies that no Postfix daemon runs +chrooted. In order to enable chroot operation, edit the file /etc/postfix/ +master.cf, and follow instructions in the file. When you're finished, execute +"postfix reload" to make the change effective. + +Note that a chrooted daemon resolves all filenames relative to the Postfix +queue directory (/var/spool/postfix). For successful use of a chroot jail, most +UNIX systems require you to bring in some files or device nodes. The examples/ +chroot-setup directory in the source code distribution has a collection of +scripts that help you set up Postfix chroot environments on different operating +systems. + +Additionally, you almost certainly need to configure syslogd so that it listens +on a socket inside the Postfix queue directory. Examples of syslogd command +line options that achieve this for specific systems: + +FreeBSD: syslogd -l /var/spool/postfix/var/run/log + +Linux, OpenBSD: syslogd -a /var/spool/postfix/dev/log + +MMyy oowwnn hhoossttnnaammee + +The myhostname parameter specifies the fully-qualified domain name of the +machine running the Postfix system. $myhostname appears as the default value in +many other Postfix configuration parameters. + +By default, myhostname is set to the local machine name. If your local machine +name is not in fully-qualified domain name form, or if you run Postfix on a +virtual interface, you will have to specify the fully-qualified domain name +that the mail system should use. + +Alternatively, if you specify mydomain in main.cf, then Postfix will use its +value to generate a fully-qualified default value for the myhostname parameter. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + myhostname = host.local.domain (machine name is not FQDN) + myhostname = host.virtual.domain (virtual interface) + myhostname = virtual.domain (virtual interface) + +MMyy oowwnn ddoommaaiinn nnaammee + +The mydomain parameter specifies the parent domain of $myhostname. By default, +it is derived from $myhostname by stripping off the first part (unless the +result would be a top-level domain). + +Conversely, if you specify mydomain in main.cf, then Postfix will use its value +to generate a fully-qualified default value for the myhostname parameter. + +Examples (specify only one of the following): + + /etc/postfix/main.cf: + mydomain = local.domain + mydomain = virtual.domain (virtual interface) + +MMyy oowwnn nneettwwoorrkk aaddddrreesssseess + +The inet_interfaces parameter specifies all network interface addresses that +the Postfix system should listen on; mail addressed to "user@[network address]" +will be delivered locally, as if it is addressed to a domain listed in +$mydestination. + +You can override the inet_interfaces setting in the Postfix master.cf file by +prepending an IP address to a server name. + +The default is to listen on all active interfaces. If you run mailers on +virtual interfaces, you will have to specify what interfaces to listen on. + +IMPORTANT: If you run MTAs on virtual interfaces you must specify explicit +inet_interfaces values for the MTA that receives mail for the machine itself: +this MTA should never listen on the virtual interfaces or you would have a +mailer loop when a virtual MTA is down. + +Example: default setting. + + /etc/postfix/main.cf: + inet_interfaces = all + +Example: host running one or more virtual mailers. For each Postfix instance, +specify only one of the following. + + /etc/postfix/main.cf: + inet_interfaces = virtual.host.tld (virtual Postfix) + inet_interfaces = $myhostname localhost... (non-virtual Postfix) + +Note: you need to stop and start Postfix after changing this parameter. + diff --git a/README_FILES/BDAT_README b/README_FILES/BDAT_README new file mode 100644 index 0000000..6ed565c --- /dev/null +++ b/README_FILES/BDAT_README @@ -0,0 +1,124 @@ +PPoossttffiixx BBDDAATT ((CCHHUUNNKKIINNGG)) ssuuppppoorrtt + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +Postfix SMTP server supports RFC 3030 CHUNKING (the BDAT command) without +BINARYMIME, in both smtpd(8) and postscreen(8). It is enabled by default. + +Topics covered in this document: + + * Disabling BDAT support + * Impact on existing configurations + * Example SMTP session + * Benefits of CHUNKING (BDAT) support without BINARYMIME + * Downsides of CHUNKING (BDAT) support + +DDiissaabblliinngg BBDDAATT ssuuppppoorrtt + +BDAT support is enabled by default. To disable BDAT support globally: + + /etc/postfix/main.cf: + # The logging alternative: + smtpd_discard_ehlo_keywords = chunking + # The non-logging alternative: + smtpd_discard_ehlo_keywords = chunking, silent-discard + +Specify '-o smtpd_discard_ehlo_keywords=' in master.cf for the submission and +smtps services, if you have clients that benefit from CHUNKING support. + +IImmppaacctt oonn eexxiissttiinngg ccoonnffiigguurraattiioonnss + + * There are no changes for smtpd_mumble_restrictions, smtpd_proxy_filter, + smtpd_milters, or for postscreen settings, except for the above mentioned + option to suppress the SMTP server's CHUNKING service announcement. + + * There are no changes in the Postfix queue file content, no changes for + down-stream SMTP servers or after-queue content filters, and no changes in + the envelope or message content that Milters will receive. + +EExxaammppllee SSMMTTPP sseessssiioonn + +The main differences are that the Postfix SMTP server announces "CHUNKING" +support in the EHLO response, and that instead of sending one DATA request, the +remote SMTP client may send one or more BDAT requests. In the example below, +"S:" indicates server responses, and "C:" indicates client requests (bold +font). + + S: 220 server.example.com + C: EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + S: 250-server.example.com + S: 250-PIPELINING + S: 250-SIZE 153600000 + S: 250-VRFY + S: 250-ETRN + S: 250-STARTTLS + S: 250-AUTH PLAIN LOGIN + S: 250-ENHANCEDSTATUSCODES + S: 250-8BITMIME + S: 250-DSN + S: 250-SMTPUTF8 + S: 250 CHUNKING + C: MMAAIILL FFRROOMM::<<sseennddeerr@@eexxaammppllee..ccoomm>> + S: 250 2.1.0 Ok + C: RRCCPPTT TTOO::<<rreecciippiieenntt@@eexxaammppllee..ccoomm>> + S: 250 2.1.5 Ok + C: BBDDAATT 1100000000 + C: ....ffoolllloowweedd bbyy 1100000000 bbyytteess...... + S: 250 2.0.0 Ok: 10000 bytes + C: BBDDAATT 112233 + C: ....ffoolllloowweedd bbyy 112233 bbyytteess...... + S: 250 2.0.0 Ok: 123 bytes + C: BBDDAATT 00 LLAASSTT + S: 250 2.0.0 Ok: 10123 bytes queued as 41yYhh41qmznjbD + C: QQUUIITT + S: 221 2.0.0 Bye + +Internally in Postfix, there is no difference between mail that was received +with BDAT or with DATA. Postfix smtpd_mumble_restrictions, policy delegation +queries, smtpd_proxy_filter and Milters all behave as if Postfix received (MAIL ++ RCPT + DATA + end-of-data). However, Postfix will log BDAT-related failures +as "xxx after BDAT" to avoid complicating troubleshooting (xxx = 'lost +connection' or 'timeout'), and will log a warning when a client sends a +malformed BDAT command. + +BBeenneeffiittss ooff CCHHUUNNKKIINNGG ((BBDDAATT)) ssuuppppoorrtt wwiitthhoouutt BBIINNAARRYYMMIIMMEE + +Support for CHUNKING (BDAT) was added to improve interoperability with some +clients, a benefit that would reportedly exist even without Postfix support for +BINARYMIME. Since June 2018, Wietse's mail server has received BDAT commands +from a variety of systems. + +Postfix does not support BINARYMIME at this time because: + + * BINARYMIME support would require moderately invasive changes to Postfix, to + support email content that is not line-oriented. With BINARYMIME, the + Content-Length: message header specifies the length of content that may or + may not have line boundaries. Without BINARYMIME support, email RFCs + require that binary content is base64-encoded, and formatted as lines of + text. + + * For delivery to non-BINARYMIME systems including UNIX mbox, the available + options are to convert binary content into 8bit text, one of the 7bit forms + (base64 or quoted-printable), or to return email as undeliverable. Any + conversion would obviously break digital signatures, so conversion would + have to happen before signing. + +DDoowwnnssiiddeess ooff CCHHUUNNKKIINNGG ((BBDDAATT)) ssuuppppoorrtt + +The RFC 3030 authors did not specify any limitations on how clients may +pipeline commands (i.e. send commands without waiting for a server response). +If a server announces PIPELINING support, like Postfix does, then a remote SMTP +client can pipeline all commands following EHLO, for example, MAIL/RCPT/BDAT/ +BDAT/MAIL/RCPT/BDAT, without ever having to wait for a server response. This +means that with BDAT, the Postfix SMTP server cannot distinguish between a +well-behaved client and a spambot, based on their command pipelining behavior. +If you require "reject_unauth_pipelining" to block spambots, then turn off +Postfix's CHUNKING announcement as described above. + +In RFC 4468, the authors write that a client may pipeline commands, and that +after sending BURL LAST or BDAT LAST, a client must wait for the server's +response. But as this text does not appear in RFC 3030 which defines BDAT, it +is a useless restriction that Postfix will not enforce. + diff --git a/README_FILES/BUILTIN_FILTER_README b/README_FILES/BUILTIN_FILTER_README new file mode 100644 index 0000000..2ce639d --- /dev/null +++ b/README_FILES/BUILTIN_FILTER_README @@ -0,0 +1,321 @@ + PPoossttffiixx BBuuiilltt--iinn CCoonntteenntt IInnssppeeccttiioonn + +------------------------------------------------------------------------------- + +BBuuiilltt--iinn ccoonntteenntt iinnssppeeccttiioonn iinnttrroodduuccttiioonn + +Postfix supports a built-in filter mechanism that examines message header and +message body content, one line at a time, before it is stored in the Postfix +queue. The filter is usually implemented with POSIX or PCRE regular +expressions, as described in the header_checks(5) manual page. + +The original purpose of the built-in filter is to stop an outbreak of specific +email worms or viruses, and it does this job well. The filter has also helped +to block bounced junk email, bounced email from worms or viruses, and +notifications from virus detection systems. Information about this secondary +application is given in the BACKSCATTER_README document. + +Because the built-in filter is optimized for stopping specific worms and virus +outbreaks, it has limitations that make it NOT suitable for general junk email +and virus detection. For that, you should use one of the external content +inspection methods that are described in the FILTER_README, SMTPD_PROXY_README +and MILTER_README documents. + +The following diagram gives an over-all picture of how Postfix built-in content +inspection works: + + Postmaster + notifications + + | + v + + Network or -> BBuuiilltt--iinn -> Postfix -> Delivery -> Network or + local users ffiilltteerr queue agents local mailbox + + ^ | + | v + + Undeliverable mail + Forwarded mail + +The picture makes clear that the filter works while Postfix is receiving new +mail. This means that Postfix can reject mail from the network without having +to return undeliverable mail to the originator address (which is often spoofed +anyway). However, this ability comes at a price: if mail inspection takes too +much time, then the remote client will time out, and the client may send the +same message repeatedly. + +Topics covered by this document: + + * What mail is subjected to header/body checks + * Limitations of Postfix header/body checks + * Preventing daily mail status reports from being blocked + * Configuring header/body checks for mail from outside users only + * Configuring different header/body checks for MX service and submission + service + * Configuring header/body checks for mail to some domains only + +WWhhaatt mmaaiill iiss ssuubbjjeecctteedd ttoo hheeaaddeerr//bbooddyy cchheecckkss + +Postfix header/body checks are implemented by the cleanup(8) server before it +injects mail into the incoming queue. The diagram below zooms in on the cleanup +(8) server, and shows that this server handles mail from many different +sources. In order to keep the diagram readable, the sources of postmaster +notifications are not shown, because they can be produced by many Postfix +daemon processes. + + bounce(8) + (undeliverable) + + ssmmttppdd((88)) | + ((nneettwwoorrkk)) \ v + + qqmmqqppdd((88)) -\ cleanup(8) -> incoming + ((nneettwwoorrkk)) -/ queue + + ppiicckkuupp((88)) / ^ + ((llooccaall)) | + + local(8) + (forwarded) + +For efficiency reasons, only mail that enters from outside of Postfix is +inspected with header/body checks. It would be inefficient to filter already +filtered mail again, and it would be undesirable to block postmaster +notifications. The table below summarizes what mail is and is not subject to +header/body checks. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |MMeessssaaggee ttyyppee |SSoouurrccee |HHeeaaddeerr//bbooddyy cchheecckkss??| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Undeliverable mail|bounce(8)|No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Network mail |smtpd(8) |Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Network mail |qmqpd(8) |Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local submission |pickup(8)|Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local forwarding |local(8) |No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Postmaster notice |many |No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +How does Postfix decide what mail needs to be filtered? It would be clumsy to +make the decision in the cleanup(8) server, as this program receives mail from +so many different sources. Instead, header/body checks are requested by the +source. Examples of how to turn off header/body checks for mail received with +smtpd(8), qmqpd(8) or pickup(8) are given below under "Configuring header/body +checks for mail from outside users only", "Configuring different header/body +checks for MX service and submission service", and "Configuring header/body +checks for mail to some domains only". + +LLiimmiittaattiioonnss ooff PPoossttffiixx hheeaaddeerr//bbooddyy cchheecckkss + + * Header/body checks do not decode message headers or message body content. + For example, if text in the message body is BASE64 encoded (RFC 2045) then + your regular expressions will have to match the BASE64 encoded form. + Likewise, message headers with encoded non-ASCII characters (RFC 2047) need + to be matched in their encoded form. + + * Header/body checks cannot filter on a combination of message headers or + body lines. Header/body checks examine content one message header at a + time, or one message body line at a time, and cannot carry a decision over + to the next message header or body line. + + * Header/body checks cannot depend on the recipient of a message. + + o One message can have multiple recipients, and all recipients of a + message receive the same treatment. Workarounds have been proposed that + involve selectively deferring some recipients of multi-recipient mail, + but that results in poor SMTP performance and does not work for non- + SMTP mail. + + o Some sources of mail send the headers and content ahead of the + recipient information. It would be inefficient to buffer up an entire + message before deciding if it needs to be filtered, and it would be + clumsy to filter mail and to buffer up all the actions until it is + known whether those actions need to be executed. + + * Despite warnings, some people try to use the built-in filter feature for + general junk email and/or virus blocking, using hundreds or even thousands + of regular expressions. This can result in catastrophic performance + failure. The symptoms are as follows: + + o The cleanup(8) processes use up all available CPU time in order to + process the regular expressions, and/or they use up all available + memory so that the system begins to swap. This slows down all incoming + mail deliveries. + + o As Postfix needs more and more time to receive an email message, the + number of simultaneous SMTP sessions increases to the point that the + SMTP server process limit is reached. + + o While all SMTP server processes are waiting for the cleanup(8) servers + to finish, new SMTP clients have to wait until an SMTP server process + becomes available. This causes mail deliveries to time out before they + have even begun. + + The remedy for this type of performance problem is simple: don't use + header/body checks for general junk email and/or virus blocking, and don't + filter mail before it is queued. When performance is a concern, use an + external content filter that runs after mail is queued, as described in the + FILTER_README document. + +PPrreevveennttiinngg ddaaiillyy mmaaiill ssttaattuuss rreeppoorrttss ffrroomm bbeeiinngg bblloocckkeedd + +The following is quoted from Jim Seymour's Pflogsumm FAQ at http:// +jimsun.linxnet.com/downloads/pflogsumm-faq.txt. Pflogsumm is a program that +analyzes Postfix logs, including the logging from rejected mail. If these logs +contain text that was rejected by Postfix body_checks patterns, then the +logging is also likely to be rejected by those same body_checks patterns. This +problem does not exist with header_checks patterns, because those are not +applied to the text that is part of the mail status report. + + You configure Postfix to do body checks, Postfix does its thing, Pflogsumm + reports it and Postfix catches the same string in the Pflogsumm report. + There are several solutions to this. + + Wolfgang Zeikat contributed this: + + #!/usr/bin/perl + use MIME::Lite; + + ### Create a new message: + $msg = MIME::Lite->new( + From => 'your@send.er', + To => 'your@recipie.nt', + # Cc => 'some@other.com, some@more.com', + Subject => 'pflogsumm', + Date => `date`, + Type => 'text/plain', + Encoding => 'base64', + Path => '/tmp/pflogg', + ); + + $msg->send; + + Where "/tmp/pflogg" is the output of Pflogsumm. This puts Pflogsumm's + output in a base64 MIME attachment. + +Note by Wietse: if you run this on a machine that is accessible by untrusted +users, it is safer to store the Pflogsumm report in a directory that is not +world writable. + + In a follow-up to a thread in the postfix-users mailing list, Ralf + Hildebrandt noted: + + "mpack does the same thing." + +And it does. Which tool one should use is a matter of preference. + +Other solutions involve additional body_checks rules that make exceptions for +daily mail status reports, but this is not recommended. Such rules slow down +all mail and complicate Postfix maintenance. + +CCoonnffiigguurriinngg hheeaaddeerr//bbooddyy cchheecckkss ffoorr mmaaiill ffrroomm oouuttssiiddee uusseerrss oonnllyy + +The following information applies to Postfix 2.1 and later. Earlier Postfix +versions do not support the receive_override_options feature. + +The easiest approach is to configure ONE Postfix instance with multiple SMTP +server IP addresses in master.cf: + + * Two SMTP server IP addresses for mail from inside users only, with header/ + body filtering turned off, and a local mail pickup service with header/body + filtering turned off. + + /etc/postfix.master.cf: + # ================================================================== + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================== + 1.2.3.4:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + 127.0.0.1:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + pickup fifo n - n 60 1 pickup + -o receive_override_options=no_header_body_checks + + * Add some firewall rule to prevent access to 1.2.3.4:smtp from the outside + world. + + * One SMTP server address for mail from outside users with header/body + filtering turned on via main.cf. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + 1.2.3.5:smtp inet n - n - - smtpd + +CCoonnffiigguurriinngg ddiiffffeerreenntt hheeaaddeerr//bbooddyy cchheecckkss ffoorr MMXX sseerrvviiccee aanndd ssuubbmmiissssiioonn sseerrvviiccee + +If authorized user submissions require different header/body checks than mail +from remote MTAs, then this is possible as long as you have separate mail +streams for authorized users and for MX service. + +The example below assumes that authorized users connect to TCP port 587 +(submission) or 465 (smtps), and that remote MTAs connect to TCP port 25 +(smtp). + +First, we define a few "user-defined" parameters that will override settings +for the submission and smtps services. + + /etc/postfix/main.cf: + msa_cleanup_service_name = msa_cleanup + msa_header_checks = pcre:/etc/postfix/msa_header_checks + msa_body_checks = pcre:/etc/postfix/msa_body_checks + +Next, we define msa_cleanup as a dedicated cleanup service that will be used +only by the submission and smtps services. This service uses the header_checks +and body_checks overrides that were defined above. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + smtp inet n - n - - smtpd + msa_cleanup unix n - n - 0 cleanup + -o header_checks=$msa_header_checks + -o body_checks=$msa_body_checks + submission inet n - n - - smtpd + -o cleanup_service_name=$msa_cleanup_service_name + -o syslog_name=postfix/submission + ...[see sample master.cf file for more]... + smtps inet n - n - - smtpd + -o cleanup_service_name=$msa_cleanup_service_name + -o syslog_name=postfix/smtps + -o smtpd_tls_wrappermode=yes + ...[see sample master.cf file for more]... + +By keeping the "msa_xxx" parameter settings in main.cf, you keep your master.cf +file simple, and you minimize the amount of duplication. + +CCoonnffiigguurriinngg hheeaaddeerr//bbooddyy cchheecckkss ffoorr mmaaiill ttoo ssoommee ddoommaaiinnss oonnllyy + +The following information applies to Postfix 2.1. Earlier Postfix versions do +not support the receive_override_options feature. + +If you are an MX service provider and want to enable header/body checks only +for some domains, you can configure ONE Postfix instance with multiple SMTP +server IP addresses in master.cf. Each address provides a different service. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + # SMTP service for domains with header/body checks turned on. + 1.2.3.4:smtp inet n - n - - smtpd + + # SMTP service for domains with header/body checks turned off. + 1.2.3.5:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + +Once this is set up you can configure MX records in the DNS that route each +domain to the proper SMTP server instance. + diff --git a/README_FILES/CDB_README b/README_FILES/CDB_README new file mode 100644 index 0000000..0f06d47 --- /dev/null +++ b/README_FILES/CDB_README @@ -0,0 +1,74 @@ +PPoossttffiixx CCDDBB HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +CDB (Constant DataBase) is an indexed file format designed by Daniel Bernstein. +CDB is optimized exclusively for read access and guarantees that each record +will be read in at most two disk accesses. This is achieved by forgoing support +for incremental updates: no single-record inserts or deletes are supported. CDB +databases can be modified only by rebuilding them completely from scratch, +hence the "constant" qualifier in the name. + +Postfix CDB databases are specified as "cdb:name", where name specifies the CDB +file name without the ".cdb" suffix (another suffix, ".tmp", is used +temporarily while a CDB file is under construction). CDB databases are +maintained with the postmap(1) or postalias(1) command. The DATABASE_README +document has general information about Postfix databases. + +CDB support is available with Postfix 2.2 and later releases. This document +describes how to build Postfix with CDB support. + +BBuuiillddiinngg PPoossttffiixx wwiitthh CCDDBB ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +Postfix is compatible with two CDB implementations: + + * The original cdb library from Daniel Bernstein, available from http:// + cr.yp.to/cdb.html, and + + * tinycdb (version 0.5 and later) from Michael Tokarev, available from http:/ + /www.corpit.ru/mjt/tinycdb.html. + +Tinycdb is preferred, since it is a bit faster, has additional useful +functionality and is much simpler to use. + +To build Postfix after you have installed tinycdb, use something like: + + % make tidy + % CDB=../../../tinycdb-0.5 + % make -f Makefile.init makefiles "CCARGS=-DHAS_CDB -I$CDB" \ + "AUXLIBS_CDB=$CDB/libcdb.a" + % make + +Alternatively, for the D.J.B. version of CDB: + + % make tidy + % CDB=../../../cdb-0.75 + % make -f Makefile.init makefiles "CCARGS=-DHAS_CDB -I$CDB" \ + "AUXLIBS_CDB=$CDB/cdb.a $CDB/alloc.a $CDB/buffer.a $CDB/unix.a $CDB/ + byte.a" + % make + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_CDB. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded CDB database client, but only the new AUXLIBS_CDB variable supports +building a dynamically-loaded or statically-loaded CDB database client. + + Failure to use the AUXLIBS_CDB variable will defeat the purpose of dynamic + database client loading. Every Postfix executable file will have CDB + database library dependencies. And that was exactly what dynamic database + client loading was meant to avoid. + +After Postfix has been built with cdb support, you can use "cdb" tables +wherever you can use read-only "hash", "btree" or "dbm" tables. However, the +"ppoossttmmaapp --ii" (incremental record insertion) and "ppoossttmmaapp --dd" (incremental +record deletion) command-line options are not available. For the same reason +the "cdb" map type cannot be used to store the persistent address verification +cache for the verify(8) service, or to store TLS session information for the +tlsmgr(8) service. + diff --git a/README_FILES/COMPATIBILITY_README b/README_FILES/COMPATIBILITY_README new file mode 100644 index 0000000..55182b7 --- /dev/null +++ b/README_FILES/COMPATIBILITY_README @@ -0,0 +1,399 @@ +PPoossttffiixx BBaacckkwwaarrddss--CCoommppaattiibbiilliittyy SSaaffeettyy NNeett + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +Postfix 3.0 introduces a safety net that runs Postfix programs with backwards- +compatible default settings after an upgrade. The safety net will log a warning +whenever a "new" default setting could have an negative effect on your mail +flow. + +This document provides information on the following topics: + + * Detailed descriptions of Postfix backwards-compatibility warnings. + + * What backwards-compatible settings you may have to make permanent in + main.cf or master.cf. + + * How to turn off Postfix backwards-compatibility warnings. + +OOvveerrvviieeww + +With backwards compatibility turned on, Postfix logs a message whenever a +backwards-compatible default setting may be required for continuity of service. +Based on this logging the system administrator can decide if any backwards- +compatible settings need to be made permanent in main.cf or master.cf, before +turning off the backwards-compatibility safety net as described at the end of +this document. + +Logged with compatibility_level < 1: + + * Using backwards-compatible default setting append_dot_mydomain=yes + + * Using backwards-compatible default setting chroot=y + +Logged with compatibility_level < 2: + + * Using backwards-compatible default setting "smtpd_relay_restrictions = + (empty)" + + * Using backwards-compatible default setting mynetworks_style=subnet + + * Using backwards-compatible default setting relay_domains=$mydestination + + * Using backwards-compatible default setting smtputf8_enable=no + +Logged with compatibility_level < 3.6: + + * Using backwards-compatible default setting smtpd_tls_fingerprint_digest=md5 + + * Using backwards-compatible default setting smtp_tls_fingerprint_digest=md5 + + * Using backwards-compatible default setting lmtp_tls_fingerprint_digest=md5 + + * Using backwards-compatible default setting + smtpd_relay_before_recipient_restrictions=no + + * Using backwards-compatible default setting respectful_logging=no + +If such a message is logged in the context of a legitimate request, the system +administrator should make the backwards-compatible setting permanent in main.cf +or master.cf, as detailed in the sections that follow. + +When no more backwards-compatible settings need to be made permanent, the +system administrator should turn off the backwards-compatibility safety net as +described at the end of this document. + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg aappppeenndd__ddoott__mmyyddoommaaiinn==yyeess + +The append_dot_mydomain default value has changed from "yes" to "no". This +could result in unexpected non-delivery of email after Postfix is updated from +an older version. The backwards-compatibility safety net is designed to prevent +such surprises. + +As long as the append_dot_mydomain parameter is left at its implicit default +value, and the compatibility_level setting is less than 1, Postfix may log one +of the following messages: + + * Messages about missing "localhost" in mydestination or other address class: + + postfix/trivial-rewrite[14777]: using backwards-compatible + default setting append_dot_mydomain=yes to rewrite + "localhost" to "localhost.example.com"; please add + "localhost" to mydestination or other address class + + If Postfix logs the above message, add "localhost" to mydestination (or + virtual_alias_domains, virtual_mailbox_domains, or relay_domains) and + execute the command "ppoossttffiixx rreellooaadd". + + * Messages about incomplete domains in email addresses: + + postfix/trivial-rewrite[25835]: using backwards-compatible + default setting append_dot_mydomain=yes to rewrite "foo" to + "foo.example.com" + + If Postfix logs the above message for domains different from "localhost", + and the sender cannot be changed to use complete domain names in email + addresses, then the system administrator should make the backwards- + compatible setting "append_dot_mydomain = yes" permanent in main.cf: + + # ppoossttccoonnff aappppeenndd__ddoott__mmyyddoommaaiinn==yyeess + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg cchhrroooott==yy + +The master.cf chroot default value has changed from "y" (yes) to "n" (no). The +new default avoids the need for copies of system files under the Postfix queue +directory. However, sites with strict security requirements may want to keep +the chroot feature enabled after updating Postfix from an older version. The +backwards-compatibility safety net is designed allow the administrator to +choose if they want to keep the old behavior. + +As long as a master.cf chroot field is left at its implicit default value, and +the compatibility_level setting is less than 1, Postfix may log the following +message while it reads the master.cf file: + + postfix/master[27664]: /etc/postfix/master.cf: line 72: using + backwards-compatible default setting chroot=y + +If this service should remain chrooted, then the system administrator should +make the backwards-compatible setting "chroot = y" permanent in master.cf. For +example, to update the chroot setting for the "smtp inet" service: + + # ppoossttccoonnff --FF ssmmttpp//iinneett//cchhrroooott==yy + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg ssmmttppdd__rreellaayy__rreessttrriiccttiioonnss == ((eemmppttyy)) + +The smtpd_relay_restrictions feature was introduced with Postfix version 2.10, +as a safety mechanism for configuration errors in smtpd_recipient_restrictions +that could make Postfix an open relay. + +The smtpd_relay_restrictions implicit default setting forbids mail to remote +destinations from clients that don't match permit_mynetworks or +permit_sasl_authenticated. This could result in unexpected 'Relay access +denied' errors after Postfix is updated from an older Postfix version. The +backwards-compatibility safety net is designed to prevent such surprises. + +When the compatibility_level less than 1, and the smtpd_relay_restrictions +parameter is left at its implicit default setting, Postfix may log the +following message: + + postfix/smtpd[38463]: using backwards-compatible default setting + "smtpd_relay_restrictions = (empty)" to avoid "Relay access + denied" error for recipient "user@example.com" from client + "host.example.net[10.0.0.2]" + +If this request should not be blocked, then the system administrator should +make the backwards-compatible setting "smtpd_relay_restrictions=" (i.e. empty) +permanent in main.cf: + + # ppoossttccoonnff ssmmttppdd__rreellaayy__rreessttrriiccttiioonnss== + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg mmyynneettwwoorrkkss__ssttyyllee==ssuubbnneett + +The mynetworks_style default value has changed from "subnet" to "host". This +parameter is used to implement the "permit_mynetworks" feature. The change +could cause unexpected 'access denied' errors after Postfix is updated from an +older version. The backwards-compatibility safety net is designed to prevent +such surprises. + +As long as the mynetworks and mynetworks_style parameters are left at their +implicit default values, and the compatibility_level setting is less than 2, +the Postfix SMTP server may log one of the following messages: + + postfix/smtpd[17375]: using backwards-compatible default setting + mynetworks_style=subnet to permit request from client + "foo.example.com[10.1.1.1]" + + postfix/postscreen[24982]: using backwards-compatible default + setting mynetworks_style=subnet to permit request from client + "10.1.1.1" + +If the client request should not be rejected, then the system administrator +should make the backwards-compatible setting "mynetworks_style = subnet" +permanent in main.cf: + + # ppoossttccoonnff mmyynneettwwoorrkkss__ssttyyllee==ssuubbnneett + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg rreellaayy__ddoommaaiinnss==$$mmyyddeessttiinnaattiioonn + +The relay_domains default value has changed from "$mydestination" to the empty +value. This could result in unexpected 'Relay access denied' errors or ETRN +errors after Postfix is updated from an older version. The backwards- +compatibility safety net is designed to prevent such surprises. + +As long as the relay_domains parameter is left at its implicit default value, +and the compatibility_level setting is less than 2, Postfix may log one of the +following messages. + + * Messages about accepting mail for a remote domain: + + postfix/smtpd[19052]: using backwards-compatible default setting + relay_domains=$mydestination to accept mail for domain + "foo.example.com" + + postfix/smtpd[19052]: using backwards-compatible default setting + relay_domains=$mydestination to accept mail for address + "user@foo.example.com" + + * Messages about providing ETRN service for a remote domain: + + postfix/smtpd[19138]: using backwards-compatible default setting + relay_domains=$mydestination to flush mail for domain + "bar.example.com" + + postfix/smtp[13945]: using backwards-compatible default setting + relay_domains=$mydestination to update fast-flush logfile for + domain "bar.example.com" + +If Postfix should continue to accept mail for that domain or continue to +provide ETRN service for that domain, then the system administrator should make +the backwards-compatible setting "relay_domains = $mydestination" permanent in +main.cf: + + # ppoossttccoonnff ''rreellaayy__ddoommaaiinnss==$$mmyyddeessttiinnaattiioonn'' + # ppoossttffiixx rreellooaadd + +Note: quotes are required as indicated above. + +Instead of $mydestination, it may be better to specify an explicit list of +domain names. + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg ssmmttppuuttff88__eennaabbllee==nnoo + +The smtputf8_enable default value has changed from "no" to "yes". With the new +"yes" setting, the Postfix SMTP server rejects non-ASCII addresses from clients +that don't request SMTPUTF8 support, after Postfix is updated from an older +version. The backwards-compatibility safety net is designed to prevent such +surprises. + +As long as the smtputf8_enable parameter is left at its implicit default value, +and the compatibility_level setting is less than 1, Postfix logs a warning each +time an SMTP command uses a non-ASCII address localpart without requesting +SMTPUTF8 support: + + postfix/smtpd[27560]: using backwards-compatible default setting + smtputf8_enable=no to accept non-ASCII sender address + "??@example.org" from localhost[127.0.0.1] + + postfix/smtpd[27560]: using backwards-compatible default setting + smtputf8_enable=no to accept non-ASCII recipient address + "??@example.com" from localhost[127.0.0.1] + +If the address should not be rejected, and the client cannot be updated to use +SMTPUTF8, then the system administrator should make the backwards-compatible +setting "smtputf8_enable = no" permanent in main.cf: + + # ppoossttccoonnff ssmmttppuuttff88__eennaabbllee==nnoo + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg ssmmttppdd__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt==mmdd55 + +The smtpd_tls_fingerprint_digest default value has changed from "md5" to +"sha256". With the new "sha256" setting, the Postfix SMTP server avoids using +the deprecated "md5" algorithm and computes a more secure digest of the client +certificate. + +If you're using the default "md5" setting, or even an explicit "sha1" (also +deprecated) setting, you should consider switching to "sha256". This will +require updating any associated lookup table keys with the "sha256" digests of +the expected client certificate or public key. + +As long as the smtpd_tls_fingerprint_digest parameter is left at its implicit +default value, and the compatibility_level setting is less than 3.6, Postfix +logs a warning each time a client certificate or public key fingerprint is +(potentially) used for access control: + + postfix/smtpd[27560]: using backwards-compatible default setting + smtpd_tls_fingerprint_digest=md5 to compute certificate fingerprints + +Since any client certificate fingerprints are passed in policy service lookups, +and Postfix doesn't know whether the fingerprint will be used, the warning may +also be logged when policy lookups are performed for connections that used a +client certificate, even if the policy service does not in fact examine the +client certificate. To reduce the noise somewhat, such warnings are issued at +most once per smtpd(8) process instance. + +If you prefer to stick with "md5", you can suppress the warnings by making that +setting explicit. After addressing any other compatibility warnings, you can +update your compatibility level. + + # ppoossttccoonnff ssmmttppdd__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt==mmdd55 + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg ssmmttpp__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt==mmdd55 + +The smtp_tls_fingerprint_digest and lmtp_tls_fingerprint_digest default values +have changed from "md5" to "sha256". With the new "sha256" setting, the Postfix +SMTP and LMTP client avoids using the deprecated "md5" algorithm and computes a +more secure digest of the server certificate. + +If you're using the default "md5" setting, or even an explicit "sha1" (also +deprecated) setting, you should consider switching to "sha256". This will +require updating any "fingerprint" security level policies in the TLS policy +table to specify matching "sha256" digests of the expected server certificates +or public keys. + +As long as the smtp_tls_fingerprint_digest (or LMTP equivalent) parameter is +left at its implicit default value, and the compatibility_level setting is less +than 3.6, Postfix logs a warning each time the "fingerprint" security level is +used to specify matching "md5" digests of trusted server certificates or public +keys: + + postfix/smtp[27560]: using backwards-compatible default setting + smtp_tls_fingerprint_digest=md5 to compute certificate fingerprints + +If you prefer to stick with "md5", you can suppress the warnings by making that +setting explicit. After addressing any other compatibility warnings, you can +update your compatibility level. + + # ppoossttccoonnff ''ssmmttpp__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt == mmdd55'' \\ + ''llmmttpp__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt == mmdd55'' + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg +ssmmttppdd__rreellaayy__bbeeffoorree__rreecciippiieenntt__rreessttrriiccttiioonnss==nnoo + +The smtpd_relay_before_recipient_restrictions feature was introduced in Postfix +version 3.6, to evaluate smtpd_relay_restrictions before +smtpd_recipient_restrictions. Historically, smtpd_relay_restrictions was +evaluated after smtpd_recipient_restrictions, contradicting documented +behavior. + + Background: smtpd_relay_restrictions is primarily designed to enforce a + mail relaying policy, while smtpd_recipient_restrictions is primarily + designed to enforce spam blocking policy. Both are evaluated while replying + to the RCPT TO command, and both support the same features. + +To maintain compatibility with earlier versions, Postfix will keep evaluating +smtpd_recipient_restrictions before smtpd_relay_restrictions, as long as the +compatibility_level is less than 3.6, and the +smtpd_relay_before_recipient_restrictions parameter is left at its implicit +default setting. As a reminder, Postfix may log the following message: + + postfix/smtpd[54696]: using backwards-compatible default setting + smtpd_relay_before_recipient_restrictions=no to reject recipient + "user@example.com" from client "host.example.net[10.0.0.2]" + +If Postfix should keep evaluating smtpd_recipient_restrictions before +smtpd_relay_restrictions, then the system administrator should make the +backwards-compatible setting "smtpd_relay_before_recipient_restrictions=no" +permanent in main.cf: + + # ppoossttccoonnff ssmmttppdd__rreellaayy__bbeeffoorree__rreecciippiieenntt__rreessttrriiccttiioonnss==nnoo + # ppoossttffiixx rreellooaadd + +UUssiinngg bbaacckkwwaarrddss--ccoommppaattiibbllee ddeeffaauulltt sseettttiinngg rreessppeeccttffuull__llooggggiinngg==nnoo + +Postfix version 3.6 deprecates configuration parameter names and logging that +suggest white is better than black. Instead it prefers 'allowlist, 'denylist', +and variations of those words. While the renamed configuration parameters have +backwards-compatible default values, the changes in logging could affect +logfile analysis tools. + +To avoid breaking existing logfile analysis tools, Postfix will keep logging +the deprecated form, as long as the respectful_logging parameter is left at its +implicit default value, and the compatibility_level setting is less than 3.6. +As a reminder, Postfix may log the following when a remote SMTP client is +allowlisted or denylisted: + + postfix/postscreen[22642]: Using backwards-compatible default setting + respectful_logging=no for client [address]:port + +If Postfix should keep logging the deprecated form, then the system +administrator should make the backwards-compatible setting "respectful_logging += no" permanent in main.cf. + + # ppoossttccoonnff ""rreessppeeccttffuull__llooggggiinngg == nnoo"" + # ppoossttffiixx rreellooaadd + +TTuurrnniinngg ooffff tthhee bbaacckkwwaarrddss--ccoommppaattiibbiilliittyy ssaaffeettyy nneett + +Backwards compatibility is turned off by updating the compatibility_level +setting in main.cf. + + # ppoossttccoonnff ccoommppaattiibbiilliittyy__lleevveell==NN + # ppoossttffiixx rreellooaadd + +For N specify the number that is logged in your postfix(1) warning message: + + warning: To disable backwards compatibility use "postconf + compatibility_level=N" and "postfix reload" + +Sites that don't care about backwards compatibility may set +"compatibility_level = 9999" at their own risk. + +Starting with Postfix version 3.6, the compatibility level in the above warning +message is the Postfix version that introduced the last incompatible change. +The level is formatted as major.minor.patch, where patch is usually omitted and +defaults to zero. Earlier compatibility levels are 0, 1 and 2. + +NOTE: Postfix 3.6 also introduces support for the "<level", "<=level", and +other operators to compare compatibility levels. With the standard operators +"<", "<=", etc., compatibility level "3.10" would be smaller than "3.9" which +is undesirable. + diff --git a/README_FILES/CONNECTION_CACHE_README b/README_FILES/CONNECTION_CACHE_README new file mode 100644 index 0000000..f578fb7 --- /dev/null +++ b/README_FILES/CONNECTION_CACHE_README @@ -0,0 +1,234 @@ +PPoossttffiixx CCoonnnneeccttiioonn CCaacchhee + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document describes the Postfix connection cache implementation, which is +available with Postfix version 2.2 and later. + +Topics covered in this document: + + * What SMTP connection caching can do for you + * Connection cache implementation + * Connection cache configuration + * Connection cache safety mechanisms + * Connection cache limitations + * Connection cache statistics + +WWhhaatt SSMMTTPP ccoonnnneeccttiioonn ccaacchhiinngg ccaann ddoo ffoorr yyoouu + +With SMTP connection caching, Postfix can deliver multiple messages over the +same SMTP connection. By default, Postfix 2.2 reuses a plaintext SMTP +connection automatically when a destination has high volume of mail in the +active queue. + +SMTP Connection caching is a performance feature. Whether or not it actually +improves performance depends on the conditions: + + * SMTP Connection caching can greatly improve performance when delivering + mail to a destination with multiple mail servers, because it can help + Postfix to skip over a non-responding server. + + * SMTP Connection caching can also help with receivers that impose rate + limits on new connections. + + * Otherwise, the benefits of SMTP connection caching are minor: it eliminates + the latency of the TCP handshake (SYN, SYN+ACK, ACK), plus the latency of + the SMTP initial handshake (220 greeting, EHLO command, EHLO response). + With TLS-encrypted connections, this can save an additional two roundtrips + that would otherwise be needed to send STARTTLS and to resume a TLS + session. + + * SMTP Connection caching gives no gains with respect to SMTP session tear- + down. The Postfix smtp(8) client normally does not wait for the server's + reply to the QUIT command, and it never waits for the TCP final handshake + to complete. + + * SMTP Connection caching introduces some overhead: the client needs to send + an RSET command to find out if a connection is still usable, before it can + send the next MAIL FROM command. This introduces one additional round-trip + delay. + +For other potential issues with SMTP connection caching, see the discussion of +limitations at the end of this document. + +CCoonnnneeccttiioonn ccaacchhee iimmpplleemmeennttaattiioonn + +For an overview of how Postfix delivers mail, see the Postfix architecture +OVERVIEW document. + +The Postfix connection cache is shared among Postfix mail delivering processes. +This maximizes the opportunity to reuse an open connection. Some MTAs such as +Sendmail have a non-shared connection cache. Here, a connection can be reused +only by the mail delivering process that creates the connection. To get the +same performance improvement as with a shared connection cache, non-shared +connections need to be kept open for a longer time. + +The scache(8) server, introduced with Postfix version 2.2, maintains the shared +connection cache. With Postfix version 2.2, only the smtp(8) client has support +to access this cache. + +When SMTP connection caching is enabled (see next section), the smtp(8) client +does not disconnect after a mail transaction, but gives the connection to the +scache(8) server which keeps the connection open for a limited amount of time. + +After handing over the open connection to the scache(8) server, the smtp(8) +client continues with some other mail delivery request. Meanwhile, any smtp(8) +client process can ask the scache(8) server for that cached connection and +reuse it for mail delivery. + + /-- smtp(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + +With TLS connection reuse (Postfix 3.4 and later), the Postfix smtp(8) client +connects to a remote SMTP server and sends plaintext EHLO and STARTTLS +commands, then inserts a tlsproxy(8) process into the connection as shown +below. + +After delivering mail, the smtp(8) client hands over the open smtp(8)-to- +tlsproxy(8) connection to the scache(8) server, and continues with some other +mail delivery request. Meanwhile, any smtp(8) client process can ask the scache +(8) server for that cached connection and reuse it for mail delivery. + + /-- smtp(8) --> tlsproxy(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + +The connection cache can be searched by destination domain name (the right-hand +side of the recipient address) and by the IP address of the host at the other +end of the connection. This allows Postfix to reuse a connection even when the +remote host is a mail server for domains with different names. + +CCoonnnneeccttiioonn ccaacchhee ccoonnffiigguurraattiioonn + +The Postfix smtp(8) client supports two connection caching strategies: + + * On-demand connection caching. This is enabled by default, and is controlled + with the smtp_connection_cache_on_demand configuration parameter. When this + feature is enabled, the Postfix smtp(8) client automatically saves a + connection to the connection cache when a destination has a high volume of + mail in the active queue. + + Example: + + /etc/postfix/main.cf: + smtp_connection_cache_on_demand = yes + + * Per-destination connection caching. This is enabled by explicitly listing + specific destinations with the smtp_connection_cache_destinations + configuration parameter. After completing delivery to a selected + destination, the Postfix smtp(8) client always saves the connection to the + connection cache. + + Specify a comma or white space separated list of destinations or pseudo- + destinations: + + o if mail is sent without a relay host: a domain name (the right-hand + side of an email address, without the [] around a numeric IP address), + + o if mail is sent via a relay host: a relay host name (without the [] or + non-default TCP port), as specified in main.cf or in the transport map, + + o a /file/name with domain names and/or relay host names as defined + above, + + o a "type:table" with domain names and/or relay host names on the left- + hand side. The right-hand side result from "type:table" lookups is + ignored. + + Examples: + + /etc/postfix/main.cf: + smtp_connection_cache_destinations = $relayhost + smtp_connection_cache_destinations = hotmail.com, ... + smtp_connection_cache_destinations = static:all (not recommended) + + See Client-side TLS connection reuse to enable multiple deliveries over a + TLS-encrypted connection (Postfix version 3.4 and later). + +CCoonnnneeccttiioonn ccaacchhee ssaaffeettyy mmeecchhaanniissmmss + +Connection caching must be used wisely. It is anti-social to keep an unused +SMTP connection open for a significant amount of time, and it is unwise to send +huge numbers of messages through the same connection. In order to avoid +problems with SMTP connection caching, Postfix implements the following safety +mechanisms: + + * The Postfix scache(8) server keeps a connection open for only a limited + time. The time limit is specified with the smtp_connection_cache_time_limit + and with the connection_cache_ttl_limit configuration parameters. This + prevents anti-social behavior. + + * The Postfix smtp(8) client reuses a session for only a limited number of + times. This avoids triggering bugs in implementations that do not correctly + handle multiple deliveries per session. + + As of Postfix 2.3 connection reuse is preferably limited with the + smtp_connection_reuse_time_limit parameter. In addition, Postfix 2.11 + provides smtp_connection_reuse_count_limit to limit how many times a + connection may be reused, but this feature is unsafe as it introduces a + "fatal attractor" failure mode (when a destination has multiple inbound + MTAs, the slowest inbound MTA will attract most connections from Postfix to + that destination). + + Postfix 2.3 logs the use count of multiply-used connections, as shown in + the following example: + + Nov 3 16:04:31 myname postfix/smtp[30840]: 19B6B2900FE: + to=<wietse@test.example.com>, orig_to=<wietse@test>, + relay=mail.example.com[1.2.3.4], ccoonnnn__uussee==22, delay=0.22, + delays=0.04/0.01/0.05/0.1, dsn=2.0.0, status=sent (250 2.0.0 Ok) + + * The connection cache explicitly labels each cached connection with + destination domain and IP address information. A connection cache lookup + succeeds only when the correct information is specified. This prevents mis- + delivery of mail. + +CCoonnnneeccttiioonn ccaacchhee lliimmiittaattiioonnss + +Postfix SMTP connection caching conflicts with certain applications: + + * With Postfix versions < 3.4, the Postfix shared connection cache cannot be + used with TLS, because an open TLS connection can be reused only in the + process that creates it. For this reason, the Postfix smtp(8) client + historically always closed the connection after completing an attempt to + deliver mail over TLS. + + * Postfix connection caching currently does not support multiple SASL + accounts per mail server. Specifically, Postfix connection caching assumes + that a SASL credential is valid for all hostnames or domain names that + deliver via the same mail server IP address and TCP port, and assumes that + the SASL credential does not depend on the message originator. + +CCoonnnneeccttiioonn ccaacchhee ssttaattiissttiiccss + +The scache(8) connection cache server logs statistics about the peak cache size +and the cache hit rates. This information is logged every +connection_cache_status_update_time seconds, when the process terminates after +the maximal idle time is exceeded, or when Postfix is reloaded. + + * Hit rates for connection cache lookups by domain will tell you how useful + connection caching is. + + * Connection cache lookups by network address will always fail, unless you're + sending mail to different domains that share the same MX hosts. + + * No statistics are logged when no attempts are made to access the connection + cache. + diff --git a/README_FILES/CONTENT_INSPECTION_README b/README_FILES/CONTENT_INSPECTION_README new file mode 100644 index 0000000..a6a6fd8 --- /dev/null +++ b/README_FILES/CONTENT_INSPECTION_README @@ -0,0 +1,56 @@ +PPoossttffiixx CCoonntteenntt IInnssppeeccttiioonn + +------------------------------------------------------------------------------- +Postfix supports three content inspection methods, ranging from light-weight +one-line-at-a-time scanning before mail is queued, to heavy duty machinery that +does sophisticated content analysis after mail is queued. Each approach serves +a different purpose. + +bbeeffoorree qquueeuuee,, bbuuiilltt--iinn,, lliigghhtt--wweeiigghhtt + This method inspects mail BEFORE it is stored in the queue, and uses + Postfix's built-in message header and message body inspection. Although the + main purpose is to stop a specific flood of mail from worms or viruses, it + is also useful to block a flood of bounced junk email and email + notifications from virus detection systems. The built-in regular + expressions are not meant to implement general SPAM and virus detection. + For that, you should use one of the content inspection methods described + below. Details are described in the BUILTIN_FILTER_README and + BACKSCATTER_README documents. + +aafftteerr qquueeuuee,, eexxtteerrnnaall,, hheeaavvyy--wweeiigghhtt + This method inspects mail AFTER it is stored in the queue, and uses + standard protocols such as SMTP or "pipe to command and wait for exit + status". After-queue inspection allows you to use content filters of + arbitrary complexity without causing timeouts while receiving mail, and + without running out of memory resources under a peak load. Details of this + approach are in the FILTER_README document. + +bbeeffoorree qquueeuuee,, eexxtteerrnnaall,, mmeeddiiuumm--wweeiigghhtt + The following two methods inspect mail BEFORE it is stored in the queue. + + * The first method uses the SMTP protocol, and is described in the + SMTPD_PROXY_README document. This approach is available with Postfix + version 2.1 and later. + + * The second method uses the Sendmail 8 Milter protocol, and is described + in the MILTER_README document. This approach is available with Postfix + version 2.3 and later. + + Although these approaches appear to be attractive, they have some serious + limitations that you need to be aware of. First, content inspection + software must finish in a limited amount of time; if content inspection + needs too much time then incoming mail deliveries will time out. Second, + content inspection software must run in a limited amount of memory; if + content inspection needs too much memory then software will crash under a + peak load. Before-queue inspection limits the peak load that your system + can handle, and limits the sophistication of the content filter that you + can use. + +The more sophisticated content filtering software is not built into Postfix for +good reasons: writing an MTA requires different skills than writing a SPAM or +virus killer. Postfix encourages the use of external filters and standard +protocols because this allows you to choose the best MTA and the best content +inspection software for your purpose. Information about external content +inspection software can be found on the Postfix website at http:// +www.postfix.org/, and on the postfix-users@postfix.org mailing list. + diff --git a/README_FILES/DATABASE_README b/README_FILES/DATABASE_README new file mode 100644 index 0000000..3fd88c3 --- /dev/null +++ b/README_FILES/DATABASE_README @@ -0,0 +1,325 @@ +PPoossttffiixx LLooookkuupp TTaabbllee OOvveerrvviieeww + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document covers the following topics: + + * The Postfix lookup table model + * Postfix lists versus tables + * Preparing Postfix for LDAP or SQL lookups + * Maintaining Postfix lookup table files + * Updating Berkeley DB files safely + * Postfix lookup table types + +TThhee PPoossttffiixx llooookkuupp ttaabbllee mmooddeell + +Postfix uses lookup tables to store and look up information for access control, +address rewriting and even for content filtering. All Postfix lookup tables are +specified as "type:table", where "type" is one of the database types described +under "Postfix lookup table types" at the end of this document, and where +"table" is the lookup table name. The Postfix documentation uses the terms +"database" and "lookup table" for the same thing. + +Examples of lookup tables that appear often in the Postfix documentation: + + /etc/postfix/main.cf: + alias_maps = hash:/etc/postfix/aliases (local aliasing) + header_checks = regexp:/etc/postfix/header_checks (content filtering) + transport_maps = hash:/etc/postfix/transport (routing table) + virtual_alias_maps = hash:/etc/postfix/virtual (address rewriting) + +All Postfix lookup tables store information as (key, value) pairs. This +interface may seem simplistic at first, but it turns out to be very powerful. +The (key, value) query interface completely hides the complexities of LDAP or +SQL from Postfix. This is a good example of connecting complex systems with +simple interfaces. + +Benefits of the Postfix (key, value) query interface: + + * You can implement Postfix lookup tables first with local Berkeley DB files + and then switch to LDAP or MySQL without any impact on the Postfix + configuration itself, as described under "Preparing Postfix for LDAP or SQL + lookups" below. + * You can use Berkeley DB files with fixed lookup strings for simple address + rewriting operations and you can use regular expression tables for the more + complicated work. In other words, you don't have to put everything into the + same table. + +PPoossttffiixx lliissttss vveerrssuuss ttaabblleess + +Most Postfix lookup tables are used to look up information. Examples are +address rewriting (the lookup string is the old address, and the result is the +new address) or access control (the lookup string is the client, sender or +recipient, and the result is an action such as "reject"). + +With some tables, however, Postfix needs to know only if the lookup key exists. +Any non-empty lookup result value may be used here: the lookup result is not +used. Examples are the local_recipient_maps that determine what local +recipients Postfix accepts in mail from the network, the mydestination +parameter that specifies what domains Postfix delivers locally for, or the +mynetworks parameter that specifies the IP addresses of trusted clients or +client networks. Technically, these are lists, not tables. Despite the +difference, Postfix lists are described here because they use the same +underlying infrastructure as Postfix lookup tables. + +PPrreeppaarriinngg PPoossttffiixx ffoorr LLDDAAPP oorr SSQQLL llooookkuuppss + +LDAP and SQL are complex systems. Trying to set up both Postfix and LDAP or SQL +at the same time is definitely not a good idea. You can save yourself a lot of +time by implementing Postfix first with local files such as Berkeley DB. Local +files have few surprises, and are easy to debug with the postmap(1) command: + + % ppoossttmmaapp --qq iinnffoo@@eexxaammppllee..ccoomm hhaasshh:://eettcc//ppoossttffiixx//vviirrttuuaall + +Once you have local files working properly you can follow the instructions in +ldap_table(5), mysql_table(5), pgsql_table(5) or sqlite_table(5) and replace +local file lookups with LDAP or SQL lookups. When you do this, you should use +the postmap(1) command again, to verify that database lookups still produce the +exact same results as local file lookup: + + % ppoossttmmaapp --qq iinnffoo@@eexxaammppllee..ccoomm llddaapp:://eettcc//ppoossttffiixx//vviirrttuuaall..ccff + +Be sure to exercise all the partial address or parent domain queries that are +documented under "table search order" in the relevant manual page: access(5), +canonical(5), virtual(5), transport(5), or under the relevant configuration +parameter: mynetworks, relay_domains, parent_domain_matches_subdomains. + +MMaaiinnttaaiinniinngg PPoossttffiixx llooookkuupp ttaabbllee ffiilleess + +When you make changes to a database while the mail system is running, it would +be desirable if Postfix avoids reading information while that information is +being changed. It would also be nice if you can change a database without +having to execute "postfix reload", in order to force Postfix to use the new +information. Each time you do "postfix reload" Postfix loses a lot of +performance. + + * If you change a network database such as LDAP, NIS or SQL, there is no need + to execute "postfix reload". The LDAP, NIS or SQL server takes care of + read/write access conflicts and gives the new data to Postfix once that + data is available. + + * If you change a regexp:, pcre:, cidr: or texthash: file then Postfix may + not pick up the file changes immediately. This is because a Postfix process + reads the entire file into memory once and never examines the file again. + + o If the file is used by a short-running process such as smtpd(8), + cleanup(8) or local(8), there is no need to execute "postfix reload" + after making a change. + + o If the file is being used by a long-running process such as trivial- + rewrite(8) on a busy server it may be necessary to execute "postfix + reload". + + * If you change a local file based database such as DBM or Berkeley DB, there + is no need to execute "postfix reload". Postfix uses file locking to avoid + read/write access conflicts, and whenever a Postfix daemon process notices + that a file has changed it will terminate before handling the next client + request, so that a new process can initialize with the new database. + +UUppddaattiinngg BBeerrkkeelleeyy DDBB ffiilleess ssaaffeellyy + +Postfix uses file locking to avoid access conflicts while updating Berkeley DB +or other local database files. This used to be safe, but as Berkeley DB has +evolved to use more aggressive caching, file locking may no longer be +sufficient. + +Furthermore, file locking would not prevent problems when the update fails +because the disk is full or something else causes a database update to fail. In +particular, commands such as postmap(1) or postalias(1) overwrite existing +files. If the overwrite fails in the middle then you have no usable database, +and Postfix will stop working. This is not an issue with the CDB database type +available with Postfix 2.2 and later: CDB creates a new file, and renames the +file upon successful completion. + +With Berkeley DB and other "one file" databases, it is possible to add some +extra robustness by using "mv" to REPLACE an existing database file instead of +overwriting it: + + # ppoossttmmaapp aacccceessss..iinn &&&& mmvv aacccceessss..iinn..ddbb aacccceessss..ddbb + +This converts the input file "access.in" into the output file "access.in.db", +and replaces the file "access.db" only when the postmap(1) command was +successful. Of course typing such commands becomes boring quickly, and this is +why people use "make" instead, as shown below. User input is shown in bold +font. + + # ccaatt MMaakkeeffiillee + all: aliases.db access.db virtual.db ...etcetera... + + # Note 1: commands are specified after a TAB character. + # Note 2: use postalias(1) for local aliases, postmap(1) for the rest. + aliases.db: aliases.in + postalias aliases.in + mv aliases.in.db aliases.db + + access.db: access.in + postmap access.in + mv access.in.db access.db + + virtual.db: virtual.in + postmap virtual.in + mv virtual.in.db virtual.db + + ...etcetera... + # vvii aacccceessss..iinn + ...editing session not shown... + # mmaakkee + postmap access.in + mv access.in.db access.db + # + +The "make" command updates only the files that have changed. In case of error, +the "make" command will stop and will not invoke the "mv" command, so that +Postfix will keep using the existing database file as if nothing happened. + +PPoossttffiixx llooookkuupp ttaabbllee ttyyppeess + +To find out what database types your Postfix system supports, use the "ppoossttccoonnff +--mm" command. Here is a list of database types that are often supported: + + bbttrreeee + A sorted, balanced tree structure. This is available only on systems + with support for Berkeley DB databases. Database files are created with + the postmap(1) or postalias(1) command. The lookup table name as used + in "btree:table" is the database file name without the ".db" suffix. + ccddbb + A read-optimized structure with no support for incremental updates. + Database files are created with the postmap(1) or postalias(1) command. + The lookup table name as used in "cdb:table" is the database file name + without the ".cdb" suffix. This feature is available with Postfix 2.2 + and later. + cciiddrr + A table that associates values with Classless Inter-Domain Routing + (CIDR) patterns. The table format is described in cidr_table(5). + ddbbmm + An indexed file type based on hashing. This is available only on + systems with support for DBM databases. Public database files are + created with the postmap(1) or postalias(1) command, and private + databases are maintained by Postfix daemons. The lookup table name as + used in "dbm:table" is the database file name without the ".dir" or + ".pag" suffix. + eennvviirroonn + The UNIX process environment array. The lookup key is the variable + name. The lookup table name in "environ:table" is ignored. + ffaaiill + A table that reliably fails all requests. The lookup table name is used + for logging only. This table exists to simplify Postfix error tests. + hhaasshh + An indexed file type based on hashing. This is available only on + systems with support for Berkeley DB databases. Public database files + are created with the postmap(1) or postalias(1) command, and private + databases are maintained by Postfix daemons. The database name as used + in "hash:table" is the database file name without the ".db" suffix. + iinnlliinnee (read-only) + A non-shared, in-memory lookup table. Example: "inline:{ key=value, + { key = text with whitespace or comma }}". Key-value pairs are + separated by whitespace or comma; with a key-value pair inside "{}", + whitespace is ignored after the opening "{", around the "=" between key + and value, and before the closing "}". Inline tables eliminate the need + to create a database file for just a few fixed elements. See also the + static: map type. + iinntteerrnnaall + A non-shared, in-memory hash table. Its contents are lost when a + process terminates. + llmmddbb + OpenLDAP LMDB database. This is available only on systems with support + for LMDB databases. Public database files are created with the postmap + (1) or postalias(1) command, and private databases are maintained by + Postfix daemons. The database name as used in "lmdb:table" is the + database file name without the ".lmdb" suffix. See lmdb_table(5) for + details. + llddaapp (read-only) + LDAP database client. Configuration details are given in the ldap_table + (5). + mmeemmccaacchhee + Memcache database client. Configuration details are given in + memcache_table(5). + mmyyssqqll (read-only) + MySQL database client. Configuration details are given in mysql_table + (5). + nneettiinnffoo (read-only) + Netinfo database client. + nniiss (read-only) + NIS database client. + nniisspplluuss (read-only) + NIS+ database client. Configuration details are given in nisplus_table + (5). + ppccrree (read-only) + A lookup table based on Perl Compatible Regular Expressions. The file + format is described in pcre_table(5). The lookup table name as used in + "pcre:table" is the name of the regular expression file. + ppiippeemmaapp (read-only) + A pipeline of lookup tables. Example: "pipemap:{type1:name1, ..., + typen:namen}". Each "pipemap:" query is given to the first table. Each + lookup result becomes the query for the next table in the pipeline, and + the last table produces the final result. When any table lookup + produces no result, the pipeline produces no result. The first and last + characters of the "pipemap:" table name must be "{" and "}". Within + these, individual maps are separated with comma or whitespace. + ppggssqqll (read-only) + PostgreSQL database client. Configuration details are given in + pgsql_table(5). + pprrooxxyy + Postfix proxymap(8) client for shared access to Postfix databases. The + lookup table name syntax is "proxy:type:table". + rraannddmmaapp (read-only) + An in-memory table that performs random selection. Example: "randmap: + {result1. ..., resultn}". Each table query returns a random choice from + the specified results. The first and last characters of the "randmap: + " table name must be "{" and "}". Within these, individual maps are + separated with comma or whitespace. To give a specific result more + weight, specify it multiple times. + rreeggeexxpp (read-only) + A lookup table based on regular expressions. The file format is + described in regexp_table(5). The lookup table name as used in "regexp: + table" is the name of the regular expression file. + ssddbbmm + An indexed file type based on hashing. This is available only on + systems with support for SDBM databases. Public database files are + created with the postmap(1) or postalias(1) command, and private + databases are maintained by Postfix daemons. The lookup table name as + used in "sdbm:table" is the database file name without the ".dir" or + ".pag" suffix. + ssoocckkeettmmaapp (read-only) + Sendmail-style socketmap client. The name of the table is either iinneett: + host:port:name for a TCP/IP server, or uunniixx:pathname:name for a UNIX- + domain server. See socketmap_table(5) for details. + ssqqlliittee (read-only) + SQLite database. Configuration details are given in sqlite_table(5). + ssttaattiicc (read-only) + A table that always returns its name as the lookup result. For example, + "static:foobar" always returns the string "foobar" as lookup result. + Specify "static:{ text with whitespace }" when the result contains + whitespace; this form ignores whitespace after the opening "{" and + before the closing "}". See also the inline: map type. + ttccpp + TCP/IP client. The protocol is described in tcp_table(5). The lookup + table name is "tcp:host:port" where "host" specifies a symbolic + hostname or a numeric IP address, and "port" specifies a symbolic + service name or a numeric port number. + tteexxtthhaasshh (read-only) + A table that produces similar results as hash: files, except that you + don't have to run the postmap(1) command before you can use the file, + and that texthash: does not detect changes after the file is read. The + lookup table name is "texthash:filename", where the file name is taken + literally; no suffix is appended. + uunniioonnmmaapp (read-only) + A table that sends each query to multiple lookup tables and that + concatenates all found results, separated by comma. The table name + syntax is the same as for pipemap tables. + uunniixx (read-only) + A limited view of the UNIX authentication database. The following + tables are implemented: + uunniixx::ppaasssswwdd..bbyynnaammee + The table is the UNIX password database. The key is a login name. + The result is a password file entry in passwd(5) format. + uunniixx::ggrroouupp..bbyynnaammee + The table is the UNIX group database. The key is a group name. The + result is a group file entry in group(5) format. + +Other lookup table types may be available depending on how Postfix was built. +With some Postfix distributions the list is dynamically extensible as support +for lookup tables is dynamically linked into Postfix. + diff --git a/README_FILES/DB_README b/README_FILES/DB_README new file mode 100644 index 0000000..2d0ac93 --- /dev/null +++ b/README_FILES/DB_README @@ -0,0 +1,173 @@ +PPoossttffiixx BBeerrkkeelleeyy DDBB HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix uses databases of various kinds to store and look up information. +Postfix databases are specified as "type:name". Berkeley DB implements the +Postfix database type "hash" and "btree". The name of a Postfix Berkeley DB +database is the name of the database file without the ".db" suffix. Berkeley DB +databases are maintained with the postmap(1) command. + +Note: Berkeley DB version 4 is not supported by Postfix versions before 2.0. + +This document describes: + + 1. How to build Postfix without Berkeley DB support even if the system comes + with Berkeley DB. + + 2. How to build Postfix on systems that normally have no Berkeley DB library. + + 3. How to build Postfix on BSD or Linux systems with multiple Berkeley DB + versions. + + 4. How to tweak performance. + + 5. Missing pthread library trouble. + +BBuuiillddiinngg PPoossttffiixx wwiitthhoouutt BBeerrkkeelleeyy DDBB ssuuppppoorrtt eevveenn iiff tthhee ssyysstteemm ccoommeess wwiitthh +BBeerrkkeelleeyy DDBB + +Note: The following instructions apply to Postfix 2.9 and later. + +Postfix will normally enable Berkeley DB support if the system is known to have +it. To build Postfix without Berkeley DB support, build the makefiles as +follows: + + % make makefiles CCARGS="-DNO_DB" + % make + +This will disable support for "hash" and "btree" files. + +BBuuiillddiinngg PPoossttffiixx oonn ssyysstteemmss tthhaatt nnoorrmmaallllyy hhaavvee nnoo BBeerrkkeelleeyy DDBB lliibbrraarryy + +Some UNIXes ship without Berkeley DB support; for historical reasons these use +DBM files instead. A problem with DBM files is that they can store only limited +amounts of data. To build Postfix with Berkeley DB support you need to download +and install the source code from http://www.oracle.com/database/berkeley-db/. + +Warning: some Linux system libraries use Berkeley DB, as do some third-party +libraries such as SASL. If you compile Postfix with a different Berkeley DB +implementation, then every Postfix program will dump core because either the +system library, the SASL library, or Postfix itself ends up using the wrong +version. + +The more recent Berkeley DB versions have a compile-time switch, "--with- +uniquename", which renames the symbols so that multiple versions of Berkeley DB +can co-exist in the same application. Although wasteful, this may be the only +way to keep things from falling apart. + +To build Postfix after you installed the Berkeley DB from source code, use +something like: + + % make makefiles CCARGS="-DHAS_DB -I/usr/local/BerkeleyDB/include" \ + AUXLIBS="-L/usr/local/BerkeleyDB/lib -ldb" + % make + +If your Berkeley DB shared library is in a directory that the RUN-TIME linker +does not know about, add a "-Wl,-R,/path/to/directory" option after "-ldb". + +Solaris needs this: + + % make makefiles CCARGS="-DHAS_DB -I/usr/local/BerkeleyDB/include" \ + AUXLIBS="-R/usr/local/BerkeleyDB/lib -L/usr/local/BerkeleyDB/lib -ldb" + % make + +The exact pathnames depend on the Berkeley DB version, and on how it was +installed. + +Warning: the file format produced by Berkeley DB version 1 is not compatible +with that of versions 2 and 3 (versions 2 and 3 have the same format). If you +switch between DB versions, then you may have to rebuild all your Postfix DB +files. + +Warning: if you use Berkeley DB version 2 or later, do not enable DB 1.85 +compatibility mode. Doing so would break fcntl file locking. + +Warning: if you use Perl to manipulate Postfix's Berkeley DB files, then you +need to use the same Berkeley DB version in Perl as in Postfix. + +BBuuiillddiinngg PPoossttffiixx oonn BBSSDD ssyysstteemmss wwiitthh mmuullttiippllee BBeerrkkeelleeyy DDBB vveerrssiioonnss + +Some BSD systems ship with multiple Berkeley DB implementations. Normally, +Postfix builds with the default DB version that ships with the system. + +To build Postfix on BSD systems with a non-default DB version, use a variant of +the following commands: + + % make makefiles CCARGS=-I/usr/include/db3 AUXLIBS=-ldb3 + % make + +Warning: the file format produced by Berkeley DB version 1 is not compatible +with that of versions 2 and 3 (versions 2 and 3 have the same format). If you +switch between DB versions, then you may have to rebuild all your Postfix DB +files. + +Warning: if you use Berkeley DB version 2 or later, do not enable DB 1.85 +compatibility mode. Doing so would break fcntl file locking. + +Warning: if you use Perl to manipulate Postfix's Berkeley DB files, then you +need to use the same Berkeley DB version in Perl as in Postfix. + +BBuuiillddiinngg PPoossttffiixx oonn LLiinnuuxx ssyysstteemmss wwiitthh mmuullttiippllee BBeerrkkeelleeyy DDBB vveerrssiioonnss + +Some Linux systems ship with multiple Berkeley DB implementations. Normally, +Postfix builds with the default DB version that ships with the system. + +Warning: some Linux system libraries use Berkeley DB. If you compile Postfix +with a non-default Berkeley DB implementation, then every Postfix program will +dump core because either the system library or Postfix itself ends up using the +wrong version. + +On Linux, you need to edit the makedefs script in order to specify a non- +default DB library. The reason is that the location of the default db.h include +file changes randomly between vendors and between versions, so that Postfix has +to choose the file for you. + +Warning: the file format produced by Berkeley DB version 1 is not compatible +with that of versions 2 and 3 (versions 2 and 3 have the same format). If you +switch between DB versions, then you may have to rebuild all your Postfix DB +files. + +Warning: if you use Berkeley DB version 2 or later, do not enable DB 1.85 +compatibility mode. Doing so would break fcntl file locking. + +Warning: if you use Perl to manipulate Postfix's Berkeley DB files, then you +need to use the same Berkeley DB version in Perl as in Postfix. + +TTwweeaakkiinngg ppeerrffoorrmmaannccee + +Postfix provides two configuration parameters that control how much buffering +memory Berkeley DB will use. + + * berkeley_db_create_buffer_size (default: 16 MBytes per table). This setting + is used by the commands that maintain Berkeley DB files: postalias(1) and + postmap(1). For "hash" files, create performance degrades rapidly unless + the memory pool is O(file size). For "btree" files, create performance is + good with sorted input even for small memory pools, but with random input + degrades rapidly unless the memory pool is O(file size). + + * berkeley_db_read_buffer_size (default: 128 kBytes per table). This setting + is used by all other Postfix programs. The buffer size is adequate for + reading. If the cache is smaller than the table, random read performance is + hardly cache size dependent, except with btree tables, where the cache size + must be large enough to contain the entire path from the root node. + Empirical evidence shows that 64 kBytes may be sufficient. We double the + size to play safe, and to anticipate changes in implementation and bloat. + +MMiissssiinngg pptthhrreeaadd lliibbrraarryy ttrroouubbllee + +When building Postfix fails with: + + undefined reference to `pthread_condattr_setpshared' + undefined reference to `pthread_mutexattr_destroy' + undefined reference to `pthread_mutexattr_init' + undefined reference to `pthread_mutex_trylock' + +Add the "-lpthread" library to the "make makefiles" command. + + % make makefiles .... AUXLIBS="... -lpthread" + +More information is available at http://www.oracle.com/database/berkeley-db/. + diff --git a/README_FILES/DEBUG_README b/README_FILES/DEBUG_README new file mode 100644 index 0000000..7221572 --- /dev/null +++ b/README_FILES/DEBUG_README @@ -0,0 +1,402 @@ +PPoossttffiixx DDeebbuuggggiinngg HHoowwttoo + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +This document describes how to debug parts of the Postfix mail system when +things do not work according to expectation. The methods vary from making +Postfix log a lot of detail, to running some daemon processes under control of +a call tracer or debugger. + +The text assumes that the Postfix main.cf and master.cf configuration files are +stored in directory /etc/postfix. You can use the command "ppoossttccoonnff +ccoonnffiigg__ddiirreeccttoorryy" to find out the actual location of this directory on your +machine. + +Listed in order of increasing invasiveness, the debugging techniques are as +follows: + + * Look for obvious signs of trouble + * Debugging Postfix from inside + * Try turning off chroot operation in master.cf + * Verbose logging for specific SMTP connections + * Record the SMTP session with a network sniffer + * Making Postfix daemon programs more verbose + * Manually tracing a Postfix daemon process + * Automatically tracing a Postfix daemon process + * Running daemon programs with the interactive ddd debugger + * Running daemon programs with the interactive gdb debugger + * Running daemon programs under a non-interactive debugger + * Unreasonable behavior + * Reporting problems to postfix-users@postfix.org + +LLooookk ffoorr oobbvviioouuss ssiiggnnss ooff ttrroouubbllee + +Postfix logs all failed and successful deliveries to a logfile. + + * When Postfix uses syslog logging (the default), the file is usually called + /var/log/maillog, /var/log/mail, or something similar; the exact pathname + is configured in a file called /etc/syslog.conf, /etc/rsyslog.conf, or + something similar. + + * When Postfix uses its own logging system (see MAILLOG_README), the location + of the logfile is configured with the Postfix maillog_file parameter. + +When Postfix does not receive or deliver mail, the first order of business is +to look for errors that prevent Postfix from working properly: + + % ggrreepp --EE ''((wwaarrnniinngg||eerrrroorr||ffaattaall||ppaanniicc))::'' //ssoommee//lloogg//ffiillee || mmoorree + +Note: the most important message is near the BEGINNING of the output. Error +messages that come later are less useful. + +The nature of each problem is indicated as follows: + + * "ppaanniicc" indicates a problem in the software itself that only a programmer + can fix. Postfix cannot proceed until this is fixed. + + * "ffaattaall" is the result of missing files, incorrect permissions, incorrect + configuration file settings that you can fix. Postfix cannot proceed until + this is fixed. + + * "eerrrroorr" reports an error condition. For safety reasons, a Postfix process + will terminate when more than 13 of these happen. + + * "wwaarrnniinngg" indicates a non-fatal error. These are problems that you may not + be able to fix (such as a broken DNS server elsewhere on the network) but + may also indicate local configuration errors that could become a problem + later. + +DDeebbuuggggiinngg PPoossttffiixx ffrroomm iinnssiiddee + +Postfix version 2.1 and later can produce mail delivery reports for debugging +purposes. These reports not only show sender/recipient addresses after address +rewriting and alias expansion or forwarding, they also show information about +delivery to mailbox, delivery to non-Postfix command, responses from remote +SMTP servers, and so on. + +Postfix can produce two types of mail delivery reports for debugging: + + * What-if: report what would happen, but do not actually deliver mail. This + mode of operation is requested with: + + % //uussrr//ssbbiinn//sseennddmmaaiill --bbvv aaddddrreessss...... + Mail Delivery Status Report will be mailed to <your login name>. + + * What happened: deliver mail and report successes and/or failures, including + replies from remote SMTP servers. This mode of operation is requested with: + + % //uussrr//ssbbiinn//sseennddmmaaiill --vv aaddddrreessss...... + Mail Delivery Status Report will be mailed to <your login name>. + +These reports contain information that is generated by Postfix delivery agents. +Since these run as daemon processes that cannot interact with users directly, +the result is sent as mail to the sender of the test message. The format of +these reports is practically identical to that of ordinary non-delivery +notifications. + +For a detailed example of a mail delivery status report, see the debugging +section at the end of the ADDRESS_REWRITING_README document. + +TTrryy ttuurrnniinngg ooffff cchhrroooott ooppeerraattiioonn iinn mmaasstteerr..ccff + +A common mistake is to turn on chroot operation in the master.cf file without +going through all the necessary steps to set up a chroot environment. This +causes Postfix daemon processes to fail due to all kinds of missing files. + +The example below shows an SMTP server that is configured with chroot turned +off: + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv cchhrroooott wakeup maxproc command + # (yes) (yes) ((yyeess)) (never) (100) + # ============================================================= + smtp inet n - nn - - smtpd + +Inspect master.cf for any processes that have chroot operation not turned off. +If you find any, save a copy of the master.cf file, and edit the entries in +question. After executing the command "ppoossttffiixx rreellooaadd", see if the problem has +gone away. + +If turning off chrooted operation made the problem go away, then +congratulations. Leaving Postfix running in this way is adequate for most +sites. If you prefer chrooted operation, see the Postfix +BASIC_CONFIGURATION_README file for information about how to prepare Postfix +for chrooted operation. + +VVeerrbboossee llooggggiinngg ffoorr ssppeecciiffiicc SSMMTTPP ccoonnnneeccttiioonnss + +In /etc/postfix/main.cf, list the remote site name or address in the +debug_peer_list parameter. For example, in order to make the software log a lot +of information to the syslog daemon for connections from or to the loopback +interface: + + /etc/postfix/main.cf: + debug_peer_list = 127.0.0.1 + +You can specify one or more hosts, domains, addresses or net/masks. To make the +change effective immediately, execute the command "ppoossttffiixx rreellooaadd". + +RReeccoorrdd tthhee SSMMTTPP sseessssiioonn wwiitthh aa nneettwwoorrkk ssnniiffffeerr + +This example uses ttccppdduummpp. In order to record a conversation you need to +specify a large enough buffer with the "--ss" option or else you will miss some +or all of the packet payload. + + # ttccppdduummpp --ww //ffiillee//nnaammee --ss 00 hhoosstt eexxaammppllee..ccoomm aanndd ppoorrtt 2255 + +Older tcpdump versions don't support "--ss 00"; in that case, use "--ss 22000000" +instead. + +Run this for a while, stop with Ctrl-C when done. To view the data use a binary +viewer, eetthheerreeaall, or good old lleessss. + +MMaakkiinngg PPoossttffiixx ddaaeemmoonn pprrooggrraammss mmoorree vveerrbboossee + +Append one or more "--vv" options to selected daemon definitions in /etc/postfix/ +master.cf and type "ppoossttffiixx rreellooaadd". This will cause a lot of activity to be +logged to the syslog daemon. For example, to make the Postfix SMTP server +process more verbose: + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd -v + +To diagnose problems with address rewriting specify a "--vv" option for the +cleanup(8) and/or trivial-rewrite(8) daemon, and to diagnose problems with mail +delivery specify a "--vv" option for the qmgr(8) or oqmgr(8) queue manager, or +for the lmtp(8), local(8), pipe(8), smtp(8), or virtual(8) delivery agent. + +MMaannuuaallllyy ttrraacciinngg aa PPoossttffiixx ddaaeemmoonn pprroocceessss + +Many systems allow you to inspect a running process with a system call tracer. +For example: + + # ttrraaccee --pp pprroocceessss--iidd (SunOS 4) + # ssttrraaccee --pp pprroocceessss--iidd (Linux and many others) + # ttrruussss --pp pprroocceessss--iidd (Solaris, FreeBSD) + # kkttrraaccee --pp pprroocceessss--iidd (generic 4.4BSD) + +Even more informative are traces of system library calls. Examples: + + # llttrraaccee --pp pprroocceessss--iidd (Linux, also ported to FreeBSD and BSD/OS) + # ssoottrruussss --pp pprroocceessss--iidd (Solaris) + +See your system documentation for details. + +Tracing a running process can give valuable information about what a process is +attempting to do. This is as much information as you can get without running an +interactive debugger program, as described in a later section. + +AAuuttoommaattiiccaallllyy ttrraacciinngg aa PPoossttffiixx ddaaeemmoonn pprroocceessss + +Postfix can attach a call tracer whenever a daemon process starts. Call tracers +come in several kinds. + + 1. System call tracers such as ttrraaccee, ttrruussss, ssttrraaccee, or kkttrraaccee. These show the + communication between the process and the kernel. + + 2. Library call tracers such as ssoottrruussss and llttrraaccee. These show calls of + library routines, and give a better idea of what is going on within the + process. + +Append a --DD option to the suspect command in /etc/postfix/master.cf, for +example: + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd -D + +Edit the debugger_command definition in /etc/postfix/main.cf so that it invokes +the call tracer of your choice, for example: + + /etc/postfix/main.cf: + debugger_command = + PATH=/bin:/usr/bin:/usr/local/bin; + (truss -p $process_id 2>&1 | logger -p mail.info) & sleep 5 + +Type "ppoossttffiixx rreellooaadd" and watch the logfile. + +RRuunnnniinngg ddaaeemmoonn pprrooggrraammss wwiitthh tthhee iinntteerraaccttiivvee dddddd ddeebbuuggggeerr + +If you have X Windows installed on the Postfix machine, then an interactive +debugger such as dddddd can be convenient. + +Edit the debugger_command definition in /etc/postfix/main.cf so that it invokes +dddddd: + + /etc/postfix/main.cf: + debugger_command = + PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin + ddd $daemon_directory/$process_name $process_id & sleep 5 + +Be sure that ggddbb is in the command search path, and export XXAAUUTTHHOORRIITTYY so that X +access control works, for example: + + % sseetteennvv XXAAUUTTHHOORRIITTYY ~~//..XXaauutthhoorriittyy (csh syntax) + $ eexxppoorrtt XXAAUUTTHHOORRIITTYY==$$HHOOMMEE//..XXaauutthhoorriittyy (sh syntax) + +Append a --DD option to the suspect daemon definition in /etc/postfix/master.cf, +for example: + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd -D + +Stop and start the Postfix system. This is necessary so that Postfix runs with +the proper XXAAUUTTHHOORRIITTYY and DDIISSPPLLAAYY settings. + +Whenever the suspect daemon process is started, a debugger window pops up and +you can watch in detail what happens. + +RRuunnnniinngg ddaaeemmoonn pprrooggrraammss wwiitthh tthhee iinntteerraaccttiivvee ggddbb ddeebbuuggggeerr + +If you have the screen command installed on the Postfix machine, then you can +run an interactive debugger such as ggddbb as follows. + +Edit the debugger_command definition in /etc/postfix/main.cf so that it runs +ggddbb inside a detached ssccrreeeenn session: + + /etc/postfix/main.cf: + debugger_command = + PATH=/bin:/usr/bin:/sbin:/usr/sbin; export PATH; HOME=/root; + export HOME; screen -e^tt -dmS $process_name gdb + $daemon_directory/$process_name $process_id & sleep 2 + +Be sure that ggddbb is in the command search path. + +Append a --DD option to the suspect daemon definition in /etc/postfix/master.cf, +for example: + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd -D + +Execute the command "ppoossttffiixx rreellooaadd" and wait until a daemon process is started +(you can see this in the maillog file). + +Then attach to the screen, and debug away: + + # HOME=/root screen -r + gdb) continue + gdb) where + +RRuunnnniinngg ddaaeemmoonn pprrooggrraammss uunnddeerr aa nnoonn--iinntteerraaccttiivvee ddeebbuuggggeerr + +If you do not have X Windows installed on the Postfix machine, or if you are +not familiar with interactive debuggers, then you can try to run ggddbb in non- +interactive mode, and have it print a stack trace when the process crashes. + +Edit the debugger_command definition in /etc/postfix/main.cf so that it invokes +the ggddbb debugger: + + /etc/postfix/main.cf: + debugger_command = + PATH=/bin:/usr/bin:/usr/local/bin; export PATH; (echo cont; echo + where; sleep 8640000) | gdb $daemon_directory/$process_name + $process_id 2>&1 + >$config_directory/$process_name.$process_id.log & sleep 5 + +Append a --DD option to the suspect daemon in /etc/postfix/master.cf, for +example: + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd -D + +Type "ppoossttffiixx rreellooaadd" to make the configuration changes effective. + +Whenever a suspect daemon process is started, an output file is created, named +after the daemon and process ID (for example, smtpd.12345.log). When the +process crashes, a stack trace (with output from the "wwhheerree" command) is +written to its logfile. + +UUnnrreeaassoonnaabbllee bbeehhaavviioorr + +Sometimes the behavior exhibited by Postfix just does not match the source +code. Why can a program deviate from the instructions given by its author? +There are two possibilities. + + * The compiler has erred. This rarely happens. + + * The hardware has erred. Does the machine have ECC memory? + +In both cases, the program being executed is not the program that was supposed +to be executed, so anything could happen. + +There is a third possibility: + + * Bugs in system software (kernel or libraries). + +Hardware-related failures usually do not reproduce in exactly the same way +after power cycling and rebooting the system. There's little Postfix can do +about bad hardware. Be sure to use hardware that at the very least can detect +memory errors. Otherwise, Postfix will just be waiting to be hit by a bit +error. Critical systems deserve real hardware. + +When a compiler makes an error, the problem can be reproduced whenever the +resulting program is run. Compiler errors are most likely to happen in the code +optimizer. If a problem is reproducible across power cycles and system reboots, +it can be worthwhile to rebuild Postfix with optimization disabled, and to see +if optimization makes a difference. + +In order to compile Postfix with optimizations turned off: + + % mmaakkee ttiiddyy + % mmaakkee mmaakkeeffiilleess OOPPTT== + +This produces a set of Makefiles that do not request compiler optimization. + +Once the makefiles are set up, build the software: + + % mmaakkee + % ssuu + Password: + # mmaakkee iinnssttaallll + +If the problem goes away, then it is time to ask your vendor for help. + +RReeppoorrttiinngg pprroobblleemmss ttoo ppoossttffiixx--uusseerrss@@ppoossttffiixx..oorrgg + +The people who participate on postfix-users@postfix.org are very helpful, +especially if YOU provide them with sufficient information. Remember, these +volunteers are willing to help, but their time is limited. + +When reporting a problem, be sure to include the following information. + + * A summary of the problem. Please do not just send some logging without + explanation of what YOU believe is wrong. + + * Complete error messages. Please use cut-and-paste, or use attachments, + instead of reciting information from memory. + + * Postfix logging. See the text at the top of the DEBUG_README document to + find out where logging is stored. Please do not frustrate the helpers by + word wrapping the logging. If the logging is more than a few kbytes of + text, consider posting an URL on a web or ftp site. + + * Consider using a test email address so that you don't have to reveal email + addresses or passwords of innocent people. + + * If you can't use a test email address, please anonymize email addresses and + host names consistently. Replace each letter by "A", each digit by "D" so + that the helpers can still recognize syntactical errors. + + * Command output from: + + o "ppoossttccoonnff --nn". Please do not send your main.cf file, or 1000+ lines of + ppoossttccoonnff command output. + + o "ppoossttccoonnff --MMff" (Postfix 2.9 or later). + + * Better, provide output from the ppoossttffiinnggeerr tool. This can be found at + https://github.com/ford--prefect/postfinger. + + * If the problem is SASL related, consider including the output from the + ssaassllffiinnggeerr tool. This can be found at https://packages.debian.org/ + search?keywords=sasl2-bin. + + * If the problem is about too much mail in the queue, consider including + output from the qqsshhaappee tool, as described in the QSHAPE_README file. + + * If the problem is protocol related (connections time out, or an SMTP server + complains about syntax errors etc.) consider recording a session with + ttccppdduummpp, as described in the DEBUG_README document. + diff --git a/README_FILES/DSN_README b/README_FILES/DSN_README new file mode 100644 index 0000000..efd7f4c --- /dev/null +++ b/README_FILES/DSN_README @@ -0,0 +1,98 @@ +PPoossttffiixx DDSSNN SSuuppppoorrtt + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix version 2.3 introduces support for Delivery Status Notifications as +described in RFC 3464. This gives senders control over successful and failed +delivery notifications. + +Specifically, DSN support gives an email sender the ability to specify: + + * What notifications are sent: success, failure, delay, or none. Normally, + Postfix informs the sender only when mail delivery is delayed or when + delivery fails. + + * What content is returned in case of failure: only the message headers, or + the full message. + + * An envelope ID that is returned as part of delivery status notifications. + This identifies the message submission transaction, and must not be + confused with the message ID, which identifies the message content. + +The implementation of DSN support involves extra parameters to the SMTP MAIL +FROM and RCPT TO commands, as well as two Postfix sendmail command line options +that provide a sub-set of the functions of the extra SMTP command parameters. + +This document has information on the following topics: + + * Restricting the scope of "success" notifications + * Postfix sendmail command-line interface + * Postfix VERP support compatibility + +RReessttrriiccttiinngg tthhee ssccooppee ooff ""ssuucccceessss"" nnoottiiffiiccaattiioonnss + +Just like reports of undeliverable mail, DSN reports of successful delivery can +give away more information about the internal infrastructure than desirable. +Unfortunately, disallowing "success" notification requests requires disallowing +other DSN requests as well. The RFCs do not offer the option to negotiate +feature subsets. + +This is not as bad as it sounds. When you turn off DSN for remote inbound mail, +remote senders with DSN support will still be informed that their mail reached +your Postfix gateway successfully; they just will not get successful delivery +notices from your internal systems. Remote senders lose very little: they can +no longer specify how Postfix should report delayed or failed delivery. + +Use the smtpd_discard_ehlo_keyword_address_maps feature if you wish to allow +DSN requests from trusted clients but not from random strangers (see below for +how to turn this off for all clients): + + /etc/postfix/main.cf: + smtpd_discard_ehlo_keyword_address_maps = + cidr:/etc/postfix/esmtp_access + + /etc/postfix/esmtp_access: + # Allow DSN requests from local subnet only + 192.168.0.0/28 silent-discard + 0.0.0.0/0 silent-discard, dsn + ::/0 silent-discard, dsn + +If you want to disallow all use of DSN requests from the network, use the +smtpd_discard_ehlo_keywords feature: + + /etc/postfix/main.cf: + smtpd_discard_ehlo_keywords = silent-discard, dsn + +PPoossttffiixx sseennddmmaaiill ccoommmmaanndd--lliinnee iinntteerrffaaccee + +Postfix has two Sendmail-compatible command-line options for DSN support. + + * The first option specifies what notifications are sent for mail that is + submitted via the Postfix sendmail(1) command line: + + $ sseennddmmaaiill --NN ssuucccceessss,,ddeellaayy,,ffaaiilluurree ...... (one or more of these) + $ sseennddmmaaiill --NN nneevveerr ...... (or just this by itself) + + The built-in default corresponds with "delay,failure". + + * The second option specifies an envelope ID which is reported in delivery + status notifications for mail that is submitted via the Postfix sendmail(1) + command line: + + $ sseennddmmaaiill --VV eennvveellooppee--iidd ...... + + Note: this conflicts with VERP support in older Postfix versions, as + discussed in the next section. + +PPoossttffiixx VVEERRPP ssuuppppoorrtt ccoommppaattiibbiilliittyy + +With Postfix versions before 2.3, the sendmail(1) command uses the -V command- +line option to request VERP-style delivery. In order to request VERP style +delivery with Postfix 2.3 and later, you must specify -XV instead of -V. + +The Postfix 2.3 sendmail(1) command will recognize if you try to use -V for +VERP-style delivery. It will do the right thing and will remind you of the new +syntax. + diff --git a/README_FILES/ETRN_README b/README_FILES/ETRN_README new file mode 100644 index 0000000..76bc8de --- /dev/null +++ b/README_FILES/ETRN_README @@ -0,0 +1,250 @@ +PPoossttffiixx EETTRRNN HHoowwttoo + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhee PPoossttffiixx ffaasstt EETTRRNN sseerrvviiccee + +The SMTP ETRN command was designed for sites that have intermittent Internet +connectivity. With ETRN, a site can tell the mail server of its provider to +"Please deliver all my mail now". The SMTP server searches the queue for mail +to the customer, and delivers that mail bbyy ccoonnnneeccttiinngg ttoo tthhee ccuussttoommeerr''ss SSMMTTPP +sseerrvveerr. The mail is not delivered via the connection that was used for sending +ETRN. + +As of version 1.0, Postfix has a fast ETRN implementation that does not require +Postfix to examine every queue file. Instead, Postfix maintains a record of +what queue files contain mail for destinations that are configured for ETRN +service. ETRN service is no longer available for domains that aren't configured +for the service. + +This document provides information on the following topics: + + * Using the Postfix fast ETRN service + * How Postfix fast ETRN works + * Postfix fast ETRN service limitations + * Configuring the Postfix fast ETRN service + * Configuring a domain for ETRN service only + * Testing the Postfix fast ETRN service + +Other documents with information on this subject: + + * flush(8), flush service implementation + +UUssiinngg tthhee PPoossttffiixx ffaasstt EETTRRNN sseerrvviiccee + +The following is an example SMTP session that shows how an SMTP client requests +the ETRN service. Client commands are shown in bold font. + + 220 my.server.tld ESMTP Postfix + HHEELLOO mmyy..cclliieenntt..ttlldd + 250 Ok + EETTRRNN ssoommee..ccuussttoommeerr..ddoommaaiinn + 250 Queuing started + QQUUIITT + 221 Bye + +As mentioned in the introduction, the mail is delivered by connecting to the +customer's SMTP server; it is not sent over the connection that was used to +send the ETRN command. + +The Postfix operator can request delivery for a specific customer by using the +command "sendmail -qRdestination" and, with Postfix version 1.1 and later, +"postqueue -sdestination". Access to this feature is controlled with the +authorized_flush_users configuration parameter (Postfix version 2.2 and later). + +HHooww PPoossttffiixx ffaasstt EETTRRNN wwoorrkkss + +When a Postfix delivery agent decides that mail must be delivered later, it +sends the destination domain name and the queue file name to the flush(8) +daemon which maintains per-destination logfiles with file names of queued mail. +These logfiles are kept below $queue_directory/flush. Per-destination logfiles +are maintained only for destinations that are listed with the +$fast_flush_domains parameter and that have syntactically valid domain names. + + Postfix Postfix One logfile + delivery -(domain, queue ID)-> flush -(queue ID)-> per eligible + agent daemon domain + +When Postfix receives a request to "deliver mail for a domain now", the flush +(8) daemon moves all deferred queue files that are listed for that domain to +the incoming queue, and requests that the queue manager deliver them. In order +to force delivery, the queue manager temporarily ignores the lists of +undeliverable destinations: the volatile in-memory list of dead domains, and +the list of message delivery transports specified with the defer_transports +configuration parameter. + +PPoossttffiixx ffaasstt EETTRRNN sseerrvviiccee lliimmiittaattiioonnss + +The design of the flush(8) server and of the flush queue introduce a few +limitations that should not be an issue unless you want to turn on fast ETRN +service for every possible destination. + + * The flush(8) daemon maintains per-destination logfiles with queue file + names. When a request to "deliver mail now" arrives, Postfix will attempt + to deliver all recipients in the queue files that have mail for the + destination in question. This does not perform well with queue files that + have recipients in many different domains, such as queue files with + outbound mailing list traffic. + + * The flush(8) daemon maintains per-destination logfiles only for + destinations listed with $fast_flush_domains. With other destinations you + cannot request delivery with "sendmail -qRdestination" or, with Postfix + version 1.1 and later, "postqueue -sdestination". + + * Up to and including early versions of Postfix version 2.1, the "fast flush" + service may not deliver some messages if the request to "deliver mail now" + is received while a deferred queue scan is already in progress. The reason + is that the queue manager does not ignore the volatile in-memory list of + dead domains, and the list of message delivery transports specified with + the defer_transports configuration parameter. + + * Up to and including Postfix version 2.3, the "fast flush" service may not + deliver some messages if the request to "deliver mail now" arrives while an + incoming queue scan is already in progress. + +CCoonnffiigguurriinngg tthhee PPoossttffiixx ffaasstt EETTRRNN sseerrvviiccee + +The behavior of the flush(8) daemon is controlled by parameters in the main.cf +configuration file. + +By default, Postfix "fast ETRN" service is available only for destinations that +Postfix is willing to relay mail to: + + /etc/postfix/main.cf: + fast_flush_domains = $relay_domains + smtpd_etrn_restrictions = permit_mynetworks, reject + +Notes: + + * The relay_domains parameter specifies what destinations Postfix will relay + to. For destinations that are not eligible for the "fast ETRN" service, + Postfix replies with an error message. + + * The smtpd_etrn_restrictions parameter limits what clients may execute the + ETRN command. By default, any client has permission. + +To enable "fast ETRN" for some other destination, specify: + + /etc/postfix/main.cf: + fast_flush_domains = $relay_domains, some.other.domain + +To disable "fast ETRN", so that Postfix rejects all ETRN requests and so that +it maintains no per-destination logfiles, specify: + + /etc/postfix/main.cf: + fast_flush_domains = + +CCoonnffiigguurriinngg aa ddoommaaiinn ffoorr EETTRRNN sseerrvviiccee oonnllyy + +While an "ETRN" customer is off-line, Postfix will make spontaneous attempts to +deliver mail to it. These attempts are separated in time by increasing time +intervals, ranging from $minimal_backoff_time to $maximal_backoff_time, and +should not be a problem unless a lot of mail is queued. + +To prevent Postfix from making spontaneous delivery attempts you can configure +Postfix to always defer mail for the "ETRN" customer. Mail is delivered only +after the ETRN command or with "sendmail -q", with "sendmail -qRdomain", or +with "postqueue -sdomain"(Postfix version 1.1 and later only), + +In the example below we configure an "etrn-only" delivery transport which is +simply a duplicate of the "smtp" and "relay" mail delivery transports. The only +difference is that mail destined for this delivery transport is deferred as +soon as it arrives. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # (yes) (yes) (yes) (never) (100) + 5 # ============================================================= + 6 smtp unix - - n - - smtp + 7 relay unix - - n - - smtp + 8 etrn-only unix - - n - - smtp + 9 + 10 /etc/postfix/main.cf: + 11 relay_domains = customer.tld ...other domains... + 12 defer_transports = etrn-only + 13 transport_maps = hash:/etc/postfix/transport + 14 + 15 /etc/postfix/transport: + 16 customer.tld etrn-only:[mailhost.customer.tld] + +Translation: + + * Line 8: The "etrn-only" mail delivery service is a copy of the "smtp" and + "relay" service. + + * Line 11: Don't forget to authorize relaying for this customer, either via + relay_domains or with the permit_mx_backup feature. + + * Line 12: The "etrn-only" mail delivery service is configured so that + spontaneous mail delivery is disabled. + + * Lines 13-16: Mail for the customer is given to the "etrn-only" mail + delivery service. + + * Line 16: The "[mailhost.customer.tld]" turns off MX record lookups; you + must specify this if your Postfix server is the primary MX host for the + customer's domain. + +TTeessttiinngg tthhee PPoossttffiixx ffaasstt EETTRRNN sseerrvviiccee + +By default, "fast ETRN" service is enabled for all domains that match +$relay_domains. If you run Postfix with "fast ETRN" service for the very first +time, you need to run "sendmail -q" once in order to populate the per-site +deferred mail logfiles. If you omit this step, no harm is done. The logfiles +will eventually become populated as Postfix routinely attempts to deliver +delayed mail, but that will take a couple hours. After the "sendmail -q" +command has completed all delivery attempts (this can take a while), you're +ready to test the "fast ETRN" service. + +To test the "fast ETRN" service, telnet to the Postfix SMTP server from a +client that is allowed to execute ETRN commands (by default, that's every +client), and type the commands shown in boldface: + + 220 my.server.tld ESMTP Postfix + HHEELLOO mmyy..cclliieenntt..ttlldd + 250 Ok + EETTRRNN ssoommee..ccuussttoommeerr..ddoommaaiinn + 250 Queuing started + +where "some.customer.domain" is the name of a domain that has a non-empty +logfile somewhere under $queue_directory/flush. + +In the maillog file, you should immediately see a couple of logfile records, as +evidence that the queue manager has opened queue files: + + Oct 2 10:51:19 myhostname postfix/qmgr[51999]: 682E8440A4: + from=<whatever>, size=12345, nrcpt=1 (queue active) + Oct 2 10:51:19 myhostname postfix/qmgr[51999]: 02249440B7: + from=<whatever>, size=4711, nrcpt=1 (queue active) + +What happens next depends on whether the destination is reachable. If it's not +reachable, the mail queue IDs will be added back to the some.customer.domain +logfile under $queue_directory/flush. + +Repeat the exercise with some other destination that your server is willing to +relay to (any domain listed in $relay_domains), but that has no mail queued. +The text in bold face stands for the commands that you type: + + 220 my.server.tld ESMTP Postfix + HHEELLOO mmyy..cclliieenntt..ttlldd + 250 Ok + EETTRRNN ssoommee..ootthheerr..ccuussttoommeerr..ddoommaaiinn + 250 Queuing started + +This time, the "ETRN"" command should trigger NO mail deliveries at all. If +this triggers delivery of all mail, then you used the wrong domain name, or +"fast ETRN" service is turned off. + +Finally, repeat the exercise with a destination that your mail server is not +willing to relay to. It does not matter if your server has mail queued for that +destination. + + 220 my.server.tld ESMTP Postfix + HHEELLOO mmyy..cclliieenntt..ttlldd + 250 Ok + EETTRRNN nnoott..aa..ccuussttoommeerr..ddoommaaiinn + 459 <not.a.customer.domain>: service unavailable + +In this case, Postfix should reject the request as shown above. + diff --git a/README_FILES/FILTER_README b/README_FILES/FILTER_README new file mode 100644 index 0000000..4a76bb9 --- /dev/null +++ b/README_FILES/FILTER_README @@ -0,0 +1,617 @@ +PPoossttffiixx AAfftteerr--QQuueeuuee CCoonntteenntt FFiilltteerr + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document requires Postfix version 2.1 or later. + +Normally, Postfix receives mail, stores it in the mail queue and then delivers +it. With the external content filter described here, mail is filtered AFTER it +is queued. This approach decouples mail receiving processes from mail filtering +processes, and gives you maximal control over how many filtering processes you +are willing to run in parallel. + +The after-queue content filter is meant to be used as follows: + + Network or -> Postfix -> CCoonntteenntt -> Postfix -> Network or + local users queue ffiilltteerr queue local mailbox + +This document describes implementations that use a single Postfix instance for +everything: receiving, filtering and delivering mail. Applications that use two +separate Postfix instances will be covered by a later version of this document. + +The after-queue content filter is not to be confused with the approaches +described in the SMTPD_PROXY_README or MILTER_README documents, where incoming +SMTP mail is filtered BEFORE it is stored into the Postfix queue. + +This document describes two approaches to content filter all email, as well as +several options to filter mail selectively: + + * Principles of operation + * Simple content filter + + o Simple content filter example + o Simple content filter performance + o Simple content filter limitations + o Turning off the simple content filter + + * Advanced content filter + + o Advanced content filter example + o Advanced content filter performance + o Turning off the advanced content filter + + * Selective content filtering + + o Filtering mail from outside users only + o Different filters for different domains + o FILTER actions in access or header/body tables + +PPrriinncciipplleess ooff ooppeerraattiioonn + +An after-queue content filter receives unfiltered mail from Postfix (as +described further below) and can do one of the following: + + 1. Re-inject the mail back into Postfix, perhaps after changing content and/or + destination. + + 2. Discard or quarantine the mail. + + 3. Reject the mail (by sending a suitable status code back to Postfix). + Postfix will send the mail back to the sender address. + +NOTE: in this time of mail worms and forged spam, it is a VERY BAD IDEA to send +viruses back to the sender address, because the sender address is almost +certainly not the originator. It is better to discard known viruses, and to +quarantine material that is suspect so that a human can decide what to do with +it. + +SSiimmppllee ccoonntteenntt ffiilltteerr eexxaammppllee + +The first example is simple to set up, but has major limitations that will be +addressed in a second example. Postfix receives unfiltered mail from the +network with the smtpd(8) server, and delivers unfiltered mail to a content +filter with the Postfix pipe(8) delivery agent. The content filter injects +filtered mail back into Postfix with the Postfix sendmail(1) command, so that +Postfix can deliver it to the final destination. + +This means that mail submitted via the Postfix sendmail(1) command cannot be +content filtered. + +In the figure below, names followed by a number represent Postfix commands or +daemon programs. See the OVERVIEW document for an introduction to the Postfix +architecture. + + Unfiltered -> smtpd(8) qmgr(8) local(8) -> Filtered + >- cleanup(8) -> Postfix -< smtp(8) -> Filtered + pickup(8) queue pipe(8) + + ^ | + | v + + maildrop Postfix Postfix Content + queue <- postdrop <- sendmail <- filter + (1) (1) + +The content filter can be a simple shell script like this: + + 1 #!/bin/sh + 2 + 3 # Simple shell-based filter. It is meant to be invoked as follows: + 4 # /path/to/script -f sender recipients... + 5 + 6 # Localize these. The -G option does nothing before Postfix 2.3. + 7 INSPECT_DIR=/var/spool/filter + 8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here. + 9 + 10 # Exit codes from <sysexits.h> + 11 EX_TEMPFAIL=75 + 12 EX_UNAVAILABLE=69 + 13 + 14 # Clean up when done or when aborting. + 15 trap "rm -f in.$$" 0 1 2 3 15 + 16 + 17 # Start processing. + 18 cd $INSPECT_DIR || { + 19 echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; } + 20 + 21 cat >in.$$ || { + 22 echo Cannot save mail to file; exit $EX_TEMPFAIL; } + 23 + 24 # Specify your content filter here. + 25 # filter <in.$$ || { + 26 # echo Message content rejected; exit $EX_UNAVAILABLE; } + 27 + 28 $SENDMAIL "$@" <in.$$ + 29 + 30 exit $? + +Notes: + + * Line 8: The -G option says the filter output is not a local mail + submission: don't do silly things like appending the local domain name to + addresses in message headers. This option does nothing before Postfix + version 2.3. + + * Line 8: The -i option says don't stop reading input when a line contains + "." only. + + * Line 8: NEVER NEVER NEVER use the "-t" command-line option here. It will + mis-deliver mail, like sending messages from a mailing list back to the + mailing list. + + * Line 21: The idea is to first capture the message to file and then run the + content through a third-party content filter program. + + * Line 22: If the message cannot be captured to file, mail delivery is + deferred by terminating with exit status 75 (EX_TEMPFAIL). Postfix places + the message in the deferred mail queue and tries again later. + + * Line 25: You will need to specify a real content filter program here that + receives the content on standard input. + + * Line 26: If the content filter program finds a problem, the mail is bounced + by terminating with exit status 69 (EX_UNAVAILABLE). Postfix will send the + message back to the sender as undeliverable mail. + + * NOTE: in this time of mail worms and spam, it is a BAD IDEA to send known + viruses or spam back to the sender, because that address is likely to be + forged. It is safer to discard known viruses and to quarantine suspicious + content so that it can be inspected by a human being. + + * Line 28: If the content is OK, it is given as input to the Postfix sendmail + command, and the exit status of the filter command is whatever exit status + the Postfix sendmail command produces. Postfix will deliver the message as + usual. + + * Line 30: Postfix returns the exit status of the Postfix sendmail command. + +I suggest that you first run this script by hand until you are satisfied with +the results. Run it with a real message (headers+body) as input: + + % /path/to/script -f sender -- recipient... <message-file + +Once you're satisfied with the content filtering script: + + * Create a dedicated local user account called "filter". This user handles + all potentially dangerous mail content - that is why it should be a + separate account. Do not use "nobody", and most certainly do not use "root" + or "postfix". + + * Create a directory /var/spool/filter that is accessible only to the + "filter" user. This is where the content filtering script is supposed to + store its temporary files. + + * Configure Postfix to deliver mail to the content filter with the pipe(8) + delivery agent (see the pipe(8) manpage for a description of the command + syntax below). + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + filter unix - n n - 10 pipe + flags=Rq user=filter null_sender= + argv=/path/to/script -f ${sender} -- ${recipient} + + This runs up to 10 content filters in parallel. Instead of a limit of 10 + concurrent processes, use whatever process limit is feasible for your + machine. Content inspection software can gobble up a lot of system + resources, so you don't want to have too much of it running at the same + time. The empty null_sender setting is required with Postfix 2.3 and later. + + * To turn on content filtering for mail arriving via SMTP only, append "- + o content_filter=filter:dummy" to the master.cf entry that defines the + Postfix SMTP server: + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + smtp inet ...other stuff here, do not change... smtpd + -o content_filter=filter:dummy + + The "-o content_filter" line causes Postfix to add one content filter + request record to each incoming mail message, with content "filter:dummy". + This record overrides the normal mail routing and causes mail to be given + to the content filter instead. + + The content_filter configuration parameter expects a value of the form + transport:destination. The transport name specifies the first field of a + mail delivery agent definition in master.cf; the syntax of the next-hop + destination is described in the manual page of the corresponding delivery + agent. + + The meaning of an empty next-hop filter destination is version dependent. + Postfix 2.7 and later will use the recipient domain; earlier versions will + use $myhostname. Specify "default_filter_nexthop = $myhostname" for + compatibility with Postfix 2.6 or earlier, or specify a non-empty next-hop + filter destination. + + The content_filter setting has lower precedence than a FILTER action that + is specified in an access(5), header_checks(5) or body_checks(5) table. + + * Execute "ppoossttffiixx rreellooaadd" to complete the change. + +SSiimmppllee ccoonntteenntt ffiilltteerr ppeerrffoorrmmaannccee + +With the shell script as shown above you will lose a factor of four in Postfix +performance for transit mail that arrives and leaves via SMTP. You will lose +another factor in transit performance for each additional temporary file that +is created and deleted in the process of content filtering. The performance +impact is less for mail that is submitted or delivered locally, because such +deliveries are already slower than SMTP transit mail. + +SSiimmppllee ccoonntteenntt ffiilltteerr lliimmiittaattiioonnss + +The problem with content filters like the one above is that they are not very +robust. The reason is that the software does not talk a well-defined protocol +with Postfix. If the filter shell script aborts because the shell runs into +some memory allocation problem, the script will not produce a nice exit status +as defined in the file /usr/include/sysexits.h. Instead of going to the +deferred queue, mail will bounce. The same lack of robustness can happen when +the content filtering software itself runs into a resource problem. + +The simple content filter method is not suitable for content filter actions +that are invoked via header_checks or body_checks patterns. These patterns will +be applied again after mail is re-injected with the Postfix sendmail command, +resulting in a mail filtering loop. The advanced content filtering method (see +below) makes it possible to turn off header_checks or body_checks patterns for +filtered mail. + +TTuurrnniinngg ooffff tthhee ssiimmppllee ccoonntteenntt ffiilltteerr + +To turn off "simple" content filtering: + + * Edit the master.cf file, remove the "-o content_filter=filter:dummy" text + from the entry that defines the Postfix SMTP server. + + * Execute "ppoossttssuuppeerr --rr AALLLL" to remove content filter request records from + existing queue files. + + * Execute another "ppoossttffiixx rreellooaadd". + +AAddvvaanncceedd ccoonntteenntt ffiilltteerr eexxaammppllee + +The second example is more complex, but can give better performance, and is +less likely to bounce mail when the machine runs into some resource problem. +This content filter receives unfiltered mail with SMTP on localhost port 10025, +and sends filtered mail back into Postfix with SMTP on localhost port 10026. + +For non-SMTP capable content filtering software, Bennett Todd's SMTP proxy +implements a nice PERL/SMTP content filtering framework. See: https:// +web.archive.org/web/20151022025756/http://bent.latency.net/smtpprox/. + +In the figure below, names followed by a number represent Postfix commands or +daemon programs. See the OVERVIEW document for an introduction to the Postfix +architecture. + + Unfiltered -> smtpd(8) qmgr(8) smtp(8) -> Filtered + >- cleanup(8) -> Postfix -< + Unfiltered -> pickup(8) queue local(8) -> Filtered + + ^ | + | v + + smtpd(8) smtp(8) + 10026 + + ^ | + | v + + content filter 10025 + +The example given here filters all mail, including mail that arrives via SMTP +and mail that is locally submitted via the Postfix sendmail command (local +submissions enter Postfix via the pickup(8) server; to keep the figure simple +we omit local submission details). See examples near the end of this document +for how to exclude local users from filtering, or how to configure a +destination dependent content filter. + +You can expect to lose about a factor of two in Postfix performance for mail +that arrives and leaves via SMTP, provided that the content filter creates no +temporary files. Each temporary file created by the content filter adds another +factor to the performance loss. + +AAddvvaanncceedd ccoonntteenntt ffiilltteerr:: rreeqquueessttiinngg tthhaatt aallll mmaaiill iiss ffiilltteerreedd + +To enable the advanced content filter method for all mail, specify in main.cf: + + /etc/postfix/main.cf: + content_filter = scan:localhost:10025 + receive_override_options = no_address_mappings + + * The "receive_override_options" line disables address manipulation before + the content filter, so that the content filter sees the original mail + addresses instead of the result of virtual alias expansion, canonical + mapping, automatic bcc, address masquerading, etc. + + * The "content_filter" line causes Postfix to add one content filter request + record to each incoming mail message, with content "scan:localhost:10025". + The content filter request records are added by the smtpd(8) and pickup(8) + servers (and qmqpd(8) if you decide to enable this service). + + * Content filter requests are stored in queue files; this is how Postfix + keeps track of what mail needs filtering. When a queue file contains a + content filter request, the queue manager will deliver the mail to the + specified content filter regardless of its final destination. + + * The content_filter configuration parameter expects a value of the form + transport:destination. The transport name specifies the first field of a + mail delivery agent definition in master.cf; the syntax of the next-hop + destination is described in the manual page of the corresponding delivery + agent. + + * The meaning of an empty next-hop filter destination is version dependent. + Postfix 2.7 and later will use the recipient domain; earlier versions will + use $myhostname. Specify "default_filter_nexthop = $myhostname" for + compatibility with Postfix 2.6 or earlier, or specify a non-empty next-hop + filter destination. + + * The content_filter setting has lower precedence than a FILTER action that + is specified in an access(5), header_checks(5) or body_checks(5) table. + +AAddvvaanncceedd ccoonntteenntt ffiilltteerr:: sseennddiinngg uunnffiilltteerreedd mmaaiill ttoo tthhee ccoonntteenntt ffiilltteerr + +In this example, "scan" is an instance of the Postfix SMTP client with slightly +different configuration parameters. This is how one would set up the service in +the Postfix master.cf file: + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + scan unix - - n - 10 smtp + -o smtp_send_xforward_command=yes + -o disable_mime_output_conversion=yes + -o smtp_generic_maps= + + * This runs up to 10 content filters in parallel. Instead of a limit of 10 + concurrent processes, use whatever process limit is feasible for your + machine. Content inspection software can gobble up a lot of system + resources, so you don't want to have too much of it running at the same + time. + + * With "-o smtp_send_xforward_command=yes", the scan transport will try to + forward the original client name and IP address through the content filter + to the after-filter smtpd process, so that filtered mail is logged with the + real client name IP address. See smtp(8) and XFORWARD_README for more + information. + + * The "-o disable_mime_output_conversion=yes" is a workaround that prevents + the breaking of domainkeys and other digital signatures. This is needed + because some SMTP-based content filters don't announce 8BITMIME support, + even though they can handle 8-bit mail. + + * The "-o smtp_generic_maps=" is a workaround that prevents local address + rewriting with generic(5) maps. Such rewriting should happen only when mail + is sent out to the Internet. + +AAddvvaanncceedd ccoonntteenntt ffiilltteerr:: rruunnnniinngg tthhee ccoonntteenntt ffiilltteerr + +The content filter can be set up with the Postfix spawn service, which is the +Postfix equivalent of inetd. For example, to instantiate up to 10 content +filtering processes on localhost port 10025: + + /etc/postfix/master.cf: + # =================================================================== + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # =================================================================== + localhost:10025 inet n n n - 10 spawn + user=filter argv=/path/to/filter localhost 10026 + + * "filter" is a dedicated local user account. The user will never log in, and + can be given a "*" password and non-existent shell and home directory. This + user handles all potentially dangerous mail content - that is why it should + be a separate account. + + * By default, Postfix will terminate a command that runs longer than + command_time_limit seconds (default: 1000s). This is a safety measure that + prevents filters from running forever. + +If you want to have your filter listening on port localhost:10025 instead of +Postfix, then you must run your filter as a stand-alone program, and must not +use the Postfix spawn service. + +AAddvvaanncceedd ffiilltteerr:: iinnjjeeccttiinngg mmaaiill bbaacckk iinnttoo PPoossttffiixx + +The job of the content filter is to either bounce mail with a suitable +diagnostic, or to feed the mail back into Postfix through a dedicated listener +on port localhost 10026. + +The simplest content filter just copies SMTP commands and data between its +inputs and outputs. If it has a problem, all it has to do is to reply to an +input of `.' from Postfix with `550 content rejected', and to disconnect +without sending `.' on the connection that injects mail back into Postfix. + + /etc/postfix/master.cf: + # =================================================================== + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # =================================================================== + localhost:10026 inet n - n - 10 smtpd + -o content_filter= + - + o + receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters + -o smtpd_helo_restrictions= + -o smtpd_client_restrictions= + -o smtpd_sender_restrictions= + # Postfix 2.10 and later: specify empty smtpd_relay_restrictions. + -o smtpd_relay_restrictions= + -o smtpd_recipient_restrictions=permit_mynetworks,reject + -o mynetworks=127.0.0.0/8 + -o smtpd_authorized_xforward_hosts=127.0.0.0/8 + + * NOTE: do not use spaces around the "=" or "," characters. + + * NOTE: the SMTP server must not have a smaller process limit than the + "filter" master.cf entry. + + * The "-o content_filter=" overrides main.cf settings, and requests no + content filtering for mail from the content filter. This is required or + else mail will loop. + + * The "-o receive_override_options" overrides main.cf settings to avoid + duplicating work that was already done before the content filter. These + options are complementary to the options that are specified in main.cf: + + o We specify "no_unknown_recipient_checks" to disable attempts to find + out if a recipient is unknown. + + o We specify "no_header_body_checks" to disable header/body checks. + + o We specify "no_milters" to disable Milter applications (this option is + available only in Postfix 2.3 and later). + + o We don't specify "no_address_mappings" here. This enables virtual alias + expansion, canonical mappings, address masquerading, and other address + mappings after the content filter. The main.cf setting of + "receive_override_options" disables these mappings before the content + filter. + + These receive override options are either implemented by the SMTP server + itself, or they are passed on to the cleanup server. + + * The "-o smtpd_xxx_restrictions" and "-o mynetworks=127.0.0.0/8" override + main.cf settings. They turn off junk mail controls that would only waste + time here. + + * With "-o smtpd_authorized_xforward_hosts=127.0.0.0/8", the scan transport + will try to forward the original client name and IP address to the after- + filter smtpd process, so that filtered mail is logged with the real client + name and IP address. See XFORWARD_README and smtpd(8). + +AAddvvaanncceedd ccoonntteenntt ffiilltteerr ppeerrffoorrmmaannccee + +With the "sandwich" approach to content filtering described here, it is +important to match the filter concurrency to the available CPU, memory and I/ +O resources. Too few content filter processes and mail accumulates in the +active queue even with low traffic volume; too much concurrency and Postfix +ends up deferring mail destined for the content filter because processes fail +due to insufficient resources. + +Currently, content filter performance tuning is a process of trial and error; +analysis is handicapped because filtered and unfiltered messages share the same +queue. As mentioned in the introduction of this document, content filtering +with multiple Postfix instances will be covered in a future version. + +TTuurrnniinngg ooffff tthhee aaddvvaanncceedd ccoonntteenntt ffiilltteerr + +To turn off "advanced" content filtering: + + * Delete or comment out the two following main.cf lines. The other changes + made for advanced content filtering have no effect when content filtering + is turned off. + + /etc/postfix/main.cf: + content_filter = scan:localhost:10025 + receive_override_options = no_address_mappings + + * Execute "ppoossttssuuppeerr --rr AALLLL" to remove content filter request records from + existing queue files. + + * Execute another "ppoossttffiixx rreellooaadd". + +FFiilltteerriinngg mmaaiill ffrroomm oouuttssiiddee uusseerrss oonnllyy + +The easiest approach is to configure ONE Postfix instance with multiple SMTP +server IP addresses in master.cf: + + * Two SMTP server IP addresses for mail from inside users only, with content + filtering turned off. + + /etc/postfix.master.cf: + # ================================================================== + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================== + 1.2.3.4:smtp inet n - n - - smtpd + -o smtpd_client_restrictions=permit_mynetworks,reject + 127.0.0.1:smtp inet n - n - - smtpd + -o smtpd_client_restrictions=permit_mynetworks,reject + + * One SMTP server address for mail from outside users with content filtering + turned on. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + 1.2.3.5:smtp inet n - n - - smtpd + -o content_filter=filter-service:filter-destination + -o receive_override_options=no_address_mappings + +After this, you can follow the same procedure as outlined in the "advanced" or +"simple" content filtering examples above, except that you must not specify +"content_filter" or "receive_override_options" in the main.cf file. + +DDiiffffeerreenntt ffiilltteerrss ffoorr ddiiffffeerreenntt ddoommaaiinnss + +If you are an MX service provider and want to apply different content filters +for different domains, you can configure ONE Postfix instance with multiple +SMTP server IP addresses in master.cf. Each address provides a different +content filter service. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + # SMTP service for domains that are filtered with service1:dest1 + 1.2.3.4:smtp inet n - n - - smtpd + -o content_filter=service1:dest1 + -o receive_override_options=no_address_mappings + + # SMTP service for domains that are filtered with service2:dest2 + 1.2.3.5:smtp inet n - n - - smtpd + -o content_filter=service2:dest2 + -o receive_override_options=no_address_mappings + +After this, you can follow the same procedure as outlined in the "advanced" or +"simple" content filtering examples above, except that you must not specify +"content_filter" or "receive_override_options" in the main.cf file. + +Set up MX records in the DNS that route each domain to the proper SMTP server +instance. + +FFIILLTTEERR aaccttiioonnss iinn aacccceessss oorr hheeaaddeerr//bbooddyy ttaabblleess + +The above filtering configurations are static. Mail that follows a given path +is either always filtered or it is never filtered. As of Postfix 2.0 you can +also turn on content filtering on the fly. + +To turn on content filtering with an access(5) table rule: + + /etc/postfix/access: + whatever FILTER foo:bar + +To turn on content filtering with a header_checks(5) or body_checks(5) table +pattern: + + /etc/postfix/header_checks: + /whatever/ FILTER foo:bar + +You can do this in smtpd access maps as well as the cleanup server's header/ +body_checks. This feature must be used with great care: you must disable all +the UCE features in the after-filter smtpd and cleanup daemons or else you will +have a content filtering loop. + +Limitations: + + * FILTER actions from smtpd access maps and header/body_checks take + precedence over filters specified with the main.cf content_filter + parameter. + + * If a message triggers more than one filter action, only the last one takes + effect. + + * The same content filter is applied to all the recipients of a given + message. + diff --git a/README_FILES/FORWARD_SECRECY_README b/README_FILES/FORWARD_SECRECY_README new file mode 100644 index 0000000..6a1e40b --- /dev/null +++ b/README_FILES/FORWARD_SECRECY_README @@ -0,0 +1,480 @@ + TTLLSS FFoorrwwaarrdd SSeeccrreeccyy iinn PPoossttffiixx + +------------------------------------------------------------------------------- + +WWaarrnniinngg + +Forward secrecy does not protect against active attacks such as forged DNS +replies or forged TLS server certificates. If such attacks are a concern, then +the SMTP client will need to authenticate the remote SMTP server in a +sufficiently-secure manner. For example, by the fingerprint of a (CA or leaf) +public key or certificate. Conventional PKI relies on many trusted parties and +is easily subverted by a state-funded adversary. + +OOvveerrvviieeww + +Postfix supports forward secrecy of TLS network communication since version +2.2. This support was adopted from Lutz Ja"nicke's "Postfix TLS patch" for +earlier Postfix versions. This document will focus on TLS Forward Secrecy in +the Postfix SMTP client and server. See TLS_README for a general description of +Postfix TLS support. + +Topics covered in this document: + + * Give me some background on forward secrecy in Postfix + + o What is Forward Secrecy + o Forward Secrecy in TLS + o Forward Secrecy in the Postfix SMTP Server + o Forward Secrecy in the Postfix SMTP Client + + * Never mind, just show me what it takes to get forward secrecy + + o Getting started, quick and dirty + o How can I see that a connection has forward secrecy? + o What ciphers provide forward secrecy? + o What do "Anonymous", "Untrusted", etc. in Postfix logging mean? + + * Credits + +WWhhaatt iiss FFoorrwwaarrdd SSeeccrreeccyy + +The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy") is used to +describe security protocols in which the confidentiality of past traffic is not +compromised when long-term keys used by either or both sides are later +disclosed. + +Forward secrecy is accomplished by negotiating session keys using per-session +cryptographically-strong random numbers that are not saved, and signing the +exchange with long-term authentication keys. Later disclosure of the long-term +keys allows impersonation of the key holder from that point on, but not +recovery of prior traffic, since with forward secrecy, the discarded random key +agreement inputs are not available to the attacker. + +Forward secrecy is only "perfect" when brute-force attacks on the key agreement +algorithm are impractical even for the best-funded adversary and the random- +number generators used by both parties are sufficiently strong. Otherwise, +forward secrecy leaves the attacker with the challenge of cracking the key- +agreement protocol, which is likely quite computationally intensive, but may be +feasible for sessions of sufficiently high value. Thus forward secrecy places +cost constraints on the efficacy of bulk surveillance, recovering all past +traffic is generally infeasible, and even recovery of individual sessions may +be infeasible given a sufficiently-strong key agreement method. + +FFoorrwwaarrdd SSeeccrreeccyy iinn TTLLSS + +Early implementations of the SSL protocol do not provide forward secrecy (some +provide it only with artificially-weakened "export" cipher suites, but we will +ignore those here). The client sends a random "pre-master secret" to the server +encrypted with the server's RSA public key. The server decrypts this with its +private key, and uses it together with other data exchanged in the clear to +generate the session key. An attacker with access to the server's private key +can perform the same computation at any later time. + +Later revisions to the TLS protocol introduced forward-secrecy cipher suites in +which the client and server implement a key exchange protocol based on +ephemeral secrets. Sessions encrypted with one of these newer cipher suites are +not compromised by future disclosure of long-term authentication keys. + +The key-exchange algorithms used for forward secrecy require the TLS server to +designate appropriate "parameters" consisting of a mathematical "group" and an +element of that group called a "generator". Presently, there are two flavors of +"groups" that work with PFS: + + * FFFFDDHHEE:: Finite-field Diffie-Hellman ephemeral key exchange groups (also EDH + or DHE). The server needs to be configured with a suitably-large prime and + a corresponding "generator". Standard choices of the prime and generator + are specified in RFC7919, and can be used in the TLS 1.3 protocol with the + server and client negotiating a mutually supported choice. In earlier + versions of TLS (1.0 through 1.2), when FFDHE key exchange is performed, + the server chooses the prime and generator unilaterally. + + * EEEECCDDHH:: This is short for Ephemeral Elliptic Curve Diffie-Hellman (also + abbreviated as ECDHE). EECDH offers better security at lower computational + cost than FFDHE. Elliptic curves used in cryptography are typically + identified by a "name" that stands for a set of well-known parameter + values, and it is these "named curves" (or, in certificates, associated + ASN.1 object identifiers) that are used in the TLS protocol. When EECDH key + exchange is used, a mutually supported named curve is negotiated as part of + the TLS handshake. + +FFoorrwwaarrdd SSeeccrreeccyy iinn tthhee PPoossttffiixx SSMMTTPP SSeerrvveerr + +The Postfix >= 2.2 SMTP server supports forward secrecy in its default +configuration. If the remote SMTP client prefers cipher suites with forward +secrecy, then the traffic between the server and client will resist decryption +even if the server's long-term authentication keys are later compromised. + +Most remote SMTP clients now support forward secrecy (the only choice as of TLS +1.3), but some may prefer cipher suites without forward secrecy. Postfix >= 2.8 +servers can be configured to override the client's preference by setting +"tls_preempt_cipherlist = yes". + +FFFFDDHHEE SSeerrvveerr ssuuppppoorrtt + +Postfix >= 3.1 supports 2048-bit-prime FFDHE out of the box, with no additional +configuration. You can also generate your own FFDHE parameters, but this is not +necessary and no longer recommended. See the quick-start section for details. + +Postfix >= 3.8 supports the finite-field Diffie-Hellman ephemeral (FFDHE) key +exchange group negotiation API of OpenSSL >= 3.0. FFDHE groups are explicitly +negotiated between client and server starting with TLS 1.3. In earlier TLS +versions, the server chooses the group unilaterally. The list of candidate +FFDHE groups can be configured via "tls_ffdhe_auto_groups", which can be used +to select a prioritized list of supported groups (most preferred first) on both +the server and client. The default list is suitable for most users. Either, but +not both of "tls_eecdh_auto_curves" and "tls_ffdhe_auto_groups" may be set +empty, disabling either EC or FFDHE key exchange in OpenSSL 3.0 with TLS 1.3. +That said, interoperability will be poor if the EC curves are all disabled or +don't include the most widely used curves. + +EEEECCDDHH SSeerrvveerr ssuuppppoorrtt + +As of Postfix 3.2 and OpenSSL 1.0.2, a range of supported EECDH curves is +enabled in the server and client, and a suitable mutually supported curve is +negotiated as part of the TLS handshake. The list of supported curves is +configurable via the "tls_eecdh_auto_curves" parameter. With TLS 1.2 the server +needs to leave its setting of "smtpd_tls_eecdh_grade" at the default value of +"auto" (earlier choices of an explicit single curve grade are deprecated). With +TLS 1.3, the "smtpd_tls_eecdh_grade" parameter is not used, and curve selection +is unconditionally negotiated. + +FFoorrwwaarrdd SSeeccrreeccyy iinn tthhee PPoossttffiixx SSMMTTPP CClliieenntt + +The Postfix >= 2.2 SMTP client supports forward secrecy in its default +configuration. All supported OpenSSL releases support both FFDHE and EECDH key +exchange. If the remote SMTP server supports cipher suites with forward secrecy +(and does not override the SMTP client's cipher preference), then the traffic +between the server and client will resist decryption even if the server's long- +term authentication keys are later compromised. Forward secrecy is always on in +TLS 1.3. + +Postfix >= 3.2 supports the curve negotiation API of OpenSSL >= 1.0.2. The list +of candidate curves can be changed via the "tls_eecdh_auto_curves" +configuration parameter, which can be used to select a prioritized list of +supported curves (most preferred first) on both the Postfix SMTP server and +SMTP client. The default list is suitable for most users. + +Postfix >= 3.8 supports the finite-field Diffie-Hellman ephemeral (FFDHE) key +exchange group negotiation API of OpenSSL >= 3.0. The list of candidate FFDHE +groups can be configured via "tls_ffdhe_auto_groups", which can be used to +select a prioritized list of supported groups (most preferred first) on both +the server and client. The default list is suitable for most users. + +The default Postfix SMTP client cipher lists are correctly ordered to prefer +EECDH and FFDHE cipher suites ahead of similar cipher suites that don't +implement forward secrecy. Administrators are strongly discouraged from +changing the cipher list definitions. + +GGeettttiinngg ssttaarrtteedd,, qquuiicckk aanndd ddiirrttyy + +EEEECCDDHH CClliieenntt ssuuppppoorrtt ((PPoossttffiixx >>== 33..22 wwiitthh OOppeennSSSSLL >>== 11..11..11)) + +This works "out of the box" with no need for additional configuration. + +Postfix >= 3.2 supports the curve negotiation API of OpenSSL >= 1.0.2. The list +of candidate curves can be changed via the "tls_eecdh_auto_curves" +configuration parameter, which can be used to select a prioritized list of +supported curves (most preferred first) on both the Postfix SMTP server and +SMTP client. The default list is suitable for most users. + +EEEECCDDHH SSeerrvveerr ssuuppppoorrtt ((PPoossttffiixx >>== 33..22 wwiitthh OOppeennSSSSLL >>== 11..11..11)) + +This works "out of the box" with no need for additional configuration. + +Postfix >= 3.2 supports the curve negotiation API of OpenSSL >= 1.0.2. The list +of candidate curves can be changed via the "tls_eecdh_auto_curves" +configuration parameter, which can be used to select a prioritized list of +supported curves (most preferred first) on both the Postfix SMTP server and +SMTP client. The default list is suitable for most users. + +FFFFDDHHEE CClliieenntt ssuuppppoorrtt ((PPoossttffiixx >>== 33..22,, OOppeennSSSSLL >>== 11..11..11)) + +In Postfix < 3.8, or OpenSSL prior to 3.0, FFDHE for TLS 1.2 or below works +"out of the box", no additional configuration is necessary. The most one can do +is (not advisable) disable all "kDHE" ciphers, which would then disable FFDHE +key exchange in TLS 1.2 and below. + +With OpenSSL 1.1.1, FFDHE is not supported for TLS 1.3, which uses only EECDH +key exchange. Support for FFDHE with TLS 1.3 was added in OpenSSL 3.0. With +OpenSSL 3.0 and Postfix 3.8 the list of supported TLS 1.3 FFDHE groups becomes +configurable via the "tls_ffdhe_auto_groups" parameter, which can be set empty +to disable FFDHE in TLS 1.3, or conversely expanded to support more groups. The +default should work well for most users. + +FFFFDDHHEE SSeerrvveerr ssuuppppoorrtt ((PPoossttffiixx >>== 22..22,, aallll ssuuppppoorrtteedd OOppeennSSSSLL vveerrssiioonnss)) + +In Postfix < 3.8, or OpenSSL prior to 3.0, FFDHE for TLS 1.2 or below works +"out of the box", no additional configuration is necessary. One can of course +(not advisable) disable all "kDHE" ciphers, which would then disable FFDHE key +exchange in TLS 1.2 and below. + +The built-in default Postfix FFDHE group is a 2048-bit group as of Postfix 3.1. +You can optionally generate non-default Postfix SMTP server FFDHE parameters +for possibly improved security against pre-computation attacks, but this is not +necessary or recommended. Just leave "smtpd_tls_dh1024_param_file" at its +default empty value. + +The set of FFDHE groups enabled for use with TLS 1.3 becomes configurable with +Postfix >= 3.8 and OpenSSL >= 3.0. The default setting of +"tls_ffdhe_auto_groups" enables the RFC7919 2048 and 3072-bit groups. If you +need more security, you should probably be using EECDH. + +HHooww ccaann II sseeee tthhaatt aa ccoonnnneeccttiioonn hhaass ffoorrwwaarrdd sseeccrreeccyy?? + +Postfix can be configured to report information about the negotiated cipher, +the corresponding key lengths, and the remote peer certificate or public-key +verification status. + + * With "smtp_tls_loglevel = 1" and "smtpd_tls_loglevel = 1", the Postfix SMTP + client and server will log TLS connection information to the maillog file. + The general logfile format is shown below. With TLS 1.3 there may be + additional properties logged after the cipher name and bits. + + postfix/smtp[process-id]: Untrusted TLS connection established + to host.example.com[192.168.0.2]:25: TLSv1 with cipher cipher-name + (actual-key-size/raw-key-size bits) + + postfix/smtpd[process-id]: Anonymous TLS connection established + from host.example.com[192.168.0.2]: TLSv1 with cipher cipher-name + (actual-key-size/raw-key-size bits) + + * With "smtpd_tls_received_header = yes", the Postfix SMTP server will record + TLS connection information in the Received: header in the form of comments + (text inside parentheses). The general format depends on the + smtpd_tls_ask_ccert setting. With TLS 1.3 there may be additional + properties logged after the cipher name and bits. + + Received: from host.example.com (host.example.com [192.168.0.2]) + (using TLSv1 with cipher cipher-name + (actual-key-size/raw-key-size bits)) + (Client CN "host.example.com", Issuer "John Doe" (not + verified)) + + Received: from host.example.com (host.example.com [192.168.0.2]) + (using TLSv1 with cipher cipher-name + (actual-key-size/raw-key-size bits)) + (No client certificate requested) + + TLS 1.3 examples. Some of the new attributes may not appear when not + applicable or not available in older versions of the OpenSSL library. + + Received: from localhost (localhost [127.0.0.1]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 + bits) + key-exchange X25519 server-signature RSA-PSS (2048 bits) + server-digest SHA256) + (No client certificate requested) + + Received: from localhost (localhost [127.0.0.1]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 + bits) + key-exchange X25519 server-signature RSA-PSS (2048 bits) + server-digest SHA256 + client-signature ECDSA (P-256) client-digest SHA256) + (Client CN "example.org", Issuer "example.org" (not verified)) + + o The "key-exchange" attribute records the type of "Diffie-Hellman" group + used for key agreement. Possible values include "DHE", "ECDHE", + "X25519" and "X448". With "DHE", the bit size of the prime will be + reported in parentheses after the algorithm name, with "ECDHE", the + curve name. + + o The "server-signature" attribute shows the public key signature + algorithm used by the server. With "RSA-PSS", the bit size of the + modulus will be reported in parentheses. With "ECDSA", the curve name. + If, for example, the server has both an RSA and an ECDSA private key + and certificate, it will be possible to track which one was used for a + given connection. + + o The new "server-digest" attribute records the digest algorithm used by + the server to prepare handshake messages for signing. The Ed25519 and + Ed448 signature algorithms do not make use of such a digest, so no + "server-digest" will be shown for these signature algorithms. + + o When a client certificate is requested with "smtpd_tls_ask_ccert" and + the client uses a TLS client-certificate, the "client-signature" and + "client-digest" attributes will record the corresponding properties of + the client's TLS handshake signature. + +The next sections will explain what cipher-name, key-size, and peer +verification status information to expect. + +WWhhaatt cciipphheerrss pprroovviiddee ffoorrwwaarrdd sseeccrreeccyy?? + +There are dozens of ciphers that support forward secrecy. What follows is the +beginning of a list of 51 ciphers available with OpenSSL 1.0.1e. The list is +sorted in the default Postfix preference order. It excludes null ciphers that +only authenticate and don't encrypt, together with export and low-grade ciphers +whose encryption is too weak to offer meaningful secrecy. The first column +shows the cipher name, and the second shows the key exchange method. + + $ openssl ciphers -v \ + 'aNULL:-aNULL:kEECDH:kEDH:+RC4:!eNULL:!EXPORT:!LOW:@STRENGTH' | + awk '{printf "%-32s %s\n", $1, $3}' + AECDH-AES256-SHA Kx=ECDH + ECDHE-RSA-AES256-GCM-SHA384 Kx=ECDH + ECDHE-ECDSA-AES256-GCM-SHA384 Kx=ECDH + ECDHE-RSA-AES256-SHA384 Kx=ECDH + ECDHE-ECDSA-AES256-SHA384 Kx=ECDH + ECDHE-RSA-AES256-SHA Kx=ECDH + ECDHE-ECDSA-AES256-SHA Kx=ECDH + ADH-AES256-GCM-SHA384 Kx=DH + ADH-AES256-SHA256 Kx=DH + ADH-AES256-SHA Kx=DH + ADH-CAMELLIA256-SHA Kx=DH + DHE-DSS-AES256-GCM-SHA384 Kx=DH + DHE-RSA-AES256-GCM-SHA384 Kx=DH + DHE-RSA-AES256-SHA256 Kx=DH + ... + +To date, all ciphers that support forward secrecy have one of five values for +the first component of their OpenSSL name: "AECDH", "ECDHE", "ADH", "EDH" or +"DHE". Ciphers that don't implement forward secrecy have names that don't start +with one of these prefixes. This pattern is likely to persist until some new +key-exchange mechanism is invented that also supports forward secrecy. + +The actual key length and raw algorithm key length are generally the same with +non-export ciphers, but they may differ for the legacy export ciphers where the +actual key is artificially shortened. + +Starting with TLS 1.3 the cipher name no longer contains enough information to +determine which forward-secrecy scheme was employed, but TLS 1.3 aallwwaayyss uses +forward-secrecy. On the client side, up-to-date Postfix releases log additional +information for TLS 1.3 connections, reporting the signature and key exchange +algorithms. Two examples below (the long single line messages are folded across +multiple lines for readability): + + postfix/smtp[process-id]: + Untrusted TLS connection established to 127.0.0.1[127.0.0.1]:25: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest + SHA256 + client-signature ECDSA (P-256) client-digest SHA256 + + postfix/smtp[process-id]: + Untrusted TLS connection established to 127.0.0.1[127.0.0.1]:25: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + key-exchange ECDHE (P-256) server-signature ECDSA (P-256) server-digest + SHA256 + +In the above connections, the "key-exchange" value records the "Diffie-Hellman" +algorithm used for key agreement. The "server-signature" value records the +public key algorithm used by the server to sign the key exchange. The "server- +digest" value records any hash algorithm used to prepare the data for signing. +With "ED25519" and "ED448", no separate hash algorithm is used. + +Examples of Postfix SMTP server logging: + + postfix/smtpd[process-id]: + Untrusted TLS connection established from localhost[127.0.0.1]:25: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest + SHA256 + client-signature ECDSA (P-256) client-digest SHA256 + + postfix/smtpd[process-id]: + Anonymous TLS connection established from localhost[127.0.0.1]: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + server-signature RSA-PSS (2048 bits) server-digest SHA256 + + postfix/smtpd[process-id]: + Anonymous TLS connection established from localhost[127.0.0.1]: + TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) + server-signature ED25519 + +Note that Postfix >= 3.4 server logging may also include a "to sni-name" +element to record the use of an alternate server certificate chain for the +connection in question. This happens when the client uses the TLS SNI +extension, and the server selects a non-default certificate chain based on the +client's SNI value: + + postfix/smtpd[process-id]: + Untrusted TLS connection established from client.example[192.0.2.1] + to server.example: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 + bits) + key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest + SHA256 + client-signature ECDSA (P-256) client-digest SHA256 + +WWhhaatt ddoo ""AAnnoonnyymmoouuss"",, ""UUnnttrruusstteedd"",, eettcc.. iinn PPoossttffiixx llooggggiinngg mmeeaann?? + +The verification levels below are subject to man-in-the-middle attacks to +different degrees. If such attacks are a concern, then the SMTP client will +need to authenticate the remote SMTP server in a sufficiently-secure manner. +For example, by the fingerprint of a (CA or leaf) public key or certificate. +Remember that conventional PKI relies on many trusted parties and is easily +subverted by a state-funded adversary. + +AAnnoonnyymmoouuss (no peer certificate) + PPoossttffiixx SSMMTTPP cclliieenntt:: With opportunistic TLS (the "may" security level) the + Postfix SMTP client does not verify any information in the peer + certificate. In this case it enables and prefers anonymous cipher suites in + which the remote SMTP server does not present a certificate (these ciphers + offer forward secrecy of necessity). When the remote SMTP server also + supports anonymous TLS, and agrees to such a cipher suite, the verification + status will be logged as "Anonymous". + + PPoossttffiixx SSMMTTPP sseerrvveerr:: This is by far most common, as client certificates are + optional, and the Postfix SMTP server does not request client certificates + by default (see smtpd_tls_ask_ccert). Even when client certificates are + requested, the remote SMTP client might not send a certificate. Unlike the + Postfix SMTP client, the Postfix SMTP server "anonymous" verification + status does not imply that the cipher suite is anonymous, which corresponds + to the server not sending a certificate. + +UUnnttrruusstteedd (peer certificate not signed by trusted CA) + PPoossttffiixx SSMMTTPP cclliieenntt:: The remote SMTP server presented a certificate, but + the Postfix SMTP client was unable to check the issuing CA signature. With + opportunistic TLS this is common with remote SMTP servers that don't + support anonymous cipher suites. + + PPoossttffiixx SSMMTTPP sseerrvveerr:: The remote SMTP client presented a certificate, but + the Postfix SMTP server was unable to check the issuing CA signature. This + can happen when the server is configured to request client certificates + (see smtpd_tls_ask_ccert). + +TTrruusstteedd (peer certificate signed by trusted CA, unverified peer name) + PPoossttffiixx SSMMTTPP cclliieenntt:: The remote SMTP server's certificate was signed by a + CA that the Postfix SMTP client trusts, but either the client was not + configured to verify the destination server name against the certificate, + or the server certificate did not contain any matching names. This is + common with opportunistic TLS (smtp_tls_security_level is "may" or else + "dane" with no usable TLSA DNS records) when the Postfix SMTP client's + trusted CAs can verify the authenticity of the remote SMTP server's + certificate, but the client is not configured or unable to verify the + server name. + + PPoossttffiixx SSMMTTPP sseerrvveerr:: The remote SMTP client certificate was signed by a CA + that the Postfix SMTP server trusts. The Postfix SMTP server never verifies + the remote SMTP client name against the names in the client certificate. + Since the client chooses to connect to the server, the Postfix SMTP server + has no expectation of a particular client hostname. + +VVeerriiffiieedd (peer certificate signed by trusted CA and verified peer name; or: +peer certificate with expected public-key or certificate fingerprint) + PPoossttffiixx SSMMTTPP cclliieenntt:: The remote SMTP server's certificate was signed by a + CA that the Postfix SMTP client trusts, and the certificate name matches + the destination or server name(s). The Postfix SMTP client was configured + to require a verified name, otherwise the verification status would have + been just "Trusted". + + PPoossttffiixx SSMMTTPP cclliieenntt:: The "Verified" status may also mean that the Postfix + SMTP client successfully matched the expected fingerprint against the + remote SMTP server public key or certificate. The expected fingerprint may + come from smtp_tls_policy_maps or from TLSA (secure) DNS records. The + Postfix SMTP client ignores the CA signature. + + PPoossttffiixx SSMMTTPP sseerrvveerr:: The status is never "Verified", because the Postfix + SMTP server never verifies the remote SMTP client name against the names in + the client certificate, and because the Postfix SMTP server does not expect + a specific fingerprint in the client public key or certificate. + +CCrreeddiittss + + * TLS support for Postfix was originally developed by Lutz Ja"nicke at + Cottbus Technical University. + * Wietse Venema adopted and restructured the code and documentation. + * Viktor Dukhovni implemented support for many subsequent TLS features, + including EECDH, and authored the initial version of this document. + diff --git a/README_FILES/INSTALL b/README_FILES/INSTALL new file mode 100644 index 0000000..09d0f80 --- /dev/null +++ b/README_FILES/INSTALL @@ -0,0 +1,1166 @@ +PPoossttffiixx IInnssttaallllaattiioonn FFrroomm SSoouurrccee CCooddee + +------------------------------------------------------------------------------- + +11 -- PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +If you are using a pre-compiled version of Postfix, you should start with +BASIC_CONFIGURATION_README and the general documentation referenced by it. +INSTALL is only a bootstrap document to get Postfix up and running from scratch +with the minimal number of steps; it should not be considered part of the +general documentation. + +This document describes how to build, install and configure a Postfix system so +that it can do one of the following: + + * Send mail only, without changing an existing Sendmail installation. + * Send and receive mail via a virtual host interface, still without any + change to an existing Sendmail installation. + * Run Postfix instead of Sendmail. + +Topics covered in this document: + + 1. Purpose of this document + 2. Typographical conventions + 3. Documentation + 4. Building on a supported system + 5. Porting Postfix to an unsupported system + 6. Installing the software after successful compilation + 7. Configuring Postfix to send mail only + 8. Configuring Postfix to send and receive mail via virtual interface + 9. Running Postfix instead of Sendmail +10. Mandatory configuration file edits +11. To chroot or not to chroot +12. Care and feeding of the Postfix system + +22 -- TTyyppooggrraapphhiiccaall ccoonnvveennttiioonnss + +In the instructions below, a command written as + + # command + +should be executed as the superuser. + +A command written as + + $ command + +should be executed as an unprivileged user. + +33 -- DDooccuummeennttaattiioonn + +Documentation is available as README files (start with the file README_FILES/ +AAAREADME), as HTML web pages (point your browser to "html/index.html") and as +UNIX-style manual pages. + +You should view the README files with a pager such as more(1) or less(1), +because the files use backspace characters in order to produce bboolldd font. To +print a README file without backspace characters, use the col(1) command. For +example: + + $ col -bx <file | lpr + +In order to view the manual pages before installing Postfix, point your MANPATH +environment variable to the "man" subdirectory; be sure to use an absolute +path. + + $ export MANPATH; MANPATH="`pwd`/man:$MANPATH" + $ setenv MANPATH "`pwd`/man:$MANPATH" + +Of particular interest is the postconf(5) manual page that lists all the 500+ +configuration parameters. The HTML version of this text makes it easy to +navigate around. + +All Postfix source files have their own built-in manual page. Tools to extract +those embedded manual pages are available in the mantools directory. + +44 -- BBuuiillddiinngg oonn aa ssuuppppoorrtteedd ssyysstteemm + +Postfix development happens on FreeBSD and MacOS X, with regular tests on Linux +(Fedora, Ubuntu) and Solaris. Support for other systems relies on feedback from +their users, and may not always be up-to-date. + +OpenBSD is partially supported. The libc resolver does not implement the +documented "internal resolver options which are [...] set by changing fields in +the _res structure" (documented in the OpenBSD 5.6 resolver(3) manpage). This +results in too many DNS queries, and false positives for queries that should +fail. + +Overview of topics: + + * 4.1 - Getting started + * 4.2 - What compiler to use + * 4.3 - Building with Postfix position-independent executables (Postfix >= + 3.0) + * 4.4 - Building with Postfix dynamically-linked libraries and database + plugins (Postfix >= 3.0) + * 4.5 - Building with optional features + * 4.6 - Overriding built-in parameter default settings + * 4.7 - Overriding other compile-time features + * 4.8 - Support for thousands of processes + * 4.9 - Compiling Postfix, at last + +44..11 -- GGeettttiinngg ssttaarrtteedd + +On Solaris, the "make" command and other development utilities are in /usr/ccs/ +bin, so you MUST have /usr/ccs/bin in your command search path. If these files +do not exist, you need to install the development packages first. + +If you need to build Postfix for multiple architectures from a single source- +code tree, use the "lndir" command to build a shadow tree with symbolic links +to the source files. + +If at any time in the build process you get messages like: "make: don't know +how to ..." you should be able to recover by running the following command from +the Postfix top-level directory: + + $ make -f Makefile.init makefiles + +If you copied the Postfix source code after building it on another machine, it +is a good idea to cd into the top-level directory and first do this: + + $ make tidy + +This will get rid of any system dependencies left over from compiling the +software elsewhere. + +44..22 -- WWhhaatt ccoommppiilleerr ttoo uussee + +To build with GCC, or with the native compiler if people told me that is better +for your system, just cd into the top-level Postfix directory of the source +tree and type: + + $ make + +To build with a non-default compiler, you need to specify the name of the +compiler. Here are a few examples: + + $ make makefiles CC=/opt/SUNWspro/bin/cc (Solaris) + $ make + + $ make makefiles CC="/opt/ansic/bin/cc -Ae" (HP-UX) + $ make + + $ make makefiles CC="purify cc" + $ make + +and so on. In some cases, optimization will be turned off automatically. + +44..33 -- BBuuiillddiinngg wwiitthh PPoossttffiixx ppoossiittiioonn--iinnddeeppeennddeenntt eexxeeccuuttaabblleess ((PPoossttffiixx >>== 33..00)) + +On some systems Postfix can be built with Position-Independent Executables. PIE +is used by the ASLR exploit mitigation technique (ASLR = Address-Space Layout +Randomization): + + $ make makefiles pie=yes ...other arguments... + +(Specify "make makefiles pie=no" to explicitly disable Postfix position- +independent executable support). + +Postfix PIE support appears to work on Fedora Core 20, Ubuntu 14.04, FreeBSD 9 +and 10, and NetBSD 6 (all with the default system compilers). + +Whether the "pie=yes" above has any effect depends on the compiler. Some +compilers always produce PIE executables, and some may even complain that the +Postfix build option is redundant. + +44..44 -- BBuuiillddiinngg wwiitthh PPoossttffiixx ddyynnaammiiccaallllyy--lliinnkkeedd lliibbrraarriieess aanndd ddaattaabbaassee pplluuggiinnss +((PPoossttffiixx >>== 33..00)) + +Postfix dynamically-linked library and database plugin support exists for +recent versions of Linux, FreeBSD and MacOS X. Dynamically-linked library +builds may become the default at some point in the future. + +Overview of topics: + + * 4.4.1 Turning on Postfix dynamically-linked library support + * 4.4.2 Turning on Postfix database-plugin support + * 4.4.3 Customizing Postfix dynamically-linked libraries and database plugins + * 4.4.4 Tips for distribution maintainers + +Note: directories with Postfix dynamically-linked libraries or database plugins +should contain only postfix-related files. Postfix dynamically-linked libraries +and database plugins should not be installed in a "public" system directory +such as /usr/lib or /usr/local/lib. Linking Postfix dynamically-linked library +or database-plugin files into non-Postfix programs is not supported. Postfix +dynamically-linked libraries and database plugins implement a Postfix-internal +API that changes without maintaining compatibility. + +44..44..11 TTuurrnniinngg oonn PPoossttffiixx ddyynnaammiiccaallllyy--lliinnkkeedd lliibbrraarryy ssuuppppoorrtt + +Postfix can be built with Postfix dynamically-linked libraries (files typically +named libpostfix-*.so). Postfix dynamically-linked libraries add minor run-time +overhead and result in significantly-smaller Postfix executable files. + +Specify "shared=yes" on the "make makefiles" command line to build Postfix with +dynamically-linked library support. + + $ make makefiles shared=yes ...other arguments... + $ make + +(Specify "make makefiles shared=no" to explicitly disable Postfix dynamically- +linked library support). + +This installs dynamically-linked libraries in $shlib_directory, typically /usr/ +lib/postfix or /usr/local/lib/postfix, with file names libpostfix-name.so, +where the name is a source-code directory name such as "util" or "global". + +See section 4.4.3 "Customizing Postfix dynamically-linked libraries and +database plugins" below for how to customize the Postfix dynamically-linked +library location, including support to upgrade a running mail system safely. + +44..44..22 TTuurrnniinngg oonn PPoossttffiixx ddaattaabbaassee--pplluuggiinn ssuuppppoorrtt + +Additionally, Postfix can be built to support dynamic loading of Postfix +database clients (database plugins) with the Debian-style dynamicmaps feature. +Postfix 3.0 supports dynamic loading of cdb:, ldap:, lmdb:, mysql:, pcre:, +pgsql:, sdbm:, and sqlite: database clients. Dynamic loading is useful when you +distribute or install pre-compiled Postfix packages. + +Specify "dynamicmaps=yes" on the "make makefiles" command line to build Postfix +with support to dynamically load Postfix database clients with the Debian-style +dynamicmaps feature. + + $ make makefiles dynamicmaps=yes ...other arguments... + $ make + +(Specify "make makefiles dynamicmaps=no" to explicitly disable Postfix +database-plugin support). + +This implicitly enables dynamically-linked library support, installs the +configuration file dynamicmaps.cf in $meta_directory (usually, /etc/postfix or +/usr/local/etc/postfix), and installs database plugins in $shlib_directory (see +above). Database plugins are named postfix-type.so where the type is a database +type such as "cdb" or "ldap". + + NOTE: The Postfix 3.0 build procedure expects that you specify database + library dependencies with variables named AUXLIBS_CDB, AUXLIBS_LDAP, etc. + With Postfix 3.0 and later, the old AUXLIBS variable still supports + building a statically-loaded database client, but only the new AUXLIBS_CDB + etc. variables support building a dynamically-loaded or statically-loaded + CDB etc. database client. See CDB_README, LDAP_README, etc. for details. + + Failure to follow this advice will defeat the purpose of dynamic database + client loading. Every Postfix executable file will have database library + dependencies. And that was exactly what dynamic database client loading was + meant to avoid. + +See the next section for how to customize the location and version of Postfix +database plugins and the location of the file dynamicmaps.cf. + +44..44..33 CCuussttoommiizziinngg PPoossttffiixx ddyynnaammiiccaallllyy--lliinnkkeedd lliibbrraarriieess aanndd ddaattaabbaassee pplluuggiinnss + +CCuussttoommiizziinngg bbuuiilldd--ttiimmee aanndd rruunn--ttiimmee ooppttiioonnss ffoorr PPoossttffiixx ddyynnaammiiccaallllyy--lliinnkkeedd +lliibbrraarriieess aanndd ddaattaabbaassee pplluuggiinnss + +The build-time environment variables SHLIB_CFLAGS, SHLIB_RPATH, and +SHLIB_SUFFIX provide control over how Postfix libraries and plugins are +compiled, linked, and named. + + $ make makefiles SHLIB_CFLAGS=flags SHLIB_RPATH=rpath SHLIB_SUFFIX=suffix + ...other arguments... + $ make + +See section 4.7 "Overriding other compile-time features" below for details. + +CCuussttoommiizziinngg tthhee llooccaattiioonn ooff PPoossttffiixx ddyynnaammiiccaallllyy--lliinnkkeedd lliibbrraarriieess aanndd ddaattaabbaassee +pplluuggiinnss + +As a reminder, the directories with Postfix dynamically-linked libraries or +database plugins should contain only Postfix-related files. Linking these files +into other programs is not supported. + +To override the default location of Postfix dynamically-linked libraries and +database plugins specify, for example: + + $ make makefiles shared=yes shlib_directory=/usr/local/lib/postfix ... + +If you intend to upgrade Postfix without stopping the mail system, then you +should append the Postfix release version to the shlib_directory pathname, to +eliminate the possibility that programs will link with dynamically-linked +libraries or database plugins from the wrong Postfix version. For example: + + $ make makefiles shared=yes \ + shlib_directory=/usr/local/lib/postfix/MAIL_VERSION ... + +The command "make makefiles name=value..." will replace the string MAIL_VERSION +at the end of a configuration parameter value with the Postfix release version. +Do not try to specify something like $mail_version on this command line. This +produces inconsistent results with different versions of the make(1) command. + +You can change the shlib_directory setting after Postfix is built, with "make +install" or "make upgrade". However, you may have to run ldconfig if you change +shlib_directory after Postfix is built (the symptom is that Postfix programs +fail because the run-time linker cannot find the files libpostfix-*.so). No +ldconfig command is needed if you keep the files libpostfix-*.so in the +compiled-in default $shlib_directory location. + + # make upgrade shlib_directory=/usr/local/lib/postfix ... + # make install shlib_directory=/usr/local/lib/postfix ... + +To append the Postfix release version to the pathname if you intend to upgrade +Postfix without stopping the mail system: + + # make upgrade shlib_directory=/usr/local/lib/postfix/MAIL_VERSION ... + # make install shlib_directory=/usr/local/lib/postfix/MAIL_VERSION ... + +See also the comments above for appending MAIL_VERSION with the "make +makefiles" command. + +CCuussttoommiizziinngg tthhee llooccaattiioonn ooff ddyynnaammiiccmmaappss..ccff aanndd ootthheerr ffiilleess + +The meta_directory parameter has the same default setting as the +config_directory parameter, typically /etc/postfix or /usr/local/etc/postfix. + +You can override the default meta_directory location at compile time or after +Postfix is built. To override the default location at compile time specify, for +example: + + % make makefiles meta_directory=/usr/libexec/postfix ... + +Here is a tip if you want to make a pathname dependent on the Postfix release +version: the command "make makefiles name=value..." will replace the string +MAIL_VERSION at the end of a configuration parameter value with the Postfix +release version. Do not try to specify something like $mail_version on this +command line. This produces inconsistent results with different versions of the +make(1) command. + +You can override the meta_directory setting after Postfix is built, with "make +install" or "make upgrade". + + # make upgrade meta_directory=/usr/libexec/postfix ... + # make install meta_directory=/usr/libexec/postfix ... + +As with the command "make makefiles", the command "make install/upgrade +name=value..." will replace the string MAIL_VERSION at the end of a +configuration parameter value with the Postfix release version. Do not try to +specify something like $mail_version on this command line. This produces +inconsistent results with different versions of the make(1) command. + +44..44..44 TTiippss ffoorr ddiissttrriibbuuttiioonn mmaaiinnttaaiinneerrss + + * The shlib_directory parameter setting also provides the default directory + for database plugin files with a relative pathname in the file + dynamicmaps.cf. + + * The meta_directory parameter specifies the location of the files + dynamicmaps.cf, postfix-files, and some multi-instance template files. The + meta_directory parameter has the same default value as the config_directory + parameter (typically, /etc/postfix or /usr/local/etc/postfix). For + backwards compatibility with Postfix 2.6 .. 2.11, specify "meta_directory = + $daemon_directory" in main.cf before installing or upgrading Postfix, or + specify "meta_directory = /path/name" on the "make makefiles", "make + install" or "make upgrade" command line. + + * The configuration file dynamicmaps.cf will automatically include files + under the directory dynamicmaps.cf.d, just like the configuration file + postfix-files will automatically include files under the directory postfix- + files.d. Thanks to this, you can install or deinstall a database plugin + package without having to edit postfix-files or dynamicmaps.cf. Instead, + you give that plugin its own configuration files under dynamicmaps.cf.d and + postfix-files.d, and you add or remove those configuration files along with + the database plugin dynamically-linked object. + + * Each configuration file under the directory dynamicmaps.cf.d must have the + same format as the configuration file dynamicmaps.cf. There is no + requirement that these configuration file *names* have a specific format. + + * Each configuration file under the directory postfix-files.d must have the + same format as the configuration file postfix-files. There is no + requirement that these configuration file *names* have a specific format. + +44..55 -- BBuuiillddiinngg wwiitthh ooppttiioonnaall ffeeaattuurreess + +By default, Postfix builds as a mail system with relatively few bells and +whistles. Support for third-party databases etc. must be configured when +Postfix is compiled. The following documents describe how to build Postfix with +support for optional features: + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |OOppttiioonnaall ffeeaattuurree |DDooccuummeenntt |AAvvaaiillaabbiilliittyy| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |Berkeley DB database |DB_README |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |LMDB database |LMDB_README |Postfix 2.11| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |LDAP database |LDAP_README |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |MySQL database |MYSQL_README |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |Perl compatible regular expression|PCRE_README |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |PostgreSQL database |PGSQL_README |Postfix 2.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |SASL authentication |SASL_README |Postfix 1.0 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |SQLite database |SQLITE_README|Postfix 2.8 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |STARTTLS session encryption |TLS_README |Postfix 2.2 | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + +Note: IP version 6 support is compiled into Postfix on operating systems that +have IPv6 support. See the IPV6_README file for details. + +44..66 -- OOvveerrrriiddiinngg bbuuiilltt--iinn ppaarraammeetteerr ddeeffaauulltt sseettttiinnggss + +44..66..11 -- PPoossttffiixx 33..00 aanndd llaatteerr + +All Postfix configuration parameters can be changed by editing a Postfix +configuration file, except for one: the parameter that specifies the location +of Postfix configuration files. In order to build Postfix with a configuration +directory other than /etc/postfix, use: + + $ make makefiles config_directory=/some/where ...other arguments... + $ make + +The command "make makefiles name=value ..." will replace the string +MAIL_VERSION at the end of a configuration parameter value with the Postfix +release version. Do not try to specify something like $mail_version on this +command line. This produces inconsistent results with different versions of the +make(1) command. + +Parameters whose defaults can be specified in this way are listed below. See +the postconf(5) manpage for a description (command: "nroff -man man/man5/ +postconf.5 | less"). + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |ppaarraammeetteerr nnaammee |ttyyppiiccaall ddeeffaauulltt | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |command_directory |/usr/sbin | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |config_directory |/etc/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |default_database_type|hash | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |daemon_directory |/usr/libexec/postfix| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |data_directory |/var/lib/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |html_directory |no | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mail_spool_directory |/var/mail | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mailq_path |/usr/bin/mailq | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |manpage_directory |/usr/local/man | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |meta_directory |/etc/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |newaliases_path |/usr/bin/newaliases | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |openssl_path |openssl | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |queue_directory |/var/spool/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |readme_directory |no | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |sendmail_path |/usr/sbin/sendmail | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |shlib_directory |/usr/lib/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +44..66..22 -- AAllll PPoossttffiixx vveerrssiioonnss + +All Postfix configuration parameters can be changed by editing a Postfix +configuration file, except for one: the parameter that specifies the location +of Postfix configuration files. In order to build Postfix with a configuration +directory other than /etc/postfix, use: + + $ make makefiles CCARGS="-DDEF_CONFIG_DIR=\\\"/some/where\\\"" + $ make + +IMPORTANT: Be sure to get the quotes right. These details matter a lot. + +Parameters whose defaults can be specified in this way are listed below. See +the postconf(5) manpage for a description (command: "nroff -man man/man5/ +postconf.5 | less"). + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |MMaaccrroo nnaammee |ddeeffaauulltt vvaalluuee ffoorr |ttyyppiiccaall ddeeffaauulltt | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_COMMAND_DIR |command_directory |/usr/sbin | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_CONFIG_DIR |config_directory |/etc/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_DB_TYPE |default_database_type|hash | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_DAEMON_DIR |daemon_directory |/usr/libexec/postfix| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_DATA_DIR |data_directory |/var/lib/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_MAILQ_PATH |mailq_path |/usr/bin/mailq | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_HTML_DIR |html_directory |no | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_MANPAGE_DIR |manpage_directory |/usr/local/man | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_NEWALIAS_PATH|newaliases_path |/usr/bin/newaliases | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_QUEUE_DIR |queue_directory |/var/spool/postfix | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_README_DIR |readme_directory |no | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |DEF_SENDMAIL_PATH|sendmail_path |/usr/sbin/sendmail | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +Note: the data_directory parameter (for caches and pseudo-random numbers) was +introduced with Postfix version 2.5. + +44..77 -- OOvveerrrriiddiinngg ootthheerr ccoommppiillee--ttiimmee ffeeaattuurreess + +The general method to override Postfix compile-time features is as follows: + + $ make makefiles name=value name=value... + $ make + +The following is an extensive list of names and values. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ +|NNaammee//VVaalluuee |DDeessccrriippttiioonn | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies one or more non-default object | +| |libraries. Postfix 3.0 and later specify some| +| |of their database library dependencies with | +|AUXLIBS="object_library..." |AUXLIBS_CDB, AUXLIBS_LDAP, AUXLIBS_LMDB, | +| |AUXLIBS_MYSQL, AUXLIBS_PCRE, AUXLIBS_PGSQL, | +| |AUXLIBS_SDBM, and AUXLIBS_SQLITE, | +| |respectively. | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|CC=compiler_command |Specifies a non-default compiler. On many | +| |systems, the default is gcc. | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies non-default compiler arguments, for| +|CCARGS="compiler_arguments..." |example, a non-default include directory. The| +| |following directives turn off Postfix | +| |features at compile time: | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with Berkeley DB support. By | +|| |default, Berkeley DB support is compiled in | +||-DNO_DB |on platforms that are known to support this | +|| |feature. If you override this, then you | +|| |probably should also override DEF_DB_TYPE as | +|| |described in section 4.6. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +||-DNO_DNSSEC |Do not build with DNSSEC support, even if the| +|| |resolver library appears to support it. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with Solaris /dev/poll support. | +||-DNO_DEVPOLL |By default, /dev/poll support is compiled in | +|| |on Solaris versions that are known to support| +|| |this feature. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with Linux EPOLL support. By | +||-DNO_EPOLL |default, EPOLL support is compiled in on | +|| |platforms that are known to support this | +|| |feature. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with EAI (SMTPUTF8) support. By | +||-DNO_EAI |default, EAI support is compiled in when the | +|| |"icuuc" library and header files are found. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not require support for C99 "inline" | +|| |functions. Instead, implement argument | +||-DNO_INLINE |typechecks for non-printf/scanf-like | +|| |functions with ternary operators and | +|| |unreachable code. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with IPv6 support. By default, | +|| |IPv6 support is compiled in on platforms that| +|| |are known to have IPv6 support. Note: this | +||-DNO_IPV6 |directive is for debugging and testing only. | +|| |It is not guaranteed to work on all | +|| |platforms. If you don't want IPv6 support, | +|| |set "inet_protocols = ipv4" in main.cf. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with FreeBSD / NetBSD / OpenBSD | +||-DNO_KQUEUE |/ MacOSX KQUEUE support. By default, KQUEUE | +|| |support is compiled in on platforms that are | +|| |known to support it. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with NIS or NISPLUS support. NIS| +||-DNO_NIS |is not available on some recent Linux | +|| |distributions. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with NISPLUS support. NISPLUS is| +||-DNO_NISPLUS |not available on some recent Solaris | +|| |distributions. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Do not build with PCRE support. By default, | +||-DNO_PCRE |PCRE support is compiled in when the pcre- | +|| |config utility is installed. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Disable support for POSIX getpwnam_r/ | +||-DNO_POSIX_GETPW_R |getpwuid_r. By default Postfix uses these | +|| |where they are known to be available. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +||-DNO_RES_NCALLS |Do not build with the threadsafe resolver(5) | +|| |API (res_ninit() etc.). | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Use setjmp()/longjmp() instead of sigsetjmp | +||-DNO_SIGSETJMP |()/siglongjmp(). By default, Postfix uses | +|| |sigsetjmp()/siglongjmp() when they are known | +|| |to be available. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +|| |Use sprintf() instead of snprintf(). By | +||-DNO_SNPRINTF |default, Postfix uses snprintf() except on | +|| |ancient systems. | +|_|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies a non-default compiler debugging | +|DEBUG=debug_level |level. The default is "-g". Specify DEBUG= to| +| |turn off debugging. | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies a non-default optimization level. | +|OPT=optimization_level |The default is "-O". Specify OPT= to turn off| +| |optimization. | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies options for the postfix-install | +|POSTFIX_INSTALL_OPTS=-option...|command, separated by whitespace. Currently, | +| |the only supported option is "-keep-build- | +| |mtime". | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies non-default compiler options for | +|SHLIB_CFLAGS=flags |building Postfix dynamically-linked libraries| +| |and database plugins. The typical default is | +| |"-fPIC". | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies a non-default runpath for Postfix | +|SHLIB_RPATH=rpath |dynamically-linked libraries. The typical | +| |default is "'-Wl,-rpath,${SHLIB_DIR}'". | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies a non-default suffix for Postfix | +|SHLIB_SUFFIX=suffix |dynamically-linked libraries and database | +| |plugins. The typical default is ".so". | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | +| |Specifies non-default compiler warning | +|WARN="warning_flags..." |options for use when "make" is invoked in a | +| |source subdirectory only. | +|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +44..88 -- SSuuppppoorrtt ffoorr tthhoouussaannddss ooff pprroocceesssseess + +The number of connections that Postfix can manage simultaneously is limited by +the number of processes that it can run. This number in turn is limited by the +number of files and sockets that a single process can open. For example, the +Postfix queue manager has a separate connection to each delivery process, and +the anvil(8) server has one connection per smtpd(8) process. + +Postfix version 2.4 and later have no built-in limits on the number of open +files or sockets, when compiled on systems that support one of the following: + + * BSD kqueue(2) (FreeBSD 4.1, NetBSD 2.0, OpenBSD 2.9), + * Solaris 8 /dev/poll, + * Linux 2.6 epoll(4). + +With other Postfix versions or operating systems, the number of file +descriptors per process is limited by the value of the FD_SETSIZE macro. If you +expect to run more than 1000 mail delivery processes, you may need to override +the definition of the FD_SETSIZE macro to make select() work correctly: + + $ make makefiles CCARGS=-DFD_SETSIZE=2048 + +Warning: the above has no effect on some Linux versions. Apparently, on these +systems the FD_SETSIZE value can be changed only by using undocumented +interfaces. Currently, that means including <bits/types.h> directly (which is +not allowed) and overriding the __FD_SETSIZE macro. Beware, undocumented +interfaces can change at any time and without warning. + +But wait, there is more: none of this will work unless the operating system is +configured to handle thousands of connections. See the TUNING_README guide for +examples of how to increase the number of open sockets or files. + +44..99 -- CCoommppiilliinngg PPoossttffiixx,, aatt llaasstt + +If the command + + $ make + +is successful, then you can proceed to install Postfix (section 6). + +If the command produces compiler error messages, it may be time to search the +web or to ask the postfix-users@postfix.org mailing list, but be sure to search +the mailing list archives first. Some mailing list archives are linked from +http://www.postfix.org/. + +55 -- PPoorrttiinngg PPoossttffiixx ttoo aann uunnssuuppppoorrtteedd ssyysstteemm + +Each system type that Postfix knows is identified by a unique name. Examples: +SUNOS5, FREEBSD4, and so on. When porting Postfix to a new system, the first +step is to choose a SYSTEMTYPE name for the new system. You must use a name +that includes at least the major version of the operating system (such as +SUNOS4 or LINUX2), so that different releases of the same system can be +supported without confusion. + +Add a case statement to the "makedefs" shell script in the source code top- +level directory that recognizes the new system reliably, and that emits the +right system-specific information. Be sure to make the code robust against user +PATH settings; if the system offers multiple UNIX flavors (e.g. BSD and SYSV) +be sure to build for the native flavor, instead of the emulated one. + +Add an "#ifdef SYSTEMTYPE" section to the central util/sys_defs.h include file. +You may have to invent new feature macro names. Please choose sensible feature +macro names such as HAS_DBM or FIONREAD_IN_SYS_FILIO_H. + +I strongly recommend against using "#ifdef SYSTEMTYPE" in individual source +files. While this may look like the quickest solution, it will create a mess +when newer versions of the same SYSTEMTYPE need to be supported. You're likely +to end up placing "#ifdef" sections all over the source code again. + +66 -- IInnssttaalllliinngg tthhee ssooffttwwaarree aafftteerr ssuucccceessssffuull ccoommppiillaattiioonn + +This text describes how to install Postfix from source code. See the +PACKAGE_README file if you are building a package for distribution to other +systems. + +66..11 -- SSaavvee eexxiissttiinngg SSeennddmmaaiill bbiinnaarriieess + +IMPORTANT: if you are REPLACING an existing Sendmail installation with Postfix, +you may need to keep the old sendmail program running for some time in order to +flush the mail queue. + + * Some systems implement a mail switch mechanism where different MTAs + (Postfix, Sendmail, etc.) can be installed at the same time, while only one + of them is actually being used. Examples of such switching mechanisms are + the FreeBSD mailwrapper(8) or the Linux mail switch. In this case you + should try to "flip" the switch to "Postfix" before installing Postfix. + + * If your system has no mail switch mechanism, execute the following commands + (your sendmail, newaliases and mailq programs may be in a different place): + + # mv /usr/sbin/sendmail /usr/sbin/sendmail.OFF + # mv /usr/bin/newaliases /usr/bin/newaliases.OFF + # mv /usr/bin/mailq /usr/bin/mailq.OFF + # chmod 755 /usr/sbin/sendmail.OFF /usr/bin/newaliases.OFF \ + /usr/bin/mailq.OFF + +66..22 -- CCrreeaattee aaccccoouunntt aanndd ggrroouuppss + +Before you install Postfix for the first time you need to create an account and +a group: + + * Create a user account "postfix" with a user id and group id that are not + used by any other user account. Preferably, this is an account that no-one + can log into. The account does not need an executable login shell, and + needs no existing home directory. My password and group file entries look + like this: + + /etc/passwd: + postfix:*:12345:12345:postfix:/no/where:/no/shell + + /etc/group: + postfix:*:12345: + + Note: there should be no whitespace before "postfix:". + + * Create a group "postdrop" with a group id that is not used by any other + user account. Not even by the postfix user account. My group file entry + looks like: + + /etc/group: + postdrop:*:54321: + + Note: there should be no whitespace before "postdrop:". + +66..33 -- IInnssttaallll PPoossttffiixx + +To install or upgrade Postfix from compiled source code, run one of the +following commands as the super-user: + + # make install (interactive version, first time install) + + # make upgrade (non-interactive version, for upgrades) + + * The interactive version ("make install") asks for pathnames for Postfix + data and program files, and stores your preferences in the main.cf file. IIff + yyoouu ddoonn''tt wwaanntt PPoossttffiixx ttoo oovveerrwwrriittee nnoonn--PPoossttffiixx ""sseennddmmaaiill"",, ""mmaaiillqq"" aanndd + ""nneewwaalliiaasseess"" ffiilleess,, ssppeecciiffyy ppaatthhnnaammeess tthhaatt eenndd iinn ""..ppoossttffiixx"". + + * The non-interactive version ("make upgrade") needs the /etc/postfix/main.cf + file from a previous installation. If the file does not exist, use + interactive installation ("make install") instead. + + * If you specify name=value arguments on the "make install" or "make upgrade" + command line, then these will take precedence over compiled-in default + settings or main.cf settings. + + The command "make install/upgrade name=value ..." will replace the string + MAIL_VERSION at the end of a configuration parameter value with the Postfix + release version. Do not try to specify something like $mail_version on this + command line. This produces inconsistent results with different versions of + the make(1) command. + +66..44 -- CCoonnffiigguurree PPoossttffiixx + +Proceed to the section on how you wish to run Postfix on your particular +machine: + + * Send mail only, without changing an existing Sendmail installation (section + 7). + + * Send and receive mail via a virtual host interface, still without any + change to an existing Sendmail installation (section 8). + + * Run Postfix instead of Sendmail (section 9). + +77 -- CCoonnffiigguurriinngg PPoossttffiixx ttoo sseenndd mmaaiill oonnllyy + +If you are going to use Postfix to send mail only, there is no need to change +your existing sendmail setup. Instead, set up your mail user agent so that it +calls the Postfix sendmail program directly. + +Follow the instructions in the "Mandatory configuration file edits" in section +10, and review the "To chroot or not to chroot" text in section 11. + +You MUST comment out the "smtp inet" entry in /etc/postfix/master.cf, in order +to avoid conflicts with the real sendmail. Put a "#" character in front of the +line that defines the smtpd service: + + /etc/postfix/master.cf: + #smtp inet n - n - - smtpd + +Start the Postfix system: + + # postfix start + +or, if you feel nostalgic, use the Postfix sendmail command: + + # sendmail -bd -qwhatever + +and watch your maillog file for any error messages. The pathname is /var/log/ +maillog, /var/log/mail, /var/log/syslog, or something else. Typically, the +pathname is defined in the /etc/syslog.conf file. + + $ grep -E '(reject|warning|error|fatal|panic):' /some/log/file + +Note: the most important error message is logged first. Later messages are not +as useful. + +In order to inspect the mail queue, use one of the following commands: + + $ mailq + + $ sendmail -bp + + $ postqueue -p + +See also the "Care and feeding" section 12 below. + +88 -- CCoonnffiigguurriinngg PPoossttffiixx ttoo sseenndd aanndd rreecceeiivvee mmaaiill vviiaa vviirrttuuaall iinntteerrffaaccee + +Alternatively, you can use the Postfix system to send AND receive mail while +leaving your Sendmail setup intact, by running Postfix on a virtual interface +address. Simply configure your mail user agent to directly invoke the Postfix +sendmail program. + +To create a virtual network interface address, study your system ifconfig +manual page. The command syntax could be any of: + + # iiffccoonnffiigg llee00::11 <<aaddddrreessss>> nneettmmaasskk <<mmaasskk>> uupp + # iiffccoonnffiigg eenn00 aalliiaass <<aaddddrreessss>> nneettmmaasskk 225555..225555..225555..225555 + +In the /etc/postfix/main.cf file, I would specify + + /etc/postfix/main.cf: + myhostname = virtual.host.tld + inet_interfaces = $myhostname + mydestination = $myhostname + +Follow the instructions in the "Mandatory configuration file edits" in section +10, and review the "To chroot or not to chroot" text in section 11. + +Start the Postfix system: + + # postfix start + +or, if you feel nostalgic, use the Postfix sendmail command: + + # sendmail -bd -qwhatever + +and watch your maillog file for any error messages. The pathname is /var/log/ +maillog, /var/log/mail, /var/log/syslog, or something else. Typically, the +pathname is defined in the /etc/syslog.conf file. + + $ grep -E '(reject|warning|error|fatal|panic):' /some/log/file + +Note: the most important error message is logged first. Later messages are not +as useful. + +In order to inspect the mail queue, use one of the following commands: + + $ mailq + + $ sendmail -bp + + $ postqueue -p + +See also the "Care and feeding" section 12 below. + +99 -- RRuunnnniinngg PPoossttffiixx iinnsstteeaadd ooff SSeennddmmaaiill + +Prior to installing Postfix you should save any existing sendmail program files +as described in section 6. Be sure to keep the old sendmail running for at +least a couple days to flush any unsent mail. To do so, stop the sendmail +daemon and restart it as: + + # /usr/sbin/sendmail.OFF -q + +Note: this is old sendmail syntax. Newer versions use separate processes for +mail submission and for running the queue. + +After you have visited the "Mandatory configuration file edits" section below, +you can start the Postfix system with: + + # postfix start + +or, if you feel nostalgic, use the Postfix sendmail command: + + # sendmail -bd -qwhatever + +and watch your maillog file for any error messages. The pathname is /var/log/ +maillog, /var/log/mail, /var/log/syslog, or something else. Typically, the +pathname is defined in the /etc/syslog.conf file. + + $ grep -E '(reject|warning|error|fatal|panic):' /some/log/file + +Note: the most important error message is logged first. Later messages are not +as useful. + +In order to inspect the mail queue, use one of the following commands: + + $ mailq + + $ sendmail -bp + + $ postqueue -p + +See also the "Care and feeding" section 12 below. + +1100 -- MMaannddaattoorryy ccoonnffiigguurraattiioonn ffiillee eeddiittss + +Note: the material covered in this section is covered in more detail in the +BASIC_CONFIGURATION_README document. The information presented below is +targeted at experienced system administrators. + +1100..11 -- PPoossttffiixx ccoonnffiigguurraattiioonn ffiilleess + +By default, Postfix configuration files are in /etc/postfix. The two most +important files are main.cf and master.cf; these files must be owned by root. +Giving someone else write permission to main.cf or master.cf (or to their +parent directories) means giving root privileges to that person. + +In /etc/postfix/main.cf, you will have to set up a minimal number of +configuration parameters. Postfix configuration parameters resemble shell +variables, with two important differences: the first one is that Postfix does +not know about quotes like the UNIX shell does. + +You specify a configuration parameter as: + + /etc/postfix/main.cf: + parameter = value + +and you use it by putting a "$" character in front of its name: + + /etc/postfix/main.cf: + other_parameter = $parameter + +You can use $parameter before it is given a value (that is the second main +difference with UNIX shell variables). The Postfix configuration language uses +lazy evaluation, and does not look at a parameter value until it is needed at +runtime. + +Whenever you make a change to the main.cf or master.cf file, execute the +following command in order to refresh a running mail system: + + # postfix reload + +1100..22 -- DDeeffaauulltt ddoommaaiinn ffoorr uunnqquuaalliiffiieedd aaddddrreesssseess + +First of all, you must specify what domain will be appended to an unqualified +address (i.e. an address without @domain.tld). The "myorigin" parameter +defaults to the local hostname, but that is probably OK only for very small +sites. + +Some examples (use only one): + + /etc/postfix/main.cf: + myorigin = $myhostname (send mail as "user@$myhostname") + myorigin = $mydomain (send mail as "user@$mydomain") + +1100..33 -- WWhhaatt ddoommaaiinnss ttoo rreecceeiivvee llooccaallllyy + +Next you need to specify what mail addresses Postfix should deliver locally. + +Some examples (use only one): + + /etc/postfix/main.cf: + mydestination = $myhostname, localhost.$mydomain, localhost + mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain + mydestination = $myhostname + +The first example is appropriate for a workstation, the second is appropriate +for the mailserver for an entire domain. The third example should be used when +running on a virtual host interface. + +1100..44 -- PPrrooxxyy//NNAATT iinntteerrffaaccee aaddddrreesssseess + +The proxy_interfaces parameter specifies all network addresses that Postfix +receives mail on by way of a proxy or network address translation unit. You may +specify symbolic hostnames instead of network addresses. + +IMPORTANT: You must specify your proxy/NAT external addresses when your system +is a backup MX host for other domains, otherwise mail delivery loops will +happen when the primary MX host is down. + +Example: host behind NAT box running a backup MX host. + + /etc/postfix/main.cf: + proxy_interfaces = 1.2.3.4 (the proxy/NAT external network address) + +1100..55 -- WWhhaatt llooccaall cclliieennttss ttoo rreellaayy mmaaiill ffrroomm + +If your machine is on an open network then you must specify what client IP +addresses are authorized to relay their mail through your machine into the +Internet. The default setting includes all subnetworks that the machine is +attached to. This may give relay permission to too many clients. My own +settings are: + + /etc/postfix/main.cf: + mynetworks = 168.100.189.0/28, 127.0.0.0/8 + +1100..66 -- WWhhaatt rreellaayy ddeessttiinnaattiioonnss ttoo aacccceepptt ffrroomm ssttrraannggeerrss + +If your machine is on an open network then you must also specify whether +Postfix will forward mail from strangers. The default setting will forward mail +to all domains (and subdomains of) what is listed in $mydestination. This may +give relay permission for too many destinations. Recommended settings (use only +one): + + /etc/postfix/main.cf: + relay_domains = (do not forward mail from strangers) + relay_domains = $mydomain (my domain and subdomains) + relay_domains = $mydomain, other.domain.tld, ... + +1100..77 -- OOppttiioonnaall:: ccoonnffiigguurree aa ssmmaarrtt hhoosstt ffoorr rreemmoottee ddeelliivveerryy + +If you're behind a firewall, you should set up a relayhost. If you can, specify +the organizational domain name so that Postfix can use DNS lookups, and so that +it can fall back to a secondary MX host when the primary MX host is down. +Otherwise just specify a hard-coded hostname. + +Some examples (use only one): + + /etc/postfix/main.cf: + relayhost = $mydomain + relayhost = [mail.$mydomain] + +The form enclosed with [] eliminates DNS MX lookups. + +By default, the SMTP client will do DNS lookups even when you specify a relay +host. If your machine has no access to a DNS server, turn off SMTP client DNS +lookups like this: + + /etc/postfix/main.cf: + disable_dns_lookups = yes + +The STANDARD_CONFIGURATION_README file has more hints and tips for firewalled +and/or dial-up networks. + +1100..88 -- CCrreeaattee tthhee aalliiaasseess ddaattaabbaassee + +Postfix uses a Sendmail-compatible aliases(5) table to redirect mail for local +(8) recipients. Typically, this information is kept in two files: in a text +file /etc/aliases and in an indexed file /etc/aliases.db. The command "postconf +alias_maps" will tell you the exact location of the text file. + +First, be sure to update the text file with aliases for root, postmaster and +"postfix" that forward mail to a real person. Postfix has a sample aliases file +/etc/postfix/aliases that you can adapt to local conditions. + + /etc/aliases: + root: you + postmaster: root + postfix: root + bin: root + etcetera... + +Note: there should be no whitespace before the ":". + +Finally, build the indexed aliases file with one of the following commands: + + # newaliases + # sendmail -bi + # postalias /etc/aliases (pathname is system dependent!) + +1111 -- TToo cchhrroooott oorr nnoott ttoo cchhrroooott + +Postfix daemon processes can be configured (via master.cf) to run in a chroot +jail. The processes run at a fixed low privilege and with access only to the +Postfix queue directories (/var/spool/postfix). This provides a significant +barrier against intrusion. The barrier is not impenetrable, but every little +bit helps. + +With the exception of Postfix daemons that deliver mail locally and/or that +execute non-Postfix commands, every Postfix daemon can run chrooted. + +Sites with high security requirements should consider to chroot all daemons +that talk to the network: the smtp(8) and smtpd(8) processes, and perhaps also +the lmtp(8) client. The author's own porcupine.org mail server runs all daemons +chrooted that can be chrooted. + +The default /etc/postfix/master.cf file specifies that no Postfix daemon runs +chrooted. In order to enable chroot operation, edit the file /etc/postfix/ +master.cf. Instructions are in the file. + +Note that a chrooted daemon resolves all filenames relative to the Postfix +queue directory (/var/spool/postfix). For successful use of a chroot jail, most +UNIX systems require you to bring in some files or device nodes. The examples/ +chroot-setup directory in the source code distribution has a collection of +scripts that help you set up Postfix chroot environments on different operating +systems. + +Additionally, you almost certainly need to configure syslogd so that it listens +on a socket inside the Postfix queue directory. Examples for specific systems: + +FreeBSD: + + # mkdir -p /var/spool/postfix/var/run + # syslogd -l /var/spool/postfix/var/run/log + +Linux, OpenBSD: + + # mkdir -p /var/spool/postfix/dev + # syslogd -a /var/spool/postfix/dev/log + +1122 -- CCaarree aanndd ffeeeeddiinngg ooff tthhee PPoossttffiixx ssyysstteemm + +Postfix daemon processes run in the background, and log problems and normal +activity to the syslog daemon. The names of logfiles are specified in /etc/ +syslog.conf. At the very least you need something like: + + /etc/syslog.conf: + mail.err /dev/console + mail.debug /var/log/maillog + +IMPORTANT: the syslogd will not create files. You must create them before +(re)starting syslogd. + +IMPORTANT: on Linux you need to put a "-" character before the pathname, e.g., +-/var/log/maillog, otherwise the syslogd will use more system resources than +Postfix does. + +Hopefully, the number of problems will be small, but it is a good idea to run +every night before the syslog files are rotated: + + # postfix check + # grep -E '(reject|warning|error|fatal|panic):' /some/log/file + + * The first line (postfix check) causes Postfix to report file permission/ + ownership discrepancies. + + * The second line looks for problem reports from the mail software, and + reports how effective the relay and junk mail access blocks are. This may + produce a lot of output. You will want to apply some postprocessing to + eliminate uninteresting information. + +The DEBUG_README document describes the meaning of the "warning" etc. labels in +Postfix logging. + diff --git a/README_FILES/IPV6_README b/README_FILES/IPV6_README new file mode 100644 index 0000000..188003b --- /dev/null +++ b/README_FILES/IPV6_README @@ -0,0 +1,249 @@ +PPoossttffiixx IIPPvv66 SSuuppppoorrtt + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix 2.2 introduces support for the IPv6 (IP version 6) protocol. IPv6 +support for older Postfix versions was available as an add-on patch. The +section "Compatibility with Postfix <2.2 IPv6 support" below discusses the +differences between these implementations. + +The main feature of interest is that IPv6 uses 128-bit IP addresses instead of +the 32-bit addresses used by IPv4. It can therefore accommodate a much larger +number of hosts and networks without ugly kluges such as NAT. A side benefit of +the much larger address space is that it makes random network scanning +impractical. + +Postfix uses the same SMTP protocol over IPv6 as it already uses over the older +IPv4 network, and does AAAA record lookups in the DNS in addition to the older +A records. Information about IPv6 can be found at http://www.ipv6.org/. + +This document provides information on the following topics: + + * Supported platforms + * Configuration + * Known limitations + * Compatibility with Postfix <2.2 IPv6 support + * IPv6 Support for unsupported platforms + * Credits + +SSuuppppoorrtteedd PPllaattffoorrmmss + +Postfix version 2.2 supports IPv4 and IPv6 on the following platforms: + + * AIX 5.1+ + * Darwin 7.3+ + * FreeBSD 4+ + * Linux 2.4+ + * NetBSD 1.5+ + * OpenBSD 2+ + * Solaris 8+ + * Tru64Unix V5.1+ + +On other platforms Postfix will simply use IPv4 as it has always done. + +See "IPv6 Support for unsupported platforms" for tips to port Postfix IPv6 +support to other environments. + +CCoonnffiigguurraattiioonn + +Postfix IPv6 support introduces two new main.cf configuration parameters, and +introduces an important change in address syntax notation in match lists such +as mynetworks or debug_peer_list. + +Postfix IPv6 address syntax is a little tricky, because there are a few places +where you must enclose an IPv6 address inside "[]" characters, and a few places +where you must not. It is a good idea to use "[]" only in the few places where +you have to. Check out the postconf(5) manual whenever you do IPv6 related +configuration work with Postfix. + + * Instead of hard-coding 127.0.0.1 and ::1 loopback addresses in master.cf, + specify "inet_interfaces = loopback-only" in main.cf. This way you can use + the same master.cf file regardless of whether or not Postfix will run on an + IPv6-enabled system. + + * The first new parameter is called inet_protocols. This specifies what + protocols Postfix will use when it makes or accepts network connections, + and also controls what DNS lookups Postfix will use when it makes network + connections. + + /etc/postfix/main.cf: + # You must stop/start Postfix after changing this parameter. + inet_protocols = all (enable IPv4, and IPv6 if supported) + inet_protocols = ipv4 (enable IPv4 only) + inet_protocols = ipv4, ipv6 (enable both IPv4 and IPv6) + inet_protocols = ipv6 (enable IPv6 only) + + The default is compile-time dependent: "all" when Postfix is built on a + software distribution with IPv6 support, "ipv4" otherwise. + + Note 1: you must stop and start Postfix after changing the inet_protocols + configuration parameter. + + Note 2: on older Linux and Solaris systems, the setting "inet_protocols = + ipv6" will not prevent Postfix from accepting IPv4 connections. + + For an unsupported test option to build Postfix without IPv6 support, see + the NO_IPV6 option in the INSTALL document. + + * The other new parameter is smtp_bind_address6. This sets the local + interface address for outgoing IPv6 SMTP connections, just like the + smtp_bind_address parameter does for IPv4: + + /etc/postfix/main.cf: + smtp_bind_address6 = 2001:240:587:0:250:56ff:fe89:1 + + * If you left the value of the mynetworks parameter at its default (i.e. no + mynetworks setting in main.cf) Postfix will figure out by itself what its + network addresses are. This is what a typical setting looks like: + + % postconf mynetworks + mynetworks = 127.0.0.0/8 168.100.189.0/28 [::1]/128 [fe80::]/10 [2001: + 240:587::]/64 + + If you did specify the mynetworks parameter value in main.cf, you need to + update the mynetworks value to include the IPv6 networks the system is in. + Be sure to specify IPv6 address information inside "[]", like this: + + /etc/postfix/main.cf: + mynetworks = ...IPv4 networks... [::1]/128 [2001:240:587::]/64 ... + +NNOOTTEE:: wwhheenn ccoonnffiigguurriinngg PPoossttffiixx mmaattcchh lliissttss ssuucchh aass mmyynneettwwoorrkkss oorr +ddeebbuugg__ppeeeerr__lliisstt,, yyoouu mmuusstt ssppeecciiffyy IIPPvv66 aaddddrreessss iinnffoorrmmaattiioonn iinnssiiddee ""[[]]"" iinn tthhee +mmaaiinn..ccff ppaarraammeetteerr vvaalluuee aanndd iinn ffiilleess ssppeecciiffiieedd wwiitthh aa ""//ffiillee//nnaammee"" ppaatttteerrnn.. +IIPPvv66 aaddddrreesssseess ccoonnttaaiinn tthhee ""::"" cchhaarraacctteerr,, aanndd wwoouulldd ootthheerrwwiissee bbee ccoonnffuusseedd wwiitthh +aa ""ttyyppee::ttaabbllee"" ppaatttteerrnn.. + +KKnnoowwnn LLiimmiittaattiioonnss + + * Postfix SMTP clients before version 2.8 try to connect over IPv6 before + trying IPv4. With more recent Postfix versions, the order of IPv6 versus + IPv4 outgoing connection attempts is configurable with the + smtp_address_preference parameter. + + * Postfix versions before 2.6 do not support DNSBL (DNS blocklist) lookups + for IPv6 client IP addresses. + + * IPv6 does not have class A, B, C, etc. networks. With IPv6 networks, the + setting "mynetworks_style = class" has the same effect as the setting + "mynetworks_style = subnet". + + * On Tru64Unix and AIX, Postfix can't figure out the local subnet mask and + always assumes a /128 network. This is a problem only with + "mynetworks_style = subnet" and no explicit mynetworks setting in main.cf. + +CCoommppaattiibbiilliittyy wwiitthh PPoossttffiixx <<22..22 IIPPvv66 ssuuppppoorrtt + +Postfix version 2.2 IPv6 support is based on the Postfix/IPv6 patch by Dean +Strik and others, but differs in a few minor ways. + + * main.cf: The inet_interfaces parameter does not support the notation "ipv6: + all" or "ipv4:all". Use the inet_protocols parameter instead. + + * main.cf: Specify "inet_protocols = all" or "inet_protocols = ipv4, ipv6" in + order to enable both IPv4 and IPv6 support. + + * main.cf: The inet_protocols parameter also controls what DNS lookups + Postfix will attempt to make when delivering or receiving mail. + + * main.cf: Specify "inet_interfaces = loopback-only" to listen on loopback + network interfaces only. + + * The lmtp_bind_address and lmtp_bind_address6 features were omitted. Postfix + version 2.3 merged the LMTP client into the SMTP client, so there was no + reason to keep adding features to the LMTP client. + + * The SMTP server now requires that IPv6 addresses in SMTP commands are + specified as [ipv6:ipv6address], as described in RFC 2821. + + * The IPv6 network address matching code was rewritten from the ground up, + and is expected to be closer to the specification. The result may be + incompatible with the Postfix/IPv6 patch. + +IIPPvv66 SSuuppppoorrtt ffoorr uunnssuuppppoorrtteedd ppllaattffoorrmmss + +Getting Postfix IPv6 working on other platforms involves the following steps: + + * Specify how Postfix should find the local network interfaces. Postfix needs + this information to avoid mailer loops and to find out if mail for user@ + [ipaddress] is a local or remote destination. + + If your system has the getifaddrs() routine then add the following to your + platform-specific section in src/util/sys_defs.h: + + #ifndef NO_IPV6 + # define HAS_IPV6 + # define HAVE_GETIFADDRS + #endif + + Otherwise, if your system has the SIOCGLIF ioctl() command in /usr/include/ + */*.h, add the following to your platform-specific section in src/util/ + sys_defs.h: + + #ifndef NO_IPV6 + # define HAS_IPV6 + # define HAS_SIOCGLIF + #endif + + Otherwise, Postfix will have to use the old SIOCGIF commands and get along + with reduced IPv6 functionality (it won't be able to figure out your IPv6 + netmasks, which are needed for "mynetworks_style = subnet". Add this to + your platform-specific section in src/util/sys_defs.h: + + #ifndef NO_IPV6 + # define HAS_IPV6 + #endif + + * Test if Postfix can figure out its interface information. + + After compiling Postfix in the usual manner, step into the src/util + directory and type "mmaakkee iinneett__aaddddrr__llooccaall". Running this file by hand should + produce all the interface addresses and network masks, for example: + + % make + % cd src/util + % make inet_addr_local + [... some messages ...] + % ./inet_addr_local + [... some messages ...] + ./inet_addr_local: inet_addr_local: configured 2 IPv4 addresses + ./inet_addr_local: inet_addr_local: configured 4 IPv6 addresses + 168.100.189.2/255.255.255.224 + 127.0.0.1/255.0.0.0 + fe80:1::2d0:b7ff:fe88:2ca7/ffff:ffff:ffff:ffff:: + 2001:240:587:0:2d0:b7ff:fe88:2ca7/ffff:ffff:ffff:ffff:: + fe80:5::1/ffff:ffff:ffff:ffff:: + ::1/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff + + The above is for an old FreeBSD machine. Other systems produce slightly + different results, but you get the idea. + +If none of all this produces a usable result, send email to the postfix- +users@postfix.org mailing list and we'll try to help you through this. + +CCrreeddiittss + +The following information is in part based on information that was compiled by +Dean Strik. + + * Mark Huizer wrote the original Postfix IPv6 patch. + + * Jun-ichiro 'itojun' Hagino of the KAME project made substantial + improvements. Since then, we speak of the KAME patch. + + * The PLD Linux Distribution ported the code to other stacks (notably USAGI). + We speak of the PLD patch. A very important feature of the PLD patch was + that it can work with Lutz Jaenicke's TLS patch for Postfix. + + * Dean Strik extended IPv6 support to platforms other than KAME and USAGI, + updated the patch to keep up with Postfix development, and provided a + combined IPv6 + TLS patch. Information about his effort can be found on + Dean Strik's Postfix website at http://www.ipnet6.org/postfix/. + + * Wietse Venema took Dean Strik's IPv6 patch, merged it into Postfix 2.2, and + took the opportunity to eliminate all IPv4-specific code from Postfix that + could be removed. For systems without IPv6 support in the kernel and system + libraries, Postfix has a simple compatibility layer, so that it will use + IPv4 as before. + diff --git a/README_FILES/LDAP_README b/README_FILES/LDAP_README new file mode 100644 index 0000000..eeef565 --- /dev/null +++ b/README_FILES/LDAP_README @@ -0,0 +1,465 @@ +PPoossttffiixx LLDDAAPP HHoowwttoo + +------------------------------------------------------------------------------- + +LLDDAAPP SSuuppppoorrtt iinn PPoossttffiixx + +Postfix can use an LDAP directory as a source for any of its lookups: aliases +(5), virtual(5), canonical(5), etc. This allows you to keep information for +your mail service in a replicated network database with fine-grained access +controls. By not storing it locally on the mail server, the administrators can +maintain it from anywhere, and the users can control whatever bits of it you +think appropriate. You can have multiple mail servers using the same +information, without the hassle and delay of having to copy it to each. + +Topics covered in this document: + + * Building Postfix with LDAP support + * Configuring LDAP lookups + * Example: aliases + * Example: virtual domains/addresses + * Example: expanding LDAP groups + * Other uses of LDAP lookups + * Notes and things to think about + * Feedback + * Credits + +BBuuiillddiinngg PPoossttffiixx wwiitthh LLDDAAPP ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +Note 1: Postfix no longer supports the LDAP version 1 interface. + +Note 2: to use LDAP with Debian GNU/Linux's Postfix, all you need is to install +the postfix-ldap package and you're done. There is no need to recompile +Postfix. + +You need to have LDAP libraries and include files installed somewhere on your +system, and you need to configure the Postfix Makefiles accordingly. + +For example, to build the OpenLDAP libraries for use with Postfix (i.e. LDAP +client code only), you could use the following command: + + % ./configure --without-kerberos --without-cyrus-sasl --without-tls \ + --without-threads --disable-slapd --disable-slurpd \ + --disable-debug --disable-shared + +If you're using the libraries from the UM distribution (http://www.umich.edu/ +~dirsvcs/ldap/ldap.html) or OpenLDAP (http://www.openldap.org), something like +this in the top level of your Postfix source tree should work: + + % make tidy + % make makefiles CCARGS="-I/usr/local/include -DHAS_LDAP" \ + AUXLIBS_LDAP="-L/usr/local/lib -lldap -L/usr/local/lib -llber" + +If your LDAP shared library is in a directory that the RUN-TIME linker does not +know about, add a "-Wl,-R,/path/to/directory" option after "-lldap". + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_LDAP. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded LDAP database client, but only the new AUXLIBS_LDAP variable supports +building a dynamically-loaded or statically-loaded LDAP database client. + + Failure to use the AUXLIBS_LDAP variable will defeat the purpose of dynamic + database client loading. Every Postfix executable file will have LDAP + database library dependencies. And that was exactly what dynamic database + client loading was meant to avoid. + +On Solaris 2.x you may have to specify run-time link information, otherwise +ld.so will not find some of the shared libraries: + + % make tidy + % make makefiles CCARGS="-I/usr/local/include -DHAS_LDAP" \ + AUXLIBS_LDAP="-L/usr/local/lib -R/usr/local/lib -lldap \ + -L/usr/local/lib -R/usr/local/lib -llber" + +The 'make tidy' command is needed only if you have previously built Postfix +without LDAP support. + +Instead of '/usr/local' specify the actual locations of your LDAP include files +and libraries. Be sure to not mix LDAP include files and LDAP libraries of +different versions!! + +If your LDAP libraries were built with Kerberos support, you'll also need to +include your Kerberos libraries in this line. Note that the KTH Kerberos IV +libraries might conflict with Postfix's lib/libdns.a, which defines dns_lookup. +If that happens, you'll probably want to link with LDAP libraries that lack +Kerberos support just to build Postfix, as it doesn't support Kerberos binds to +the LDAP server anyway. Sorry about the bother. + +If you're using one of the Netscape LDAP SDKs, you'll need to change the +AUXLIBS line to point to libldap10.so or libldapssl30.so or whatever you have, +and you may need to use the appropriate linker option (e.g. '-R') so the +executables can find it at runtime. + +If you are using OpenLDAP, and the libraries were built with SASL support, you +can add -DUSE_LDAP_SASL to the CCARGS to enable SASL support. For example: + + CCARGS="-I/usr/local/include -DHAS_LDAP -DUSE_LDAP_SASL" + +CCoonnffiigguurriinngg LLDDAAPP llooookkuuppss + +In order to use LDAP lookups, define an LDAP source as a table lookup in +main.cf, for example: + + alias_maps = hash:/etc/aliases, ldap:/etc/postfix/ldap-aliases.cf + +The file /etc/postfix/ldap-aliases.cf can specify a great number of parameters, +including parameters that enable LDAP SSL or STARTTLS, and LDAP SASL. For a +complete description, see the ldap_table(5) manual page. + +EExxaammppllee:: llooccaall((88)) aalliiaasseess + +Here's a basic example for using LDAP to look up local(8) aliases. Assume that +in main.cf, you have: + + alias_maps = hash:/etc/aliases, ldap:/etc/postfix/ldap-aliases.cf + +and in ldap:/etc/postfix/ldap-aliases.cf you have: + + server_host = ldap.example.com + search_base = dc=example, dc=com + +Upon receiving mail for a local address "ldapuser" that isn't found in the / +etc/aliases database, Postfix will search the LDAP server listening at port 389 +on ldap.example.com. It will bind anonymously, search for any directory entries +whose mailacceptinggeneralid attribute is "ldapuser", read the "maildrop" +attributes of those found, and build a list of their maildrops, which will be +treated as RFC822 addresses to which the message will be delivered. + +EExxaammppllee:: vviirrttuuaall ddoommaaiinnss//aaddddrreesssseess + +If you want to keep information for virtual lookups in your directory, it's +only a little more complicated. First, you need to make sure Postfix knows +about the virtual domain. An easy way to do that is to add the domain to the +mailacceptinggeneralid attribute of some entry in the directory. Next, you'll +want to make sure all of your virtual recipient's mailacceptinggeneralid +attributes are fully qualified with their virtual domains. Finally, if you want +to designate a directory entry as the default user for a virtual domain, just +give it an additional mailacceptinggeneralid (or the equivalent in your +directory) of "@fake.dom". That's right, no user part. If you don't want a +catchall user, omit this step and mail to unknown users in the domain will +simply bounce. + +In summary, you might have a catchall user for a virtual domain that looks like +this: + + dn: cn=defaultrecipient, dc=fake, dc=dom + objectclass: top + objectclass: virtualaccount + cn: defaultrecipient + owner: uid=root, dc=someserver, dc=isp, dc=dom + 1 -> mailacceptinggeneralid: fake.dom + 2 -> mailacceptinggeneralid: @fake.dom + 3 -> maildrop: realuser@real.dom + + 1: Postfix knows fake.dom is a valid virtual domain when it looks for this + and gets something (the maildrop) back. + + 2: This causes any mail for unknown users in fake.dom to go to this entry + ... + + 3: ... and then to its maildrop. + +Normal users might simply have one mailacceptinggeneralid and maildrop, e.g. +"normaluser@fake.dom" and "normaluser@real.dom". + +EExxaammppllee:: eexxppaannddiinngg LLDDAAPP ggrroouuppss + +LDAP is frequently used to store group member information. There are a number +of ways of handling LDAP groups. We will show a few examples in order of +increasing complexity, but owing to the number of independent variables, we can +only present a tiny portion of the solution space. We show how to: + + 1. query groups as lists of addresses; + + 2. query groups as lists of user objects containing addresses; + + 3. forward special lists unexpanded to a separate list server, for moderation + or other processing; + + 4. handle complex schemas by controlling expansion and by treating leaf nodes + specially, using features that are new in Postfix 2.4. + +The example LDAP entries and implied schema below show two group entries +("agroup" and "bgroup") and four user entries ("auser", "buser", "cuser" and +"duser"). The group "agroup" has the users "auser" (1) and "buser" (2) as +members via DN references in the multi-valued attribute "memberdn", and direct +email addresses of two external users "auser@example.org" (3) and +"buser@example.org" (4) stored in the multi-valued attribute "memberaddr". The +same is true of "bgroup" and "cuser"/"duser" (6)/(7)/(8)/(9), but "bgroup" also +has a "maildrop" attribute of "bgroup@mlm.example.com" (5): + + dn: cn=agroup, dc=example, dc=com + objectclass: top + objectclass: ldapgroup + cn: agroup + mail: agroup@example.com + 1 -> memberdn: uid=auser, dc=example, dc=com + 2 -> memberdn: uid=buser, dc=example, dc=com + 3 -> memberaddr: auser@example.org + 4 -> memberaddr: buser@example.org + + dn: cn=bgroup, dc=example, dc=com + objectclass: top + objectclass: ldapgroup + cn: bgroup + mail: bgroup@example.com + 5 -> maildrop: bgroup@mlm.example.com + 6 -> memberdn: uid=cuser, dc=example, dc=com + 7 -> memberdn: uid=duser, dc=example, dc=com + 8 -> memberaddr: cuser@example.org + 9 -> memberaddr: duser@example.org + + dn: uid=auser, dc=example, dc=com + objectclass: top + objectclass: ldapuser + uid: auser + 10 -> mail: auser@example.com + 11 -> maildrop: auser@mailhub.example.com + + dn: uid=buser, dc=example, dc=com + objectclass: top + objectclass: ldapuser + uid: buser + 12 -> mail: buser@example.com + 13 -> maildrop: buser@mailhub.example.com + + dn: uid=cuser, dc=example, dc=com + objectclass: top + objectclass: ldapuser + uid: cuser + 14 -> mail: cuser@example.com + + dn: uid=duser, dc=example, dc=com + objectclass: top + objectclass: ldapuser + uid: duser + 15 -> mail: duser@example.com + +Our first use case ignores the "memberdn" attributes, and assumes that groups +hold only direct "memberaddr" strings as in (3), (4), (8) and (9). The goal is +to map the group address to the list of constituent "memberaddr" values. This +is simple, ignoring the various connection related settings (hosts, ports, bind +settings, timeouts, ...) we have: + + simple.cf: + ... + search_base = dc=example, dc=com + query_filter = mail=%s + result_attribute = memberaddr + $ postmap -q agroup@example.com ldap:/etc/postfix/simple.cf \ + auser@example.org,buser@example.org + +We search "dc=example, dc=com". The "mail" attribute is used in the +query_filter to locate the right group, the "result_attribute" setting +described in ldap_table(5) is used to specify that "memberaddr" values from the +matching group are to be returned as a comma separated list. Always check +tables using postmap(1) with the "-q" option, before deploying them into +production use in main.cf. + +Our second use case instead expands "memberdn" attributes (1), (2), (6) and +(7), follows the DN references and returns the "maildrop" of the referenced +user entries. Here we use the "special_result_attribute" setting from +ldap_table(5) to designate the "memberdn" attribute as holding DNs of the +desired member entries. The "result_attribute" setting selects which attributes +are returned from the selected DNs. It is important to choose a result +attribute that is not also present in the group object, because result +attributes are collected from both the group and the member DNs. In this case +we choose "maildrop" and assume for the moment that groups never have a +"maildrop" (the "bgroup" "maildrop" attribute is for a different use case). The +returned data for "auser" and "buser" is from items (11) and (13) in the +example data. + + special.cf: + ... + search_base = dc=example, dc=com + query_filter = mail=%s + result_attribute = maildrop + special_result_attribute = memberdn + $ postmap -q agroup@example.com ldap:/etc/postfix/special.cf \ + auser@mailhub.example.com,buser@mailhub.example.com + +Note: if the desired member object result attribute is always also present in +the group, you get surprising results: the expansion also returns the address +of the group. This is a known limitation of Postfix releases prior to 2.4, and +is addressed in the new with Postfix 2.4 "leaf_result_attribute" feature +described in ldap_table(5). + +Our third use case has some groups that are expanded immediately, and other +groups that are forwarded to a dedicated mailing list manager host for delayed +expansion. This uses two LDAP tables, one for users and forwarded groups and a +second for groups that can be expanded immediately. It is assumed that groups +that require forwarding are never nested members of groups that are directly +expanded. + + no_expand.cf: + ... + search_base = dc=example, dc=com + query_filter = mail=%s + result_attribute = maildrop + expand.cf + ... + search_base = dc=example, dc=com + query_filter = mail=%s + result_attribute = maildrop + special_result_attribute = memberdn + $ postmap -q auser@example.com \ + ldap:/etc/postfix/no_expand.cf ldap:/etc/postfix/expand.cf \ + auser@mailhub.example.com + $ postmap -q agroup@example.com \ + ldap:/etc/postfix/no_expand.cf ldap:/etc/postfix/expand.cf \ + auser@mailhub.example.com,buser@mailhub.example.com + $ postmap -q bgroup@example.com \ + ldap:/etc/postfix/no_expand.cf ldap:/etc/postfix/expand.cf \ + bgroup@mlm.example.com + +Non-group objects and groups with delayed expansion (those that have a maildrop +attribute) are rewritten to a single maildrop value. Groups that don't have a +maildrop are expanded as the second use case. This admits a more elegant +solution with Postfix 2.4 and later. + +Our final use case is the same as the third, but this time uses new features in +Postfix 2.4. We now are able to use just one LDAP table and no longer need to +assume that forwarded groups are never nested inside expanded groups. + + fancy.cf: + ... + search_base = dc=example, dc=com + query_filter = mail=%s + result_attribute = memberaddr + special_result_attribute = memberdn + terminal_result_attribute = maildrop + leaf_result_attribute = mail + $ postmap -q auser@example.com ldap:/etc/postfix/fancy.cf \ + auser@mailhub.example.com + $ postmap -q cuser@example.com ldap:/etc/postfix/fancy.cf \ + cuser@example.com + $ postmap -q agroup@example.com ldap:/etc/postfix/fancy.cf \ + + auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org + $ postmap -q bgroup@example.com ldap:/etc/postfix/fancy.cf \ + bgroup@mlm.example.com + +Above, delayed expansion is enabled via "terminal_result_attribute", which, if +present, is used as the sole result and all other expansion is suppressed. +Otherwise, the "leaf_result_attribute" is only returned for leaf objects that +don't have a "special_result_attribute" (non-groups), while the +"result_attribute" (direct member address of groups) is returned at every level +of recursive expansion, not just the leaf nodes. This fancy example illustrates +all the features of Postfix 2.4 group expansion. + +OOtthheerr uusseess ooff LLDDAAPP llooookkuuppss + +Other common uses for LDAP lookups include rewriting senders and recipients +with Postfix's canonical lookups, for example in order to make mail leaving +your site appear to be coming from "First.Last@example.com" instead of +"userid@example.com". + +NNootteess aanndd tthhiinnggss ttoo tthhiinnkk aabboouutt + + * The bits of schema and attribute names used in this document are just + examples. There's nothing special about them, other than that some are the + defaults in the LDAP configuration parameters. You can use whatever schema + you like, and configure Postfix accordingly. + + * You probably want to make sure that mailacceptinggeneralids are unique, and + that not just anyone can specify theirs as postmaster or root, say. + + * An entry can have an arbitrary number of mailacceptinggeneralids or + maildrops. Maildrops can also be comma-separated lists of addresses. They + will all be found and returned by the lookups. For example, you could + define an entry intended for use as a mailing list that looks like this + (Warning! Schema made up just for this example): + + dn: cn=Accounting Staff List, dc=example, dc=com + cn: Accounting Staff List + o: example.com + objectclass: maillist + mailacceptinggeneralid: accountingstaff + mailacceptinggeneralid: accounting-staff + maildrop: mylist-owner + maildrop: an-accountant + maildrop: some-other-accountant + maildrop: this, that, theother + + * If you use an LDAP map for lookups other than aliases, you may have to make + sure the lookup makes sense. In the case of virtual lookups, maildrops + other than mail addresses are pretty useless, because Postfix can't know + how to set the ownership for program or file delivery. Your qquueerryy__ffiilltteerr + should probably look something like this: + + query_filter = (&(mailacceptinggeneralid=%s)(!(|(maildrop="*|*") + (maildrop="*:*")(maildrop="*/*")))) + + * And for that matter, even for aliases, you may not want users to be able to + specify their maildrops as programs, includes, etc. This might be + particularly pertinent on a "sealed" server where they don't have local + UNIX accounts, but exist only in LDAP and Cyrus. You might allow the fun + stuff only for directory entries owned by an administrative account, so + that if the object had a program as its maildrop and weren't owned by + "cn=root" it wouldn't be returned as a valid local user. This will require + some thought on your part to implement safely, considering the + ramifications of this type of delivery. You may decide it's not worth the + bother to allow any of that nonsense in LDAP lookups, ban it in the + qquueerryy__ffiilltteerr, and keep things like majordomo lists in local alias + databases. + + query_filter = (&(mailacceptinggeneralid=%s)(!(|(maildrop="*|*") + (maildrop="*:*")(maildrop="*/*"))(owner=cn=root, dc=your, dc=com))) + + * LDAP lookups are slower than local DB or DBM lookups. For most sites they + won't be a bottleneck, but it's a good idea to know how to tune your + directory service. + + * Multiple LDAP maps share the same LDAP connection if they differ only in + their query related parameters: base, scope, query_filter, and so on. To + take advantage of this, avoid spurious differences in the definitions of + LDAP maps: host selection order, version, bind, tls parameters, ... should + be the same for multiple maps whenever possible. + +FFeeeeddbbaacckk + +If you have questions, send them to postfix-users@postfix.org. Please include +relevant information about your Postfix setup: LDAP-related output from +postconf, which LDAP libraries you built with, and which directory server +you're using. If your question involves your directory contents, please include +the applicable bits of some directory entries. + +CCrreeddiittss + + * Manuel Guesdon: Spotted a bug with the timeout attribute. + * John Hensley: Multiple LDAP sources with more configurable attributes. + * Carsten Hoeger: Search scope handling. + * LaMont Jones: Domain restriction, URL and DN searches, multiple result + attributes. + * Mike Mattice: Alias dereferencing control. + * Hery Rakotoarisoa: Patches for LDAPv3 updating. + * Prabhat K Singh: Wrote the initial Postfix LDAP lookups and connection + caching. + * Keith Stevenson: RFC 2254 escaping in queries. + * Samuel Tardieu: Noticed that searches could include wildcards, prompting + the work on RFC 2254 escaping in queries. Spotted a bug in binding. + * Sami Haahtinen: Referral chasing and v3 support. + * Victor Duchovni: ldap_bind() timeout. With fixes from LaMont Jones: + OpenLDAP cache deprecation. Limits on recursion, expansion and search + results size. LDAP connection sharing for maps differing only in the query + parameters. + * Liviu Daia: Support for SSL/STARTTLS. Support for storing map definitions + in external files (ldap:/path/ldap.cf) needed to securely store passwords + for plain auth. + * Liviu Daia revised the configuration interface and added the main.cf + configuration feature. + * Liviu Daia with further refinements from Jose Luis Tallon and Victor + Duchovni developed the common query, result_format, domain and + expansion_limit interface for LDAP, MySQL and PosgreSQL. + * Gunnar Wrobel provided a first implementation of a feature to limit LDAP + search results to leaf nodes only. Victor generalized this into the Postfix + 2.4 "leaf_result_attribute" feature. + * Quanah Gibson-Mount contributed support for advanced LDAP SASL mechanisms, + beyond the password-based LDAP "simple" bind. + +And of course Wietse. + diff --git a/README_FILES/LINUX_README b/README_FILES/LINUX_README new file mode 100644 index 0000000..b828879 --- /dev/null +++ b/README_FILES/LINUX_README @@ -0,0 +1,74 @@ +PPoossttffiixx aanndd LLiinnuuxx + +------------------------------------------------------------------------------- + +HHoosstt llooookkuupp iissssuueess + +By default Linux /etc/hosts lookups do not support multiple IP addresses per +hostname. This causes warnings from the Postfix SMTP server that "hostname XXX +does not resolve to address YYY", and is especially a problem with hosts that +have both IPv4 and IPv6 addresses. To fix this, turn on support for multiple IP +addresses: + + /etc/host.conf: + ... + # We have machines with multiple IP addresses. + multi on + ... + +Alternatively, specify the RESOLV_MULTI environment variable in main.cf: + + /etc/postfix/main.cf: + import_environment = MAIL_CONFIG MAIL_DEBUG MAIL_LOGTAG TZ XAUTHORITY + DISPLAY LANG=C RESOLV_MULTI=on + +BBeerrkkeelleeyy DDBB iissssuueess + +If you can't compile Postfix because the file "db.h" isn't found, then you MUST +install the Berkeley DB development package (name: db???-devel-???) that +matches your system library. You can find out what is installed with the rpm +command. For example: + + $ rrppmm --qqff //uussrr//lliibb//lliibbddbb..ssoo + db4-4.3.29-2 + +This means that you need to install db4-devel-4.3.29-2 (on some systems, +specify "rrppmm --qqff //lliibb//lliibbddbb..ssoo" instead). + +DO NOT download some Berkeley DB version from the network. Every Postfix +program will dump core when it is built with a different Berkeley DB version +than the version that is used by the system library routines. See the DB_README +file for further information. + +PPrrooccmmaaiill iissssuueess + +On RedHat Linux 7.1 and later pprrooccmmaaiill no longer has permission to write to the +mail spool directory. Workaround: + + # chmod 1777 /var/spool/mail + +LLooggggiinngg iinn aa ccoonnttaaiinneerr + +When running Postfix inside a container, you can use stdout logging as +described in MAILLOG_README. Alternatives: run syslogd inside the container, or +mount the host's syslog socket inside the container. + +SSyyssllooggdd ppeerrffoorrmmaannccee + +LINUX ssyyssllooggdd uses synchronous writes by default. Because of this, ssyyssllooggdd can +actually use more system resources than Postfix. To avoid such badness, disable +synchronous mail logfile writes by editing /etc/syslog.conf and by prepending a +- to the logfile name: + + /etc/syslog.conf: + mail.* -/var/log/mail.log + +Send a "kkiillll --HHUUPP" to the ssyyssllooggdd to make the change effective. + +OOtthheerr llooggggiinngg ppeerrffoorrmmaannccee iissssuueess + +LINUX ssyysstteemmdd intercepts all logging and enforces its own rate limits before +handing off requests to a backend such as rrssyyssllooggdd or ssyysslloogg--nngg. On a busy mail +server this can result in information loss. As a workaround, you can use +Postfix's built-in logging as described in MAILLOG_README. + diff --git a/README_FILES/LMDB_README b/README_FILES/LMDB_README new file mode 100644 index 0000000..193880b --- /dev/null +++ b/README_FILES/LMDB_README @@ -0,0 +1,126 @@ +PPoossttffiixx OOppeennLLDDAAPP LLMMDDBB HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix uses databases of various kinds to store and look up information. +Postfix databases are specified as "type:name". OpenLDAP LMDB (called "LMDB" +from here on) implements the Postfix database type "lmdb". The name of a +Postfix LMDB database is the name of the database file without the ".lmdb" +suffix. + +This document describes: + + * Building Postfix with LMDB support. + + * Configuring LMDB settings. + + * Using LMDB maps with non-Postfix programs. + + * Required minimum LMDB patchlevel. + + * Credits. + +BBuuiillddiinngg PPoossttffiixx wwiitthh LLMMDDBB ssuuppppoorrtt + +Postfix normally does not enable LMDB support. To build Postfix with LMDB +support, use something like: + + % make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \ + AUXLIBS_LMDB="-L/usr/local/lib -llmdb" + % make + +If your LMDB shared library is in a directory that the RUN-TIME linker does not +know about, add a "-Wl,-R,/path/to/directory" option after "-llmdb". + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_LMDB. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded LMDB database client, but only the new AUXLIBS_LMDB variable supports +building a dynamically-loaded or statically-loaded LMDB database client. + + Failure to use the AUXLIBS_LMDB variable will defeat the purpose of dynamic + database client loading. Every Postfix executable file will have LMDB + database library dependencies. And that was exactly what dynamic database + client loading was meant to avoid. + +Solaris may need this: + + % make makefiles CCARGS="-DHAS_LMDB -I/usr/local/include" \ + AUXLIBS_LMDB="-R/usr/local/lib -L/usr/local/lib -llmdb" + % make + +The exact pathnames depend on how LMDB was installed. + +When building Postfix fails with: + + undefined reference to `pthread_mutexattr_destroy' + undefined reference to `pthread_mutexattr_init' + undefined reference to `pthread_mutex_lock' + +Add the "-lpthread" library to the "make makefiles" command. + + % make makefiles .... AUXLIBS_LMDB="... -lpthread" + +CCoonnffiigguurriinngg LLMMDDBB sseettttiinnggss + +Postfix provides one configuration parameter that controls LMDB database +behavior. + + * lmdb_map_size (default: 16777216). This setting specifies the initial LMDB + database size limit in bytes. Each time a database becomes "full", its size + limit is doubled. The maximum size is the largest signed integer value of + "long". + +UUssiinngg LLMMDDBB mmaappss wwiitthh nnoonn--PPoossttffiixx pprrooggrraammss + +Programs that use LMDB's built-in locking protocol will corrupt a Postfix LMDB +database or will read garbage. + +Postfix does not use LMDB's built-in locking protocol, because that would +require world-writable lockfiles, and would violate Postfix security policy. +Instead, Postfix uses external locks based on fcntl(2) to prevent writers from +corrupting the database, and to prevent readers from receiving garbage. + +See lmdb_table(5) for a detailed description of the locking protocol that all +programs must use when they access a Postfix LMDB database. + +RReeqquuiirreedd mmiinniimmuumm LLMMDDBB ppaattcchhlleevveell + +Currently, Postfix requires LMDB 0.9.11 or later. The required minimum LMDB +patchlevel has evolved over time, as the result of Postfix deployment +experience: + + * LMDB 0.9.11 allows Postfix daemons to log an LMDB error message, instead of + falling out of the sky without any notification. + + * LMDB 0.9.10 closes an information leak where LMDB was writing up to 4-kbyte + chunks of uninitialized heap memory to the database. This would persist + information that was not meant to be persisted, or share information that + was not meant to be shared. + + * LMDB 0.9.9 allows Postfix to use external (fcntl()-based) locks, instead of + having to use world-writable LMDB lock files, violating the Postfix + security model in multiple ways. + + * LMDB 0.9.8 allows Postfix to recover from a "database full" error without + having to close the database. This version adds support to update the + database size limit on-the-fly. This is necessary because Postfix database + sizes vary with mail server load. + + * LMDB 0.9.7 allows the postmap(1) and postalias(1) commands to use a bulk- + mode transaction larger than the amount of physical memory. This is + necessary because LMDB supports databases larger than physical memory. + +CCrreeddiittss + + * Howard Chu contributed the initial Postfix dict_lmdb driver. + + * Wietse Venema wrote an abstraction layer (slmdb) that behaves more like + Berkeley DB, NDBM, etc. This layer automatically retries an LMDB request + when a database needs to be resized, or after a database was resized by a + different process. + + * Howard and Wietse went through many iterations with changes to both LMDB + and Postfix, with input from Viktor Dukhovni. + diff --git a/README_FILES/LOCAL_RECIPIENT_README b/README_FILES/LOCAL_RECIPIENT_README new file mode 100644 index 0000000..ea4ce25 --- /dev/null +++ b/README_FILES/LOCAL_RECIPIENT_README @@ -0,0 +1,117 @@ +RReejjeeccttiinngg UUnnkknnoowwnn LLooccaall RReecciippiieennttss wwiitthh PPoossttffiixx + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +As of Postfix version 2.0, the Postfix SMTP server rejects mail for unknown +recipients in local domains (domains that match $mydestination or the IP +addresses in $inet_interfaces or $proxy_interfaces) with "User unknown in local +recipient table". This feature was optional with earlier Postfix versions. + +The good news is that this keeps undeliverable mail out of your queue, so that +your mail queue is not clogged up with undeliverable MAILER-DAEMON messages. + +The bad news is that it may cause mail to be rejected when you upgrade from a +Postfix system that was not configured to reject mail for unknown local +recipients. + +This document describes what steps are needed in order to reject unknown local +recipients correctly. + + * Configuring local_recipient_maps in main.cf + * When you need to change the local_recipient_maps setting in main.cf + * Local recipient table format + +CCoonnffiigguurriinngg llooccaall__rreecciippiieenntt__mmaappss iinn mmaaiinn..ccff + +The local_recipient_maps parameter specifies lookup tables with all names or +addresses of local recipients. A recipient address is local when its domain +matches $mydestination, $inet_interfaces or $proxy_interfaces. If a local +username or address is not listed in $local_recipient_maps, then the Postfix +SMTP server will reject the address with "User unknown in local recipient +table". + +The default setting, shown below, assumes that you use the default Postfix +local(8) delivery agent for local delivery, where recipients are either UNIX +accounts or local aliases: + + /etc/postfix/main.cf: + local_recipient_maps = proxy:unix:passwd.byname $alias_maps + +To turn off unknown local recipient rejects by the SMTP server, specify: + + /etc/postfix/main.cf: + local_recipient_maps = + +That is, an empty value. With this setting, the Postfix SMTP server will not +reject mail with "User unknown in local recipient table". DDoonn''tt ddoo tthhiiss oonn +ssyysstteemmss tthhaatt rreecceeiivvee mmaaiill ddiirreeccttllyy ffrroomm tthhee IInntteerrnneett.. WWiitthh ttooddaayy''ss wwoorrmmss aanndd +vviirruusseess,, PPoossttffiixx wwiillll bbeeccoommee aa bbaacckkssccaatttteerr ssoouurrccee:: iitt aacccceeppttss mmaaiill ffoorr nnoonn-- +eexxiisstteenntt rreecciippiieennttss aanndd tthheenn ttrriieess ttoo rreettuurrnn tthhaatt mmaaiill aass ""uunnddeelliivveerraabbllee"" ttoo +tthhee oofftteenn ffoorrggeedd sseennddeerr aaddddrreessss. + +WWhheenn yyoouu nneeeedd ttoo cchhaannggee tthhee llooccaall__rreecciippiieenntt__mmaappss sseettttiinngg iinn mmaaiinn..ccff + + * Problem: you don't use the default Postfix local(8) delivery agent for + domains matching $mydestination, $inet_interfaces, or $proxy_interfaces. + For example, you redefined the "local_transport" setting in main.cf. + + Solution: your local_recipient_maps setting needs to specify a database + that lists all the known user names or addresses for that delivery agent. + For example, if you deliver users in $mydestination etc. domains via the + virtual(8) delivery agent, specify: + + /etc/postfix/main.cf + mydestination = $myhostname localhost.$mydomain localhost ... + local_transport = virtual + local_recipient_maps = $virtual_mailbox_maps + + If you use a different delivery agent for $mydestination etc. domains, see + the section "Local recipient table format" below for a description of how + the table should be populated. + + * Problem: you use the mailbox_transport or fallback_transport feature of the + Postfix local(8) delivery agent in order to deliver mail to non-UNIX + accounts. + + Solution: you need to add the database that lists the non-UNIX users: + + /etc/postfix/main.cf + local_recipient_maps = proxy:unix:passwd.byname, $alias_maps, + <the database with non-UNIX accounts> + + See the section "Local recipient table format" below for a description of + how the table should be populated. + + * Problem: you use the luser_relay feature of the Postfix local delivery + agent. + + Solution: you must disable the local_recipient_maps feature completely, so + that Postfix accepts mail for all local addresses: + + /etc/postfix/main.cf + local_recipient_maps = + +LLooccaall rreecciippiieenntt ttaabbllee ffoorrmmaatt + +If you use local files in postmap(1) format, then local_recipient_maps expects +the following table format: + + * In the left-hand side, specify a bare username, an "@domain.tld" wild-card, + or specify a complete "user@domain.tld" address. + + * You have to specify something on the right-hand side of the table, but the + value is ignored by local_recipient_maps. + +If you use lookup tables based on NIS, LDAP, MYSQL, or PGSQL, then +local_recipient_maps does the same queries as for local files in postmap(1) +format, and expects the same results. + +With regular expression tables, Postfix only queries with the full recipient +address, and not with the bare username or the "@domain.tld" wild-card. + +NOTE: a lookup table should always return a result when the address exists, and +should always return "not found" when the address does not exist. In +particular, a zero-length result does not count as a "not found" result. + diff --git a/README_FILES/MAILDROP_README b/README_FILES/MAILDROP_README new file mode 100644 index 0000000..e1d320e --- /dev/null +++ b/README_FILES/MAILDROP_README @@ -0,0 +1,129 @@ +PPoossttffiixx ++ MMaaiillddrroopp HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document discusses various options to plug the maildrop delivery agent +into Postfix: + + * Direct delivery without the local delivery agent + * Indirect delivery via the local delivery agent + * Credits + +DDiirreecctt ddeelliivveerryy wwiitthhoouutt tthhee llooccaall ddeelliivveerryy aaggeenntt + +Postfix can be configured to deliver mail directly to maildrop, without using +the local(8) delivery agent as an intermediate. This means that you do not get +local aliases(5) expansion or $HOME/.forward file processing. You would +typically do this for hosted domains with recipients that don't have UNIX home +directories. + +The following example shows how to use maildrop for some.domain and for +someother.domain. The example comes in two parts. + +Part 1 describes changes to the main.cf file: + + 1 /etc/postfix/main.cf: + 2 maildrop_destination_recipient_limit = 1 + 3 virtual_mailbox_domains = some.domain someother.domain + 4 virtual_transport = maildrop + 5 virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox + 6 virtual_alias_maps = hash:/etc/postfix/virtual_alias + 7 + 8 /etc/postfix/virtual_mailbox: + 9 user1@some.domain ...text here does not matter... + 10 user2@some.domain ...text here does not matter... + 11 user3@someother.domain ...text here does not matter... + 12 + 13 /etc/postfix/virtual_alias: + 14 postmaster@some.domain postmaster + 15 postmaster@someother.domain postmaster + + * Line 2 is needed so that Postfix will provide one recipient at a time to + the maildrop delivery agent. + + * Line 3 informs Postfix that some.domain and someother.domain are so-called + virtual mailbox domains. Instead of listing the names in main.cf you can + also list them in a file; see the virtual_mailbox_domains documentation for + details. + + * Line 4 specifies that mail for some.domain and someother.domain should be + delivered by the maildrop delivery agent. + + * Lines 5 and 8-11 specify what recipients the Postfix SMTP server should + receive mail for. This prevents the mail queue from becoming clogged with + undeliverable messages. Specify an empty value ("virtual_mailbox_maps =") + to disable this feature. + + * Lines 6 and 13-15 redirect mail for postmaster to the local postmaster. RFC + 821 requires that every domain has a postmaster address. + +The vmail userid as used below is the user that maildrop should run as. This +would be the owner of the virtual mailboxes if they all have the same owner. If +maildrop is suid (see maildrop documentation), then maildrop will change to the +appropriate owner to deliver the mail. + +Note: Do not use the postfix user as the maildrop user. + +Part 2 describes changes to the master.cf file: + + /etc/postfix/master.cf: + maildrop unix - n n - - pipe + flags=ODRhu user=vmail argv=/path/to/maildrop -d ${recipient} + +The pipe(8) manual page gives a detailed description of the above command line +arguments, and more. + +If you want to support user+extension@domain style addresses, use the following +instead: + + /etc/postfix/master.cf: + maildrop unix - n n - - pipe + flags=ODRhu user=vmail argv=/path/to/maildrop + -d ${user}@${domain} ${extension} ${recipient} ${user} ${nexthop} + +The mail is delivered to ${user}@${domain} (search key for maildrop userdb +lookup). The ${extension} and the other address components are available to +maildrop rules as $1, $2, $3, ... and can be omitted from master.cf or ignored +by maildrop when not needed. + +With Postfix 2.4 and earlier, use ${nexthop} instead of ${domain}. + +IInnddiirreecctt ddeelliivveerryy vviiaa tthhee llooccaall ddeelliivveerryy aaggeenntt + +Postfix can be configured to deliver mail to maildrop via the local delivery +agent. This is slightly less efficient than the "direct" approach discussed +above, but gives you the convenience of local aliases(5) expansion and +$HOME/.forward file processing. You would typically use this for domains that +are listed in mydestination and that have users with a UNIX system account. + +To configure maildrop delivery for all UNIX system accounts: + + /etc/postfix/main.cf: + mailbox_command = /path/to/maildrop -d ${USER} + +Note: ${USER} is spelled in upper case. + +To enable maildrop delivery for specific users only, you can use the Postfix +local(8) delivery agent's mailbox_command_maps feature: + + /etc/postfix/main.cf: + mailbox_command_maps = hash:/etc/postfix/mailbox_commands + + /etc/postfix/mailbox_commands: + you /path/to/maildrop -d ${USER} + +Maildrop delivery for specific users is also possible by invoking it from the +user's $HOME/.forward file: + + /home/you/.forward: + "|/path/to/maildrop -d ${USER}" + +CCrreeddiittss + + * The original text was kindly provided by Russell Mosemann. + * Victor Duchovni provided tips for supporting user+foo@domain addresses. + * Tonni Earnshaw contributed text about delivery via the local(8) delivery + agent. + diff --git a/README_FILES/MAILLOG_README b/README_FILES/MAILLOG_README new file mode 100644 index 0000000..6f8fead --- /dev/null +++ b/README_FILES/MAILLOG_README @@ -0,0 +1,113 @@ +PPoossttffiixx llooggggiinngg ttoo ffiillee oorr ssttddoouutt + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +Postfix supports its own logging system as an alternative to syslog (which +remains the default). This is available with Postfix version 3.4 or later. + +Topics covered in this document: + + * Configuring logging to file + * Configuring logging to stdout + * Rotating logs + * Limitations + +CCoonnffiigguurriinngg llooggggiinngg ttoo ffiillee + +Logging to file solves a usability problem for MacOS, and eliminates multiple +problems for systemd-based systems. + + 1. Add the following line to master.cf if not already present (note: there + must be no whitespace at the start of the line): + + postlog unix-dgram n - n - 1 postlogd + + Note: the service type "uunniixx--ddggrraamm" was introduced with Postfix 3.4. Remove + the above line before backing out to an older Postfix version. + + 2. Configure Postfix to write logging, to, for example, /var/log/postfix.log. + See also the "Logfile rotation" section below for logfile management. + + # postfix stop + # postconf maillog_file=/var/log/postfix.log + # postfix start + + By default, the logfile name must start with "/var" or "/dev/stdout" (the + list of allowed prefixes is configured with the maillog_file_prefixes + parameter). This safety mechanism limits the damage from a single + configuration mistake. + +CCoonnffiigguurriinngg llooggggiinngg ttoo ssttddoouutt + +Logging to stdout is useful when Postfix runs in a container, as it eliminates +a syslogd dependency. + + 1. Add the following line to master.cf if not already present (note: there + must be no whitespace at the start of the line): + + postlog unix-dgram n - n - 1 postlogd + + Note: the service type "uunniixx--ddggrraamm" was introduced with Postfix 3.4. Remove + the above line before backing out to an older Postfix version. + + 2. Configure main.cf with "maillog_file = /dev/stdout". + + 3. Start Postfix with "ppoossttffiixx ssttaarrtt--ffgg". + +RRoottaattiinngg llooggss + +The command "ppoossttffiixx llooggrroottaattee" may be run by hand or by a cronjob. It logs all +errors, and reports errors to stderr if run from a terminal. This command +implements the following steps: + + * Rename the current logfile by appending a suffix that contains the date and + time. This suffix is configured with the maillog_file_rotate_suffix + parameter (default: %Y%m%d-%H%M%S). + + * Reload Postfix so that postlogd(8) immediately closes the old logfile. + + * After a brief pause, compress the old logfile. The compression program is + configured with the maillog_file_compressor parameter (default: gzip). + +Notes: + + * This command will not rotate a logfile with a pathname under the /dev + directory, such as /dev/stdout. + + * This command does not (yet) remove old logfiles. + +LLiimmiittaattiioonnss + +Background: + + * Postfix consists of a number of daemon programs that run in the background, + as well as non-daemon programs for local mail submission or Postfix + management. + + * Logging to the Postfix logfile or stdout requires the Postfix postlogd(8) + service. This ensures that simultaneous logging from different programs + will not get mixed up. + + * All Postfix programs can log to syslog, but not all programs have + sufficient privileges to use the Postfix logging service, and many non- + daemon programs must not log to stdout as that would corrupt their output. + +Limitations: + + * Non-daemon Postfix programs will log errors to syslogd(8) before they have + processed command-line options and main.cf parameters. + + * If Postfix is down, the non-daemon programs postfix(1), postsuper(1), + postmulti(1), and postlog(1), will log directly to $maillog_file. These + programs expect to run with root privileges, for example during Postfix + start-up, reload, or shutdown. + + * Other non-daemon Postfix programs will never write directly to + $maillog_file (also, logging to stdout would interfere with the operation + of some of these programs). These programs can log to postlogd(8) if they + are run by the super-user, or if their executable file has set-gid + permission. Do not set this permission on programs other than postdrop(1), + postqueue(1), and (Postfix >= 3.7) postlog(1). + diff --git a/README_FILES/MEMCACHE_README b/README_FILES/MEMCACHE_README new file mode 100644 index 0000000..f89e148 --- /dev/null +++ b/README_FILES/MEMCACHE_README @@ -0,0 +1,50 @@ +PPoossttffiixx mmeemmccaacchhee cclliieenntt HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +The Postfix memcache client allows you to hook up Postfix to a memcache server. +The current implementation supports one memcache server per Postfix table, with +one optional Postfix database that provides persistent backup. The Postfix +memcache client supports the lookup, update, delete and sequence operations. +The sequence (i.e. first/next) operation requires a backup database that +supports this operation. + +Typically, the Postfix memcache client is used to reduce query load on a +persistent database, but it may also be used to query a memory-only database +for low-value, easy-to-recreate, information such as a reputation cache for +postscreen(8), verify(8) or greylisting. + +LLiimmiittaattiioonnss + + * The Postfix memcache client cannot be used for security-sensitive tables + such as alias_maps (these may contain "|command" and "/file/name" + destinations), or virtual_uid_maps, virtual_gid_maps and + virtual_mailbox_maps (these specify UNIX process privileges or "/file/name" + destinations). Typically, a memcache database is writable by any process + that can talk to the memcache server; in contrast, security-sensitive + tables must never be writable by the unprivileged Postfix user. + + * The Postfix memcache client requires additional configuration when used as + postscreen(8) or verify(8) cache. For details see the backup and ttl + parameter discussions in the memcache_table(5) manual page. + +BBuuiillddiinngg PPoossttffiixx wwiitthh mmeemmccaacchhee ssuuppppoorrtt + +The Postfix memcache client has no external dependencies, and is therefore +built into Postfix by default. + +CCoonnffiigguurriinngg mmeemmccaacchhee llooookkuupp ttaabblleess + +Configuration is described in the memcache_table(5) manpage. + +CCrreeddiittss + +The first memcache client for Postfix was written by Omar Kilani, and was based +on the libmemcache library. + +Wietse wrote the current memcache client from the ground up for Postfix version +2.9. This implementation does not use libmemcache, and bears no resemblance to +earlier work. + diff --git a/README_FILES/MILTER_README b/README_FILES/MILTER_README new file mode 100644 index 0000000..4ace868 --- /dev/null +++ b/README_FILES/MILTER_README @@ -0,0 +1,684 @@ +PPoossttffiixx bbeeffoorree--qquueeuuee MMiilltteerr ssuuppppoorrtt + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +Postfix implements support for the Sendmail version 8 Milter (mail filter) +protocol. This protocol is used by applications that run outside the MTA to +inspect SMTP events (CONNECT, DISCONNECT), SMTP commands (HELO, MAIL FROM, +etc.) as well as mail content (headers and body). All this happens before mail +is queued. + +The reason for adding Milter support to Postfix is that there exists a large +collection of applications, not only to block unwanted mail, but also to verify +authenticity (examples: OpenDKIM and DMARC) or to digitally sign mail (example: +OpenDKIM). Having yet another Postfix-specific version of all that software is +a poor use of human and system resources. + +The Milter protocol has evolved over time, and different Postfix versions +implement different feature sets. See the workarounds and limitations sections +at the end of this document for differences between Postfix and Sendmail +implementations. + +This document provides information on the following topics: + + * How Milter applications plug into Postfix + * When Postfix and Milters inspect an SMTP session + * Building Milter applications + * Running Milter applications + * Configuring Postfix + * Workarounds + * Limitations + +HHooww MMiilltteerr aapppplliiccaattiioonnss pplluugg iinnttoo PPoossttffiixx + +The Postfix Milter implementation uses two different lists of mail filters: one +list of filters for SMTP mail only, and one list of filters for non-SMTP mail. +The two lists have different capabilities, which is unfortunate. Avoiding this +would require major restructuring of Postfix. + + * The SMTP-only filters handle mail that arrives via the Postfix smtpd(8) + server. They are typically used to filter unwanted mail and to sign mail + from authorized SMTP clients. You specify SMTP-only Milter applications + with the smtpd_milters parameter as described in a later section. Mail that + arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP + filters that are described next. + + * The non-SMTP filters handle mail that arrives via the Postfix sendmail(1) + command-line or via the Postfix qmqpd(8) server. They are typically used to + digitally sign mail only. Although non-SMTP filters can be used to filter + unwanted mail, they have limitations compared to the SMTP-only filters. You + specify non-SMTP Milter applications with the non_smtpd_milters parameter + as described in a later section. + +For those who are familiar with the Postfix architecture, the figure below +shows how Milter applications plug into Postfix. Names followed by a number are +Postfix commands or server programs, while unnumbered names inside shaded areas +represent Postfix queues. To avoid clutter, the path for local submission is +simplified (the OVERVIEW document has a more complete description of the +Postfix architecture). + + SMTP-only non-SMTP + filters filters + + ^ | + | v + ^ | + | | + Network -> smtpd(8) | | + | v + + \ + + Network -> qmqpd(8) -> cleanup(8) -> incoming + + / + + pickup(8) + + : + + Local -> sendmail(1) + +WWhheenn PPoossttffiixx aanndd MMiilltteerrss iinnssppeecctt aann SSMMTTPP sseessssiioonn + +Generally, Postfix inspects information first, then the first configured +Milter, the second configured Milter, and so on. + + * With most SMTP commands: Postfix reviews one SMTP command, and if Postfix + does not reject it, Postfix passes the command to the first configured + Milter. If the first Milter does not reject the command, Postfix passes it + to the second configured Milter, and so on. This includes commands with an + envelope sender (MAIL FROM) or envelope recipient (RCPT TO). Postfix stores + the same envelope records in a queue file as when no Milters are + configured, including rewritten envelope addresses, expanded virtual + aliases, BCC addresses from sender/recipient_bcc_maps, and so on. + + * With header/body content: Postfix may rewrite or reject header/body content + before it stores that content in the queue file; Postfix stores the same + header/body content as when no Milters are configured. If Postfix does not + reject the header/body content, Postfix passes it to the first configured + Milter which may modify or reject that content or may modify the stored + envelope. If the first Milter does not reject the header/body content, + Postfix passes it to the second configured Milter, and so on. + +Details: + + * Postfix hides its own Postfix-prepended Received: header, for compatibility + with Sendmail. Postfix does not hide other headers that Postfix or Milters + added or modified. + + * When the Postfix SMTP server receives a sequence of one or more valid BDAT + commands, it generates one DATA command for the Milters. + + * The Milter API does not support inspection of SMTP commands such as QUIT, + NOOP, or VRFY; the API supports only commands that are needed for email + delivery. + +BBuuiillddiinngg MMiilltteerr aapppplliiccaattiioonnss + +Milter applications have been written in C, Haskell, Java, Perl, Python, Rust, +and more, but this document covers C applications only. For these, you need an +object library that implements the Sendmail 8 Milter protocol. Postfix +currently does not provide such a library, but Sendmail does. + +Some systems install the Sendmail libmilter library by default. With other +systems, libmilter may be provided by a package (called "sendmail-devel" on +some Linux systems). + +Once libmilter is installed, applications such as OpenDKIM and OpenDMARC build +out of the box without requiring any tinkering: + + $ ggzzccaatt ooppeennddkkiimm--xx..yy..zz..ttaarr..ggzz || ttaarr xxff -- + $ ccdd ooppeennddkkiimm--xx..yy..zz + $ ..//ccoonnffiigguurree ......ooppttiioonnss...... + $ mmaakkee + [...lots of output omitted...] + $ mmaakkee iinnssttaallll + +RRuunnnniinngg MMiilltteerr aapppplliiccaattiioonnss + +To run a Milter application, see the documentation of the filter for options. A +typical command looks like this: + + # //ssoommee//wwhheerree//ooppeennddkkiimm --ll --uu uusseerriidd --pp iinneett::ppoorrttnnuummbbeerr@@llooccaallhhoosstt ......ootthheerr + ooppttiioonnss...... + +Please specify a userid value that isn't used for other applications (not +"postfix", not "www", etc.). + +CCoonnffiigguurriinngg PPoossttffiixx + +Like Sendmail, Postfix has a lot of configuration options that control how it +talks to Milter applications. Besides global options that apply to all Milter +applications, Postfix 3.0 and later support per-Milter timeouts, per-Milter +error handling, etc. + +Information in this section: + + * SMTP-Only Milter applications + * Non-SMTP Milter applications + * Milter error handling + * Milter protocol version + * Milter protocol timeouts + * Different settings for different Milter applications + * Different settings for different SMTP clients + * Sendmail macro emulation + * What macros will Postfix send to Milters? + +SSMMTTPP--OOnnllyy MMiilltteerr aapppplliiccaattiioonnss + +The SMTP-only Milter applications handle mail that arrives via the Postfix +smtpd(8) server. They are typically used to filter unwanted mail, and to sign +mail from authorized SMTP clients. Mail that arrives via the Postfix smtpd(8) +server is not filtered by the non-SMTP filters that are described in the next +section. + + NOTE for Postfix versions that have a mail_release_date before 20141018: do + not use the header_checks(5) IGNORE action to remove Postfix's own + Received: message header. This causes problems with mail signing filters. + Instead, keep Postfix's own Received: message header and use the + header_checks(5) REPLACE action to sanitize information. + +You specify SMTP-only Milter applications (there can be more than one) with the +smtpd_milters parameter. Each Milter application is identified by the name of +its listening socket; other Milter configuration options will be discussed in +later sections. Postfix sends commands to each Milter application in the order +as configured with smtpd_milters. When a Milter application rejects a command, +that will override responses from other Milter applications. + + /etc/postfix/main.cf: + # Milters for mail that arrives via the smtpd(8) server. + # See below for socket address syntax. + smtpd_milters = inet:localhost:portnumber ...other filters... + +The general syntax for listening sockets is as follows: + + uunniixx::pathname + Connect to the local UNIX-domain server that is bound to the specified + pathname. If the smtpd(8) or cleanup(8) process runs chrooted, an + absolute pathname is interpreted relative to the Postfix queue + directory. On many systems, llooccaall is a synonym for uunniixx + + iinneett::host::port + Connect to the specified TCP port on the specified local or remote + host. The host and port can be specified in numeric or symbolic form. + + NOTE: Postfix syntax differs from Milter syntax which has the form + iinneett::port@@host. + +For advanced configuration see "Different settings for different SMTP clients" +and "Different settings for different Milter applications". + +NNoonn--SSMMTTPP MMiilltteerr aapppplliiccaattiioonnss + +The non-SMTP Milter applications handle mail that arrives via the Postfix +sendmail(1) command-line or via the Postfix qmqpd(8) server. They are typically +used to digitally sign mail. Although non-SMTP filters can be used to filter +unwanted mail, there are limitations as discussed later in this section. Mail +that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP +filters. + +NOTE: Do not use the header_checks(5) IGNORE action to remove Postfix's own +Received: message header. This causes problems with mail signing filters. +Instead, keep Postfix's own Received: message header and use the header_checks +(5) REPLACE action to sanitize information. + +You specify non-SMTP Milter applications with the non_smtpd_milters parameter. +This parameter uses the same syntax as the smtpd_milters parameter in the +previous section. As with the SMTP-only filters, you can specify more than one +Milter application. Postfix sends commands to each Milter application in the +order as configured with non_smtpd_milters. When a Milter application rejects a +command, that will override responses from other Milter applications. + + /etc/postfix/main.cf: + # Milters for non-SMTP mail. + # See below for socket address syntax. + non_smtpd_milters = inet:localhost:portnumber ...other filters... + +There's one small complication when using Milter applications for non-SMTP +mail: there is no SMTP session. To keep Milter applications happy, the Postfix +cleanup(8) server actually has to simulate the SMTP client CONNECT and +DISCONNECT events, and the SMTP client EHLO, MAIL FROM, RCPT TO and DATA +commands. + + * When new mail arrives via the sendmail(1) command line, the Postfix cleanup + (8) server pretends that the mail arrives with ESMTP from "localhost" with + IP address "127.0.0.1". The result is very similar to what happens with + command line submissions in Sendmail version 8.12 and later, although + Sendmail uses a different mechanism to achieve this result. + + * When new mail arrives via the qmqpd(8) server, the Postfix cleanup(8) + server pretends that the mail arrives with ESMTP, and uses the QMQPD client + hostname and IP address. + + * When old mail is re-injected into the queue with "postsuper -r", the + Postfix cleanup(8) server uses the same client information that was used + when the mail arrived as new mail. + +This generally works as expected, with only one exception: non-SMTP filters +must not REJECT or TEMPFAIL simulated RCPT TO commands. When a +non_smtpd_milters application REJECTs or TEMPFAILs a recipient, Postfix will +report a configuration error, and mail will stay in the queue. + +SSiiggnniinngg iinntteerrnnaallllyy--ggeenneerraatteedd bboouunnccee mmeessssaaggeess + +Postfix normally does not apply content filters to mail that is generated +internally such as bounces or Postmaster notifications. Filtering internally- +generated bounces would result in loss of mail when a filter rejects a message, +as the resulting double-bounce message would almost certainly also be blocked. + +To sign Postfix's own bounce messages, enable filtering of internally-generated +bounces (line 2 below), and don't reject any internally-generated bounces with +non_smtpd_milters, header_checks or body_checks (lines 3-5 below). + + 1 /etc/postfix/main.cf: + 2 internal_mail_filter_classes = bounce + 3 non_smtpd_milters = don't reject internally-generated bounces + 4 header_checks = don't reject internally-generated bounces + 5 body_checks = don't reject internally-generated bounces + +MMiilltteerr eerrrroorr hhaannddlliinngg + +The milter_default_action parameter specifies how Postfix handles Milter +application errors. The default action is to respond with a temporary error +status, so that the client will try again later. Specify "accept" if you want +to receive mail as if the filter does not exist, and "reject" to reject mail +with a permanent status. The "quarantine" action is like "accept" but freezes +the message in the "hold" queue, and is available with Postfix 2.6 or later. + + /etc/postfix/main.cf: + # What to do in case of errors? Specify accept, reject, tempfail, + # or quarantine (Postfix 2.6 or later). + milter_default_action = tempfail + +See "Different settings for different Milter applications" for advanced +configuration options. + +MMiilltteerr pprroottooccooll vveerrssiioonn + +As Postfix is not built with the Sendmail libmilter library, you may need to +configure the Milter protocol version that Postfix should use. The default +version is 6 (before Postfix 2.6 the default version is 2). + + /etc/postfix/main.cf: + # Postfix >= 2.6 + milter_protocol = 6 + # 2.3 <= Postfix <= 2.5 + milter_protocol = 2 + +If the Postfix milter_protocol setting specifies a too low version, the +libmilter library will log an error message like this: + + application name: st_optionneg[xxxxx]: 0xyy does not fulfill action + requirements 0xzz + +The remedy is to increase the Postfix milter_protocol version number. See, +however, the limitations section below for features that aren't supported by +Postfix. + +With Postfix 2.7 and earlier, if the Postfix milter_protocol setting specifies +a too high version, the libmilter library simply hangs up without logging a +warning, and you see a Postfix warning message like one of the following: + + warning: milter inet:host:port: can't read packet header: Unknown error : 0 + warning: milter inet:host:port: can't read packet header: Success + warning: milter inet:host:port: can't read SMFIC_DATA reply packet header: + No such file or directory + +The remedy is to lower the Postfix milter_protocol version number. Postfix 2.8 +and later will automatically turn off protocol features that the application's +libmilter library does not expect. + +See "Different settings for different Milter applications" for advanced +configuration options. + +MMiilltteerr pprroottooccooll ttiimmeeoouuttss + +Postfix uses different time limits at different Milter protocol stages. The +table shows the timeout settings and the corresponding protocol stages (EOH = +end of headers; EOM = end of message). + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPoossttffiixx ppaarraammeetteerr |TTiimmee lliimmiitt|MMiilltteerr pprroottooccooll ssttaaggee | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_connect_timeout|30s |CONNECT | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_command_timeout|30s |HELO, MAIL, RCPT, DATA, UNKNOWN| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_content_timeout|300s |HEADER, EOH, BODY, EOM | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +Beware: 30s may be too short for Milter applications that do lots of DNS +lookups. However, if you increase the above timeouts too much, remote SMTP +clients may hang up and mail may be delivered multiple times. This is an +inherent problem with before-queue filtering. + +See "Different settings for different Milter applications" for advanced +configuration options. + +DDiiffffeerreenntt sseettttiinnggss ffoorr ddiiffffeerreenntt MMiilltteerr aapppplliiccaattiioonnss + +The previous sections list a number of Postfix main.cf parameters that control +time limits and other settings for all Postfix Milter clients. This is +sufficient for simple configurations. With more complex configurations it +becomes desirable to have different settings for different Milter clients. This +is supported with Postfix 3.0 and later. + +The following example shows a "non-critical" Milter client with a short connect +timeout, and with "accept" as default action when the service is unvailable. + + 1 /etc/postfix/main.cf: + 2 smtpd_milters = { inet:host:port, + 3 connect_timeout=10s, default_action=accept } + +Instead of a server endpoint, we now have a list enclosed in {}. + + * Line 2: The first item in the list is the server endpoint. This supports + the exact same "inet" and "unix" syntax as described earlier. + + * Line 3: The remainder of the list contains per-Milter settings. These + settings override global main.cf parameters, and have the same name as + those parameters, without the "milter_" prefix. The per-Milter settings + that are supported as of Postfix 3.0 are command_timeout, connect_timeout, + content_timeout, default_action, and protocol. + +Inside the list, syntax is similar to what we already know from main.cf: items +separated by space or comma. There is one difference: yyoouu mmuusstt eenncclloossee aa +sseettttiinngg iinn ppaarreenntthheesseess,, aass iinn ""{{ nnaammee == vvaalluuee }}"",, iiff yyoouu wwaanntt ttoo hhaavvee ssppaaccee oorr +ccoommmmaa wwiitthhiinn aa vvaalluuee oorr aarroouunndd ""=="". + +DDiiffffeerreenntt sseettttiinnggss ffoorr ddiiffffeerreenntt SSMMTTPP cclliieennttss + +The smtpd_milter_maps feature supports different Milter settings for different +client IP addresses. Lookup results override the the global smtpd_milters +setting, and have the same syntax. For example, to disable Milter settings for +local address ranges: + +/etc/postfix/main.cf: + smtpd_milter_maps = cidr:/etc/postfix/smtpd_milter_map + smtpd_milters = inet:host:port, { inet:host:port, ... }, ... + +/etc/postfix/smtpd_milter_map: + # Disable Milters for local clients. + 127.0.0.0/8 DISABLE + 192.168.0.0/16 DISABLE + ::/64 DISABLE + 2001:db8::/32 DISABLE + +This feature is available with Postfix 3.2 and later. + +SSeennddmmaaiill mmaaccrroo eemmuullaattiioonn + +Postfix emulates a limited number of Sendmail macros, as shown in the table. +Some macro values depend on whether a recipient is rejected (rejected +recipients are available on request by the Milter application). Different +macros are available at different Milter protocol stages (EOH = end-of-header, +EOM = end-of-message); their availability is not always the same as in +Sendmail. See the workarounds section below for solutions. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |SSeennddmmaaiill mmaaccrroo |MMiilltteerr pprroottooccooll ssttaaggee |DDeessccrriippttiioonn | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |i |DATA, EOH, EOM |Queue ID, also Postfix | + | | |queue file name | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |j |Always |Value of myhostname | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |_ |Always |The validated client name | + | | |and address | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{auth_authen} |MAIL, DATA, EOH, EOM |SASL login name | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{auth_author} |MAIL, DATA, EOH, EOM |SASL sender | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{auth_type} |MAIL, DATA, EOH, EOM |SASL login method | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{client_addr} |Always |Remote client IP address | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Connection concurrency for| + | | |this client (zero if the | + |{client_connections}|CONNECT |client is excluded from | + | | |all smtpd_client_* | + | | |limits). | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Remote client hostname | + | | |When address -> name | + |{client_name} |Always |lookup or name -> address | + | | |verification fails: | + | | |"unknown" | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{client_port} |Always (Postfix >=2.5) |Remote client TCP port | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Client name from address -| + |{client_ptr} |CONNECT, HELO, MAIL, DATA|> name lookup | + | | |When address -> name | + | | |lookup fails: "unknown" | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{cert_issuer} |HELO, MAIL, DATA, EOH, |TLS client certificate | + | |EOM |issuer | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{cert_subject} |HELO, MAIL, DATA, EOH, |TLS client certificate | + | |EOM |subject | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{cipher_bits} |HELO, MAIL, DATA, EOH, |TLS session key size | + | |EOM | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{cipher} |HELO, MAIL, DATA, EOH, |TLS cipher | + | |EOM | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{daemon_addr} |Always (Postfix >=3.2) |Local server IP address | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{daemon_name} |Always |value of | + | | |milter_macro_daemon_name | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{daemon_port} |Always (Postfix >=3.2) |Local server TCP port | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{mail_addr} |MAIL |Sender address | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{mail_host} |MAIL (Postfix >= 2.6, |Sender next-hop | + | |only with smtpd_milters) |destination | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{mail_mailer} |MAIL (Postfix >= 2.6, |Sender mail delivery | + | |only with smtpd_milters) |transport | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Recipient address | + |{rcpt_addr} |RCPT |With rejected recipient: | + | | |descriptive text | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Recipient next-hop | + |{rcpt_host} |RCPT (Postfix >= 2.6, |destination | + | |only with smtpd_milters) |With rejected recipient: | + | | |enhanced status code | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | | |Recipient mail delivery | + |{rcpt_mailer} |RCPT (Postfix >= 2.6, |transport | + | |only with smtpd_milters) |With rejected recipient: | + | | |"error" | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |{tls_version} |HELO, MAIL, DATA, EOH, |TLS protocol version | + | |EOM | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |v |Always |value of milter_macro_v | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +WWhhaatt mmaaccrrooss wwiillll PPoossttffiixx sseenndd ttoo MMiilltteerrss?? + +Postfix sends specific sets of macros at different Milter protocol stages. The +names of these macros are configured with the parameters shown in the table +below (EOH = end of headers; EOM = end of message). Some lists require a +minimum Milter protocol version. + +As of Sendmail 8.14.0, Milter applications can specify what macros they want to +receive at different Milter protocol stages. An application-specified list +takes precedence over a Postfix-specified list. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPoossttffiixx ppaarraammeetteerr |MMiilltteerr pprroottooccooll|MMiilltteerr pprroottooccooll ssttaaggee| + | |vveerrssiioonn | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_connect_macros |2 or higher |CONNECT | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_helo_macros |2 or higher |HELO/EHLO | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_mail_macros |2 or higher |MAIL FROM | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_rcpt_macros |2 or higher |RCPT TO | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_data_macros |4 or higher |DATA | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_end_of_header_macros |6 or higher |EOH | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_end_of_data_macros |2 or higher |EOM | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |milter_unknown_command_macros|3 or higher |unknown command | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +By default, Postfix will send only macros whose values have been updated with +information from main.cf or master.cf, from an SMTP session (for example; SASL +login, or TLS certificates) or from a Mail delivery transaction (for example; +queue ID, sender, or recipient). + +To force a macro to be sent even when its value has not been updated, you may +specify macro default values with the milter_macro_defaults parameter. Specify +zero or more name=value pairs separated by comma or whitespace; you may even +specify macro names that Postfix does not know about! + +WWoorrkkaarroouunnddss + + * To avoid breaking DKIM etc. signatures with an SMTP-based content filter, + update the before-filter SMTP client in master.cf, and add a line with "- + o disable_mime_output_conversion=yes" (note: no spaces around the "="). For + details, see the advanced content filter example. + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + scan unix - - n - 10 smtp + -o smtp_send_xforward_command=yes + -o disable_mime_output_conversion=yes + -o smtp_generic_maps= + + * Some Milter applications use the "{if_addr}" macro to recognize local mail; + this macro does not exist in Postfix. Workaround: use the "{daemon_addr}" + (Postfix >= 3.2) or "{client_addr}" macro instead. + + * Some Milter applications log a warning that looks like this: + + sid-filter[36540]: WARNING: sendmail symbol 'i' not available + + And they may insert an ugly message header with "unknown-msgid" like this: + + X-SenderID: Sendmail Sender-ID Filter vx.y.z host.example.com <unknown- + msgid> + + The problem is that Milter applications expect that the queue ID is known + before the MTA accepts the MAIL FROM (sender) command. Postfix does not + choose a queue ID, which is used as the queue file name, until after it + accepts the first valid RCPT TO (recipient) command. + + If you experience the ugly header problem, see if a recent version of the + Milter application fixes it. For example, current versions of dkim-filter + and dk-filter already have code that looks up the Postfix queue ID at a + later protocol stage, and sid-filter version 1.0.0 no longer includes the + queue ID in the message header. + + To fix the ugly message header, you will need to add code that looks up the + Postfix queue ID at some later point in time. The example below adds the + lookup after the end-of-message. + + o Edit the filter source file (typically named xxx-filter/xxx-filter.c or + similar). + + o Look up the mlfi_eom() function and add code near the top shown as bboolldd + text below: + + dfc = cc->cctx_msg; + assert(dfc != NULL); + + //** DDeetteerrmmiinnee tthhee jjoobb IIDD ffoorr llooggggiinngg.. **// + iiff ((ddffcc-->>mmccttxx__jjoobbiidd ==== 00 |||| ssttrrccmmpp((ddffcc-->>mmccttxx__jjoobbiidd,, JJOOBBIIDDUUNNKKNNOOWWNN)) ==== 00)) + {{ + cchhaarr **jjoobbiidd == ssmmffii__ggeettssyymmvvaall((ccttxx,, ""ii""));; + iiff ((jjoobbiidd !!== 00)) + ddffcc-->>mmccttxx__jjoobbiidd == jjoobbiidd;; + }} + + NOTES: + + o Different mail filters use slightly different names for variables. If + the above code does not compile, look elsewhere in the mail filter + source file for code that looks up the "i" macro value, and copy that + code. + + o This change fixes only the ugly message header, but not the WARNING + message. Fortunately, many Milters log that message only once. + +LLiimmiittaattiioonnss + +This section lists limitations of the Postfix Milter implementation. Some +limitations will be removed as the implementation is extended over time. Of +course the usual limitations of before-queue filtering will always apply. See +the CONTENT_INSPECTION_README document for a discussion. + + * The Milter protocol has evolved over time. Therefore, different Postfix + versions implement different feature sets. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPoossttffiixx|SSuuppppoorrtteedd MMiilltteerr rreeqquueessttss | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | 2.6 |All Milter requests of Sendmail 8.14.0 (see notes below). | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | |All Milter requests of Sendmail 8.14.0, except: | + | |SMFIP_RCPT_REJ (report rejected recipients to the mail filter), | + | 2.5 |SMFIR_CHGFROM (replace sender, with optional ESMTP parameters), | + | |SMFIR_ADDRCPT_PAR (add recipient, with optional ESMTP | + | |parameters). | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | 2.4 |All Milter requests of Sendmail 8.13.0. | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + | 2.3 |All Milter requests of Sendmail 8.13.0, except: | + | |SMFIR_REPLBODY (replace message body). | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + + * For Milter applications that are written in C, you need to use the Sendmail + libmilter library. + + * Postfix has TWO sets of mail filters: filters that are used for SMTP mail + only (specified with the smtpd_milters parameter), and filters for non-SMTP + mail (specified with the non_smtpd_milters parameter). The non-SMTP filters + are primarily for local submissions. + + When mail is filtered by non_smtpd_milters, the Postfix cleanup(8) server + has to simulate SMTP client requests. This works as expected, with only one + exception: non_smtpd_milters must not REJECT or TEMPFAIL simulated RCPT TO + commands. When this rule is violated, Postfix will report a configuration + error, and mail will stay in the queue. + + * When you use the before-queue content filter for incoming SMTP mail (see + SMTPD_PROXY_README), Milter applications have access only to the SMTP + command information; they have no access to the message header or body, and + cannot make modifications to the message or to the envelope. + + * Postfix 2.6 ignores the optional ESMTP parameters in requests to replace + the sender (SMFIR_CHGFROM) or to append a recipient (SMFIR_ADDRCPT_PAR). + Postfix logs a warning message when a Milter application supplies such + ESMTP parameters: + + warning: queue-id: cleanup_chg_from: ignoring ESMTP arguments "whatever" + warning: queue-id: cleanup_add_rcpt: ignoring ESMTP arguments "whatever" + + * Postfix 2.3 does not implement requests to replace the message body. Milter + applications log a warning message when they need this unsupported + operation: + + st_optionneg[134563840]: 0x3d does not fulfill action requirements 0x1e + + The solution is to use Postfix version 2.4 or later. + + * Postfix versions before 3.0 did not support per-Milter timeouts, per-Milter + error handling, etc. + diff --git a/README_FILES/MULTI_INSTANCE_README b/README_FILES/MULTI_INSTANCE_README new file mode 100644 index 0000000..1f261f6 --- /dev/null +++ b/README_FILES/MULTI_INSTANCE_README @@ -0,0 +1,981 @@ +MMaannaaggiinngg mmuullttiippllee PPoossttffiixx iinnssttaanncceess oonn aa ssiinnggllee hhoosstt + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document is a guide to managing multiple Postfix instances on a single +host using the postmulti(1) instance manager. Multi-instance support is +available with Postfix version 2.6 and later. See the postfix-wrapper(5) manual +page for background on the instance management framework, and on how to deploy +a custom instance manager. + +Topics covered in this document: + + * Why multiple Postfix instances + * Null-client instances versus service instances + * Multi-instance walk-through + * Components of a Postfix system + * The default Postfix instance + * Instance groups + * Multi-instance configuration parameters + * Using the postmulti(1) command + * Credits + +WWhhyy mmuullttiippllee PPoossttffiixx iinnssttaanncceess + +Postfix is a general-purpose mail system that can be configured to serve a +variety of needs. Examples of Postfix applications are: + + * Local mail submission for shell users and system processes. + + * Incoming (MX host) email from the Internet. + + * Outbound mail relay for a corporate network. + + * Authenticated submission for roaming users. + + * Before/after content-filter mail. + +A single Postfix configuration can provide many or all of these services, but a +complex interplay of settings may be required, for example with master.cf +options overriding main.cf settings. In this document we take the view that +multiple Postfix instances may be a simpler way to configure a multi-function +Postfix system. With multiple Postfix instances, each instance has its own +directories for configuration, queue and data files, but it shares all Postfix +program and documentation files with other instances. + +Since there is no single right way to configure your system, we recommend that +you choose what makes you most comfortable. If different Postfix services don't +involve incompatible main.cf or master.cf settings, and if they can be combined +together without complex tricks, then a single monolithic configuration may be +the simplest approach. + +The purpose of multi-instance support in Postfix is not to force you to create +multiple Postfix instances, but rather to give you a choice. Multiple instances +give you the freedom to tune each Postfix instance to a single task that it +does well and to combine instances into complete systems. + +With the introduction of the postmulti(1) utility and the reduction of the per- +instance configuration footprint of a secondary Postfix instance to just a +main.cf and master.cf file (other files are now in shared locations), we hope +that multiple instances will be easier to use than ever before. + +NNuullll--cclliieenntt iinnssttaanncceess vveerrssuuss sseerrvviiccee iinnssttaanncceess + +In the multi-instance approach to configuring Postfix, the first simplification +is with the default local-submission Postfix instance. + +Most UNIX systems require support for email submission with the sendmail(1) +command so that system processes such as cron jobs can send status reports, and +so that system users can send email with command-line utilities. Such email can +be handled with a null-client Postfix configuration that forwards all mail to a +central mail hub. The null client will typically either not run an SMTP +listener at all (master_service_disable = inet), or it will listen only on the +loopback interface (inet_interfaces = loopback-only). + +When implementing specialized servers for inbound Internet email, outbound +MTAs, internal mail hubs, and so on, we recommend using a null client for local +submission and creating single-function secondary Postfix instances to serve +the specialized needs. + + Note: usually, you need to use different "myhostname" settings when you run + multiple instances on the same host. Otherwise, there will be false "mail + loops back to myself" alarms when one instance tries to send mail into + another instance. Typically, the null-client instance will use the system's + hostname, and other instances will use their own dedicated "myhostname" + settings. Different names are not needed when instances send mail to each + other with a protocol other than SMTP, or with SMTP over a TCP port other + than 25 as is usual with SMTP-based content filters. + +MMuullttii--iinnssttaannccee wwaallkk--tthhrroouugghh + +Before discussing the fine details of multi-instance operation we first show +the steps for creating a border mail server. This server has with a null-client +Postfix instance for local submission, an input Postfix instance to receive +mail from the Internet, plus an advanced SMTP content-filter and an output +Postfix instance to deliver filtered email to its internal destination. + +SSeettttiinngg uupp tthhee nnuullll--cclliieenntt PPoossttffiixx iinnssttaannccee + +On a border mail hub, while mail from the Internet requires a great deal of +scrutiny, locally submitted messages are typically limited to mail from cron +jobs and other system services. In this regard the border MTA is not different +from other Unix hosts in your environment. For this reason, it will submit +locally-generated email to the internal mail hub. We start the construction of +the border mail server with the default instance, which will be a local- +submission null client: + + /etc/postfix/main.cf: + # We are mta1.example.com + # + myhostname = mta1.example.com + mydomain = example.com + + # Flat user-account namespace in example.com: + # + # user@example.com not user@host.example.com + # + myorigin = $mydomain + + # Postfix 2.6+, disable inet services, specifically disable smtpd(8) + # + master_service_disable = inet + + # No local delivery: + # + mydestination = + local_transport = error:5.1.1 Mailbox unavailable + alias_database = + alias_maps = + local_recipient_maps = + + # Send everything to the internal mailhub + # + relayhost = [mailhub.example.com] + + # Indexed table macro: + # (use "hash", ... when cdb is not available) + # + default_database_type = cdb + indexed = ${default_database_type}:${config_directory}/ + + # Expose origin host of mail from "root", ... + # + smtp_generic_maps = ${indexed}generic + + # Send messages addressed to "root", ... to the MTA support team + # + virtual_alias_maps = ${indexed}virtual + + /etc/postfix/generic: + # The smarthost supports "+" addressing (recipient_delimiter = +). + # Mail from "root" exposes the origin host, without replies + # and bounces going back to the same host. + # + # On clustered MTAs this file is typically machine-built from + # a template file. The build process expands the template into + # "mtaadmin+root=mta1" + # + root mtaadmin+root=mta1 + + /etc/postfix/virtual: + # Caretaker aliases: + # + root mtaadmin + postmaster root + +You would typically also add a Makefile, to automatically run postmap(1) +commands when source files change. This Makefile also creates a "generic" +database when none exists. + + /etc/postfix/Makefile: + MTAADMIN=mtaadmin + + all: virtual.cdb generic.cdb + + generic: Makefile + @echo Creating $@ + @rm -f $@.tmp + @printf '%s\t%s+root=%s\n' root ${MTAADMIN} `uname -n` > $@.tmp + @mv $@.tmp generic + + %.cdb: % + postmap cdb:$< + +Construct the "virtual" and "generic" databases (the latter is created by +running "make"), then start and test the null-client: + + # cd /etc/postfix; make + # postfix start + # sendmail -i -f root -t <<EOF + From: root + To: root + Subject: test + + testing + EOF + +The test message should be delivered to the members of the "mtaadmin" address +group (or whatever address group you choose) with the following headers: + + From: mtaadmin+root=mta1@example.com + To: mtadmin+root=mta1@example.com + Subject: test + +SSeettttiinngg uupp tthhee ""oouuttppuutt"" PPoossttffiixx iinnssttaannccee + +With the null-client instance out of the way, we can create the MTA "output" +instance that will deliver filtered mail to the inside network. We add the +"output" instance first, because the output instance needs to be up and running +before the input instance can be fully tested, and when the system boots, the +"output" instance must start before the input instance. We will put the output +and input instances into a single instance group named "mta". + +Just once, when adding the first secondary instance, enable multi-instance +support in the default (null-client) instance: + + # postmulti -e init + +Then create the output instance: + + # postmulti -I postfix-out -G mta -e create + +The instance configuration directory defaults to /etc/postfix-out, more +precisely, the "postfix-out" subdirectory of the parent directory of the +default-instance configuration directory. The new instance will be created in a +"disabled" state: + + /etc/postfix-out/main.cf + # + # ... "stock" main.cf settings ... + # + multi_instance_name = postfix-out + queue_directory = /var/spool/postfix-out + data_directory = /var/lib/postfix-out + # + multi_instance_enable = no + master_service_disable = inet + authorized_submit_users = + +This instance has a "stock" master.cf file, and its queue and data directories, +also named "postfix-out", will be located in the same parent directories as the +corresponding directories of the default instance (e.g., /var/spool/postfix-out +and /var/lib/postfix-out). + +While this instance is immediately safe to start, it is not yet usefully +configured. It needs to be customized to fit the role of a post-filter re- +injection SMTP service. Typical additions include: + + /etc/postfix-out/master.cf: + # Replace default "smtp inet" entry with one listening on port 10026. + 127.0.0.1:10026 inet n - n - - smtpd + + /etc/postfix-out/main.cf + # ... + + # Comment out if you don't use IPv6 internally + # inet_protocols = ipv4 + inet_interfaces = loopback-only + mynetworks_style = host + smtpd_authorized_xforward_hosts = $mynetworks + + # Don't anvil(8) control the re-injection port. + # + smtpd_client_connection_count_limit = 0 + smtpd_client_event_limit_exceptions = $mynetworks + + # Best practice when inet_interfaces is set, as this is not a + # "secondary IP personality" configuration. + # + smtp_bind_address = 0.0.0.0 + + # All header rewriting happens upstream + # + local_header_rewrite_clients = + + # No local delivery on border gateway + # + mydestination = + alias_maps = + alias_database = + local_recipient_maps = + local_transport = error:5.1.1 Mailbox unavailable + + # May need a recipient_delimiter for per-user transport lookups: + # + recipient_delimiter = + + + # Only one (unrestricted client) + # With multiple instances, rarely need "-o param=value" overrides + # in master.cf, each instance gets its own main.cf file. + # + # Postfix 2.10 and later: specify empty smtpd_relay_restrictions. + smtpd_relay_restrictions = + smtpd_recipient_restrictions = permit_mynetworks, reject + + # Tolerate occasional high latency in the content filter. + # + smtpd_timeout = 1200s + + # Best when empty, with all parent domain matches explicit. + # + parent_domain_matches_subdomains = + + # Use the "relay" transport for inbound mail, and the default + # "smtp" transport for outbound mail (bounces, ...). The latter + # won't starve the former of delivery agent slots. + # + relay_domains = example.com, .example.com + + # With xforward, match the input instance setting, if you + # want "yes", set both to "yes". + # + smtpd_client_port_logging = no + + # Transport settings ... + # Message size limit + # Concurrency tuning for "relay" and "smtp" transport + # ... + +With the "output" configuration in place, enable and start the instance: + + 1 # postmulti -i postfix-out -x postconf -e \ + 2 "master_service_disable =" "authorized_submit_users = root" + 3 # postmulti -i postfix-out -e enable + 4 # postmulti -i postfix-out -p start + +This uses the postmulti(1) command to invoke postconf(1) in the context +(MAIL_CONFIG=/etc/postfix-out) of the output instance. + + * Lines 1-2: With "authorized_submit_users = root", the superuser can test + the postfix-out instance with "postmulti -i postfix-out -x sendmail -bv + recipient...", but otherwise local submission remains disabled. + + * Lines 1-2: With "master_service_disable =", the "inet" listeners are re- + enabled. + + * Line 3: The output instance is enabled for multi-instance start/stop. + + * Line 4: The output instance is started. + +Test the output instance by submitting probe messages via "sendmail -bv" and +"telnet". For production systems, in-depth configuration tests should be done +on a lab system. The simple tests just suggested will only confirm successful +deployment of a configuration that should already be known good. + +SSeettttiinngg uupp tthhee ccoonntteenntt--ffiilltteerr pprrooxxyy + +With the output instance ready, deploy your content-filter proxy. Most proxies +will need their own /etc/rc* start/stop script. Some proxies, however, are +started on demand by the Postfix spawn(8) service, in which case you need to +add the relevant spawn(8) entry to the output instance master.cf file. + +Configure the proxy to listen on 127.0.0.1:10025 and to re-inject filtered +email to 127.0.0.1:10026. Start the proxy service if necessary, then test the +proxy via "telnet" or automated SMTP injectors. The proxy should support the +following ESMTP features: DSN, 8BITMIME, and XFORWARD. In addition, the proxy +should support multiple mail deliveries within an SMTP session. + +SSeettttiinngg uupp tthhee iinnppuutt PPoossttffiixx iinnssttaannccee + +The input Postfix instance receives mail from the network and sends it through +the content filter. Now we create the input instance, also part of the "mta" +instance group: + + # postmulti -I postfix-in -G mta -e create + +The new instance configuration directory defaults to /etc/postfix-in, more +precisely, the "postfix-in" subdirectory of the parent directory of the +default-instance configuration directory. The new instance will be created in a +"disabled" state: + + /etc/postfix-in/main.cf + # + # ... "stock" main.cf settings ... + # + multi_instance_name = postfix-in + queue_directory = /var/spool/postfix-in + data_directory = /var/lib/postfix-in + # + multi_instance_enable = no + master_service_disable = inet + authorized_submit_users = + +As before, make appropriate changes to main.cf and master.cf to make the +instance production ready. Consider setting "soft_bounce = yes" during the +first few hours of deployment, so you can iron-out any unexpected "kinks". + +Manual testing can start with: + + /etc/postfix-in/main.cf + # Accept only local traffic, but allow impersonation: + inet_interfaces = 127.0.0.1 + smtpd_authorized_xclient_hosts = 127.0.0.1 + +This allows you to use the Postfix-specific XCLIENT SMTP command to safely +simulate connections from remote systems before any remote systems are able to +connect. If the test results look good, revert the above settings to the +required production values. Typical settings in the pre-filter input instance +include: + + /etc/postfix-in/main.cf + # + # ... + # + + # No local delivery on border gateway + # + mydestination = + alias_maps = + alias_database = + local_recipient_maps = + local_transport = error:5.1.1 Mailbox unavailable + + # Don't rewrite remote headers + # + local_header_rewrite_clients = + + # All recipients of not yet filtered email go to the same filter + together. + # + # With multiple instances, the content-filter is specified + # via transport settings not the "content_filter" transport + # switch override! Here the filter listens on local port 10025. + # + # If you need to route some users or recipient domains directly to the + # output instance bypassing the filter, just define a transport table + # with suitable entries. + # + default_transport = smtp:[127.0.0.1]:10025 + relay_transport = $default_transport + virtual_transport = $default_transport + transport_maps = + + # Pass original client log information through the filter. + # + smtp_send_xforward_command = yes + + # Avoid splitting the envelope and scanning messages multiple times. + # Match the re-injection server's recipient limit. + # + smtp_destination_recipient_limit = 1000 + + # Tolerate occasional high latency in the content filter. + # + smtp_data_done_timeout = 1200s + + # With xforward, match the output instance setting, if you + # want "yes", set both to "yes". + # + smtpd_client_port_logging = no + + # ... Lots of settings for inbound MX host ... + +With the "input" instance configured, enable and start it: + + # postmulti -i postfix-in -x postconf -e \ + "master_service_disable =" "authorized_submit_users = root" + # postmulti -i postfix-in -e enable + # postmulti -i postfix-in -p start + +That's it. You now have a 3-instance configuration. A null-client sending all +locally submitted mail to the internal mail hub and a pair of "mta" instances +that receive mail from the Internet, pass it through a content-filter, and then +deliver it to the internal destination. + +Running "postfix start" or "postfix stop" will now start/stop all three Postfix +instances. You can use "postfix -c /config/path start" to start just one +instance, or use the instance name (or instance group name) via postmulti(1): + + # postmulti -i - -p stop + # postmulti -g mta -p status + # postmulti -i postfix-out -p flush + # postmulti -i postfix-in -p reload + # ... + +This example ends the multi-instance "walk through". The remainder of this +document provides background information on Postfix multi-instance support +features and options. + +CCoommppoonneennttss ooff aa PPoossttffiixx ssyysstteemm + +A Postfix system consists of the following components: + +Shared among all instances: + + * Command-line utilities for administrators and users installed in + $command_directory, $sendmail_path, $mailq_path and $newaliases_path. + + * Daemon executables, and run-time support files installed in + $daemon_directory. + + * Bundled documentation, installed in $html_directory, $manpage_directory and + $readme_directory. + + * Entries in /etc/passwd and /etc/group for the $mail_owner user and + $setgid_group group. The $mail_owner user provides the mail system with a + protected (non-root) execution context. The $setgid_group group is used + exclusively to support the setgid postdrop(1) and postqueue(1) utilities + (it mmuusstt nnoott be the primary group or secondary group of any users, + including the $mail_owner user). + +Private to each instance: + + * The main.cf, master.cf (and other optional) configuration files in + $config_directory. + + * The maildrop, incoming, active, deferred and hold queues in + $queue_directory (which contains additional directories needed by Postfix, + and which optionally doubles as a chroot jail for Postfix daemon + processes). + + * Various caches (TLS session, address verification, ...) in $data_directory. + +The Postfix configuration parameters mentioned above are collectively referred +to as "installation parameters". Their default values are set when the Postfix +software is built from source, and all but one may be optionally set to a non- +default value via the main.cf file. The one parameter that (catch-22) cannot be +set in main.cf is $config_directory, as this defines the location of the +main.cf file itself. + +Though config_directory cannot be set in main.cf, postfix(1) and most of the +other command-line Postfix utilities allow you to specify a non-default +configuration directory via a command line option (typically --cc) or via the +MAIL_CONFIG environment variable. In this way, it is possible to have multiple +configuration directories on the same machine, and to have multiple running +master(8) daemons each with its own configuration files, queue directory and +data directory. + +These multiple running copies of master(8) share the base Postfix software. +They do not (and cannot) share their configuration directories, queue +directories or data directories. + +Each combination of configuration directory, together with the queue directory +and data directory (specified in the corresponding main.cf file) make up a +Postfix iinnssttaannccee. + +TThhee ddeeffaauulltt PPoossttffiixx iinnssttaannccee + +One Postfix instance is special: this is the instance whose configuration +directory is the default one compiled into the Postfix utilities. The location +of the default configuration directory is typically /etc/postfix, and can be +queried via the "postconf -d config_directory" command. We call the instance +with this configuration directory the "default instance". + +The default instance is responsible for local mail submission. The setgid +postdrop(1) utility is used by the sendmail(1) local submission program to +spool messages into the mmaaiillddrroopp sub-directory of the queue directory of the +default instance. + +Even in the rare case when "sendmail -C" is used to submit local mail into a +non-default Postfix instance, for security reasons, postdrop(1) will consult +the default main.cf file to check the validity of the requested non-default +configuration directory. + +So, while in most other respects, all instances are equal, the default instance +is "more equal than others". You may choose to create additional instances, but +you must have at least the default instance, with its configuration directory +in the default compiled-in location. + +IInnssttaannccee ggrroouuppss + +The postmulti(1) multi-instance manager supports the notion of an instance +"group". Typically, the member instances of an instance group constitute a +logical service, and are expected to all be running or all be stopped. + +In many cases a single Postfix instance will be a complete logical "service". +You should define such instances as stand-alone instances that are not members +of any instance "group". The null-client instance is an example of a non-group +instance. + +When a logical service consists of multiple Postfix instances, often a pair of +pre-filter and post-filter instances with a content filter proxy between them, +the related instances should be members of a single instance group (however, +the content filter usually has its own start/stop procedure that is separate +from any Postfix instance). + +The default instance main.cf file's $multi_instance_directories configuration +parameter lists the configuration directories of all secondary (non-default) +instances. Together with the default instance, these secondary instances are +managed by the multi-instance manager. Instances are started in the order +listed, and stopped in the opposite order. For instances that are members of a +service "group", you should arrange to start the service back-to-front, with +the output stages started and ready to receive mail before the input stages are +started. + +MMuullttii--iinnssttaannccee ccoonnffiigguurraattiioonn ppaarraammeetteerrss + +multi_instance_wrapper + This default-instance configuration parameter must be set to a suitable + multi-instance manager's "wrapper" program that controls the starting, + stopping, etc. of a multi-instance Postfix system. To use the postmulti(1) + manager described in this document, this parameter should be set with the + "postmulti -e init" command. + +multi_instance_directories + This default-instance configuration parameter specifies an optional list of + the secondary instances controlled via the multi-instance manager. + Instances are listed in their "start" order, with the default instance + always started first (if enabled). If $multi_instance_directories is left + empty, the postfix(1) command runs with multi-instance support turned off, + and none of the multi_instance_ configuration parameters will have any + effect. + + Do not assign a non-empty list of secondary instance configuration + directories to multi_instance_directories until you have configured a + suitable multi_instance_wrapper setting! This is best accomplished via the + "postmulti -e init" command. + +multi_instance_name + Each Postfix instance may be assigned a distinct name (with "postfix - + e create/import/assign -I name..."). This name can be used with the + postmulti(1) command-line utility to perform tasks on the instance by name + (rather than the full pathname of its configuration directory). Choose a + name that concisely captures the role of the instance (it must start with + "postfix-"). It is an error for two instances to have the same + $multi_instance_name. You can leave an instance "nameless" by leaving this + parameter at the default empty setting. + + To avoid confusion in your logs, if you don't assign each secondary + instance a non-empty (distinct) $multi_instance_name, you should make sure + that the $syslog_name setting is different for each instance. The + $syslog_name parameter defaults to $multi_instance_name when the latter is + non-empty. If at all possible, the syslog_name should start with "postfix- + ", this helps log parsers to identify log entries from secondary Postfix + instances. + +multi_instance_group + Each Postfix instance may be assigned an "instance group" name (with + "postfix -e create/import/assign -G name..."). The default (empty) value of + multi_instance_group parameter indicates a stand-alone instance that is not + part of any group. The group name can be used with the postmulti(1) + command-line utility to perform a task on the members of a group by name. + Choose a single-word group name that concisely captures the role of the + group. + +multi_instance_enable + This parameter controls whether a Postfix instance will be started by a + Postfix multi-instance manager. The default value is "no". The instance can + be started explicitly with "postfix -c /path/to/config/directory"; this is + useful for testing. + + When an instance is disabled, the postfix(1) "start" command is replaced by + "check". + + Some postfix(1) commands (such as "stop", "flush", ...) require a running + Postfix instance, and skip instances that are disabled. + + Other postfix(1) commands (such as "status", "set-permissions", "upgrade- + configuration", ...) do not require a running Postfix system, and apply to + all instances whether enabled or not. + +The postmulti(1) utility can be used to create (or destroy) instances. It can +also be used to "import" or "deport" existing instances into or from the list +of managed instances. When using postmulti(1) to manage instances, the above +configuration parameters are managed for you automatically. See below. + +UUssiinngg tthhee ppoossttmmuullttii((11)) ccoommmmaanndd + + * Initializing the multi-instance manager + * Listing managed instances + * Starting or stopping a multi-instance system + * Ad-hoc multi-instance operations + * Creating a new Postfix instance + * Destroying a Postfix instance + * Importing an existing Postfix instance + * Deporting a managed Postfix instance + * Assigning a new name or group name + * Enabling/disabling managed instances + +IInniittiiaalliizziinngg tthhee mmuullttii--iinnssttaannccee mmaannaaggeerr + +Before postmulti(1) is used for the first time, you must install it as the +multi_instance_wrapper for your Postfix system and enable multi-instance +operation of the default Postfix instance. You can then proceed to add new or +existing instances to the multi-instance configuration. This initial +installation is accomplished as follows: + + # postmulti -e init + +This updates the default instance main.cf file as follows: + + # Use postmulti(1) as a postfix-wrapper(5) + # + multi_instance_wrapper = ${command_directory}/postmulti -p -- + + # Configure the default instance to start when in multi-instance mode + # + multi_instance_enable = yes + +If you prefer, you can make these changes by editing the default main.cf +directly, or by using "postconf -e". + +LLiissttiinngg mmaannaaggeedd iinnssttaanncceess + +The list of managed instances consists of the default instance and the +additional instances whose configuration directories are listed (in start +order) under the multi_instance_directories parameter of the default main.cf +configuration file. + +You can list selected instances, groups of instances or all instances by +specifying only the instance matching options with the "-l" option. The "-a" +option is assumed if no other instance selection options are specified (this +behavior changes with the "-e" option). As a special case, even if it has an +explicit name, the default instance can always be selected via "-i -". + + # postmulti -l -a + # postmulti -l -g a_group + # postmulti -l -i an_instance + +The output is one line per instance (in "postfix start" order): + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |nnaammee |ggrroouupp|eennaabblleedd|ccoonnffiigg__ddiirreeccttoorryy | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |- |- |yes |/etc/postfix | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mta-out|mta |yes |/etc/postfix/mta-out| + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mta-in |mta |yes |/etc/postfix-mta-in | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |msa-out|msa |yes |/etc/postfix-msa-out| + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |msa-in |msa |yes |/etc/postfix-msa-in | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |test |- |no |/etc/postfix-test | + |_ _ _ _ _ _ _ _|_ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +The first line showing the column headings is not part of the output. When +either the instance name or the instance group is not set, it is shown as a "- +". + +When selecting an existing instance via the "-i" option, you can always use the +full pathname of its configuration directory instead of the instance (short) +name. This is the only way to select a non-default nameless instance. The +default instance can be selected via "-i -", whether it has a name or not. + +To list instances in reverse start order, include the "-R" option together with +the instance selection options. + +SSttaarrttiinngg oorr ssttooppppiinngg aa mmuullttii--iinnssttaannccee ssyysstteemm + +To start, stop, reload, etc. the complete (already configured as above) multi- +instance system just use postfix(1) as you would with a single-instance system. +The Postfix multi-instance wrapper framework insulates Postfix init.d start and +package upgrade scripts from the details of multi-instance management! + +The --pp option of postmulti(1) turns on postfix(1) compatibility mode. With this +option the remaining arguments are exactly those supported by postfix(1), but +commands are applied to all instances or all enabled instances as appropriate. +As described above, this switch is required when using postmulti(1) as the +multi_instance_wrapper. + +If you want to specify a subset of instances by name, or group name, or run +arbitrary commands (not just "postfix stop/start/etc. in the context +(MAIL_CONFIG environment variable setting) of a particular instance or group of +instances, then you can use the instance-aware postmulti(1) utility directly. + +AAdd--hhoocc mmuullttii--iinnssttaannccee ooppeerraattiioonnss + +The postmulti(1) command can be used by the administrator to run arbitrary +commands in the context of one or more Postfix instances. The most common use- +case is stopping or starting a group of Postfix instances: + + # postmulti -g mygroup -p start + # postmulti -g mygroup -p flush + # postmulti -g mygroup -p reload + # postmulti -g mygroup -p status + # postmulti -g mygroup -p stop + # postmulti -g mygroup -p upgrade-configuration + +The --pp option is essentially a short-hand for a leading ppoossttffiixx command +argument, but with appropriate additional options turned on depending on the +first argument. In the case of "start", disabled instances are "checked" +(postfix check) rather than simply skipped. + +The resulting command is executed for each candidate instance with the +MMAAIILL__CCOONNFFIIGG environment variable set to the configuration directory of the +corresponding Postfix instance. + +The postmulti(1) utility is able to launch commands other than postfix(1), Use +the --xx option to ask postmulti to execute an ad-hoc command for all instances, +a group of instances, or just one instance. With ad-hoc commands the +multi_instance_enable parameter is ignored: the command is unconditionally +executed for the instances selected via -a, -g or -i. In addition to +MAIL_CONFIG, the following instance parameters are exported into the command +environment: + + command_directory=$command_directory + daemon_directory=$daemon_directory + config_directory=$config_directory + queue_directory=$queue_directory + data_directory=$data_directory + multi_instance_name=$multi_instance_name + multi_instance_group=$multi_instance_group + multi_instance_enable=$multi_instance_enable + +The config_directory setting is of course the same as MAIL_CONFIG, and is +arguably redundant, but leaving it in is less surprising. If you want to skip +disabled instances, just check multi_instance_enable environment variable and +exit if it is set to "no". + +The ability to run ad-hoc commands opens up a wealth of additional +possibilities: + + * Specify an instance by name rather than configuration directory when using + sendmail(1) to send a verification probe: + + $ postmulti -i postfix-myinst -x sendmail -bv test@example.net + + * Display non-default main.cf settings of all Postfix instances. This uses an + inline shell script to package together multiple shell commands to execute + for each instance: + + $ postmulti -x sh -c 'echo "-- $MAIL_CONFIG"; postconf -n' + + * Put all mail in enabled member instances of a group on hold: + + # postmulti -g group_name -x \ + sh -c 'test $multi_instance_enable = yes && postsuper -h ALL' + + * Show top 10 domains in the deferred queue of all instances: + + # postmulti -x sh -c 'echo "-- $MAIL_CONFIG"; qshape deferred | head - + 12' + +CCrreeaattiinngg aa nneeww PPoossttffiixx iinnssttaannccee + +The postmulti(1) command can be used to create additional Postfix instances. +New instances are created with local submission and all "inet" services +disabled via the following non-default parameter settings in the main.cf file: + + authorized_submit_users = + master_service_disable = inet + +The above settings ensure that new instances are safe to start immediately: +they will not conflict with inet listeners in existing Postfix instances. They +will also not accept any mail until they are fully configured, at which point +you can do away with one or both of the above safety measures. + +The postmulti(1) command encourages a preferred way of organizing the +configuration directories, queue directories and data directories of non- +default instances. If the default instance settings are: + + config_directory = /conf-path/postfix + queue_directory = /queue-path/postfix + data_directory = /data-path/postfix + +A newly-created instance named postfix-myinst will by default have: + + multi_instance_enable = no + multi_instance_name = postfix-myinst + config_directory = /conf-path/postfix-myinst + queue_directory = /queue-path/postfix-myinst + data_directory = /data-path/postfix-myinst + +You can override any of these defaults when creating the instance, but unless +you want to spread instance queue directories over multiple file-systems, use +the default naming strategy. It keeps the multiple instances organized in a +uniform, predictable fashion. + +When specifying the instance name later, you can refer to it either as +"postfix-myinst", or via the full path of the configuration directory. + +To create a new instance just use the --ee ccrreeaattee option: + + # postmulti -I postfix-myinst -e create + +If the new instance is to belong to a group of related instances that implement +a single logical service, assign it to a group: + + # postmulti -I postfix-myinst -G mygroup -e create + +If you want to override the conventional values of the instance installation +parameters, specify their values on the command-line: + + # postmulti [-I postfix-myinst] [-G mygroup] -e create \ + "config_directory = /path/to/config_directory" \ + "queue_directory = /path/to/queue_directory" \ + "data_directory = /path/to/data_directory" + +A note on the --II and --GG options above. These are always used to assign a name +or group name to an instance, while the --ii and --gg options always select +existing instances. By default, the configuration directories of newly managed +instances are appended to the instance list. You can use the "-i" or "-g" or "- +a" options to insert the new instance before the specified instance or group, +or at the beginning of the instance list (multi_instance_directories parameter +of the default instance). + +If you do specify a name (use "-I" with a name that is not "-") for the new +instance, you may omit any of the 3 instance installation parameters whose +instance-name based value is acceptable. Otherwise, all three instance +installation parameters are required. You should set the "syslog_name" +explicitly in the main.cf file of a "nameless" instance, in order to avoid +confusion in the mail logs when multiple instances are in use. + +DDeessttrrooyyiinngg aa PPoossttffiixx iinnssttaannccee + +If you no longer need an instance, you can destroy it via: + + # postmulti -i postfix-myinst -p stop + # postmulti -i postfix-myinst -e disable + # postmulti -i postfix-myinst -e destroy + +The instance must be stopped, disabled and have no queued messages. This is +expected to fully delete a just created instance that has never been used. If +the instance is not freshly created, files added after the instance was created +will remain in the configuration, queue or data directories, in which case the +corresponding directory may not be fully removed and a warning to that effect +will be displayed. You can complete the destruction of the instance manually by +removing any unwanted remnants of the instance-specific "private" directories. + +IImmppoorrttiinngg aann eexxiissttiinngg PPoossttffiixx iinnssttaannccee + +If you already have an existing secondary Postfix instance that is not yet +managed via postmulti(1), you can "import" it into the list of managed +instances. If your instance is already using the default configuration +directory naming scheme, just specify the corresponding instance name (the +multi_instance_name parameter in its configuration file will be adjusted to +match this name if necessary): + + # postmulti -I postfix-myinst [-G mygroup] -e import + +Otherwise, you must specify the location of its configuration directory: + + # postmulti [-I postfix-myinst] [-G mygroup] -e import \ + "config_directory = /path/of/config_directory" + +When the instance is imported, you can assign a name or a group. As with +"create", you can control the placement of the new instance in the start order +by using "-i", "-g" or "-a" to prepend before the selected instance or +instances. + +An imported instance is usually not multi-instance "enabled", unless it was +part of a multi-instance configuration at an earlier time. If it is fully +configured and ready to run, don't forget to enable it and if necessary start +it. When other enabled instances are already running, new instances need to be +started individually when they are first created or imported. + +To find out what instances are running, use: + + # postfix status + +DDeeppoorrttiinngg aa mmaannaaggeedd PPoossttffiixx iinnssttaannccee + +You can "deport" an existing instance from the list of managed instances. This +does not destroy the instance, rather the instance just becomes a stand-alone +Postfix instance not registered with the multi-instance manager. postmulti(1) +will refuse to "deport" an instance that is not stopped and disabled. + + # postmulti -i postfix-myinst -p stop + # postmulti -i postfix-myinst -e disable + # postmulti -i postfix-myinst -e deport + +AAssssiiggnniinngg aa nneeww nnaammee oorr ggrroouupp nnaammee + +You can assign a new name or new group to a managed instance. Use "-" as the +new value to assign the instance to no group or make it nameless. To specify a +nameless secondary instance use the configuration directory path instead of the +old name: + + # postmulti -i postfix-old [-I postfix-new] [-G newgroup] -e assign + +EEnnaabblliinngg//ddiissaabblliinngg mmaannaaggeedd iinnssttaanncceess + +You can enable or disable a managed instance. As documented in postfix-wrapper +(5), disabled instances are skipped with actions that start, stop or control +running Postfix instances. + + # postmulti -i postfix-myinst -e enable + # postmulti -i postfix-myinst -e disable + +CCrreeddiittss + +Wietse Venema created Postfix, designed and implemented the multi-instance +wrapper framework and provided design feedback that made the postmulti(1) +utility much more general and useful than originally envisioned. + +The postmulti(1) utility was developed by Victor Duchovni of Morgan Stanley, +who also wrote the initial version of this document. + diff --git a/README_FILES/MYSQL_README b/README_FILES/MYSQL_README new file mode 100644 index 0000000..1769923 --- /dev/null +++ b/README_FILES/MYSQL_README @@ -0,0 +1,135 @@ +PPoossttffiixx MMyySSQQLL HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +The Postfix mysql map type allows you to hook up Postfix to a MySQL database. +This implementation allows for multiple mysql databases: you can use one for a +virtual(5) table, one for an access(5) table, and one for an aliases(5) table +if you want. You can specify multiple servers for the same database, so that +Postfix can switch to a good database server if one goes bad. + +Busy mail servers using mysql maps will generate lots of concurrent mysql +clients, so the mysql server(s) should be run with this fact in mind. You can +reduce the number of concurrent mysql clients by using the Postfix proxymap(8) +service. + +BBuuiillddiinngg PPoossttffiixx wwiitthh MMyySSQQLL ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +Note: to use mysql with Debian GNU/Linux's Postfix, all you need is to install +the postfix-mysql package and you're done. There is no need to recompile +Postfix. + +The Postfix MySQL client utilizes the mysql client library, which can be +obtained from: + + http://www.mysql.com/downloads/ + +In order to build Postfix with mysql map support, you will need to add - +DHAS_MYSQL and -I for the directory containing the mysql headers, and the +mysqlclient library (and libm) to AUXLIBS_MYSQL, for example: + + make -f Makefile.init makefiles \ + "CCARGS=-DHAS_MYSQL -I/usr/local/mysql/include" \ + "AUXLIBS_MYSQL=-L/usr/local/mysql/lib -lmysqlclient -lz -lm" + +If your MySQL shared library is in a directory that the RUN-TIME linker does +not know about, add a "-Wl,-R,/path/to/directory" option after "-lmysqlclient". + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_MYSQL. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded MySQL database client, but only the new AUXLIBS_MYSQL variable supports +building a dynamically-loaded or statically-loaded MySQL database client. + + Failure to use the AUXLIBS_MYSQL variable will defeat the purpose of + dynamic database client loading. Every Postfix executable file will have + MYSQL database library dependencies. And that was exactly what dynamic + database client loading was meant to avoid. + +On Solaris, use this instead: + + make -f Makefile.init makefiles \ + "CCARGS=-DHAS_MYSQL -I/usr/local/mysql/include" \ + "AUXLIBS_MYSQL=-L/usr/local/mysql/lib -R/usr/local/mysql/lib \ + -lmysqlclient -lz -lm" + +Then, just run 'make'. This requires libz, the compression library. Older mysql +implementations build without libz. + +UUssiinngg MMyySSQQLL ttaabblleess + +Once Postfix is built with mysql support, you can specify a map type in main.cf +like this: + + alias_maps = mysql:/etc/postfix/mysql-aliases.cf + +The file /etc/postfix/mysql-aliases.cf specifies lots of information telling +Postfix how to reference the mysql database. For a complete description, see +the mysql_table(5) manual page. + +EExxaammppllee:: llooccaall aalliiaasseess + +# +# mysql config file for local(8) aliases(5) lookups +# + +# The user name and password to log into the mysql server. +user = someone +password = some_password + +# The database name on the servers. +dbname = customer_database + +# For Postfix 2.2 and later The SQL query template. +# See mysql_table(5) for details. +query = SELECT forw_addr FROM mxaliases WHERE alias='%s' AND status='paid' + +# For Postfix releases prior to 2.2. See mysql_table(5) for details. +select_field = forw_addr +table = mxaliases +where_field = alias +# Don't forget the leading "AND"! +additional_conditions = AND status = 'paid' + +# This is necessary to make UTF8 queries work for Postfix 2.11 .. 3.1, +# and is the default setting as of Postfix 3.2. +option_group = client + +AAddddiittiioonnaall nnootteess + +Postfix 3.2 and later read [[cclliieenntt]] option group settings by default. To +disable this, specify no ooppttiioonn__ffiillee and specify "ooppttiioonn__ggrroouupp ==" (i.e. an +empty value). + +Postfix 3.1 and earlier don't read [[cclliieenntt]] option group settings unless a non- +empty ooppttiioonn__ffiillee or ooppttiioonn__ggrroouupp value are specified. To enable this, specify, +for example "ooppttiioonn__ggrroouupp == cclliieenntt". + +The MySQL configuration interface setup allows for multiple mysql databases: +you can use one for a virtual table, one for an access table, and one for an +aliases table if you want. + +Since sites that have a need for multiple mail exchangers may enjoy the +convenience of using a networked mailer database, but do not want to introduce +a single point of failure to their system, we've included the ability to have +Postfix reference multiple hosts for access to a single mysql map. This will +work if sites set up mirrored mysql databases on two or more hosts. Whenever +queries fail with an error at one host, the rest of the hosts will be tried in +random order. If no mysql server hosts are reachable, then mail will be +deferred until at least one of those hosts is reachable. + +CCrreeddiittss + + * The initial version was contributed by Scott Cotton and Joshua Marcus, IC + Group, Inc. + * Liviu Daia revised the configuration interface and added the main.cf + configuration feature. + * Liviu Daia with further refinements from Jose Luis Tallon and Victor + Duchovni developed the common query, result_format, domain and + expansion_limit interface for LDAP, MySQL and PostgreSQL. + diff --git a/README_FILES/NFS_README b/README_FILES/NFS_README new file mode 100644 index 0000000..60d0578 --- /dev/null +++ b/README_FILES/NFS_README @@ -0,0 +1,102 @@ +PPoossttffiixx aanndd NNFFSS + +------------------------------------------------------------------------------- + +PPoossttffiixx ssuuppppoorrtt ssttaattuuss ffoorr NNFFSS + +What is the status of support for Postfix on NFS? The answer is that Postfix +itself is supported when you use NFS, but there is no promise that an NFS- +related problem will promptly receive a Postfix workaround, or that a +workaround will even be possible. + +That said, Postfix will in many cases work very well on NFS, because Postfix +implements a number of workarounds (see below). Good NFS implementations seldom +if ever give problems with Postfix, so Wietse recommends that you spend your +money wisely. + +PPoossttffiixx ffiillee lloocckkiinngg aanndd NNFFSS + +For the Postfix mail queue, it does not matter how well NFS file locking works. +The reason is that you cannot share Postfix queues among multiple running +Postfix instances. You can use NFS to switch a Postfix mail queue from one NFS +client to another one, but only one NFS client can access a Postfix mail queue +at any particular point in time. + +For mailbox file sharing with NFS, your options are to use ffccnnttll (kernel +locks), ddoottlloocckk (username.lock files), to use both locking methods +simultaneously, or to switch to maildir format. The maildir format uses one +file per message and needs no file locking support in Postfix or in other mail +software. + +Many sites that use mailbox format play safe and use both locking methods +simultaneously. + + /etc/postfix/main.cf: + virtual_mailbox_lock = fcntl, dotlock + mailbox_delivery_lock = fcntl, dotlock + +PPoossttffiixx NNFFSS wwoorrkkaarroouunnddss + +The list below summarizes the workarounds that exist for running Postfix on NFS +as of the middle of 2003. As a reminder, Postfix itself is still supported when +it runs on NFS, but there is no promise that an NFS-related problem will +promptly receive a Postfix workaround, or that a workaround will even be +possible. + + * Problem: when renaming a file, the operation may succeed but report an + error anyway[1]. + + Workaround: when rename(old, new) reports an error, Postfix checks if the + new name exists and the old name is gone. If the check succeeds, Postfix + assumes that the rename() operation completed normally. + + * Problem: when creating a directory, the operation may succeed but report an + error anyway[1]. + + Workaround: when mkdir(new) reports an EEXIST error, Postfix checks if the + new name resolves to a directory. If the check succeeds, Postfix assumes + that the mkdir() operation completed normally. + + * Problem: when creating a hardlink to a file, the operation may succeed but + report an error anyway[1]. + + Workaround: when link(old, new) fails, Postfix compares the device and + inode number of the old and new files. When the two files are identical, + Postfix assumes that the link() operation completed normally. + + * Problem: when creating a dotlock (username.lock) file, the operation may + succeed but report an error anyway[1]. + + Workaround: in this case, the only safe action is to back off and try again + later. + + * Problem: when a file server's "time of day" clock is not synchronized with + the client's "time of day" clock, email deliveries are delayed by a minute + or more. + + Workaround: Postfix explicitly sets file time stamps to avoid delays with + new mail (Postfix uses "last modified" file time stamps to decide when a + queue file is ready for delivery). + +[1] How can an operation succeed and report an error anyway? + +Suppose that an NFS server executes a client request successfully, and that the +server's reply to the client is lost. After some time the client retransmits +the request to the server. Normally, the server remembers that it already +completed the request (it keeps a list of recently-completed requests and +replies), and simply retransmits the reply. + +However, when the server has rebooted or when it has been very busy, the server +no longer remembers that it already completed the request, and repeats the +operation. This causes no problems with file read/write requests (they contain +a file offset and can therefore be repeated safely), but fails with non- +idempotent operations. For example, when the server executes a retransmitted +rename() request, the server reports an ENOENT error because the old name does +not exist; and when the server executes a retransmitted link(), mkdir() or +create() request, the server reports an EEXIST error because the name already +exists. + +Thus, successful, non-idempotent, NFS operations will report false errors when +the server reply is lost, the client retransmits the request, and the server +does not remember that it already completed the request. + diff --git a/README_FILES/OVERVIEW b/README_FILES/OVERVIEW new file mode 100644 index 0000000..71976d4 --- /dev/null +++ b/README_FILES/OVERVIEW @@ -0,0 +1,492 @@ +PPoossttffiixx AArrcchhiitteeccttuurree OOvveerrvviieeww + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document presents an overview of the Postfix architecture, and provides +pointers to descriptions of every Postfix command or server program. The text +gives the general context in which each command or server program is used, and +provides pointers to documents with specific usage examples and background +information. + +Topics covered by this document: + + * How Postfix receives mail + * How Postfix delivers mail + * Postfix behind the scenes + * Postfix support commands + +HHooww PPoossttffiixx rreecceeiivveess mmaaiill + +When a message enters the Postfix mail system, the first stop on the inside is +the incoming queue. The figure below shows the main processes that are involved +with new mail. Names followed by a number are Postfix commands or server +programs, while unnumbered names inside shaded areas represent Postfix queues. + + trivial- + rewrite(8) + + Network -> smtpd(8) + + ^ | + \ | v + + Network -> qmqpd(8) -> cleanup(8) -> incoming + + / + + pickup(8) <- maildrop + + ^ + | + + Local -> sendmail(1) -> postdrop(1) + + * Network mail enters Postfix via the smtpd(8) or qmqpd(8) servers. These + servers remove the SMTP or QMQP protocol encapsulation, enforce some sanity + checks to protect Postfix, and give the sender, recipients and message + content to the cleanup(8) server. The smtpd(8) server can be configured to + block unwanted mail, as described in the SMTPD_ACCESS_README document. + + * Local submissions are received with the Postfix sendmail(1) compatibility + command, and are queued in the maildrop queue by the privileged postdrop(1) + command. This arrangement even works while the Postfix mail system is not + running. The local pickup(8) server picks up local submissions, enforces + some sanity checks to protect Postfix, and gives the sender, recipients and + message content to the cleanup(8) server. + + * Mail from internal sources is given directly to the cleanup(8) server. + These sources are not shown in the figure, and include: mail that is + forwarded by the local(8) delivery agent (see next section), messages that + are returned to the sender by the bounce(8) server (see second-next + section), and postmaster notifications about problems with Postfix. + + * The cleanup(8) server implements the final processing stage before mail is + queued. It adds missing From: and other message headers, and transforms + addresses as described in the ADDRESS_REWRITING_README document. + Optionally, the cleanup(8) server can be configured to do light-weight + content inspection with regular expressions as described in the + BUILTIN_FILTER_README document. The cleanup(8) server places the result as + a single file into the incoming queue, and notifies the queue manager (see + next section) of the arrival of new mail. + + * The trivial-rewrite(8) server rewrites addresses to the standard + "user@fully.qualified.domain" form, as described in the + ADDRESS_REWRITING_README document. Postfix currently does not implement a + rewriting language, but a lot can be done via table lookups and, if need + be, regular expressions. + +HHooww PPoossttffiixx ddeelliivveerrss mmaaiill + +Once a message has reached the incoming queue the next step is to deliver it. +The figure shows the main components of the Postfix mail delivery apparatus. +Names followed by a number are Postfix commands or server programs, while +unnumbered names inside shaded areas represent Postfix queues. + + trivial- smtp(8) -> Network + rewrite(8) + / + + - lmtp(8) -> Network + ^ | + | v / + + incoming -> active -> qmgr(8) --- local(8) -> File, command + + \ + ^ | + | v - virtual(8) -> File + + deferred \ + + pipe(8) -> Command + + * The queue manager (the qmgr(8) server process in the figure) is the heart + of Postfix mail delivery. It contacts the smtp(8), lmtp(8), local(8), + virtual(8), pipe(8), discard(8) or error(8) delivery agents, and sends a + delivery request for one or more recipient addresses. The discard(8) and + error(8) delivery agents are special: they discard or bounce all mail, and + are not shown in the figure above. + + The queue manager maintains a small active queue with the messages that it + has opened for delivery. The active queue acts as a limited window on + potentially large incoming or deferred queues. The limited active queue + prevents the queue manager from running out of memory under heavy load. + + The queue manager maintains a separate deferred queue for mail that cannot + be delivered, so that a large mail backlog will not slow down normal queue + accesses. The queue manager's strategy for delayed mail delivery attempts + is described in the QSHAPE_README and TUNING_README documents. + + * The trivial-rewrite(8) server resolves each recipient address according to + its local or remote address class, as defined in the ADDRESS_CLASS_README + document. Additional routing information can be specified with the optional + transport(5) table. The trivial-rewrite(8) server optionally queries the + relocated(5) table for recipients whose address has changed; mail for such + recipients is returned to the sender with an explanation. + + * The smtp(8) client looks up a list of mail exchangers for the destination + host, sorts the list by preference, and tries each server in turn until it + finds a server that responds. It then encapsulates the sender, recipient + and message content as required by the SMTP protocol; this includes + conversion of 8-bit MIME to 7-bit encoding. + + * The lmtp(8) client speaks a protocol similar to SMTP that is optimized for + delivery to mailbox servers such as Cyrus. The advantage of this setup is + that one Postfix machine can feed multiple mailbox servers over LMTP. The + opposite is true as well: one mailbox server can be fed over LMTP by + multiple Postfix machines. + + * The local(8) delivery agent understands UNIX-style mailboxes, qmail- + compatible maildir files, Sendmail-style system-wide aliases(5) databases, + and Sendmail-style per-user .forward files. Multiple local delivery agents + can be run in parallel, but parallel delivery to the same user is usually + limited. + + The local(8) delivery agent has hooks for alternative forms of local + delivery: you can configure it to deliver to mailbox files in user home + directories, you can configure it to delegate mailbox delivery to an + external command such as procmail, or you can delegate delivery to a + different Postfix delivery agent. + + * The virtual(8) delivery agent is a bare-bones delivery agent that delivers + to UNIX-style mailbox or qmail-style maildir files only. This delivery + agent can deliver mail for multiple domains, which makes it especially + suitable for hosting lots of small domains on a single machine. This is + described in the VIRTUAL_README document. + + * The pipe(8) mailer is the outbound interface to other mail processing + systems (the Postfix sendmail(1) command being the inbound interface). The + interface is UNIX compatible: it provides information on the command line + and on the standard input stream, and expects a process exit status code as + defined in <sysexits.h>. Examples of delivery via the pipe(8) mailer are in + the MAILDROP_README and UUCP_README documents. + +PPoossttffiixx bbeehhiinndd tthhee sscceenneess + +The previous sections gave an overview of how Postfix server processes send and +receive mail. These server processes rely on other server processes that do +things behind the scenes. The text below attempts to visualize each service in +its own context. As before, names followed by a number are Postfix commands or +server programs, while unnumbered names inside shaded areas represent Postfix +queues. + + * The resident master(8) server is the supervisor that keeps an eye on the + well-being of the Postfix mail system. It is typically started at system + boot time with the "postfix start" command, and keeps running until the + system goes down. The master(8) server is responsible for starting Postfix + server processes to receive and deliver mail, and for restarting servers + that terminate prematurely because of some problem. The master(8) server is + also responsible for enforcing the server process count limits as specified + in the mmaasstteerr..ccff configuration file. The picture below gives the program + hierarchy when Postfix is started up. Only some of the mail handling daemon + processes are shown. + + postfix(1) + + | + | + + postfix-script(1) + + / | \ + | + / \ + + postsuper(1) master(8) postlog(1) + + / | \ + | + / \ + + smtpd(8) qmgr(8) local(8) + + * The anvil(8) server implements client connection and request rate limiting + for all smtpd(8) servers. The TUNING_README document provides guidance for + dealing with mis-behaving SMTP clients. The anvil(8) service is available + in Postfix version 2.2 and later. + + Network -> smtpd(8) <-> anvil(8) + + * The bounce(8), defer(8) and trace(8) services each maintain their own queue + directory trees with per-message logfiles. Postfix uses this information + when sending "failed", "delayed" or "success" delivery status notifications + to the sender. + + The trace(8) service also implements support for the Postfix "sendmail -bv" + and "sendmail -v" commands which produce reports about how Postfix delivers + mail, and is available with Postfix version 2.1 and later. See DEBUG_README + for examples. + + qmgr(8) Delivery + cleanup(8) -> Postfix -> agents + queue + + ^ | | + | v v + + (Non-) bounce(8) Queue id, + delivery <- defer(8) <- recipient, + notice trace(8) status + + ^ | + | v + + Per- + message + logfiles + + * The flush(8) servers maintain per-destination logs and implement both ETRN + and "sendmail -qRdestination", as described in the ETRN_README document. + This moves selected queue files from the deferred queue back to the + incoming queue and requests their delivery. The flush(8) service is + available with Postfix version 1.0 and later. + + incoming + ^ + deferred + + ^ + | + + smtpd(8) Destination Deferred Delivery + sendmail(1) - to flush -> flush(8) <- destination, - agents, + postqueue(1) queue id qmgr(8) + + ^ | + | v + + Per-dest- + ination + logs + + * The proxymap(8) servers provide read-only and read-write table lookup + service to Postfix processes. This overcomes chroot restrictions, reduces + the number of open lookup tables by sharing one open table among multiple + processes, and implements single-updater tables. + + * The scache(8) server maintains the connection cache for the Postfix smtp(8) + client. When connection caching is enabled for selected destinations, the + smtp(8) client does not disconnect immediately after a mail transaction, + but gives the connection to the connection cache server which keeps the + connection open for a limited amount of time. The smtp(8) client continues + with some other mail delivery request. Meanwhile, any smtp(8) process can + ask the scache(8) server for that cached connection and reuse it for mail + delivery. As a safety measure, Postfix limits the number of times that a + connection may be reused. + + When delivering mail to a destination with multiple mail servers, + connection caching can help to skip over a non-responding server, and thus + dramatically speed up delivery. SMTP connection caching is available in + Postfix version 2.2 and later. More information about this feature is in + the CONNECTION_CACHE_README document. + + /-- smtp(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + + A Postfix smtp(8) client can reuse a TLS-encrypted connection (with + "smtp_tls_connection_reuse = yes"). This can greatly reduce the overhead of + connection setup and improves message delivery rates. After a Postfix smtp + (8) client connects to a remote SMTP server and sends plaintext EHLO and + STARTTLS commands, the smtp(8) client inserts a tlsproxy(8) process into + the connection as shown below. + + After the mail transaction completes, the Postfix smtp(8) client gives the + smtp(8)-to-tlsproxy(8) connection to the scache(8) server, which keeps the + connection open for a limited amount of time. The smtp(8) client continues + with some other mail delivery request. Meanwhile, any Postfix smtp(8) + client can ask the scache(8) server for that cached connection and reuse it + for mail delivery. + + /-- smtp(8) --> tlsproxy(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + + * The showq(8) servers list the Postfix queue status. This is the queue + listing service that does the work for the mailq(1) and postqueue(1) + commands. + + mailq(1) Postfix + Output <- post- <- showq(8) <- queue + queue(1) + + * The spawn(8) servers run non-Postfix commands on request, with the client + connected via socket or FIFO to the command's standard input, output and + error streams. You can find examples of its use in the SMTPD_POLICY_README + document. + + * The tlsmgr(8) server runs when TLS (Transport Layer Security, formerly + known as SSL) is turned on in the Postfix smtp(8) client or smtpd(8) + server. This process has two duties: + + o Maintain the pseudo-random number generator (PRNG) that is used to seed + the TLS engines in Postfix smtp(8) client or smtpd(8) server processes. + The state of this PRNG is periodically saved to a file, and is read + when tlsmgr(8) starts up. + + o Maintain the optional Postfix smtp(8) client or smtpd(8) server caches + with TLS session keys. Saved keys can improve performance by reducing + the amount of computation at the start of a TLS session. + + TLS support is available in Postfix version 2.2 and later. Information + about the Postfix TLS implementation is in the TLS_README document. + + <---seed--- ---seed---> + Network-> smtpd(8) tlsmgr(8) smtp(8) ->Network + <-session-> <-session-> + + / | \ + | + / \ + + smtpd PRNG smtp + session state session + cache file cache + + * The verify(8) server verifies that a sender or recipient address is + deliverable before the smtpd(8) server accepts it. The verify(8) server + queries a cache with address verification results. If a result is not + found, the verify(8) server injects a probe message into the Postfix queue + and processes the status update from a delivery agent or queue manager. + This process is described in the ADDRESS_VERIFICATION_README document. The + verify(8) service is available with Postfix version 2.1 and later. + + probe Postfix + message -> mail + Network -> smtpd(8) <-> verify(8) -> queue + + | + v + + <- probe Postfix -> Local + status <- delivery -> Network + ^ agents + | + v + + Address + verification + cache + + * The postscreen(8) server can be put "in front" of Postfix smtpd(8) + processes. Its purpose is to accept connections from the network and to + decide what SMTP clients are allowed to talk to Postfix. According to the + 2008 MessageLabs annual report, 81% of all email was spam, and 90% of that + was sent by botnets; by 2010, those numbers were 92% and 95%, respectively. + While postscreen(8) keeps the zombies away, more smtpd(8) processes remain + available for legitimate clients. + + postscreen(8) maintains a temporary allowlist for clients that pass its + tests; by allowing allowlisted clients to skip tests, postscreen(8) + minimizes its impact on legitimate email traffic. + + The postscreen(8) server is available with Postfix 2.8 and later. To keep + the implementation simple, postscreen(8) delegates DNS allow/denylist + lookups to dnsblog(8) server processes, and delegates TLS encryption/ + decryption to tlsproxy(8) server processes. This delegation is invisible to + the remote SMTP client. + + zombie + + \ + + zombie - tlsproxy(8) - - smtpd(8) + + \ / + + other --- postscreen(8) + + / \ + + other - - smtpd(8) + + / + + zombie + + * The postlogd(8) server provides an alternative to syslog logging, which + remains the default. This feature is available with Postfix version 3.4 or + later, and supports the following modes: + + o Logging to file, which addresses a usability problem with MacOS, and + eliminates information loss caused by systemd rate limits. + + commands -> postlogd(8) -> /path/to/file + or daemons + + o Logging to stdout, which eliminates a syslog dependency when Postfix + runs inside a container. + + commands -> postlogd(8) -> stdout inherited + or daemons from "postfix start-fg" + + See MAILLOG_README for details and limitations. + +PPoossttffiixx ssuuppppoorrtt ccoommmmaannddss + +The Postfix architecture overview ends with a summary of command-line utilities +for day-to-day use of the Postfix mail system. Besides the Sendmail-compatible +sendmail(1), mailq(1), and newaliases(1) commands, the Postfix system comes +with it own collection of command-line utilities. For consistency, these are +all named postsomething. + + * The postfix(1) command controls the operation of the mail system. It is the + interface for starting, stopping, and restarting the mail system, as well + as for some other administrative operations. This command is reserved to + the super-user. + + * The postalias(1) command maintains Postfix aliases(5) type databases. This + is the program that does the work for the newaliases(1) command. + + * The postcat(1) command displays the contents of Postfix queue files. This + is a limited, preliminary utility. This program is likely to be superseded + by something more powerful that can also edit Postfix queue files. + + * The postconf(1) command displays or updates Postfix main.cf parameters and + displays system dependent information about the supported file locking + methods, and the supported types of lookup tables. + + * The postdrop(1) command is the mail posting utility that is run by the + Postfix sendmail(1) command in order to deposit mail into the maildrop + queue directory. + + * The postkick(1) command makes some Postfix internal communication channels + available for use in, for example, shell scripts. + + * The postlock(1) command provides Postfix-compatible mailbox locking for use + in, for example, shell scripts. + + * The postlog(1) command provides Postfix-compatible logging for shell + scripts. + + * The postmap(1) command maintains Postfix lookup tables such as canonical + (5), virtual(5) and others. It is a cousin of the UNIX makemap command. + + * The postmulti(1) command repeats the "postfix start" etc. command for each + Postfix instance, and supports creation, deletion etc. of Postfix + instances. For a tutorial, see MULTI_INSTANCE_README. + + * The postqueue(1) command is the privileged command that is run by Postfix + sendmail(1) and mailq(1) in order to flush or list the mail queue. + + * The postsuper(1) command maintains the Postfix queue. It removes old + temporary files, and moves queue files into the right directory after a + change in the hashing depth of queue directories. This command is run at + mail system startup time and when Postfix is restarted. + diff --git a/README_FILES/PACKAGE_README b/README_FILES/PACKAGE_README new file mode 100644 index 0000000..c64604b --- /dev/null +++ b/README_FILES/PACKAGE_README @@ -0,0 +1,109 @@ +GGuuiiddeelliinneess ffoorr PPaacckkaaggee BBuuiillddeerrss + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +This document has hints and tips for those who manage their own Postfix binary +distribution for internal use, and for those who maintain Postfix binary +distributions for general use. + +GGeenneerraall ddiissttrriibbuuttiioonnss:: pplleeaassee pprroovviiddee aa ssmmaallll ddeeffaauulltt mmaaiinn..ccff ffiillee + +The installed main.cf file must be small. PLEASE resist the temptation to list +all parameters in the main.cf file. Postfix is supposed to be easy to +configure. Listing all parameters in main.cf defeats the purpose. It is an +invitation for hobbyists to make random changes without understanding what they +do, and gets them into endless trouble. + +GGeenneerraall ddiissttrriibbuuttiioonnss:: pplleeaassee iinncclluuddee RREEAADDMMEE oorr HHTTMMLL ffiilleess + +Please provide the applicable README or HTML files. They are referenced by the +Postfix manual pages and by other files. Without README or HTML files, Postfix +will be difficult if not impossible to configure. + +PPoossttffiixx IInnssttaallllaattiioonn ppaarraammeetteerrss + +Postfix installation is controlled by a dozen installation parameters. See the +postfix-install and post-install files for details. Most parameters have +system-dependent default settings that are configurable at compile time, as +described in the INSTALL file. + +PPrreeppaarriinngg aa pprree--bbuuiilltt ppaacckkaaggee ffoorr ddiissttrriibbuuttiioonn ttoo ootthheerr ssyysstteemmss + +You can build a Postfix package on a machine that does not have Postfix +installed on it. All you need is Postfix source code and a compilation +environment that is compatible with the target system. + +You can build a pre-built Postfix package as an unprivileged user. + +First compile Postfix. After successful compilation, execute: + + % mmaakkee ppaacckkaaggee + +With Postfix versions before 2.2 you must invoke the post-install script +directly (% sshh ppoosstt--iinnssttaallll). + +You will be prompted for installation parameters. Specify an install_root +directory other than /. The mail_owner and setgid_group installation parameter +settings will be recorded in the main.cf file, but they won't take effect until +the package is unpacked and installed on the destination machine. + +If you want to fully automate this process, specify all the non-default +installation parameters on the command line: + + % mmaakkee nnoonn--iinntteerraaccttiivvee--ppaacckkaaggee iinnssttaallll__rroooott==//ssoommee//wwhheerree... + +With Postfix versions before 2.2 you must invoke the post-install script +directly (% sshh ppoosstt--iinnssttaallll --nnoonn--iinntteerraaccttiivvee iinnssttaallll__rroooott......). + +With Postfix 3.0 and later, the command "make package name=value ..." will +replace the string MAIL_VERSION in a configuration parameter value with the +Postfix release version. Do not try to specify something like $mail_version on +this command line. This produces inconsistent results with different versions +of the make(1) command. + +BBeeggiinn SSeeccuurriittyy AAlleerrtt + +WWhheenn bbuuiillddiinngg aann aarrcchhiivvee ffoorr ddiissttrriibbuuttiioonn,, bbee ssuurree ttoo aarrcchhiivvee oonnllyy ffiilleess aanndd +ssyymmbboolliicc lliinnkkss,, nnoott tthheeiirr ppaarreenntt ddiirreeccttoorriieess.. OOtthheerrwwiissee,, uunnppaacckkiinngg aa pprree--bbuuiilltt +PPoossttffiixx ppaacckkaaggee mmaayy mmeessss uupp ppeerrmmiissssiioonn aanndd//oorr oowwnneerrsshhiipp ooff ssyysstteemm ddiirreeccttoorriieess +ssuucchh aass // //eettcc //uussrr //uussrr//bbiinn //vvaarr //vvaarr//ssppooooll aanndd ssoo oonn.. TThhiiss iiss eessppeecciiaallllyy aann +iissssuuee iiff yyoouu eexxeeccuutteedd ppoossttffiixx--iinnssttaallll ((sseeee aabboovvee)) aass aann uunnpprriivviilleeggeedd uusseerr.. + +EEnndd SSeeccuurriittyy AAlleerrtt + +Thus, to tar up the pre-built package, take the following steps: + + % cd INSTALL_ROOT + % rm -f SOMEWHERE/outputfile + % find . \! -type d -print | xargs tar rf SOMEWHERE/outputfile + % gzip SOMEWHERE/outputfile + +This way you will not include any directories that might cause trouble upon +extraction. + +IInnssttaalllliinngg aa pprree--bbuuiilltt PPoossttffiixx ppaacckkaaggee + + * To unpack a pre-built Postfix package, execute the equivalent of: + + # umask 022 + # gzip -d <outputfile.tar.gz | (cd / ; tar xvpf -) + + The umask command is necessary for getting the correct permissions on non- + Postfix directories that need to be created in the process. + + * Create the necessary mail_owner account and setgid_group group for + exclusive use by Postfix. + + * Execute the postfix command to set ownership and permission of Postfix + files and directories, and to update Postfix configuration files. If + necessary, specify any non-default settings for mail_owner or setgid_group + on the postfix command line: + + # postfix set-permissions upgrade-configuration \ + setgid_group=xxx mail_owner=yyy + + With Postfix versions before 2.1 you achieve the same result by invoking + the post-install script directly. + diff --git a/README_FILES/PCRE_README b/README_FILES/PCRE_README new file mode 100644 index 0000000..6dffb4b --- /dev/null +++ b/README_FILES/PCRE_README @@ -0,0 +1,78 @@ +PPoossttffiixx PPCCRREE SSuuppppoorrtt + +------------------------------------------------------------------------------- + +PPCCRREE ((PPeerrll CCoommppaattiibbllee RReegguullaarr EExxpprreessssiioonnss)) mmaapp ssuuppppoorrtt + +The optional "pcre" map type allows you to specify regular expressions with the +PERL style notation such as \s for space and \S for non-space. The main +benefit, however, is that pcre lookups are often faster than regexp lookups. +This is because the pcre implementation is often more efficient than the POSIX +regular expression implementation that you find on many systems. + +A description of how to use pcre tables, including examples, is given in the +pcre_table(5) manual page. Information about PCRE itself can be found at http:/ +/www.pcre.org/. + +UUssiinngg PPoossttffiixx ppaacckkaaggeess wwiitthh PPCCRREE ssuuppppoorrtt + +To use pcre with Debian GNU/Linux's Postfix, or with Fedora or RHEL Postfix, +all you need is to install the postfix-pcre package and you're done. There is +no need to recompile Postfix. + +BBuuiillddiinngg PPoossttffiixx ffrroomm ssoouurrccee wwiitthh PPCCRREE ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. + +To build Postfix from source with pcre support, you need a pcre library. +Install a vendor package, or download the source code from locations in https:/ +/www.pcre.org/ and build that yourself. + +Postfix can build with the pcre2 library or the legacy pcre library. It's +probably easiest to let the Postfix build procedure pick one. The following +commands will first discover if the pcre2 library is installed, and if that is +not available, will discover if the legacy pcre library is installed. + + $ make -f Makefile.init makefiles + $ make + +To build Postfix explicitly with a pcre2 library (Postfix 3.7 and later): + + $ make -f Makefile.init makefiles \ + "CCARGS=-DHAS_PCRE=2 `pcre2-config --cflags`" \ + "AUXLIBS_PCRE=`pcre2-config --libs8`" + $ make + +To build Postfix explicitly with a legacy pcre library (all Postfix versions): + + $ make -f Makefile.init makefiles \ + "CCARGS=-DHAS_PCRE=1 `pcre-config --cflags`" \ + "AUXLIBS_PCRE=`pcre-config --libs`" + $ make + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_PCRE. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded PCRE database client, but only the new AUXLIBS_PCRE variable supports +building a dynamically-loaded or statically-loaded PCRE database client. + + Failure to use the AUXLIBS_PCRE variable will defeat the purpose of dynamic + database client loading. Every Postfix executable file will have PCRE + library dependencies. And that was exactly what dynamic database client + loading was meant to avoid. + +TThhiinnggss ttoo kknnooww + + * When Postfix searches a pcre: or regexp: lookup table, each pattern is + applied to the entire input string. Depending on the application, that + string is an entire client hostname, an entire client IP address, or an + entire mail address. Thus, no parent domain or parent network search is + done, "user@domain" mail addresses are not broken up into their user and + domain constituent parts, and "user+foo" is not broken up into user and + foo. + + * Regular expression tables such as pcre: or regexp: are not allowed to do + $number substitution in lookup results that can be security sensitive: + currently, that restriction applies to the local aliases(5) database or the + virtual(8) delivery agent tables. + diff --git a/README_FILES/PGSQL_README b/README_FILES/PGSQL_README new file mode 100644 index 0000000..f68536c --- /dev/null +++ b/README_FILES/PGSQL_README @@ -0,0 +1,126 @@ +PPoossttffiixx PPoossttggrreeSSQQLL HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +The Postfix pgsql map type allows you to hook up Postfix to a PostgreSQL +database. This implementation allows for multiple pgsql databases: you can use +one for a virtual(5) table, one for an access(5) table, and one for an aliases +(5) table if you want. You can specify multiple servers for the same database, +so that Postfix can switch to a good database server if one goes bad. + +Busy mail servers using pgsql maps will generate lots of concurrent pgsql +clients, so the pgsql server(s) should be run with this fact in mind. You can +reduce the number of concurrent pgsql clients by using the Postfix proxymap(8) +service. + +BBuuiillddiinngg PPoossttffiixx wwiitthh PPoossttggrreeSSQQLL ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +Note: to use pgsql with Debian GNU/Linux's Postfix, all you need to do is to +install the postfix-pgsql package and you're done. There is no need to +recompile Postfix. + +In order to build Postfix with pgsql map support, you specify -DHAS_PGSQL, the +directory with the PostgreSQL header files, and the location of the libpq +library file. + +For example: + + % make tidy + % make -f Makefile.init makefiles \ + "CCARGS=-DHAS_PGSQL -I/usr/local/include/pgsql" \ + "AUXLIBS_PGSQL=-L/usr/local/lib -lpq" + +If your PostgreSQL shared library is in a directory that the RUN-TIME linker +does not know about, add a "-Wl,-R,/path/to/directory" option after "-lpq". + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_PGSQL. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded PostgreSQL database client, but only the new AUXLIBS_PGSQL variable +supports building a dynamically-loaded or statically-loaded PostgreSQL database +client. + + Failure to use the AUXLIBS_PGSQL variable will defeat the purpose of + dynamic database client loading. Every Postfix executable file will have + PGSQL database library dependencies. And that was exactly what dynamic + database client loading was meant to avoid. + +Then just run 'make'. + +CCoonnffiigguurriinngg PPoossttggrreeSSQQLL llooookkuupp ttaabblleess + +Once Postfix is built with pgsql support, you can specify a map type in main.cf +like this: + + /etc/postfix/main.cf: + alias_maps = pgsql:/etc/postfix/pgsql-aliases.cf + +The file /etc/postfix/pgsql-aliases.cf specifies lots of information telling +postfix how to reference the pgsql database. For a complete description, see +the pgsql_table(5) manual page. + +EExxaammppllee:: llooccaall aalliiaasseess + +# +# pgsql config file for local(8) aliases(5) lookups +# + +# +# The hosts that Postfix will try to connect to +hosts = host1.some.domain host2.some.domain + +# The user name and password to log into the pgsql server. +user = someone +password = some_password + +# The database name on the servers. +dbname = customer_database + +# Postfix 2.2 and later The SQL query template. See pgsql_table(5). +query = SELECT forw_addr FROM mxaliases WHERE alias='%s' AND status='paid' + +# For Postfix releases prior to 2.2. See pgsql_table(5) for details. +select_field = forw_addr +table = mxaliases +where_field = alias +# Don't forget the leading "AND"! +additional_conditions = AND status = 'paid' + +UUssiinngg mmiirrrroorreedd ddaattaabbaasseess + +Sites that have a need for multiple mail exchangers may enjoy the convenience +of using a networked mailer database, but do not want to introduce a single +point of failure to their system. + +For this reason we've included the ability to have Postfix reference multiple +hosts for access to a single pgsql map. This will work if sites set up mirrored +pgsql databases on two or more hosts. + +Whenever queries fail with an error at one host, the rest of the hosts will be +tried in random order. If no pgsql server hosts are reachable, then mail will +be deferred until at least one of those hosts is reachable. + +CCrreeddiittss + + * This code is based upon the Postfix mysql map by Scott Cotton and Joshua + Marcus, IC Group, Inc. + * The PostgreSQL changes were done by Aaron Sethman. + * Updates for Postfix 1.1.x and PostgreSQL 7.1+ and support for calling + stored procedures were added by Philip Warner. + * LaMont Jones was the initial Postfix pgsql maintainer. + * Liviu Daia revised the configuration interface and added the main.cf + configuration feature. + * Liviu Daia revised the configuration interface and added the main.cf + configuration feature. + * Liviu Daia with further refinements from Jose Luis Tallon and Victor + Duchovni developed the common query, result_format, domain and + expansion_limit interface for LDAP, MySQL and PosgreSQL. + * Leandro Santi updated the PostgreSQL client after the PostgreSQL developers + made major database API changes in response to SQL injection problems, and + made PQexec() handling more robust. + diff --git a/README_FILES/POSTSCREEN_3_5_README b/README_FILES/POSTSCREEN_3_5_README new file mode 100644 index 0000000..ab67800 --- /dev/null +++ b/README_FILES/POSTSCREEN_3_5_README @@ -0,0 +1,863 @@ +PPoossttffiixx PPoossttssccrreeeenn HHoowwttoo ((PPoossttffiixx 22..88 -- 33..55)) + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document describes features that are available in Postfix 2.8 - 3.5. + +The Postfix postscreen(8) daemon provides additional protection against mail +server overload. One postscreen(8) process handles multiple inbound SMTP +connections, and decides which clients may talk to a Postfix SMTP server +process. By keeping spambots away, postscreen(8) leaves more SMTP server +processes available for legitimate clients, and delays the onset of server +overload conditions. + +postscreen(8) should not be used on SMTP ports that receive mail from end-user +clients (MUAs). In a typical deployment, postscreen(8) handles the MX service +on TCP port 25, while MUA clients submit mail via the submission service on TCP +port 587 which requires client authentication. Alternatively, a site could set +up a dedicated, non-postscreen, "port 25" server that provides submission +service and client authentication, but no MX service. + +postscreen(8) maintains a temporary allowlist for clients that pass its tests; +by allowing allowlisted clients to skip tests, postscreen(8) minimizes its +impact on legitimate email traffic. + +postscreen(8) is part of a multi-layer defense. + + * As the first layer, postscreen(8) blocks connections from zombies and other + spambots that are responsible for about 90% of all spam. It is implemented + as a single process to make this defense as inexpensive as possible. + + * The second layer implements more complex SMTP-level access checks with + Postfix SMTP servers, policy daemons, and Milter applications. + + * The third layer performs light-weight content inspection with the Postfix + built-in header_checks and body_checks. This can block unacceptable + attachments such as executable programs, and worms or viruses with easy-to- + recognize signatures. + + * The fourth layer provides heavy-weight content inspection with external + content filters. Typical examples are Amavisd-new, SpamAssassin, and Milter + applications. + +Each layer reduces the spam volume. The general strategy is to use the less +expensive defenses first, and to use the more expensive defenses only for the +spam that remains. + +Topics in this document: + + * Introduction + * The basic idea behind postscreen(8) + * General operation + * Quick tests before everything else + * Tests before the 220 SMTP server greeting + * Tests after the 220 SMTP server greeting + * Other errors + * When all tests succeed + * Configuring the postscreen(8) service + * Historical notes and credits + +TThhee bbaassiicc iiddeeaa bbeehhiinndd ppoossttssccrreeeenn((88)) + +Most email is spam, and most spam is sent out by zombies (malware on +compromised end-user computers). Wietse expects that the zombie problem will +get worse before things improve, if ever. Without a tool like postscreen(8) +that keeps the zombies away, Postfix would be spending most of its resources +not receiving email. + +The main challenge for postscreen(8) is to make an is-a-zombie decision based +on a single measurement. This is necessary because many zombies try to fly +under the radar and avoid spamming the same site repeatedly. Once postscreen(8) +decides that a client is not-a-zombie, it allowlists the client temporarily to +avoid further delays for legitimate mail. + +Zombies have challenges too: they have only a limited amount of time to deliver +spam before their IP address becomes denylisted. To speed up spam deliveries, +zombies make compromises in their SMTP protocol implementation. For example, +they speak before their turn, or they ignore responses from SMTP servers and +continue sending mail even when the server tells them to go away. + +postscreen(8) uses a variety of measurements to recognize zombies. First, +postscreen(8) determines if the remote SMTP client IP address is denylisted. +Second, postscreen(8) looks for protocol compromises that are made to speed up +delivery. These are good indicators for making is-a-zombie decisions based on +single measurements. + +postscreen(8) does not inspect message content. Message content can vary from +one delivery to the next, especially with clients that (also) send legitimate +email. Content is not a good indicator for making is-a-zombie decisions based +on single measurements, and that is the problem that postscreen(8) is focused +on. + +GGeenneerraall ooppeerraattiioonn + +For each connection from an SMTP client, postscreen(8) performs a number of +tests in the order as described below. Some tests introduce a delay of a few +seconds. postscreen(8) maintains a temporary allowlist for clients that pass +its tests; by allowing allowlisted clients to skip tests, postscreen(8) +minimizes its impact on legitimate email traffic. + +By default, postscreen(8) hands off all connections to a Postfix SMTP server +process after logging its findings. This mode is useful for non-destructive +testing. + +In a typical production setting, postscreen(8) is configured to reject mail +from clients that fail one or more tests, after logging the helo, sender and +recipient information. + +Note: postscreen(8) is not an SMTP proxy; this is intentional. The purpose is +to keep zombies away from Postfix, with minimal overhead for legitimate +clients. + +QQuuiicckk tteessttss bbeeffoorree eevveerryytthhiinngg eellssee + +Before engaging in SMTP-level tests. postscreen(8) queries a number of local +deny and allowlists. These tests speed up the handling of known clients. + + * Permanent allow/denylist test + * Temporary allowlist test + * MX Policy test + +PPeerrmmaanneenntt aallllooww//ddeennyylliisstt tteesstt + +The postscreen_access_list parameter (default: permit_mynetworks) specifies a +permanent access list for SMTP client IP addresses. Typically one would specify +something that allowlists local networks, followed by a CIDR table for +selective allow- and denylisting. + +Example: + +/etc/postfix/main.cf: + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + +/etc/postfix/postscreen_access.cidr: + # Rules are evaluated in the order as specified. + # Denylist 192.168.* except 192.168.0.1. + 192.168.0.1 permit + 192.168.0.0/16 reject + +See the postscreen_access_list manpage documentation for more details. + +When the SMTP client address matches a "permit" action, postscreen(8) logs this +with the client address and port number as: + + WWHHIITTEELLIISSTTEEDD [address]:port + +The allowlist action is not configurable: immediately hand off the connection +to a Postfix SMTP server process. + +When the SMTP client address matches a "reject" action, postscreen(8) logs this +with the client address and port number as: + + BBLLAACCKKLLIISSTTEEDD [address]:port + +The postscreen_blacklist_action parameter specifies the action that is taken +next. See "When tests fail before the 220 SMTP server greeting" below. + +TTeemmppoorraarryy aalllloowwlliisstt tteesstt + +The postscreen(8) daemon maintains a temporary allowlist for SMTP client IP +addresses that have passed all the tests described below. The +postscreen_cache_map parameter specifies the location of the temporary +allowlist. The temporary allowlist is not used for SMTP client addresses that +appear on the permanent access list. + +By default the temporary allowlist is not shared with other postscreen(8) +daemons. See Sharing the temporary allowlist below for alternatives. + +When the SMTP client address appears on the temporary allowlist, postscreen(8) +logs this with the client address and port number as: + + PPAASSSS OOLLDD [address]:port + +The action is not configurable: immediately hand off the connection to a +Postfix SMTP server process. The client is excluded from further tests until +its temporary allowlist entry expires, as controlled with the postscreen_*_ttl +parameters. Expired entries are silently renewed if possible. + +MMXX PPoolliiccyy tteesstt + +When the remote SMTP client is not on the static access list or temporary +allowlist, postscreen(8) can implement a number of allowlist tests, before it +grants the client a temporary allowlist status that allows it to talk to a +Postfix SMTP server process. + +When postscreen(8) is configured to monitor all primary and backup MX +addresses, it can refuse to allowlist clients that connect to a backup MX +address only (an old spammer trick to take advantage of backup MX hosts with +weaker anti-spam policies than primary MX hosts). + + NOTE: The following solution is for small sites. Larger sites would have to + share the postscreen(8) cache between primary and backup MTAs, which would + introduce a common point of failure. + + * First, configure the host to listen on both primary and backup MX + addresses. Use the appropriate ifconfig or ip command for the local + operating system, or update the appropriate configuration files and + "refresh" the network protocol stack. + + Second, configure Postfix to listen on the new IP address (this step is + needed when you have specified inet_interfaces in main.cf). + + * Then, configure postscreen(8) to deny the temporary allowlist status on the + backup MX address(es). An example for Wietse's server is: + + /etc/postfix/main.cf: + postscreen_whitelist_interfaces = !168.100.189.8 static:all + + Translation: allow clients to obtain the temporary allowlist status on all + server IP addresses except 168.100.189.8, which is a backup MX address. + +When a non-allowlisted client connects the backup MX address, postscreen(8) +logs this with the client address and port number as: + + CCOONNNNEECCTT ffrroomm [address]:port ttoo [[116688..110000..118899..88]]::2255 + WWHHIITTEELLIISSTT VVEETTOO [address]:port + +Translation: the client at [address]:port connected to the backup MX address +168.100.189.8 while it was not allowlisted. The client will not be granted the +temporary allowlist status, even if passes all the allowlist tests described +below. + +TTeessttss bbeeffoorree tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +The postscreen_greet_wait parameter specifies a short time interval before the +"220 text..." server greeting, where postscreen(8) can run a number of tests in +parallel. + +When a good client passes these tests, and no "deep protocol tests" are +configured, postscreen(8) adds the client to the temporary allowlist and hands +off the "live" connection to a Postfix SMTP server process. The client can then +continue as if postscreen(8) never even existed (except of course for the short +postscreen_greet_wait delay). + + * Pregreet test + * DNS Allow/denylist test + * When tests fail before the 220 SMTP server greeting + +PPrreeggrreeeett tteesstt + +The SMTP protocol is a classic example of a protocol where the server speaks +before the client. postscreen(8) detects zombies that are in a hurry and that +speak before their turn. This test is enabled by default. + +The postscreen_greet_banner parameter specifies the text portion of a "220- +text..." teaser banner (default: $smtpd_banner). Note that this becomes the +first part of a multi-line server greeting. The postscreen(8) daemon sends this +before the postscreen_greet_wait timer is started. The purpose of the teaser +banner is to confuse zombies so that they speak before their turn. It has no +effect on SMTP clients that correctly implement the protocol. + +To avoid problems with poorly-implemented SMTP engines in network appliances or +network testing tools, either exclude them from all tests with the +postscreen_access_list feature or else specify an empty teaser banner: + +/etc/postfix/main.cf: + # Exclude broken clients by allowlisting. Clients in mynetworks + # should always be allowlisted. + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + +/etc/postfix/postscreen_access.cidr: + 192.168.254.0/24 permit + +/etc/postfix/main.cf: + # Disable the teaser banner (try allowlisting first if you can). + postscreen_greet_banner = + +When an SMTP client sends a command before the postscreen_greet_wait time has +elapsed, postscreen(8) logs this as: + + PPRREEGGRREEEETT count aafftteerr time ffrroomm [address]:port text... + +Translation: the client at [address]:port sent count bytes before its turn to +speak. This happened time seconds after the postscreen_greet_wait timer was +started. The text is what the client sent (truncated to 100 bytes, and with +non-printable characters replaced with C-style escapes such as \r for carriage- +return and \n for newline). + +The postscreen_greet_action parameter specifies the action that is taken next. +See "When tests fail before the 220 SMTP server greeting" below. + +DDNNSS AAllllooww//ddeennyylliisstt tteesstt + +The postscreen_dnsbl_sites parameter (default: empty) specifies a list of DNS +blocklist servers with optional filters and weight factors (positive weights +for denylisting, negative for allowlisting). These servers will be queried in +parallel with the reverse client IP address. This test is disabled by default. + + CAUTION: when postscreen rejects mail, its SMTP reply contains the DNSBL + domain name. Use the postscreen_dnsbl_reply_map feature to hide "password" + information in DNSBL domain names. + +When the postscreen_greet_wait time has elapsed, and the combined DNSBL score +is equal to or greater than the postscreen_dnsbl_threshold parameter value, +postscreen(8) logs this as: + + DDNNSSBBLL rraannkk count ffoorr [address]:port + +Translation: the SMTP client at [address]:port has a combined DNSBL score of +count. + +The postscreen_dnsbl_action parameter specifies the action that is taken when +the combined DNSBL score is equal to or greater than the threshold. See "When +tests fail before the 220 SMTP server greeting" below. + +WWhheenn tteessttss ffaaiill bbeeffoorree tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +When the client address matches the permanent denylist, or when the client +fails the pregreet or DNSBL tests, the action is specified with +postscreen_blacklist_action, postscreen_greet_action, or +postscreen_dnsbl_action, respectively. + +iiggnnoorree (default) + Ignore the failure of this test. Allow other tests to complete. Repeat this + test the next time the client connects. This option is useful for testing + and collecting statistics without blocking mail. +eennffoorrccee + Allow other tests to complete. Reject attempts to deliver mail with a 550 + SMTP reply, and log the helo/sender/recipient information. Repeat this test + the next time the client connects. +ddrroopp + Drop the connection immediately with a 521 SMTP reply. Repeat this test the + next time the client connects. + +TTeessttss aafftteerr tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +In this phase of the protocol, postscreen(8) implements a number of "deep +protocol" tests. These tests use an SMTP protocol engine that is built into the +postscreen(8) server. + +Important note: these protocol tests are disabled by default. They are more +intrusive than the pregreet and DNSBL tests, and they have limitations as +discussed next. + + * The main limitation of "after 220 greeting" tests is that a new client must + disconnect after passing these tests (reason: postscreen is not a proxy). + Then the client must reconnect from the same IP address before it can + deliver mail. The following measures may help to avoid email delays: + + o Allow "good" clients to skip tests with the + postscreen_dnsbl_whitelist_threshold feature (Postfix 2.11 and later). + This is especially effective for sites such as Google that never retry + immediately from the same IP address. + + o Small sites: Configure postscreen(8) to listen on multiple IP + addresses, published in DNS as different IP addresses for the same MX + hostname or for different MX hostnames. This avoids mail delivery + delays with clients that reconnect immediately from the same IP + address. + + o Large sites: Share the postscreen(8) cache between different Postfix + MTAs with a large-enough memcache_table(5). Again, this avoids mail + delivery delays with clients that reconnect immediately from the same + IP address. + + * postscreen(8)'s built-in SMTP engine does not implement the AUTH, XCLIENT, + and XFORWARD features. If you need to make these services available on port + 25, then do not enable the tests after the 220 server greeting. + + * End-user clients should connect directly to the submission service, so that + they never have to deal with postscreen(8)'s tests. + +The following "after 220 greeting" tests are available: + + * Command pipelining test + * Non-SMTP command test + * Bare newline test + * When tests fail after the 220 SMTP server greeting + +CCoommmmaanndd ppiippeelliinniinngg tteesstt + +By default, SMTP is a half-duplex protocol: the sender and receiver send one +command and one response at a time. Unlike the Postfix SMTP server, postscreen +(8) does not announce support for ESMTP command pipelining. Therefore, clients +are not allowed to send multiple commands. postscreen(8)'s deep protocol test +for this is disabled by default. + +With "postscreen_pipelining_enable = yes", postscreen(8) detects zombies that +send multiple commands, instead of sending one command and waiting for the +server to reply. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends multiple commands, postscreen(8) logs this as: + + CCOOMMMMAANNDD PPIIPPEELLIINNIINNGG ffrroomm [address]:port aafftteerr command: text + +Translation: the SMTP client at [address]:port sent multiple SMTP commands, +instead of sending one command and then waiting for the server to reply. This +happened after the client sent command. The text shows part of the input that +was sent too early; it is not logged with Postfix 2.8. + +The postscreen_pipelining_action parameter specifies the action that is taken +next. See "When tests fail after the 220 SMTP server greeting" below. + +NNoonn--SSMMTTPP ccoommmmaanndd tteesstt + +Some spambots send their mail through open proxies. A symptom of this is the +usage of commands such as CONNECT and other non-SMTP commands. Just like the +Postfix SMTP server's smtpd_forbidden_commands feature, postscreen(8) has an +equivalent postscreen_forbidden_commands feature to block these clients. +postscreen(8)'s deep protocol test for this is disabled by default. + +With "postscreen_non_smtp_command_enable = yes", postscreen(8) detects zombies +that send commands specified with the postscreen_forbidden_commands parameter. +This also detects commands with the syntax of a message header label. The +latter is a symptom that the client is sending message content after ignoring +all the responses from postscreen(8) that reject mail. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends non-SMTP commands, postscreen(8) logs this as: + + NNOONN--SSMMTTPP CCOOMMMMAANNDD ffrroomm [address]:port aafftteerr command: text + +Translation: the SMTP client at [address]:port sent a command that matches the +postscreen_forbidden_commands parameter, or that has the syntax of a message +header label (text followed by optional space and ":"). The "aafftteerr command" +portion is logged with Postfix 2.10 and later. + +The postscreen_non_smtp_command_action parameter specifies the action that is +taken next. See "When tests fail after the 220 SMTP server greeting" below. + +BBaarree nneewwlliinnee tteesstt + +SMTP is a line-oriented protocol: lines have a limited length, and are +terminated with <CR><LF>. Lines ending in a "bare" <LF>, that is newline not +preceded by carriage return, are not allowed in SMTP. postscreen(8)'s deep +protocol test for this is disabled by default. + +With "postscreen_bare_newline_enable = yes", postscreen(8) detects clients that +send lines ending in bare newline characters. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends bare newline characters, postscreen(8) logs this as: + + BBAARREE NNEEWWLLIINNEE ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port sent a bare newline character, +that is newline not preceded by carriage return. The "aafftteerr command" portion is +logged with Postfix 2.10 and later. + +The postscreen_bare_newline_action parameter specifies the action that is taken +next. See "When tests fail after the 220 SMTP server greeting" below. + +WWhheenn tteessttss ffaaiill aafftteerr tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +When the client fails the pipelining, non-SMTP command or bare newline tests, +the action is specified with postscreen_pipelining_action, +postscreen_non_smtp_command_action or postscreen_bare_newline_action, +respectively. + +iiggnnoorree (default for bare newline) + Ignore the failure of this test. Allow other tests to complete. Do NOT + repeat this test before the result from some other test expires. This + option is useful for testing and collecting statistics without blocking + mail permanently. +eennffoorrccee (default for pipelining) + Allow other tests to complete. Reject attempts to deliver mail with a 550 + SMTP reply, and log the helo/sender/recipient information. Repeat this test + the next time the client connects. +ddrroopp (default for non-SMTP commands) + Drop the connection immediately with a 521 SMTP reply. Repeat this test the + next time the client connects. This action is compatible with the Postfix + SMTP server's smtpd_forbidden_commands feature. + +OOtthheerr eerrrroorrss + +When an SMTP client hangs up unexpectedly, postscreen(8) logs this as: + + HHAANNGGUUPP aafftteerr time ffrroomm [address]:port iinn test name + +Translation: the SMTP client at [address]:port disconnected unexpectedly, time +seconds after the start of the test named test name. + +There is no punishment for hanging up. A client that hangs up without sending +the QUIT command can still pass all postscreen(8) tests. + +The following errors are reported by the built-in SMTP engine. This engine +never accepts mail, therefore it has per-session limits on the number of +commands and on the session length. + + CCOOMMMMAANNDD TTIIMMEE LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-command time +limit as specified with the postscreen_command_time_limit parameter. The +session is terminated immediately. The "aafftteerr command" portion is logged with +Postfix 2.10 and later. + + CCOOMMMMAANNDD CCOOUUNNTT LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-session command +count limit as specified with the postscreen_command_count_limit parameter. The +session is terminated immediately. The "aafftteerr command" portion is logged with +Postfix 2.10 and later. + + CCOOMMMMAANNDD LLEENNGGTTHH LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-command length +limit, as specified with the line_length_limit parameter. The session is +terminated immediately. The "aafftteerr command" portion is logged with Postfix 2.10 +and later. + +When an SMTP client makes too many connections at the same time, postscreen(8) +rejects the connection with a 421 status code and logs: + + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: ttoooo mmaannyy ccoonnnneeccttiioonnss + +The postscreen_client_connection_count_limit parameter controls this limit. + +When an SMTP client connects after postscreen(8) has reached a connection count +limit, postscreen(8) rejects the connection with a 421 status code and logs: + + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: aallll ssccrreeeenniinngg ppoorrttss bbuussyy + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: aallll sseerrvveerr ppoorrttss bbuussyy + +The postscreen_pre_queue_limit and postscreen_post_queue_limit parameters +control these limits. + +WWhheenn aallll tteessttss ssuucccceeeedd + +When a new SMTP client passes all tests (i.e. it is not allowlisted via some +mechanism), postscreen(8) logs this as: + + PPAASSSS NNEEWW [address]:port + +Where [address]:port are the client IP address and port. Then, postscreen(8) +creates a temporary allowlist entry that excludes the client IP address from +further tests until the temporary allowlist entry expires, as controlled with +the postscreen_*_ttl parameters. + +When no "deep protocol tests" are configured, postscreen(8) hands off the +"live" connection to a Postfix SMTP server process. The client can then +continue as if postscreen(8) never even existed (except for the short +postscreen_greet_wait delay). + +When any "deep protocol tests" are configured, postscreen(8) cannot hand off +the "live" connection to a Postfix SMTP server process in the middle of the +session. Instead, postscreen(8) defers mail delivery attempts with a 4XX +status, logs the helo/sender/recipient information, and waits for the client to +disconnect. The next time the client connects it will be allowed to talk to a +Postfix SMTP server process to deliver its mail. postscreen(8) mitigates the +impact of this limitation by giving deep protocol tests a long expiration time. + +CCoonnffiigguurriinngg tthhee ppoossttssccrreeeenn((88)) sseerrvviiccee + +postscreen(8) has been tested on FreeBSD [4-8], Linux 2.[4-6] and Solaris 9 +systems. + + * Turning on postscreen(8) without blocking mail + * postscreen(8) TLS configuration + * Blocking mail with postscreen(8) + * Turning off postscreen(8) + * Sharing the temporary allowlist + +TTuurrnniinngg oonn ppoossttssccrreeeenn((88)) wwiitthhoouutt bblloocckkiinngg mmaaiill + +To enable the postscreen(8) service and log client information without blocking +mail: + + 1. Make sure that local clients and systems with non-standard SMTP + implementations are excluded from any postscreen(8) tests. The default is + to exclude all clients in mynetworks. To exclude additional clients, for + example, third-party performance monitoring tools (these tend to have + broken SMTP implementations): + + /etc/postfix/main.cf: + # Exclude broken clients by allowlisting. Clients in mynetworks + # should always be allowlisted. + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + + /etc/postfix/postscreen_access.cidr: + 192.168.254.0/24 permit + + 2. Comment out the "smtp inet ... smtpd" service in master.cf, including any + "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtp inet n - n - - smtpd + # -o parameter=value ... + + 3. Uncomment the new "smtpd pass ... smtpd" service in master.cf, and + duplicate any "-o parameter=value" entries from the smtpd service that was + commented out in the previous step. + + /etc/postfix/master.cf: + smtpd pass - - n - - smtpd + -o parameter=value ... + + 4. Uncomment the new "smtp inet ... postscreen" service in master.cf. + + /etc/postfix/master.cf: + smtp inet n - n - 1 postscreen + + 5. Uncomment the new "tlsproxy unix ... tlsproxy" service in master.cf. This + service implements STARTTLS support for postscreen(8). + + /etc/postfix/master.cf: + tlsproxy unix - - n - 0 tlsproxy + + 6. Uncomment the new "dnsblog unix ... dnsblog" service in master.cf. This + service does DNSBL lookups for postscreen(8) and logs results. + + /etc/postfix/master.cf: + dnsblog unix - - n - 0 dnsblog + + 7. To enable DNSBL lookups, list some DNS blocklist sites in main.cf, + separated by whitespace. Different sites can have different weights. For + example: + + /etc/postfix/main.cf: + postscreen_dnsbl_threshold = 2 + postscreen_dnsbl_sites = zen.spamhaus.org*2 + bl.spamcop.net*1 b.barracudacentral.org*1 + + Note: if your DNSBL queries have a "secret" in the domain name, you must + censor this information from the postscreen(8) SMTP replies. For example: + + /etc/postfix/main.cf: + postscreen_dnsbl_reply_map = texthash:/etc/postfix/dnsbl_reply + + /etc/postfix/dnsbl_reply: + # Secret DNSBL name Name in postscreen(8) replies + secret.zen.dq.spamhaus.net zen.spamhaus.org + + The texthash: format is similar to hash: except that there is no need to + run postmap(1) before the file can be used, and that it does not detect + changes after the file is read. It is new with Postfix version 2.8. + + 8. Read the new configuration with "postfix reload". + +Notes: + + * Some postscreen(8) configuration parameters implement stress-dependent + behavior. This is supported only when the default value is stress-dependent + (that is, "postconf -d parametername" output shows "parametername = $ + {stress?something}${stress:something}" or "parametername = ${stress? + {something}:{something}}"). Other parameters always evaluate as if the + stress value is the empty string. + + * See "Tests before the 220 SMTP server greeting" for details about the + logging from these postscreen(8) tests. + + * If you run Postfix 2.6 or earlier you must stop and start the master daemon + ("postfix stop; postfix start"). This is needed because the Postfix "pass" + master service type did not work reliably on all systems. + +ppoossttssccrreeeenn((88)) TTLLSS ccoonnffiigguurraattiioonn + +postscreen(8) TLS support is available for remote SMTP clients that aren't +allowlisted, including clients that need to renew their temporary allowlist +status. When a remote SMTP client requests TLS service, postscreen(8) invisibly +hands off the connection to a tlsproxy(8) process. Then, tlsproxy(8) encrypts +and decrypts the traffic between postscreen(8) and the remote SMTP client. One +tlsproxy(8) process can handle multiple SMTP sessions. The number of tlsproxy +(8) processes slowly increases with server load, but it should always be much +smaller than the number of postscreen(8) TLS sessions. + +TLS support for postscreen(8) and tlsproxy(8) uses the same parameters as with +smtpd(8). We recommend that you keep the relevant configuration parameters in +main.cf. If you must specify "-o smtpd_mumble=value" parameter overrides in +master.cf for a postscreen-protected smtpd(8) service, then you should specify +those same parameter overrides for the postscreen(8) and tlsproxy(8) services. + +BBlloocckkiinngg mmaaiill wwiitthh ppoossttssccrreeeenn((88)) + +For compatibility with smtpd(8), postscreen(8) implements the soft_bounce +safety feature. This causes Postfix to reject mail with a "try again" reply +code. + + * To turn this on for all of Postfix, specify "soft_bounce = yes" in main.cf. + + * To turn this on for postscreen(8) only, append "-o soft_bounce=yes" (note: + NO SPACES around '=') to the postscreen entry in master.cf. + +Execute "postfix reload" to make the change effective. + +After testing, do not forget to remove the soft_bounce feature, otherwise +senders won't receive their non-delivery notification until many days later. + +To use the postscreen(8) service to block mail, edit main.cf and specify one or +more of: + + * "postscreen_dnsbl_action = enforce", to reject clients that are on DNS + blocklists, and to log the helo/sender/recipient information. With good + DNSBLs this reduces the amount of load on Postfix SMTP servers + dramatically. + + * "postscreen_greet_action = enforce", to reject clients that talk before + their turn, and to log the helo/sender/recipient information. This stops + over half of all known-to-be illegitimate connections to Wietse's mail + server. It is backup protection for zombies that haven't yet been + denylisted. + + * You can also enable "deep protocol tests", but these are more intrusive + than the pregreet or DNSBL tests. + + When a good client passes the "deep protocol tests", postscreen(8) adds the + client to the temporary allowlist but it cannot hand off the "live" + connection to a Postfix SMTP server process in the middle of the session. + Instead, postscreen(8) defers mail delivery attempts with a 4XX status, + logs the helo/sender/recipient information, and waits for the client to + disconnect. + + When the good client comes back in a later session, it is allowed to talk + directly to a Postfix SMTP server. See "Tests after the 220 SMTP server + greeting" above for limitations with AUTH and other features that clients + may need. + + An unexpected benefit from "deep protocol tests" is that some "good" + clients don't return after the 4XX reply; these clients were not so good + after all. + + Unfortunately, some senders will retry requests from different IP + addresses, and may never get allowlisted. For this reason, Wietse stopped + using "deep protocol tests" on his own internet-facing mail server. + + * There is also support for permanent denylisting and allowlisting; see the + description of the postscreen_access_list parameter for details. + +TTuurrnniinngg ooffff ppoossttssccrreeeenn((88)) + +To turn off postscreen(8) and handle mail directly with Postfix SMTP server +processes: + + 1. Comment out the "smtp inet ... postscreen" service in master.cf, including + any "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtp inet n - n - 1 postscreen + # -o parameter=value ... + + 2. Comment out the "dnsblog unix ... dnsblog" service in master.cf. + + /etc/postfix/master.cf: + #dnsblog unix - - n - 0 dnsblog + + 3. Comment out the "smtpd pass ... smtpd" service in master.cf, including any + "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtpd pass - - n - - smtpd + # -o parameter=value ... + + 4. Comment out the "tlsproxy unix ... tlsproxy" service in master.cf, + including any "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #tlsproxy unix - - n - 0 tlsproxy + # -o parameter=value ... + + 5. Uncomment the "smtp inet ... smtpd" service in master.cf, including any "- + o parameter=value" entries that may follow. + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd + -o parameter=value ... + + 6. Read the new configuration with "postfix reload". + +SShhaarriinngg tthhee tteemmppoorraarryy aalllloowwlliisstt + +By default, the temporary allowlist is not shared between multiple postscreen +(8) daemons. To enable sharing, choose one of the following options: + + * A non-persistent memcache: temporary allowlist can be shared between + postscreen(8) daemons on the same host or different hosts. Disable cache + cleanup (postscreen_cache_cleanup_interval = 0) in all postscreen(8) + daemons because memcache: has no first-next API (but see example 4 below + for memcache: with persistent backup). This requires Postfix 2.9 or later. + + # Example 1: non-persistent memcache: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = memcache:/etc/postfix/postscreen_cache + postscreen_cache_cleanup_interval = 0 + + /etc/postfix/postscreen_cache: + memcache = inet:127.0.0.1:11211 + key_format = postscreen:%s + + * A persistent lmdb: temporary allowlist can be shared between postscreen(8) + daemons that run under the same master(8) daemon, or under different master + (8) daemons on the same host. Disable cache cleanup + (postscreen_cache_cleanup_interval = 0) in all postscreen(8) daemons except + one that is responsible for cache cleanup. This requires Postfix 2.11 or + later. + + # Example 2: persistent lmdb: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = lmdb:$data_directory/postscreen_cache + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + * Other kinds of persistent temporary allowlist can be shared only between + postscreen(8) daemons that run under the same master(8) daemon. In this + case, temporary allowlist access must be shared through the proxymap(8) + daemon. This requires Postfix 2.9 or later. + + # Example 3: proxied btree: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = + proxy:btree:/var/lib/postfix/postscreen_cache + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + # Example 4: proxied btree: allowlist with memcache: accelerator. + /etc/postfix/main.cf: + postscreen_cache_map = memcache:/etc/postfix/postscreen_cache + proxy_write_maps = + proxy:btree:/var/lib/postfix/postscreen_cache + ... other proxied tables ... + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + /etc/postfix/postscreen_cache: + # Note: the $data_directory macro is not defined in this context. + memcache = inet:127.0.0.1:11211 + backup = proxy:btree:/var/lib/postfix/postscreen_cache + key_format = postscreen:%s + + Note 1: disable cache cleanup (postscreen_cache_cleanup_interval = 0) in + all postscreen(8) daemons except one that is responsible for cache cleanup. + + Note 2: postscreen(8) cache sharing via proxymap(8) requires Postfix 2.9 or + later; earlier proxymap(8) implementations don't support cache cleanup. + +HHiissttoorriiccaall nnootteess aanndd ccrreeddiittss + +Many ideas in postscreen(8) were explored in earlier work by Michael Tokarev, +in OpenBSD spamd, and in MailChannels Traffic Control. + +Wietse threw together a crude prototype with pregreet and dnsbl support in June +2009, because he needed something new for a Mailserver conference presentation +in July. Ralf Hildebrandt ran this code on several servers to collect real- +world statistics. This version used the dnsblog(8) ad-hoc DNS client program. + +Wietse needed new material for a LISA conference presentation in November 2010, +so he added support for DNSBL weights and filters in August, followed by a +major code rewrite, deep protocol tests, helo/sender/recipient logging, and +stress-adaptive behavior in September. Ralf Hildebrandt ran this code on +several servers to collect real-world statistics. This version still used the +embarrassing dnsblog(8) ad-hoc DNS client program. + +Wietse added STARTTLS support in December 2010. This makes postscreen(8) usable +for sites that require TLS support. The implementation introduces the tlsproxy +(8) event-driven TLS proxy that decrypts/encrypts the sessions for multiple +SMTP clients. + +The tlsproxy(8) implementation led to the discovery of a "new" class of +vulnerability (CVE-2011-0411) that affected multiple implementations of SMTP, +POP, IMAP, NNTP, and FTP over TLS. + +postscreen(8) was officially released as part of the Postfix 2.8 stable release +in January 2011. + diff --git a/README_FILES/POSTSCREEN_README b/README_FILES/POSTSCREEN_README new file mode 100644 index 0000000..9467e68 --- /dev/null +++ b/README_FILES/POSTSCREEN_README @@ -0,0 +1,876 @@ +PPoossttffiixx PPoossttssccrreeeenn HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document describes features that are available in Postfix 3.6 and later. +See POSTSCREEN_3_5_README.html for Postfix versions 2.8 - 3.5. + +The Postfix postscreen(8) daemon provides additional protection against mail +server overload. One postscreen(8) process handles multiple inbound SMTP +connections, and decides which clients may talk to a Postfix SMTP server +process. By keeping spambots away, postscreen(8) leaves more SMTP server +processes available for legitimate clients, and delays the onset of server +overload conditions. + +postscreen(8) should not be used on SMTP ports that receive mail from end-user +clients (MUAs). In a typical deployment, postscreen(8) handles the MX service +on TCP port 25, while MUA clients submit mail via the submission service on TCP +port 587 which requires client authentication. Alternatively, a site could set +up a dedicated, non-postscreen, "port 25" server that provides submission +service and client authentication, but no MX service. + +postscreen(8) maintains a temporary allowlist for clients that pass its tests; +by allowing allowlisted clients to skip tests, postscreen(8) minimizes its +impact on legitimate email traffic. + +postscreen(8) is part of a multi-layer defense. + + * As the first layer, postscreen(8) blocks connections from zombies and other + spambots that are responsible for about 90% of all spam. It is implemented + as a single process to make this defense as inexpensive as possible. + + * The second layer implements more complex SMTP-level access checks with + Postfix SMTP servers, policy daemons, and Milter applications. + + * The third layer performs light-weight content inspection with the Postfix + built-in header_checks and body_checks. This can block unacceptable + attachments such as executable programs, and worms or viruses with easy-to- + recognize signatures. + + * The fourth layer provides heavy-weight content inspection with external + content filters. Typical examples are Amavisd-new, SpamAssassin, and Milter + applications. + +Each layer reduces the spam volume. The general strategy is to use the less +expensive defenses first, and to use the more expensive defenses only for the +spam that remains. + +Topics in this document: + + * Introduction + * The basic idea behind postscreen(8) + * General operation + * Quick tests before everything else + * Tests before the 220 SMTP server greeting + * Tests after the 220 SMTP server greeting + * Other errors + * When all tests succeed + * Configuring the postscreen(8) service + * Historical notes and credits + +TThhee bbaassiicc iiddeeaa bbeehhiinndd ppoossttssccrreeeenn((88)) + +Most email is spam, and most spam is sent out by zombies (malware on +compromised end-user computers). Wietse expects that the zombie problem will +get worse before things improve, if ever. Without a tool like postscreen(8) +that keeps the zombies away, Postfix would be spending most of its resources +not receiving email. + +The main challenge for postscreen(8) is to make an is-a-zombie decision based +on a single measurement. This is necessary because many zombies try to fly +under the radar and avoid spamming the same site repeatedly. Once postscreen(8) +decides that a client is not-a-zombie, it allowlists the client temporarily to +avoid further delays for legitimate mail. + +Zombies have challenges too: they have only a limited amount of time to deliver +spam before their IP address becomes denylisted. To speed up spam deliveries, +zombies make compromises in their SMTP protocol implementation. For example, +they speak before their turn, or they ignore responses from SMTP servers and +continue sending mail even when the server tells them to go away. + +postscreen(8) uses a variety of measurements to recognize zombies. First, +postscreen(8) determines if the remote SMTP client IP address is denylisted. +Second, postscreen(8) looks for protocol compromises that are made to speed up +delivery. These are good indicators for making is-a-zombie decisions based on +single measurements. + +postscreen(8) does not inspect message content. Message content can vary from +one delivery to the next, especially with clients that (also) send legitimate +email. Content is not a good indicator for making is-a-zombie decisions based +on single measurements, and that is the problem that postscreen(8) is focused +on. + +GGeenneerraall ooppeerraattiioonn + +For each connection from an SMTP client, postscreen(8) performs a number of +tests in the order as described below. Some tests introduce a delay of a few +seconds. postscreen(8) maintains a temporary allowlist for clients that pass +its tests; by allowing allowlisted clients to skip tests, postscreen(8) +minimizes its impact on legitimate email traffic. + +By default, postscreen(8) hands off all connections to a Postfix SMTP server +process after logging its findings. This mode is useful for non-destructive +testing. + +In a typical production setting, postscreen(8) is configured to reject mail +from clients that fail one or more tests, after logging the helo, sender and +recipient information. + +Note: postscreen(8) is not an SMTP proxy; this is intentional. The purpose is +to keep zombies away from Postfix, with minimal overhead for legitimate +clients. + +QQuuiicckk tteessttss bbeeffoorree eevveerryytthhiinngg eellssee + +Before engaging in SMTP-level tests. postscreen(8) queries a number of local +deny and allowlists. These tests speed up the handling of known clients. + + * Permanent allow/denylist test + * Temporary allowlist test + * MX Policy test + +PPeerrmmaanneenntt aallllooww//ddeennyylliisstt tteesstt + +The postscreen_access_list parameter (default: permit_mynetworks) specifies a +permanent access list for SMTP client IP addresses. Typically one would specify +something that allowlists local networks, followed by a CIDR table for +selective allow- and denylisting. + +Example: + +/etc/postfix/main.cf: + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + +/etc/postfix/postscreen_access.cidr: + # Rules are evaluated in the order as specified. + # Denylist 192.168.* except 192.168.0.1. + 192.168.0.1 permit + 192.168.0.0/16 reject + +See the postscreen_access_list manpage documentation for more details. + +When the SMTP client address matches a "permit" action, postscreen(8) logs this +with the client address and port number as: + + AALLLLOOWWLLIISSTTEEDD [address]:port + + Use the respectful_logging configuration parameter to select a deprecated + form of this logging. + +The allowlist action is not configurable: immediately hand off the connection +to a Postfix SMTP server process. + +When the SMTP client address matches a "reject" action, postscreen(8) logs this +with the client address and port number as: + + DDEENNYYLLIISSTTEEDD [address]:port + + Use the respectful_logging configuration parameter to select a deprecated + form of this logging. + +The postscreen_denylist_action parameter specifies the action that is taken +next. See "When tests fail before the 220 SMTP server greeting" below. + +TTeemmppoorraarryy aalllloowwlliisstt tteesstt + +The postscreen(8) daemon maintains a temporary allowlist for SMTP client IP +addresses that have passed all the tests described below. The +postscreen_cache_map parameter specifies the location of the temporary +allowlist. The temporary allowlist is not used for SMTP client addresses that +appear on the permanent access list. + +By default the temporary allowlist is not shared with other postscreen(8) +daemons. See Sharing the temporary allowlist below for alternatives. + +When the SMTP client address appears on the temporary allowlist, postscreen(8) +logs this with the client address and port number as: + + PPAASSSS OOLLDD [address]:port + +The action is not configurable: immediately hand off the connection to a +Postfix SMTP server process. The client is excluded from further tests until +its temporary allowlist entry expires, as controlled with the postscreen_*_ttl +parameters. Expired entries are silently renewed if possible. + +MMXX PPoolliiccyy tteesstt + +When the remote SMTP client is not on the static access list or temporary +allowlist, postscreen(8) can implement a number of allowlist tests, before it +grants the client a temporary allowlist status that allows it to talk to a +Postfix SMTP server process. + +When postscreen(8) is configured to monitor all primary and backup MX +addresses, it can refuse to allowlist clients that connect to a backup MX +address only (an old spammer trick to take advantage of backup MX hosts with +weaker anti-spam policies than primary MX hosts). + + NOTE: The following solution is for small sites. Larger sites would have to + share the postscreen(8) cache between primary and backup MTAs, which would + introduce a common point of failure. + + * First, configure the host to listen on both primary and backup MX + addresses. Use the appropriate ifconfig or ip command for the local + operating system, or update the appropriate configuration files and + "refresh" the network protocol stack. + + Second, configure Postfix to listen on the new IP address (this step is + needed when you have specified inet_interfaces in main.cf). + + * Then, configure postscreen(8) to deny the temporary allowlist status on the + backup MX address(es). An example for Wietse's server is: + + /etc/postfix/main.cf: + postscreen_allowlist_interfaces = !168.100.189.8 static:all + + Translation: allow clients to obtain the temporary allowlist status on all + server IP addresses except 168.100.189.8, which is a backup MX address. + +When a non-allowlisted client connects the backup MX address, postscreen(8) +logs this with the client address and port number as: + + CCOONNNNEECCTT ffrroomm [address]:port ttoo [[116688..110000..118899..88]]::2255 + AALLLLOOWWLLIISSTT VVEETTOO [address]:port + + Use the respectful_logging configuration parameter to select a deprecated + form of this logging. + +Translation: the client at [address]:port connected to the backup MX address +168.100.189.8 while it was not allowlisted. The client will not be granted the +temporary allowlist status, even if passes all the allowlist tests described +below. + +TTeessttss bbeeffoorree tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +The postscreen_greet_wait parameter specifies a short time interval before the +"220 text..." server greeting, where postscreen(8) can run a number of tests in +parallel. + +When a good client passes these tests, and no "deep protocol tests" are +configured, postscreen(8) adds the client to the temporary allowlist and hands +off the "live" connection to a Postfix SMTP server process. The client can then +continue as if postscreen(8) never even existed (except of course for the short +postscreen_greet_wait delay). + + * Pregreet test + * DNS Allow/denylist test + * When tests fail before the 220 SMTP server greeting + +PPrreeggrreeeett tteesstt + +The SMTP protocol is a classic example of a protocol where the server speaks +before the client. postscreen(8) detects zombies that are in a hurry and that +speak before their turn. This test is enabled by default. + +The postscreen_greet_banner parameter specifies the text portion of a "220- +text..." teaser banner (default: $smtpd_banner). Note that this becomes the +first part of a multi-line server greeting. The postscreen(8) daemon sends this +before the postscreen_greet_wait timer is started. The purpose of the teaser +banner is to confuse zombies so that they speak before their turn. It has no +effect on SMTP clients that correctly implement the protocol. + +To avoid problems with poorly-implemented SMTP engines in network appliances or +network testing tools, either exclude them from all tests with the +postscreen_access_list feature or else specify an empty teaser banner: + +/etc/postfix/main.cf: + # Exclude broken clients by allowlisting. Clients in mynetworks + # should always be allowlisted. + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + +/etc/postfix/postscreen_access.cidr: + 192.168.254.0/24 permit + +/etc/postfix/main.cf: + # Disable the teaser banner (try allowlisting first if you can). + postscreen_greet_banner = + +When an SMTP client sends a command before the postscreen_greet_wait time has +elapsed, postscreen(8) logs this as: + + PPRREEGGRREEEETT count aafftteerr time ffrroomm [address]:port text... + +Translation: the client at [address]:port sent count bytes before its turn to +speak. This happened time seconds after the postscreen_greet_wait timer was +started. The text is what the client sent (truncated to 100 bytes, and with +non-printable characters replaced with C-style escapes such as \r for carriage- +return and \n for newline). + +The postscreen_greet_action parameter specifies the action that is taken next. +See "When tests fail before the 220 SMTP server greeting" below. + +DDNNSS AAllllooww//ddeennyylliisstt tteesstt + +The postscreen_dnsbl_sites parameter (default: empty) specifies a list of DNS +blocklist servers with optional filters and weight factors (positive weights +for denylisting, negative for allowlisting). These servers will be queried in +parallel with the reverse client IP address. This test is disabled by default. + + CAUTION: when postscreen rejects mail, its SMTP reply contains the DNSBL + domain name. Use the postscreen_dnsbl_reply_map feature to hide "password" + information in DNSBL domain names. + +When the postscreen_greet_wait time has elapsed, and the combined DNSBL score +is equal to or greater than the postscreen_dnsbl_threshold parameter value, +postscreen(8) logs this as: + + DDNNSSBBLL rraannkk count ffoorr [address]:port + +Translation: the SMTP client at [address]:port has a combined DNSBL score of +count. + +The postscreen_dnsbl_action parameter specifies the action that is taken when +the combined DNSBL score is equal to or greater than the threshold. See "When +tests fail before the 220 SMTP server greeting" below. + +WWhheenn tteessttss ffaaiill bbeeffoorree tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +When the client address matches the permanent denylist, or when the client +fails the pregreet or DNSBL tests, the action is specified with +postscreen_denylist_action, postscreen_greet_action, or +postscreen_dnsbl_action, respectively. + +iiggnnoorree (default) + Ignore the failure of this test. Allow other tests to complete. Repeat this + test the next time the client connects. This option is useful for testing + and collecting statistics without blocking mail. +eennffoorrccee + Allow other tests to complete. Reject attempts to deliver mail with a 550 + SMTP reply, and log the helo/sender/recipient information. Repeat this test + the next time the client connects. +ddrroopp + Drop the connection immediately with a 521 SMTP reply. Repeat this test the + next time the client connects. + +TTeessttss aafftteerr tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +In this phase of the protocol, postscreen(8) implements a number of "deep +protocol" tests. These tests use an SMTP protocol engine that is built into the +postscreen(8) server. + +Important note: these protocol tests are disabled by default. They are more +intrusive than the pregreet and DNSBL tests, and they have limitations as +discussed next. + + * The main limitation of "after 220 greeting" tests is that a new client must + disconnect after passing these tests (reason: postscreen is not a proxy). + Then the client must reconnect from the same IP address before it can + deliver mail. The following measures may help to avoid email delays: + + o Allow "good" clients to skip tests with the + postscreen_dnsbl_allowlist_threshold feature. This is especially + effective for large providers that usually don't retry from the same IP + address. + + o Small sites: Configure postscreen(8) to listen on multiple IP + addresses, published in DNS as different IP addresses for the same MX + hostname or for different MX hostnames. This avoids mail delivery + delays with clients that reconnect immediately from the same IP + address. + + o Large sites: Share the postscreen(8) cache between different Postfix + MTAs with a large-enough memcache_table(5). Again, this avoids mail + delivery delays with clients that reconnect immediately from the same + IP address. + + * postscreen(8)'s built-in SMTP engine does not implement the AUTH, XCLIENT, + and XFORWARD features. If you need to make these services available on port + 25, then do not enable the tests after the 220 server greeting. + + * End-user clients should connect directly to the submission service, so that + they never have to deal with postscreen(8)'s tests. + +The following "after 220 greeting" tests are available: + + * Command pipelining test + * Non-SMTP command test + * Bare newline test + * When tests fail after the 220 SMTP server greeting + +CCoommmmaanndd ppiippeelliinniinngg tteesstt + +By default, SMTP is a half-duplex protocol: the sender and receiver send one +command and one response at a time. Unlike the Postfix SMTP server, postscreen +(8) does not announce support for ESMTP command pipelining. Therefore, clients +are not allowed to send multiple commands. postscreen(8)'s deep protocol test +for this is disabled by default. + +With "postscreen_pipelining_enable = yes", postscreen(8) detects zombies that +send multiple commands, instead of sending one command and waiting for the +server to reply. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends multiple commands, postscreen(8) logs this as: + + CCOOMMMMAANNDD PPIIPPEELLIINNIINNGG ffrroomm [address]:port aafftteerr command: text + +Translation: the SMTP client at [address]:port sent multiple SMTP commands, +instead of sending one command and then waiting for the server to reply. This +happened after the client sent command. The text shows part of the input that +was sent too early; it is not logged with Postfix 2.8. + +The postscreen_pipelining_action parameter specifies the action that is taken +next. See "When tests fail after the 220 SMTP server greeting" below. + +NNoonn--SSMMTTPP ccoommmmaanndd tteesstt + +Some spambots send their mail through open proxies. A symptom of this is the +usage of commands such as CONNECT and other non-SMTP commands. Just like the +Postfix SMTP server's smtpd_forbidden_commands feature, postscreen(8) has an +equivalent postscreen_forbidden_commands feature to block these clients. +postscreen(8)'s deep protocol test for this is disabled by default. + +With "postscreen_non_smtp_command_enable = yes", postscreen(8) detects zombies +that send commands specified with the postscreen_forbidden_commands parameter. +This also detects commands with the syntax of a message header label. The +latter is a symptom that the client is sending message content after ignoring +all the responses from postscreen(8) that reject mail. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends non-SMTP commands, postscreen(8) logs this as: + + NNOONN--SSMMTTPP CCOOMMMMAANNDD ffrroomm [address]:port aafftteerr command: text + +Translation: the SMTP client at [address]:port sent a command that matches the +postscreen_forbidden_commands parameter, or that has the syntax of a message +header label (text followed by optional space and ":"). The "aafftteerr command" +portion is logged with Postfix 2.10 and later. + +The postscreen_non_smtp_command_action parameter specifies the action that is +taken next. See "When tests fail after the 220 SMTP server greeting" below. + +BBaarree nneewwlliinnee tteesstt + +SMTP is a line-oriented protocol: lines have a limited length, and are +terminated with <CR><LF>. Lines ending in a "bare" <LF>, that is newline not +preceded by carriage return, are not allowed in SMTP. postscreen(8)'s deep +protocol test for this is disabled by default. + +With "postscreen_bare_newline_enable = yes", postscreen(8) detects clients that +send lines ending in bare newline characters. + +This test is opportunistically enabled when postscreen(8) has to use the built- +in SMTP engine anyway. This is to make postscreen(8) logging more informative. + +When a client sends bare newline characters, postscreen(8) logs this as: + + BBAARREE NNEEWWLLIINNEE ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port sent a bare newline character, +that is newline not preceded by carriage return. The "aafftteerr command" portion is +logged with Postfix 2.10 and later. + +The postscreen_bare_newline_action parameter specifies the action that is taken +next. See "When tests fail after the 220 SMTP server greeting" below. + +WWhheenn tteessttss ffaaiill aafftteerr tthhee 222200 SSMMTTPP sseerrvveerr ggrreeeettiinngg + +When the client fails the pipelining, non-SMTP command or bare newline tests, +the action is specified with postscreen_pipelining_action, +postscreen_non_smtp_command_action or postscreen_bare_newline_action, +respectively. + +iiggnnoorree (default for bare newline) + Ignore the failure of this test. Allow other tests to complete. Do NOT + repeat this test before the result from some other test expires. This + option is useful for testing and collecting statistics without blocking + mail permanently. +eennffoorrccee (default for pipelining) + Allow other tests to complete. Reject attempts to deliver mail with a 550 + SMTP reply, and log the helo/sender/recipient information. Repeat this test + the next time the client connects. +ddrroopp (default for non-SMTP commands) + Drop the connection immediately with a 521 SMTP reply. Repeat this test the + next time the client connects. This action is compatible with the Postfix + SMTP server's smtpd_forbidden_commands feature. + +OOtthheerr eerrrroorrss + +When an SMTP client hangs up unexpectedly, postscreen(8) logs this as: + + HHAANNGGUUPP aafftteerr time ffrroomm [address]:port iinn test name + +Translation: the SMTP client at [address]:port disconnected unexpectedly, time +seconds after the start of the test named test name. + +There is no punishment for hanging up. A client that hangs up without sending +the QUIT command can still pass all postscreen(8) tests. + +The following errors are reported by the built-in SMTP engine. This engine +never accepts mail, therefore it has per-session limits on the number of +commands and on the session length. + + CCOOMMMMAANNDD TTIIMMEE LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-command time +limit as specified with the postscreen_command_time_limit parameter. The +session is terminated immediately. The "aafftteerr command" portion is logged with +Postfix 2.10 and later. + + CCOOMMMMAANNDD CCOOUUNNTT LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-session command +count limit as specified with the postscreen_command_count_limit parameter. The +session is terminated immediately. The "aafftteerr command" portion is logged with +Postfix 2.10 and later. + + CCOOMMMMAANNDD LLEENNGGTTHH LLIIMMIITT ffrroomm [address]:port aafftteerr command + +Translation: the SMTP client at [address]:port reached the per-command length +limit, as specified with the line_length_limit parameter. The session is +terminated immediately. The "aafftteerr command" portion is logged with Postfix 2.10 +and later. + +When an SMTP client makes too many connections at the same time, postscreen(8) +rejects the connection with a 421 status code and logs: + + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: ttoooo mmaannyy ccoonnnneeccttiioonnss + +The postscreen_client_connection_count_limit parameter controls this limit. + +When an SMTP client connects after postscreen(8) has reached a connection count +limit, postscreen(8) rejects the connection with a 421 status code and logs: + + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: aallll ssccrreeeenniinngg ppoorrttss bbuussyy + NNOOQQUUEEUUEE:: rreejjeecctt:: CCOONNNNEECCTT ffrroomm [address]:port:: aallll sseerrvveerr ppoorrttss bbuussyy + +The postscreen_pre_queue_limit and postscreen_post_queue_limit parameters +control these limits. + +WWhheenn aallll tteessttss ssuucccceeeedd + +When a new SMTP client passes all tests (i.e. it is not allowlisted via some +mechanism), postscreen(8) logs this as: + + PPAASSSS NNEEWW [address]:port + +Where [address]:port are the client IP address and port. Then, postscreen(8) +creates a temporary allowlist entry that excludes the client IP address from +further tests until the temporary allowlist entry expires, as controlled with +the postscreen_*_ttl parameters. + +When no "deep protocol tests" are configured, postscreen(8) hands off the +"live" connection to a Postfix SMTP server process. The client can then +continue as if postscreen(8) never even existed (except for the short +postscreen_greet_wait delay). + +When any "deep protocol tests" are configured, postscreen(8) cannot hand off +the "live" connection to a Postfix SMTP server process in the middle of the +session. Instead, postscreen(8) defers mail delivery attempts with a 4XX +status, logs the helo/sender/recipient information, and waits for the client to +disconnect. The next time the client connects it will be allowed to talk to a +Postfix SMTP server process to deliver its mail. postscreen(8) mitigates the +impact of this limitation by giving deep protocol tests a long expiration time. + +CCoonnffiigguurriinngg tthhee ppoossttssccrreeeenn((88)) sseerrvviiccee + +postscreen(8) has been tested on FreeBSD [4-8], Linux 2.[4-6] and Solaris 9 +systems. + + * Turning on postscreen(8) without blocking mail + * postscreen(8) TLS configuration + * Blocking mail with postscreen(8) + * Turning off postscreen(8) + * Sharing the temporary allowlist + +TTuurrnniinngg oonn ppoossttssccrreeeenn((88)) wwiitthhoouutt bblloocckkiinngg mmaaiill + +To enable the postscreen(8) service and log client information without blocking +mail: + + 1. Make sure that local clients and systems with non-standard SMTP + implementations are excluded from any postscreen(8) tests. The default is + to exclude all clients in mynetworks. To exclude additional clients, for + example, third-party performance monitoring tools (these tend to have + broken SMTP implementations): + + /etc/postfix/main.cf: + # Exclude broken clients by allowlisting. Clients in mynetworks + # should always be allowlisted. + postscreen_access_list = permit_mynetworks, + cidr:/etc/postfix/postscreen_access.cidr + + /etc/postfix/postscreen_access.cidr: + 192.168.254.0/24 permit + + 2. Comment out the "smtp inet ... smtpd" service in master.cf, including any + "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtp inet n - n - - smtpd + # -o parameter=value ... + + 3. Uncomment the new "smtpd pass ... smtpd" service in master.cf, and + duplicate any "-o parameter=value" entries from the smtpd service that was + commented out in the previous step. + + /etc/postfix/master.cf: + smtpd pass - - n - - smtpd + -o parameter=value ... + + 4. Uncomment the new "smtp inet ... postscreen" service in master.cf. + + /etc/postfix/master.cf: + smtp inet n - n - 1 postscreen + + 5. Uncomment the new "tlsproxy unix ... tlsproxy" service in master.cf. This + service implements STARTTLS support for postscreen(8). + + /etc/postfix/master.cf: + tlsproxy unix - - n - 0 tlsproxy + + 6. Uncomment the new "dnsblog unix ... dnsblog" service in master.cf. This + service does DNSBL lookups for postscreen(8) and logs results. + + /etc/postfix/master.cf: + dnsblog unix - - n - 0 dnsblog + + 7. To enable DNSBL lookups, list some DNS blocklist sites in main.cf, + separated by whitespace. Different sites can have different weights. For + example: + + /etc/postfix/main.cf: + postscreen_dnsbl_threshold = 2 + postscreen_dnsbl_sites = zen.spamhaus.org*2 + bl.spamcop.net*1 b.barracudacentral.org*1 + + Note: if your DNSBL queries have a "secret" in the domain name, you must + censor this information from the postscreen(8) SMTP replies. For example: + + /etc/postfix/main.cf: + postscreen_dnsbl_reply_map = texthash:/etc/postfix/dnsbl_reply + + /etc/postfix/dnsbl_reply: + # Secret DNSBL name Name in postscreen(8) replies + secret.zen.dq.spamhaus.net zen.spamhaus.org + + The texthash: format is similar to hash: except that there is no need to + run postmap(1) before the file can be used, and that it does not detect + changes after the file is read. It is new with Postfix version 2.8. + + 8. Read the new configuration with "postfix reload". + +Notes: + + * Some postscreen(8) configuration parameters implement stress-dependent + behavior. This is supported only when the default value is stress-dependent + (that is, "postconf -d parametername" output shows "parametername = $ + {stress?something}${stress:something}" or "parametername = ${stress? + {something}:{something}}"). Other parameters always evaluate as if the + stress value is the empty string. + + * See "Tests before the 220 SMTP server greeting" for details about the + logging from these postscreen(8) tests. + + * If you run Postfix 2.6 or earlier you must stop and start the master daemon + ("postfix stop; postfix start"). This is needed because the Postfix "pass" + master service type did not work reliably on all systems. + +ppoossttssccrreeeenn((88)) TTLLSS ccoonnffiigguurraattiioonn + +postscreen(8) TLS support is available for remote SMTP clients that aren't +allowlisted, including clients that need to renew their temporary allowlist +status. When a remote SMTP client requests TLS service, postscreen(8) invisibly +hands off the connection to a tlsproxy(8) process. Then, tlsproxy(8) encrypts +and decrypts the traffic between postscreen(8) and the remote SMTP client. One +tlsproxy(8) process can handle multiple SMTP sessions. The number of tlsproxy +(8) processes slowly increases with server load, but it should always be much +smaller than the number of postscreen(8) TLS sessions. + +TLS support for postscreen(8) and tlsproxy(8) uses the same parameters as with +smtpd(8). We recommend that you keep the relevant configuration parameters in +main.cf. If you must specify "-o smtpd_mumble=value" parameter overrides in +master.cf for a postscreen-protected smtpd(8) service, then you should specify +those same parameter overrides for the postscreen(8) and tlsproxy(8) services. + +BBlloocckkiinngg mmaaiill wwiitthh ppoossttssccrreeeenn((88)) + +For compatibility with smtpd(8), postscreen(8) implements the soft_bounce +safety feature. This causes Postfix to reject mail with a "try again" reply +code. + + * To turn this on for all of Postfix, specify "soft_bounce = yes" in main.cf. + + * To turn this on for postscreen(8) only, append "-o soft_bounce=yes" (note: + NO SPACES around '=') to the postscreen entry in master.cf. + +Execute "postfix reload" to make the change effective. + +After testing, do not forget to remove the soft_bounce feature, otherwise +senders won't receive their non-delivery notification until many days later. + +To use the postscreen(8) service to block mail, edit main.cf and specify one or +more of: + + * "postscreen_dnsbl_action = enforce", to reject clients that are on DNS + blocklists, and to log the helo/sender/recipient information. With good + DNSBLs this reduces the amount of load on Postfix SMTP servers + dramatically. + + * "postscreen_greet_action = enforce", to reject clients that talk before + their turn, and to log the helo/sender/recipient information. This stops + over half of all known-to-be illegitimate connections to Wietse's mail + server. It is backup protection for zombies that haven't yet been + denylisted. + + * You can also enable "deep protocol tests", but these are more intrusive + than the pregreet or DNSBL tests. + + When a good client passes the "deep protocol tests", postscreen(8) adds the + client to the temporary allowlist but it cannot hand off the "live" + connection to a Postfix SMTP server process in the middle of the session. + Instead, postscreen(8) defers mail delivery attempts with a 4XX status, + logs the helo/sender/recipient information, and waits for the client to + disconnect. + + When the good client comes back in a later session, it is allowed to talk + directly to a Postfix SMTP server. See "Tests after the 220 SMTP server + greeting" above for limitations with AUTH and other features that clients + may need. + + An unexpected benefit from "deep protocol tests" is that some "good" + clients don't return after the 4XX reply; these clients were not so good + after all. + + Unfortunately, some senders will retry requests from different IP + addresses, and may never get allowlisted. For this reason, Wietse stopped + using "deep protocol tests" on his own internet-facing mail server. + + * There is also support for permanent denylisting and allowlisting; see the + description of the postscreen_access_list parameter for details. + +TTuurrnniinngg ooffff ppoossttssccrreeeenn((88)) + +To turn off postscreen(8) and handle mail directly with Postfix SMTP server +processes: + + 1. Comment out the "smtp inet ... postscreen" service in master.cf, including + any "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtp inet n - n - 1 postscreen + # -o parameter=value ... + + 2. Comment out the "dnsblog unix ... dnsblog" service in master.cf. + + /etc/postfix/master.cf: + #dnsblog unix - - n - 0 dnsblog + + 3. Comment out the "smtpd pass ... smtpd" service in master.cf, including any + "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #smtpd pass - - n - - smtpd + # -o parameter=value ... + + 4. Comment out the "tlsproxy unix ... tlsproxy" service in master.cf, + including any "-o parameter=value" entries that follow. + + /etc/postfix/master.cf: + #tlsproxy unix - - n - 0 tlsproxy + # -o parameter=value ... + + 5. Uncomment the "smtp inet ... smtpd" service in master.cf, including any "- + o parameter=value" entries that may follow. + + /etc/postfix/master.cf: + smtp inet n - n - - smtpd + -o parameter=value ... + + 6. Read the new configuration with "postfix reload". + +SShhaarriinngg tthhee tteemmppoorraarryy aalllloowwlliisstt + +By default, the temporary allowlist is not shared between multiple postscreen +(8) daemons. To enable sharing, choose one of the following options: + + * A non-persistent memcache: temporary allowlist can be shared between + postscreen(8) daemons on the same host or different hosts. Disable cache + cleanup (postscreen_cache_cleanup_interval = 0) in all postscreen(8) + daemons because memcache: has no first-next API (but see example 4 below + for memcache: with persistent backup). This requires Postfix 2.9 or later. + + # Example 1: non-persistent memcache: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = memcache:/etc/postfix/postscreen_cache + postscreen_cache_cleanup_interval = 0 + + /etc/postfix/postscreen_cache: + memcache = inet:127.0.0.1:11211 + key_format = postscreen:%s + + * A persistent lmdb: temporary allowlist can be shared between postscreen(8) + daemons that run under the same master(8) daemon, or under different master + (8) daemons on the same host. Disable cache cleanup + (postscreen_cache_cleanup_interval = 0) in all postscreen(8) daemons except + one that is responsible for cache cleanup. This requires Postfix 2.11 or + later. + + # Example 2: persistent lmdb: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = lmdb:$data_directory/postscreen_cache + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + * Other kinds of persistent temporary allowlist can be shared only between + postscreen(8) daemons that run under the same master(8) daemon. In this + case, temporary allowlist access must be shared through the proxymap(8) + daemon. This requires Postfix 2.9 or later. + + # Example 3: proxied btree: allowlist. + /etc/postfix/main.cf: + postscreen_cache_map = + proxy:btree:/var/lib/postfix/postscreen_cache + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + # Example 4: proxied btree: allowlist with memcache: accelerator. + /etc/postfix/main.cf: + postscreen_cache_map = memcache:/etc/postfix/postscreen_cache + proxy_write_maps = + proxy:btree:/var/lib/postfix/postscreen_cache + ... other proxied tables ... + # See note 1 below. + # postscreen_cache_cleanup_interval = 0 + + /etc/postfix/postscreen_cache: + # Note: the $data_directory macro is not defined in this context. + memcache = inet:127.0.0.1:11211 + backup = proxy:btree:/var/lib/postfix/postscreen_cache + key_format = postscreen:%s + + Note 1: disable cache cleanup (postscreen_cache_cleanup_interval = 0) in + all postscreen(8) daemons except one that is responsible for cache cleanup. + + Note 2: postscreen(8) cache sharing via proxymap(8) requires Postfix 2.9 or + later; earlier proxymap(8) implementations don't support cache cleanup. + +HHiissttoorriiccaall nnootteess aanndd ccrreeddiittss + +Many ideas in postscreen(8) were explored in earlier work by Michael Tokarev, +in OpenBSD spamd, and in MailChannels Traffic Control. + +Wietse threw together a crude prototype with pregreet and dnsbl support in June +2009, because he needed something new for a Mailserver conference presentation +in July. Ralf Hildebrandt ran this code on several servers to collect real- +world statistics. This version used the dnsblog(8) ad-hoc DNS client program. + +Wietse needed new material for a LISA conference presentation in November 2010, +so he added support for DNSBL weights and filters in August, followed by a +major code rewrite, deep protocol tests, helo/sender/recipient logging, and +stress-adaptive behavior in September. Ralf Hildebrandt ran this code on +several servers to collect real-world statistics. This version still used the +embarrassing dnsblog(8) ad-hoc DNS client program. + +Wietse added STARTTLS support in December 2010. This makes postscreen(8) usable +for sites that require TLS support. The implementation introduces the tlsproxy +(8) event-driven TLS proxy that decrypts/encrypts the sessions for multiple +SMTP clients. + +The tlsproxy(8) implementation led to the discovery of a "new" class of +vulnerability (CVE-2011-0411) that affected multiple implementations of SMTP, +POP, IMAP, NNTP, and FTP over TLS. + +postscreen(8) was officially released as part of the Postfix 2.8 stable release +in January 2011. + +Noel Jones helped with the Postfix 3.6 transition towards respectful +documentation. + diff --git a/README_FILES/QMQP_README b/README_FILES/QMQP_README new file mode 100644 index 0000000..6bb664f --- /dev/null +++ b/README_FILES/QMQP_README @@ -0,0 +1,5 @@ +PPoossttffiixx qqmmaaiill aanndd eezzmmllmm ssuuppppoorrtt + +------------------------------------------------------------------------------- +This document will be made available via http://www.postfix.org/. + diff --git a/README_FILES/QSHAPE_README b/README_FILES/QSHAPE_README new file mode 100644 index 0000000..28c2386 --- /dev/null +++ b/README_FILES/QSHAPE_README @@ -0,0 +1,741 @@ +PPoossttffiixx BBoottttlleenneecckk AAnnaallyyssiiss + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +This document is an introduction to Postfix queue congestion analysis. It +explains how the qshape(1) program can help to track down the reason for queue +congestion. qshape(1) is bundled with Postfix 2.1 and later source code, under +the "auxiliary" directory. This document describes qshape(1) as bundled with +Postfix 2.4. + +This document covers the following topics: + + * Introducing the qshape tool + * Trouble shooting with qshape + * Example 1: Healthy queue + * Example 2: Deferred queue full of dictionary attack bounces + * Example 3: Congestion in the active queue + * Example 4: High volume destination backlog + * Postfix queue directories + + o The "maildrop" queue + o The "hold" queue + o The "incoming" queue + o The "active" queue + o The "deferred" queue + + * Credits + +IInnttrroodduucciinngg tthhee qqsshhaappee ttooooll + +When mail is draining slowly or the queue is unexpectedly large, run qshape(1) +as the super-user (root) to help zero in on the problem. The qshape(1) program +displays a tabular view of the Postfix queue contents. + + * On the horizontal axis, it displays the queue age with fine granularity for + recent messages and (geometrically) less fine granularity for older + messages. + + * The vertical axis displays the destination (or with the "-s" switch the + sender) domain. Domains with the most messages are listed first. + +For example, in the output below we see the top 10 lines of the (mostly forged) +sender domain distribution for captured spam in the "hold" queue: + + $ qshape -s hold | head + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 486 0 0 1 0 0 2 4 20 40 419 + yahoo.com 14 0 0 1 0 0 0 0 1 0 12 + extremepricecuts.net 13 0 0 0 0 0 0 0 2 0 11 + ms35.hinet.net 12 0 0 0 0 0 0 0 0 1 11 + winnersdaily.net 12 0 0 0 0 0 0 0 2 0 10 + hotmail.com 11 0 0 0 0 0 0 0 0 1 10 + worldnet.fr 6 0 0 0 0 0 0 0 0 0 6 + ms41.hinet.net 6 0 0 0 0 0 0 0 0 0 6 + osn.de 5 0 0 0 0 0 1 0 0 0 4 + + * The "T" column shows the total (in this case sender) count for each domain. + The columns with numbers above them, show counts for messages aged fewer + than that many minutes, but not younger than the age limit for the previous + column. The row labeled "TOTAL" shows the total count for all domains. + + * In this example, there are 14 messages allegedly from yahoo.com, 1 between + 10 and 20 minutes old, 1 between 320 and 640 minutes old and 12 older than + 1280 minutes (1440 minutes in a day). + +When the output is a terminal intermediate results showing the top 20 domains +(-n option) are displayed after every 1000 messages (-N option) and the final +output also shows only the top 20 domains. This makes qshape useful even when +the "deferred" queue is very large and it may otherwise take prohibitively long +to read the entire "deferred" queue. + +By default, qshape shows statistics for the union of both the "incoming" and +"active" queues which are the most relevant queues to look at when analyzing +performance. + +One can request an alternate list of queues: + + $ qshape deferred + $ qshape incoming active deferred + +this will show the age distribution of the "deferred" queue or the union of the +"incoming", "active" and "deferred" queues. + +Command line options control the number of display "buckets", the age limit for +the smallest bucket, display of parent domain counts and so on. The "-h" option +outputs a summary of the available switches. + +TTrroouubbllee sshhoooottiinngg wwiitthh qqsshhaappee + +Large numbers in the qshape output represent a large number of messages that +are destined to (or alleged to come from) a particular domain. It should be +possible to tell at a glance which domains dominate the queue sender or +recipient counts, approximately when a burst of mail started, and when it +stopped. + +The problem destinations or sender domains appear near the top left corner of +the output table. Remember that the "active" queue can accommodate up to 20000 +($qmgr_message_active_limit) messages. To check whether this limit has been +reached, use: + + $ qshape -s active (show sender statistics) + +If the total sender count is below 20000 the "active" queue is not yet +saturated, any high volume sender domains show near the top of the output. + +With oqmgr(8) the "active" queue is also limited to at most 20000 recipient +addresses ($qmgr_message_recipient_limit). To check for exhaustion of this +limit use: + + $ qshape active (show recipient statistics) + +Having found the high volume domains, it is often useful to search the logs for +recent messages pertaining to the domains in question. + + # Find deliveries to example.com + # + $ tail -10000 /var/log/maillog | + grep -E -i ': to=<.*@example\.com>,' | + less + + # Find messages from example.com + # + $ tail -10000 /var/log/maillog | + grep -E -i ': from=<.*@example\.com>,' | + less + +You may want to drill in on some specific queue ids: + + # Find all messages for a specific queue id. + # + $ tail -10000 /var/log/maillog | grep -E ': 2B2173FF68: ' + +Also look for queue manager warning messages in the log. These warnings can +suggest strategies to reduce congestion. + + $ grep -E 'qmgr.*(panic|fatal|error|warning):' /var/log/maillog + +When all else fails try the Postfix mailing list for help, but please don't +forget to include the top 10 or 20 lines of qshape(1) output. + +EExxaammppllee 11:: HHeeaalltthhyy qquueeuuee + +When looking at just the "incoming" and "active" queues, under normal +conditions (no congestion) the "incoming" and "active" queues are nearly empty. +Mail leaves the system almost as quickly as it comes in or is deferred without +congestion in the "active" queue. + + $ qshape (show "incoming" and "active" queue status) + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 5 0 0 0 1 0 0 0 1 1 2 + meri.uwasa.fi 5 0 0 0 1 0 0 0 1 1 2 + +If one looks at the two queues separately, the "incoming" queue is empty or +perhaps briefly has one or two messages, while the "active" queue holds more +messages and for a somewhat longer time: + + $ qshape incoming + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 0 0 0 0 0 0 0 0 0 0 0 + + $ qshape active + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 5 0 0 0 1 0 0 0 1 1 2 + meri.uwasa.fi 5 0 0 0 1 0 0 0 1 1 2 + +EExxaammppllee 22:: DDeeffeerrrreedd qquueeuuee ffuullll ooff ddiiccttiioonnaarryy aattttaacckk bboouunncceess + +This is from a server where recipient validation is not yet available for some +of the hosted domains. Dictionary attacks on the unvalidated domains result in +bounce backscatter. The bounces dominate the queue, but with proper tuning they +do not saturate the "incoming" or "active" queues. The high volume of deferred +mail is not a direct cause for alarm. + + $ qshape deferred | head + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 2234 4 2 5 9 31 57 108 201 464 1353 + heyhihellothere.com 207 0 0 1 1 6 6 8 25 68 92 + pleazerzoneprod.com 105 0 0 0 0 0 0 0 5 44 56 + groups.msn.com 63 2 1 2 4 4 14 14 14 8 0 + orion.toppoint.de 49 0 0 0 1 0 2 4 3 16 23 + kali.com.cn 46 0 0 0 0 1 0 2 6 12 25 + meri.uwasa.fi 44 0 0 0 0 1 0 2 8 11 22 + gjr.paknet.com.pk 43 1 0 0 1 1 3 3 6 12 16 + aristotle.algonet.se 41 0 0 0 0 0 1 2 11 12 15 + +The domains shown are mostly bulk-mailers and all the volume is the tail end of +the time distribution, showing that short term arrival rates are moderate. +Larger numbers and lower message ages are more indicative of current trouble. +Old mail still going nowhere is largely harmless so long as the "active" and +"incoming" queues are short. We can also see that the groups.msn.com +undeliverables are low rate steady stream rather than a concentrated dictionary +attack that is now over. + + $ qshape -s deferred | head + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 2193 4 4 5 8 33 56 104 205 465 1309 + MAILER-DAEMON 1709 4 4 5 8 33 55 101 198 452 849 + example.com 263 0 0 0 0 0 0 0 0 2 261 + example.org 209 0 0 0 0 0 1 3 6 11 188 + example.net 6 0 0 0 0 0 0 0 0 0 6 + example.edu 3 0 0 0 0 0 0 0 0 0 3 + example.gov 2 0 0 0 0 0 0 0 1 0 1 + example.mil 1 0 0 0 0 0 0 0 0 0 1 + +Looking at the sender distribution, we see that as expected most of the +messages are bounces. + +EExxaammppllee 33:: CCoonnggeessttiioonn iinn tthhee aaccttiivvee qquueeuuee + +This example is taken from a Feb 2004 discussion on the Postfix Users list. +Congestion was reported with the "active" and "incoming" queues large and not +shrinking despite very large delivery agent process limits. The thread is +archived at: http://groups.google.com/ +groups?threadm=c0b7js$2r65$1@FreeBSD.csie.NCTU.edu.tw and http:// +archives.neohapsis.com/archives/postfix/2004-02/thread.html#1371 + +Using an older version of qshape(1) it was quickly determined that all the +messages were for just a few destinations: + + $ qshape (show "incoming" and "active" queue status) + + T A 5 10 20 40 80 160 320 320+ + TOTAL 11775 9996 0 0 1 1 42 94 221 1420 + user.sourceforge.net 7678 7678 0 0 0 0 0 0 0 0 + lists.sourceforge.net 2313 2313 0 0 0 0 0 0 0 0 + gzd.gotdns.com 102 0 0 0 0 0 0 0 2 100 + +The "A" column showed the count of messages in the "active" queue, and the +numbered columns showed totals for the "deferred" queue. At 10000 messages +(Postfix 1.x "active" queue size limit) the "active" queue is full. The +"incoming" queue was growing rapidly. + +With the trouble destinations clearly identified, the administrator quickly +found and fixed the problem. It is substantially harder to glean the same +information from the logs. While a careful reading of mailq(1) output should +yield similar results, it is much harder to gauge the magnitude of the problem +by looking at the queue one message at a time. + +EExxaammppllee 44:: HHiigghh vvoolluummee ddeessttiinnaattiioonn bbaacckklloogg + +When a site you send a lot of email to is down or slow, mail messages will +rapidly build up in the "deferred" queue, or worse, in the "active" queue. The +qshape output will show large numbers for the destination domain in all age +buckets that overlap the starting time of the problem: + + $ qshape deferred | head + + T 5 10 20 40 80 160 320 640 1280 1280+ + TOTAL 5000 200 200 400 800 1600 1000 200 200 200 200 + highvolume.com 4000 160 160 320 640 1280 1440 0 0 0 0 + ... + +Here the "highvolume.com" destination is continuing to accumulate deferred +mail. The "incoming" and "active" queues are fine, but the "deferred" queue +started growing some time between 1 and 2 hours ago and continues to grow. + +If the high volume destination is not down, but is instead slow, one might see +similar congestion in the "active" queue. "Active" queue congestion is a +greater cause for alarm; one might need to take measures to ensure that the +mail is deferred instead or even add an access(5) rule asking the sender to try +again later. + +If a high volume destination exhibits frequent bursts of consecutive +connections refused by all MX hosts or "421 Server busy errors", it is possible +for the queue manager to mark the destination as "dead" despite the transient +nature of the errors. The destination will be retried again after the +expiration of a $minimal_backoff_time timer. If the error bursts are frequent +enough it may be that only a small quantity of email is delivered before the +destination is again marked "dead". In some cases enabling static (not on +demand) connection caching by listing the appropriate nexthop domain in a table +included in "smtp_connection_cache_destinations" may help to reduce the error +rate, because most messages will re-use existing connections. + +The MTA that has been observed most frequently to exhibit such bursts of errors +is Microsoft Exchange, which refuses connections under load. Some proxy virus +scanners in front of the Exchange server propagate the refused connection to +the client as a "421" error. + +Note that it is now possible to configure Postfix to exhibit similarly erratic +behavior by misconfiguring the anvil(8) service. Do not use anvil(8) for +steady-state rate limiting, its purpose is (unintentional) DoS prevention and +the rate limits set should be very generous! + +If one finds oneself needing to deliver a high volume of mail to a destination +that exhibits frequent brief bursts of errors and connection caching does not +solve the problem, there is a subtle workaround. + + * Postfix version 2.5 and later: + + o In master.cf set up a dedicated clone of the "smtp" transport for the + destination in question. In the example below we will call it + "fragile". + + o In master.cf configure a reasonable process limit for the cloned smtp + transport (a number in the 10-20 range is typical). + + o IMPORTANT!!! In main.cf configure a large per-destination pseudo-cohort + failure limit for the cloned smtp transport. + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + fragile_destination_concurrency_failed_cohort_limit = 100 + fragile_destination_concurrency_limit = 20 + + /etc/postfix/transport: + example.com fragile: + + /etc/postfix/master.cf: + # service type private unpriv chroot wakeup maxproc command + fragile unix - - n - 20 smtp + + See also the documentation for + default_destination_concurrency_failed_cohort_limit and + default_destination_concurrency_limit. + + * Earlier Postfix versions: + + o In master.cf set up a dedicated clone of the "smtp" transport for the + destination in question. In the example below we will call it + "fragile". + + o In master.cf configure a reasonable process limit for the transport (a + number in the 10-20 range is typical). + + o IMPORTANT!!! In main.cf configure a very large initial and destination + concurrency limit for this transport (say 2000). + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + initial_destination_concurrency = 2000 + fragile_destination_concurrency_limit = 2000 + + /etc/postfix/transport: + example.com fragile: + + /etc/postfix/master.cf: + # service type private unpriv chroot wakeup maxproc command + fragile unix - - n - 20 smtp + + See also the documentation for default_destination_concurrency_limit. + +The effect of this configuration is that up to 2000 consecutive errors are +tolerated without marking the destination dead, while the total concurrency +remains reasonable (10-20 processes). This trick is only for a very specialized +situation: high volume delivery into a channel with multi-error bursts that is +capable of high throughput, but is repeatedly throttled by the bursts of +errors. + +When a destination is unable to handle the load even after the Postfix process +limit is reduced to 1, a desperate measure is to insert brief delays between +delivery attempts. + + * Postfix version 2.5 and later: + + o In master.cf set up a dedicated clone of the "smtp" transport for the + problem destination. In the example below we call it "slow". + + o In main.cf configure a short delay between deliveries to the same + destination. + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + slow_destination_rate_delay = 1 + slow_destination_concurrency_failed_cohort_limit = 100 + + /etc/postfix/transport: + example.com slow: + + /etc/postfix/master.cf: + # service type private unpriv chroot wakeup maxproc command + slow unix - - n - - smtp + + See also the documentation for default_destination_rate_delay. + + This solution forces the Postfix smtp(8) client to wait for + $slow_destination_rate_delay seconds between deliveries to the same + destination. + + IMPORTANT!! The large slow_destination_concurrency_failed_cohort_limit + value is needed. This prevents Postfix from deferring all mail for the same + destination after only one connection or handshake error (the reason for + this is that non-zero slow_destination_rate_delay forces a per-destination + concurrency of 1). + + * Earlier Postfix versions: + + o In the transport map entry for the problem destination, specify a dead + host as the primary nexthop. + + o In the master.cf entry for the transport specify the problem + destination as the fallback_relay and specify a small + smtp_connect_timeout value. + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + + /etc/postfix/transport: + example.com slow:[dead.host] + + /etc/postfix/master.cf: + # service type private unpriv chroot wakeup maxproc command + slow unix - - n - 1 smtp + -o fallback_relay=problem.example.com + -o smtp_connect_timeout=1 + -o smtp_connection_cache_on_demand=no + + This solution forces the Postfix smtp(8) client to wait for + $smtp_connect_timeout seconds between deliveries. The connection caching + feature is disabled to prevent the client from skipping over the dead host. + +PPoossttffiixx qquueeuuee ddiirreeccttoorriieess + +The following sections describe Postfix queues: their purpose, what normal +behavior looks like, and how to diagnose abnormal behavior. + +TThhee ""mmaaiillddrroopp"" qquueeuuee + +Messages that have been submitted via the Postfix sendmail(1) command, but not +yet brought into the main Postfix queue by the pickup(8) service, await +processing in the "maildrop" queue. Messages can be added to the "maildrop" +queue even when the Postfix system is not running. They will begin to be +processed once Postfix is started. + +The "maildrop" queue is drained by the single threaded pickup(8) service +scanning the queue directory periodically or when notified of new message +arrival by the postdrop(1) program. The postdrop(1) program is a setgid helper +that allows the unprivileged Postfix sendmail(1) program to inject mail into +the "maildrop" queue and to notify the pickup(8) service of its arrival. + +All mail that enters the main Postfix queue does so via the cleanup(8) service. +The cleanup service is responsible for envelope and header rewriting, header +and body regular expression checks, automatic bcc recipient processing, milter +content processing, and reliable insertion of the message into the Postfix +"incoming" queue. + +In the absence of excessive CPU consumption in cleanup(8) header or body +regular expression checks or other software consuming all available CPU +resources, Postfix performance is disk I/O bound. The rate at which the pickup +(8) service can inject messages into the queue is largely determined by disk +access times, since the cleanup(8) service must commit the message to stable +storage before returning success. The same is true of the postdrop(1) program +writing the message to the "maildrop" directory. + +As the pickup service is single threaded, it can only deliver one message at a +time at a rate that does not exceed the reciprocal disk I/O latency (+ CPU if +not negligible) of the cleanup service. + +Congestion in this queue is indicative of an excessive local message submission +rate or perhaps excessive CPU consumption in the cleanup(8) service due to +excessive body_checks, or (Postfix >= 2.3) high latency milters. + +Note, that once the "active" queue is full, the cleanup service will attempt to +slow down message injection by pausing $in_flow_delay for each message. In this +case "maildrop" queue congestion may be a consequence of congestion downstream, +rather than a problem in its own right. + +Note, you should not attempt to deliver large volumes of mail via the pickup(8) +service. High volume sites should avoid using "simple" content filters that re- +inject scanned mail via Postfix sendmail(1) and postdrop(1). + +A high arrival rate of locally submitted mail may be an indication of an +uncaught forwarding loop, or a run-away notification program. Try to keep the +volume of local mail injection to a moderate level. + +The "postsuper -r" command can place selected messages into the "maildrop" +queue for reprocessing. This is most useful for resetting any stale +content_filter settings. Requeuing a large number of messages using "postsuper +-r" can clearly cause a spike in the size of the "maildrop" queue. + +TThhee ""hhoolldd"" qquueeuuee + +The administrator can define "smtpd" access(5) policies, or cleanup(8) header/ +body checks that cause messages to be automatically diverted from normal +processing and placed indefinitely in the "hold" queue. Messages placed in the +"hold" queue stay there until the administrator intervenes. No periodic +delivery attempts are made for messages in the "hold" queue. The postsuper(1) +command can be used to manually release messages into the "deferred" queue. + +Messages can potentially stay in the "hold" queue longer than +$maximal_queue_lifetime. If such "old" messages need to be released from the +"hold" queue, they should typically be moved into the "maildrop" queue using +"postsuper -r", so that the message gets a new timestamp and is given more than +one opportunity to be delivered. Messages that are "young" can be moved +directly into the "deferred" queue using "postsuper -H". + +The "hold" queue plays little role in Postfix performance, and monitoring of +the "hold" queue is typically more closely motivated by tracking spam and +malware, than by performance issues. + +TThhee ""iinnccoommiinngg"" qquueeuuee + +All new mail entering the Postfix queue is written by the cleanup(8) service +into the "incoming" queue. New queue files are created owned by the "postfix" +user with an access bitmask (or mode) of 0600. Once a queue file is ready for +further processing the cleanup(8) service changes the queue file mode to 0700 +and notifies the queue manager of new mail arrival. The queue manager ignores +incomplete queue files whose mode is 0600, as these are still being written by +cleanup. + +The queue manager scans the "incoming" queue bringing any new mail into the +"active" queue if the "active" queue resource limits have not been exceeded. By +default, the "active" queue accommodates at most 20000 messages. Once the +"active" queue message limit is reached, the queue manager stops scanning the +"incoming" queue (and the "deferred" queue, see below). + +Under normal conditions the "incoming" queue is nearly empty (has only mode +0600 files), with the queue manager able to import new messages into the +"active" queue as soon as they become available. + +The "incoming" queue grows when the message input rate spikes above the rate at +which the queue manager can import messages into the "active" queue. The main +factors slowing down the queue manager are disk I/O and lookup queries to the +trivial-rewrite service. If the queue manager is routinely not keeping up, +consider not using "slow" lookup services (MySQL, LDAP, ...) for transport +lookups or speeding up the hosts that provide the lookup service. If the +problem is I/O starvation, consider striping the queue over more disks, faster +controllers with a battery write cache, or other hardware improvements. At the +very least, make sure that the queue directory is mounted with the "noatime" +option if applicable to the underlying filesystem. + +The in_flow_delay parameter is used to clamp the input rate when the queue +manager starts to fall behind. The cleanup(8) service will pause for +$in_flow_delay seconds before creating a new queue file if it cannot obtain a +"token" from the queue manager. + +Since the number of cleanup(8) processes is limited in most cases by the SMTP +server concurrency, the input rate can exceed the output rate by at most "SMTP +connection count" / $in_flow_delay messages per second. + +With a default process limit of 100, and an in_flow_delay of 1s, the coupling +is strong enough to limit a single run-away injector to 1 message per second, +but is not strong enough to deflect an excessive input rate from many sources +at the same time. + +If a server is being hammered from multiple directions, consider raising the +in_flow_delay to 10 seconds, but only if the "incoming" queue is growing even +while the "active" queue is not full and the trivial-rewrite service is using a +fast transport lookup mechanism. + +TThhee ""aaccttiivvee"" qquueeuuee + +The queue manager is a delivery agent scheduler; it works to ensure fast and +fair delivery of mail to all destinations within designated resource limits. + +The "active" queue is somewhat analogous to an operating system's process run +queue. Messages in the "active" queue are ready to be sent (runnable), but are +not necessarily in the process of being sent (running). + +While most Postfix administrators think of the "active" queue as a directory on +disk, the real "active" queue is a set of data structures in the memory of the +queue manager process. + +Messages in the "maildrop", "hold", "incoming" and "deferred" queues (see +below) do not occupy memory; they are safely stored on disk waiting for their +turn to be processed. The envelope information for messages in the "active" +queue is managed in memory, allowing the queue manager to do global scheduling, +allocating available delivery agent processes to an appropriate message in the +"active" queue. + +Within the "active" queue, (multi-recipient) messages are broken up into groups +of recipients that share the same transport/nexthop combination; the group size +is capped by the transport's recipient concurrency limit. + +Multiple recipient groups (from one or more messages) are queued for delivery +grouped by transport/nexthop combination. The ddeessttiinnaattiioonn concurrency limit for +the transports caps the number of simultaneous delivery attempts for each +nexthop. Transports with a rreecciippiieenntt concurrency limit of 1 are special: these +are grouped by the actual recipient address rather than the nexthop, yielding +per-recipient concurrency limits rather than per-domain concurrency limits. +Per-recipient limits are appropriate when performing final delivery to +mailboxes rather than when relaying to a remote server. + +Congestion occurs in the "active" queue when one or more destinations drain +slower than the corresponding message input rate. + +Input into the "active" queue comes both from new mail in the "incoming" queue, +and retries of mail in the "deferred" queue. Should the "deferred" queue get +really large, retries of old mail can dominate the arrival rate of new mail. +Systems with more CPU, faster disks and more network bandwidth can deal with +larger "deferred" queues, but as a rule of thumb the "deferred" queue scales to +somewhere between 100,000 and 1,000,000 messages with good performance unlikely +above that "limit". Systems with queues this large should typically stop +accepting new mail, or put the backlog "on hold" until the underlying issue is +fixed (provided that there is enough capacity to handle just the new mail). + +When a destination is down for some time, the queue manager will mark it dead, +and immediately defer all mail for the destination without trying to assign it +to a delivery agent. In this case the messages will quickly leave the "active" +queue and end up in the "deferred" queue (with Postfix < 2.4, this is done +directly by the queue manager, with Postfix >= 2.4 this is done via the "retry" +delivery agent). + +When the destination is instead simply slow, or there is a problem causing an +excessive arrival rate the "active" queue will grow and will become dominated +by mail to the congested destination. + +The only way to reduce congestion is to either reduce the input rate or +increase the throughput. Increasing the throughput requires either increasing +the concurrency or reducing the latency of deliveries. + +For high volume sites a key tuning parameter is the number of "smtp" delivery +agents allocated to the "smtp" and "relay" transports. High volume sites tend +to send to many different destinations, many of which may be down or slow, so a +good fraction of the available delivery agents will be blocked waiting for slow +sites. Also mail destined across the globe will incur large SMTP command- +response latencies, so high message throughput can only be achieved with more +concurrent delivery agents. + +The default "smtp" process limit of 100 is good enough for most sites, and may +even need to be lowered for sites with low bandwidth connections (no use +increasing concurrency once the network pipe is full). When one finds that the +queue is growing on an "idle" system (CPU, disk I/O and network not exhausted) +the remaining reason for congestion is insufficient concurrency in the face of +a high average latency. If the number of outbound SMTP connections (either +ESTABLISHED or SYN_SENT) reaches the process limit, mail is draining slowly and +the system and network are not loaded, raise the "smtp" and/or "relay" process +limits! + +When a high volume destination is served by multiple MX hosts with typically +low delivery latency, performance can suffer dramatically when one of the MX +hosts is unresponsive and SMTP connections to that host timeout. For example, +if there are 2 equal weight MX hosts, the SMTP connection timeout is 30 seconds +and one of the MX hosts is down, the average SMTP connection will take +approximately 15 seconds to complete. With a default per-destination +concurrency limit of 20 connections, throughput falls to just over 1 message +per second. + +The best way to avoid bottlenecks when one or more MX hosts is non-responsive +is to use connection caching. Connection caching was introduced with Postfix +2.2 and is by default enabled on demand for destinations with a backlog of mail +in the "active" queue. When connection caching is in effect for a particular +destination, established connections are re-used to send additional messages, +this reduces the number of connections made per message delivery and maintains +good throughput even in the face of partial unavailability of the destination's +MX hosts. + +If connection caching is not available (Postfix < 2.2) or does not provide a +sufficient latency reduction, especially for the "relay" transport used to +forward mail to "your own" domains, consider setting lower than default SMTP +connection timeouts (1-5 seconds) and higher than default destination +concurrency limits. This will further reduce latency and provide more +concurrency to maintain throughput should latency rise. + +Setting high concurrency limits to domains that are not your own may be viewed +as hostile by the receiving system, and steps may be taken to prevent you from +monopolizing the destination system's resources. The defensive measures may +substantially reduce your throughput or block access entirely. Do not set +aggressive concurrency limits to remote domains without coordinating with the +administrators of the target domain. + +If necessary, dedicate and tune custom transports for selected high volume +destinations. The "relay" transport is provided for forwarding mail to domains +for which your server is a primary or backup MX host. These can make up a +substantial fraction of your email traffic. Use the "relay" and not the "smtp" +transport to send email to these domains. Using the "relay" transport allocates +a separate delivery agent pool to these destinations and allows separate tuning +of timeouts and concurrency limits. + +Another common cause of congestion is unwarranted flushing of the entire +"deferred" queue. The "deferred" queue holds messages that are likely to fail +to be delivered and are also likely to be slow to fail delivery (time out). As +a result the most common reaction to a large "deferred" queue (flush it!) is +more than likely counter-productive, and typically makes the congestion worse. +Do not flush the "deferred" queue unless you expect that most of its content +has recently become deliverable (e.g. relayhost back up after an outage)! + +Note that whenever the queue manager is restarted, there may already be +messages in the "active" queue directory, but the "real" "active" queue in +memory is empty. In order to recover the in-memory state, the queue manager +moves all the "active" queue messages back into the "incoming" queue, and then +uses its normal "incoming" queue scan to refill the "active" queue. The process +of moving all the messages back and forth, redoing transport table (trivial- +rewrite(8) resolve service) lookups, and re-importing the messages back into +memory is expensive. At all costs, avoid frequent restarts of the queue manager +(e.g. via frequent execution of "postfix reload"). + +TThhee ""ddeeffeerrrreedd"" qquueeuuee + +When all the deliverable recipients for a message are delivered, and for some +recipients delivery failed for a transient reason (it might succeed later), the +message is placed in the "deferred" queue. + +The queue manager scans the "deferred" queue periodically. The scan interval is +controlled by the queue_run_delay parameter. While a "deferred" queue scan is +in progress, if an "incoming" queue scan is also in progress (ideally these are +brief since the "incoming" queue should be short), the queue manager alternates +between looking for messages in the "incoming" queue and in the "deferred" +queue. This "round-robin" strategy prevents starvation of either the "incoming" +or the "deferred" queues. + +Each "deferred" queue scan only brings a fraction of the "deferred" queue back +into the "active" queue for a retry. This is because each message in the +"deferred" queue is assigned a "cool-off" time when it is deferred. This is +done by time-warping the modification time of the queue file into the future. +The queue file is not eligible for a retry if its modification time is not yet +reached. + +The "cool-off" time is at least $minimal_backoff_time and at most +$maximal_backoff_time. The next retry time is set by doubling the message's age +in the queue, and adjusting up or down to lie within the limits. This means +that young messages are initially retried more often than old messages. + +If a high volume site routinely has large "deferred" queues, it may be useful +to adjust the queue_run_delay, minimal_backoff_time and maximal_backoff_time to +provide short enough delays on first failure (Postfix >= 2.4 has a sensibly low +minimal backoff time by default), with perhaps longer delays after multiple +failures, to reduce the retransmission rate of old messages and thereby reduce +the quantity of previously deferred mail in the "active" queue. If you want a +really low minimal_backoff_time, you may also want to lower queue_run_delay, +but understand that more frequent scans will increase the demand for disk I/O. + +One common cause of large "deferred" queues is failure to validate recipients +at the SMTP input stage. Since spammers routinely launch dictionary attacks +from unrepliable sender addresses, the bounces for invalid recipient addresses +clog the "deferred" queue (and at high volumes proportionally clog the "active" +queue). Recipient validation is strongly recommended through use of the +local_recipient_maps and relay_recipient_maps parameters. Even when bounces +drain quickly they inundate innocent victims of forgery with unwanted email. To +avoid this, do not accept mail for invalid recipients. + +When a host with lots of deferred mail is down for some time, it is possible +for the entire "deferred" queue to reach its retry time simultaneously. This +can lead to a very full "active" queue once the host comes back up. The +phenomenon can repeat approximately every maximal_backoff_time seconds if the +messages are again deferred after a brief burst of congestion. Perhaps, a +future Postfix release will add a random offset to the retry time (or use a +combination of strategies) to reduce the odds of repeated complete "deferred" +queue flushes. + +CCrreeddiittss + +The qshape(1) program was developed by Victor Duchovni of Morgan Stanley, who +also wrote the initial version of this document. + diff --git a/README_FILES/RELEASE_NOTES b/README_FILES/RELEASE_NOTES new file mode 120000 index 0000000..577eefe --- /dev/null +++ b/README_FILES/RELEASE_NOTES @@ -0,0 +1 @@ +../RELEASE_NOTES
\ No newline at end of file diff --git a/README_FILES/RESTRICTION_CLASS_README b/README_FILES/RESTRICTION_CLASS_README new file mode 100644 index 0000000..9c78684 --- /dev/null +++ b/README_FILES/RESTRICTION_CLASS_README @@ -0,0 +1,166 @@ +PPoossttffiixx PPeerr--CClliieenntt//UUsseerr//eettcc.. AAcccceessss CCoonnttrrooll + +------------------------------------------------------------------------------- + +PPoossttffiixx rreessttrriiccttiioonn ccllaasssseess + +The Postfix SMTP server supports access restrictions such as reject_rbl_client +or reject_unknown_client_hostname on the right-hand side of SMTP server access +(5) tables. This allows you to implement different junk mail restrictions for +different clients or users. + +Having to specify lists of access restrictions for every recipient becomes +tedious quickly. Postfix restriction classes allow you to give easy-to-remember +names to groups of UCE restrictions (such as "permissive", "restrictive", and +so on). + +The real reason for the existence of Postfix restriction classes is more +mundane: you can't specify a lookup table on the right-hand side of a Postfix +access table. This is because Postfix needs to open lookup tables ahead of +time, but the reader probably does not care about these low-level details. + +Example: + + /etc/postfix/main.cf: + smtpd_restriction_classes = restrictive, permissive + # With Postfix < 2.3 specify reject_unknown_client. + restrictive = reject_unknown_sender_domain + reject_unknown_client_hostname ... + permissive = permit + + smtpd_recipient_restrictions = + permit_mynetworks + # reject_unauth_destination is not needed here if the mail + # relay policy is specified with smtpd_relay_restrictions + # (available with Postfix 2.10 and later). + reject_unauth_destination + check_recipient_access hash:/etc/postfix/recipient_access + ... + + /etc/postfix/recipient_access: + joe@my.domain permissive + jane@my.domain restrictive + +With this in place, you can use "restrictive" or "permissive" on the right-hand +side of your per-client, helo, sender, or recipient SMTPD access tables. + +The remainder of this document gives examples of how Postfix access restriction +classes can be used to: + + * Shield an internal mailing list from outside posters, + * Prevent external access by internal senders. + +These questions come up frequently, and the examples hopefully make clear that +Postfix restriction classes aren't really the right solution. They should be +used for what they were designed to do, different junk mail restrictions for +different clients or users. + +PPrrootteeccttiinngg iinntteerrnnaall eemmaaiill ddiissttrriibbuuttiioonn lliissttss + + We want to implement an internal email distribution list. Something like + all@our.domain.com, which aliases to all employees. My first thought was to + use the aliases map, but that would lead to "all" being accessible from the + "outside", and this is not desired... :-) + +Postfix can implement per-address access controls. What follows is based on the +SMTP client IP address, and therefore is subject to IP spoofing. + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + ... + check_recipient_access hash:/etc/postfix/access + ...the usual stuff... + + /etc/postfix/access: + all@my.domain permit_mynetworks,reject + all@my.hostname permit_mynetworks,reject + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what map types Postfix supports, use the command ppoossttccoonnff --mm. + +Now, that would be sufficient when your machine receives all Internet mail +directly from the Internet. That's unlikely if your network is a bit larger +than an office. For example, your backup MX hosts would "launder" the client IP +address of mail from the outside so it would appear to come from a trusted +machine. + +In the general case you need two lookup tables: one table that lists +destinations that need to be protected, and one table that lists domains that +are allowed to send to the protected destinations. + +What follows is based on the sender SMTP envelope address, and therefore is +subject to SMTP sender spoofing. + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + ... + check_recipient_access hash:/etc/postfix/protected_destinations + ...the usual stuff... + + smtpd_restriction_classes = insiders_only + insiders_only = check_sender_access hash:/etc/postfix/insiders, reject + + /etc/postfix/protected_destinations: + all@my.domain insiders_only + all@my.hostname insiders_only + + /etc/postfix/insiders: + my.domain OK matches my.domain and subdomains + another.domain OK matches another.domain and subdomains + +Getting past this scheme is relatively easy, because all one has to do is to +spoof the SMTP sender address. + +If the internal list is a low-volume one, perhaps it makes more sense to make +it moderated. + +RReessttrriiccttiinngg wwhhaatt uusseerrss ccaann sseenndd mmaaiill ttoo ooffff--ssiittee ddeessttiinnaattiioonnss + + How can I configure Postfix in a way that some users can send mail to the + internet and other users not. The users with no access should receive a + generic bounce message. Please don't discuss whether such access + restrictions are necessary, it was not my decision. + +Postfix has support for per-user restrictions. The restrictions are implemented +by the SMTP server. Thus, users that violate the policy have their mail +rejected by the SMTP server. Like this: + + 554 <user@remote>: Access denied + +The implementation uses two lookup tables. One table defines what users are +restricted in where they can send mail, and the other table defines what +destinations are local. It is left as an exercise for the reader to change this +into a scheme where only some users have permission to send mail to off-site +destinations, and where most users are restricted. + +The example assumes DB/DBM files, but this could also be done with LDAP or SQL. + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + ... + check_sender_access hash:/etc/postfix/restricted_senders + ...other stuff... + + smtpd_restriction_classes = local_only + local_only = + check_recipient_access hash:/etc/postfix/local_domains, reject + + /etc/postfix/restricted_senders: + foo@domain local_only + bar@domain local_only + + /etc/postfix/local_domains: + this.domain OK matches this.domain and subdomains + that.domain OK matches that.domain and subdomains + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what map types Postfix supports, use the command ppoossttccoonnff --mm. + +Note: this scheme does not authenticate the user, and therefore it can be +bypassed in several ways: + + * By sending mail via a less restrictive mail relay host. + + * By sending mail as someone else who does have permission to send mail to + off-site destinations. + diff --git a/README_FILES/SASL_README b/README_FILES/SASL_README new file mode 100644 index 0000000..83c0b29 --- /dev/null +++ b/README_FILES/SASL_README @@ -0,0 +1,1436 @@ +PPoossttffiixx SSAASSLL HHoowwttoo + +------------------------------------------------------------------------------- + +HHooww PPoossttffiixx uusseess SSAASSLL aauutthheennttiiccaattiioonn + +SMTP servers need to decide whether an SMTP client is authorized to send mail +to remote destinations, or only to destinations that the server itself is +responsible for. Usually, SMTP servers accept mail to remote destinations when +the client's IP address is in the "same network" as the server's IP address. + +SMTP clients outside the SMTP server's network need a different way to get +"same network" privileges. To address this need, Postfix supports SASL +authentication (RFC 4954, formerly RFC 2554). With this a remote SMTP client +can authenticate to the Postfix SMTP server, and the Postfix SMTP client can +authenticate to a remote SMTP server. Once a client is authenticated, a server +can give it "same network" privileges. + +Postfix does not implement SASL itself, but instead uses existing +implementations as building blocks. This means that some SASL-related +configuration files will belong to Postfix, while other configuration files +belong to the specific SASL implementation that Postfix will use. This document +covers both the Postfix and non-Postfix configuration. + +NOTE: People who go to the trouble of installing Postfix may have the +expectation that Postfix is more secure than some other mailers. The Cyrus SASL +library contains a lot of code. With this, Postfix becomes as secure as other +mail systems that use the Cyrus SASL library. Dovecot provides an alternative +that may be worth considering. + +You can read more about the following topics: + + * Configuring SASL authentication in the Postfix SMTP server + * Configuring SASL authentication in the Postfix SMTP/LMTP client + * Building Postfix with SASL support + * Using Cyrus SASL version 1.5.x + * Credits + +CCoonnffiigguurriinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +As mentioned earlier, SASL is implemented separately from Postfix. For this +reason, configuring SASL authentication in the Postfix SMTP server involves two +different steps: + + * Configuring the SASL implementation to offer a list of mechanisms that are + suitable for SASL authentication and, depending on the SASL implementation + used, configuring authentication backends that verify the remote SMTP + client's authentication data against the system password file or some other + database. + + * Configuring the Postfix SMTP server to enable SASL authentication, and to + authorize clients to relay mail or to control what envelope sender + addresses the client may use. + +Successful authentication in the Postfix SMTP server requires a functional SASL +framework. Configuring SASL should therefore always be the first step, before +configuring Postfix. + +You can read more about the following topics: + + * Which SASL Implementations are supported? + * Configuring Dovecot SASL + + o Postfix to Dovecot SASL communication + + * Configuring Cyrus SASL + + o Cyrus SASL configuration file name + o Cyrus SASL configuration file location + o Postfix to Cyrus SASL communication + + * Enabling SASL authentication and authorization in the Postfix SMTP server + + o Enabling SASL authentication in the Postfix SMTP server + o Postfix SMTP Server policy - SASL mechanism properties + o Enabling SASL authorization in the Postfix SMTP server + o Additional SMTP Server SASL options + + * Testing SASL authentication in the Postfix SMTP server + +WWhhiicchh SSAASSLL IImmpplleemmeennttaattiioonnss aarree ssuuppppoorrtteedd?? + +Currently the Postfix SMTP server supports the Cyrus SASL and Dovecot SASL +implementations. + + NNoottee + + Current Postfix versions have a plug-in architecture that can support + multiple SASL implementations. Before Postfix version 2.3, Postfix had + support only for Cyrus SASL. + +To find out what SASL implementations are compiled into Postfix, use the +following commands: + + % ppoossttccoonnff --aa (SASL support in the SMTP server) + % ppoossttccoonnff --AA (SASL support in the SMTP+LMTP client) + +These commands are available only with Postfix version 2.3 and later. + +CCoonnffiigguurriinngg DDoovveeccoott SSAASSLL + +Dovecot is a POP/IMAP server that has its own configuration to authenticate +POP/IMAP clients. When the Postfix SMTP server uses Dovecot SASL, it reuses +parts of this configuration. Consult the Dovecot documentation for how to +configure and operate the Dovecot authentication server. + +PPoossttffiixx ttoo DDoovveeccoott SSAASSLL ccoommmmuunniiccaattiioonn + +Communication between the Postfix SMTP server and Dovecot SASL happens over a +UNIX-domain socket or over a TCP socket. We will be using a UNIX-domain socket +for better privacy. + +The following fragment for Dovecot version 2 assumes that the Postfix queue is +under /var/spool/postfix/. + + 1 conf.d/10-master.conf: + 2 service auth { + 3 ... + 4 unix_listener /var/spool/postfix/private/auth { + 5 mode = 0660 + 6 # Assuming the default Postfix user and group + 7 user = postfix + 8 group = postfix + 9 } + 10 ... + 11 } + 12 + 13 conf.d/10-auth.conf + 14 auth_mechanisms = plain login + +Line 4 places the Dovecot SASL socket in /var/spool/postfix/private/auth, lines +5-8 limit read+write permissions to user and group postfix only, and line 14 +provides plain and login as mechanisms for the Postfix SMTP server. + +Proceed with the section "Enabling SASL authentication and authorization in the +Postfix SMTP server" to turn on and use SASL in the Postfix SMTP server. + +CCoonnffiigguurriinngg CCyyrruuss SSAASSLL + +The Cyrus SASL framework supports a wide variety of applications (POP, IMAP, +SMTP, etc.). Different applications may require different configurations. As a +consequence each application may have its own configuration file. + +The first step configuring Cyrus SASL is to determine name and location of a +configuration file that describes how the Postfix SMTP server will use the SASL +framework. + +CCyyrruuss SSAASSLL ccoonnffiigguurraattiioonn ffiillee nnaammee + +The name of the configuration file (default: smtpd.conf) is configurable. It is +a concatenation from a value that the Postfix SMTP server sends to the Cyrus +SASL library, and the suffix .conf, added by Cyrus SASL. + +The value sent by Postfix is the name of the server component that will use +Cyrus SASL. It defaults to smtpd and is configured with one of the following +variables: + + /etc/postfix/main.cf: + # Postfix 2.3 and later + smtpd_sasl_path = smtpd + + # Postfix < 2.3 + smtpd_sasl_application_name = smtpd + +CCyyrruuss SSAASSLL ccoonnffiigguurraattiioonn ffiillee llooccaattiioonn + +The location where Cyrus SASL searches for the named file depends on the Cyrus +SASL version and the OS/distribution used. + +You can read more about the following topics: + + * Cyrus SASL version 2.x searches for the configuration file in /usr/lib/ + sasl2/. + + * Cyrus SASL version 2.1.22 and newer additionally search in /etc/sasl2/. + + * With Postfix 2.5 and later you can explicitly configure the search path via + the cyrus_sasl_config_path configuration parameter. Specify zero or more + colon-separated directories. If set empty (the default value) the search + path is the one compiled into the Cyrus SASL library. + + * Some Postfix distributions employ a non-empty default value for + cyrus_sasl_config_path to look for the Cyrus SASL configuration file in / + etc/postfix/sasl/, /var/lib/sasl2/ etc. See the output of postconf + cyrus_sasl_config_path and/or the distribution-specific documentation to + determine the expected location. + + * Some Debian-based Postfix distributions patch Postfix to hardcode a non- + default search path, making it impossible to set an alternate search path + via the "cyrus_sasl_config_path" parameter. This is likely to be the case + when the distribution documents a Postfix-specific path (e.g. /etc/postfix/ + sasl/) that is different from the default value of "cyrus_sasl_config_path" + (which then is likely to be empty). + + NNoottee + + Cyrus SASL searches /usr/lib/sasl2/ first. If it finds the specified + configuration file there, it will not examine other locations. + +PPoossttffiixx ttoo CCyyrruuss SSAASSLL ccoommmmuunniiccaattiioonn + +As the Postfix SMTP server is linked with the Cyrus SASL library libsasl, +communication between Postfix and Cyrus SASL takes place by calling functions +in the SASL library. + +The SASL library may use an external password verification service, or an +internal plugin to connect to authentication backends and verify the SMTP +client's authentication data against the system password file or other +databases. + +The following table shows typical combinations discussed in this document: + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + | aauutthheennttiiccaattiioonn bbaacckkeenndd |ppaasssswwoorrdd vveerriiffiiccaattiioonn sseerrvviiccee // pplluuggiinn| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |/etc/shadow |saslauthd | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |PAM |saslauthd | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |IMAP server |saslauthd | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |sasldb |sasldb | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |MySQL, PostgreSQL, SQLite|sql | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |LDAP |ldapdb | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + + NNoottee + + Read the Cyrus SASL documentation for other backends it can use. + +ssaassllaauutthhdd -- CCyyrruuss SSAASSLL ppaasssswwoorrdd vveerriiffiiccaattiioonn sseerrvviiccee + +Communication between the Postfix SMTP server (read: Cyrus SASL's libsasl) and +the saslauthd server takes place over a UNIX-domain socket. + +saslauthd usually establishes the UNIX domain socket in /var/run/saslauthd/ and +waits for authentication requests. The Postfix SMTP server must have +read+execute permission to this directory or authentication attempts will fail. + + IImmppoorrttaanntt + + Some distributions require the user postfix to be member of a special group + e.g. sasl, otherwise it will not be able to access the saslauthd socket + directory. + +The following example configures the Cyrus SASL library to contact saslauthd as +its password verification service: + + /etc/sasl2/smtpd.conf: + pwcheck_method: saslauthd + mech_list: PLAIN LOGIN + + IImmppoorrttaanntt + + Do not specify any other mechanisms in mech_list than PLAIN or LOGIN when + using saslauthd! It can only handle these two mechanisms, and + authentication will fail if clients are allowed to choose other mechanisms. + + IImmppoorrttaanntt + + Plaintext mechanisms (PLAIN, LOGIN) send credentials unencrypted. This + information should be protected by an additional security layer such as a + TLS-encrypted SMTP session (see: TLS_README). + +Additionally the saslauthd server itself must be configured. It must be told +which authentication backend to turn to for password verification. The backend +is selected with a saslauthd command-line option and will be shown in the +following examples. + + NNoottee + + Some distributions use a configuration file to provide saslauthd command + line options to set e.g. the authentication backend. Typical locations are + /etc/sysconfig/saslauthd or /etc/default/saslauthd. + +UUssiinngg ssaassllaauutthhdd wwiitthh //eettcc//sshhaaddooww + +Access to the /etc/shadow system password file requires root privileges. The +Postfix SMTP server (and in consequence libsasl linked to the server) runs with +the least privilege possible. Direct access to /etc/shadow would not be +possible without breaking the Postfix security architecture. + +The saslauthd socket builds a safe bridge. Postfix, running as limited user +postfix, can access the UNIX-domain socket that saslauthd receives commands on; +saslauthd, running as privileged user root, has the privileges required to +access the shadow file. + +The saslauthd server verifies passwords against the authentication backend / +etc/shadow if started like this: + + % ssaassllaauutthhdd --aa sshhaaddooww + +See section "Testing saslauthd authentication" for test instructions. + +UUssiinngg ssaassllaauutthhdd wwiitthh PPAAMM + +Cyrus SASL can use the PAM framework to authenticate credentials. saslauthd +uses the PAM framework when started like this: + + % ssaassllaauutthhdd --aa ppaamm + + NNoottee + + PAM configuration for the Postfix SMTP server is usually given in /etc/ + pam.d/smtp and is beyond the scope of this document. + +See section "Testing saslauthd authentication" for test instructions. + +UUssiinngg ssaassllaauutthhdd wwiitthh aann IIMMAAPP sseerrvveerr + +saslauthd can verify the SMTP client credentials by using them to log into an +IMAP server. If the login succeeds, SASL authentication also succeeds. +saslauthd contacts an IMAP server when started like this: + + % ssaassllaauutthhdd --aa rriimmaapp --OO iimmaapp..eexxaammppllee..ccoomm + + NNoottee + + The option "-O imap.example.com" specifies the IMAP server saslauthd should + contact when it verifies credentials. + + IImmppoorrttaanntt + + saslauthd sends IMAP login information unencrypted. Any IMAP session + leaving the local host should be protected by an additional security layer + such as an SSL tunnel. + +See section "Testing saslauthd authentication" for test instructions. + +TTeessttiinngg ssaassllaauutthhdd aauutthheennttiiccaattiioonn + +Cyrus SASL provides the testsaslauthd utility to test saslauthd authentication. +The username and password are given as command line arguments. The example +shows the response when authentication is successful: + + % tteessttssaassllaauutthhdd --uu uusseerrnnaammee --pp ppaasssswwoorrdd + 0: OK "Success." + + NNoottee + + Sometimes the testsaslauthd program is not distributed with a the Cyrus + SASL main package. In that case, it may be distributed with -devel, -dev or + -debug packages. + +Specify an additional "-s smtp" if saslauthd was configured to contact the PAM +authentication framework, and specify an additional "-f //ppaatthh//ttoo//ssoocckkeettddiirr//mmuuxx" +if saslauthd establishes the UNIX-domain socket in a non-default location. + +If authentication succeeds, proceed with the section "Enabling SASL +authentication and authorization in the Postfix SMTP server". + +CCyyrruuss SSAASSLL PPlluuggiinnss -- aauuxxiilliiaarryy pprrooppeerrttyy pplluuggiinnss + +Cyrus SASL uses a plugin infrastructure (called auxprop) to expand libsasl's +capabilities. Currently Cyrus SASL sources provide three authentication +plugins. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPlluuggiinn|DDeessccrriippttiioonn | + |_ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |sasldb|Accounts are stored stored in a Cyrus SASL Berkeley DB database| + |_ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |sql |Accounts are stored in a SQL database | + |_ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |ldapdb|Accounts are stored stored in an LDAP database | + |_ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + + IImmppoorrttaanntt + + These three plugins support shared-secret mechanisms i.e. CRAM-MD5, DIGEST- + MD5 and NTLM. These mechanisms send credentials encrypted but their + verification process requires the password to be available in plaintext. + Consequently passwords cannot (!) be stored in encrypted form. + +TThhee ssaassllddbb pplluuggiinn + +The sasldb auxprop plugin authenticates SASL clients against credentials that +are stored in a Berkeley DB database. The database schema is specific to Cyrus +SASL. The database is usually located at /etc/sasldb2. + + NNoottee + + The sasldb2 file contains passwords in plaintext, and should have + read+write access only to user postfix or a group that postfix is member + of. + +The saslpasswd2 command-line utility creates and maintains the database: + + % ssaassllppaasssswwdd22 --cc --uu eexxaammppllee..ccoomm uusseerrnnaammee + Password: + Again (for verification): + +This command creates an account uusseerrnnaammee@@eexxaammppllee..ccoomm. + + IImmppoorrttaanntt + + users must specify uusseerrnnaammee@@eexxaammppllee..ccoomm as login name, not uusseerrnnaammee. + +Run the following command to reuse the Postfix mydomain parameter value as the +login domain: + + % ssaassllppaasssswwdd22 --cc --uu ``ppoossttccoonnff --hh mmyyddoommaaiinn`` uusseerrnnaammee + Password: + Again (for verification): + + NNoottee + + Run saslpasswd2 without any options for further help on how to use the + command. + +The sasldblistusers2 command lists all existing users in the sasldb database: + + % ssaassllddbblliissttuusseerrss22 + username1@example.com: password1 + username2@example.com: password2 + +Configure libsasl to use sasldb with the following instructions: + + /etc/sasl2/smtpd.conf: + pwcheck_method: auxprop + auxprop_plugin: sasldb + mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 NTLM + + NNoottee + + In the above example adjust mech_list to the mechanisms that are applicable + for your environment. + +TThhee ssqqll pplluuggiinn + +The sql auxprop plugin is a generic SQL plugin. It provides access to +credentials stored in a MySQL, PostgreSQL or SQLite database. This plugin +requires that SASL client passwords are stored as plaintext. + + TTiipp + + If you must store encrypted passwords, you cannot use the sql auxprop + plugin. Instead, see section "Using saslauthd with PAM", and configure PAM + to look up the encrypted passwords with, for example, the pam_mysql module. + You will not be able to use any of the methods that require access to + plaintext passwords, such as the shared-secret methods CRAM-MD5 and DIGEST- + MD5. + +The following example configures libsasl to use the sql plugin and connects it +to a PostgreSQL server: + + /etc/sasl2/smtpd.conf: + pwcheck_method: auxprop + auxprop_plugin: sql + mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 NTLM + sql_engine: pgsql + sql_hostnames: 127.0.0.1, 192.0.2.1 + sql_user: username + sql_passwd: secret + sql_database: dbname + sql_select: SELECT password FROM users WHERE user = '%u@%r' + + NNoottee + + Set appropriate permissions if smtpd.conf contains a password. The file + should be readable by the postfix user. + + NNoottee + + In the above example, adjust mech_list to the mechanisms that are + applicable for your environment. + +The sql plugin has the following configuration options: + + sql_engine + Specify mysql to connect to a MySQL server, pgsql for a PostgreSQL + server or sqlite for an SQLite database + + sql_hostnames + Specify one or more servers (hostname or hostname:port) separated by + commas. + + NNoottee + + With MySQL servers, specify localhost to connect over a UNIX-domain + socket, and specify 127.0.0.1 to connect over a TCP socket. + + sql_user + The login name to gain access to the database. + + sql_passwd + The password to gain access to the database. + + sql_database + The name of the database to connect to. + + sql_select + The SELECT statement that should retrieve the plaintext password from a + database table. + + IImmppoorrttaanntt + + Do not enclose the statement in quotes! Use single quotes to escape + macros! + +The sql plugin provides macros to build sql_select statements. They will be +replaced with arguments sent from the client. The following macros are +available: + + %u + The name of the user whose properties are being selected. + + %p + The name of the property being selected. While this could technically + be anything, Cyrus SASL will try userPassword and cmusaslsecretMECHNAME + (where MECHNAME is the name of a SASL mechanism). + + %r + The name of the realm to which the user belongs. This could be the + KERBEROS realm, the fully-qualified domain name of the computer the + SASL application is running on, or the domain after the "@" in a + username. + +TThhee llddaappddbb pplluuggiinn + +The ldapdb auxprop plugin provides access to credentials stored in an LDAP +server. This plugin requires that SASL client passwords are stored as +plaintext. + + TTiipp + + If you must store encrypted passwords, you cannot use the ldapdb auxprop + plugin. Instead, you can use "saslauthd -a ldap" to query the LDAP database + directly, with appropriate configuration in saslauthd.conf, as described + here. You will not be able to use any of the methods that require access to + plaintext passwords, such as the shared-secret methods CRAM-MD5 and DIGEST- + MD5. + +The ldapdb plugin implements proxy authorization. This means that the ldapdb +plugin uses its own username and password to authenticate with the LDAP server, +before it asks the LDAP server for the remote SMTP client's password. The LDAP +server then decides if the ldapdb plugin is authorized to read the remote SMTP +client's password. + +In a nutshell: Configuring ldapdb means authentication and authorization must +be configured twice - once in the Postfix SMTP server to authenticate and +authorize the remote SMTP client, and once in the LDAP server to authenticate +and authorize the ldapdb plugin. + +This example configures libsasl to use the ldapdb plugin and the plugin to +connect to an LDAP server: + + /etc/sasl2/smtpd.conf: + pwcheck_method: auxprop + auxprop_plugin: ldapdb + mech_list: PLAIN LOGIN NTLM CRAM-MD5 DIGEST-MD5 + ldapdb_uri: ldap://localhost + ldapdb_id: proxyuser + ldapdb_pw: password + ldapdb_mech: DIGEST-MD5 + + IImmppoorrttaanntt + + Set appropriate permissions if smtpd.conf contains a password. The file + should be readable by the postfix user. + + NNoottee + + The shared-secret mechanisms (CRAM-MD5, etc.) require that the SASL client + passwords are stored as plaintext. + +The following is a summary of applicable smtpd.conf file entries: + + auxprop_plugin + Specify ldapdb to enable the plugin. + + ldapdb_uri + Specify either ldapi:// to connect over a UNIX-domain socket, ldap:/ + / for an unencrypted TCP connection, or ldaps:// for an encrypted TCP + connection. + + ldapdb_id + The login name to authenticate the ldapdb plugin to the LDAP server + (proxy authorization). + + ldapdb_pw + The password (in plaintext) to authenticate the ldapdb plugin to the + LDAP server (proxy authorization). + + ldapdb_mech + The mechanism to authenticate the ldapdb plugin to the LDAP server. + + NNoottee + + Specify a mechanism here that is supported by the LDAP server. + + ldapdb_rc (optional) + The path to a file containing individual configuration options for the + ldapdb LDAP client (libldap). This allows to specify a TLS client + certificate which in turn can be used to use the SASL EXTERNAL + mechanism. + + NNoottee + + This mechanism supports authentication over an encrypted transport + layer, which is recommended if the plugin must connect to an + OpenLDAP server on a remote machine. + + ldapdb_starttls (optional) + The TLS policy for connecting to the LDAP server. Specify either try or + demand. If the option is try the plugin will attempt to establish a + TLS-encrypted connection with the LDAP server, and will fallback to an + unencrypted connection if TLS fails. If the policy is demand and a TLS- + encrypted connection cannot be established, the connection fails + immediately. + +When the ldapdb plugin connects to the OpenLDAP server and successfully +authenticates, the OpenLDAP server decides if the plugin user is authorized to +read SASL account information. + +The following configuration gives an example of authorization configuration in +the OpenLDAP slapd server: + + /etc/openldap/slapd.conf: + authz-regexp + uid=(.*),cn=.*,cn=auth + ldap:///dc=example,dc=com??sub?cn=$1 + authz-policy to + +Here, the authz-regexp option serves for authentication of the ldapdb user. It +maps its login name to a DN in the LDAP directory tree where slapd can look up +the SASL account information. The authz-policy options defines the +authentication policy. In this case it grants authentication privileges "to" +the ldapdb plugin. + +The last configuration step is to tell the OpenLDAP slapd server where ldapdb +may search for usernames matching the one given by the mail client. The example +below adds an additional attribute ldapdb user object (here: authzTo because +the authz-policy is "to") and configures the scope where the login name +"proxyuser" may search: + + dn: cn=proxyuser,dc=example,dc=com + changetype: modify + add: authzTo + authzTo: dn.regex:uniqueIdentifier=(.*),ou=people,dc=example,dc=com + +Use the ldapmodify or ldapadd command to add the above attribute. + + NNoottee + + Read the chapter "Using SASL" in the OpenLDAP Admin Guide for more detailed + instructions to set up SASL authentication in OpenLDAP. + +EEnnaabblliinngg SSAASSLL aauutthheennttiiccaattiioonn aanndd aauutthhoorriizzaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +By default the Postfix SMTP server uses the Cyrus SASL implementation. If the +Dovecot SASL implementation should be used, specify an smtpd_sasl_type value of +dovecot instead of cyrus: + + /etc/postfix/main.cf: + smtpd_sasl_type = dovecot + +Additionally specify how Postfix SMTP server can find the Dovecot +authentication server. This depends on the settings that you have selected in +the section "Postfix to Dovecot SASL communication". + + * If you configured Dovecot for UNIX-domain socket communication, configure + Postfix as follows: + + /etc/postfix/main.cf: + smtpd_sasl_path = private/auth + + NNoottee + This example uses a pathname relative to the Postfix queue directory, so + that it will work whether or not the Postfix SMTP server runs chrooted. + + * If you configured Dovecot for TCP socket communication, configure Postfix + as follows. If Dovecot runs on a different machine, replace 127.0.0.1 by + that machine's IP address. + + /etc/postfix/main.cf: + smtpd_sasl_path = inet:127.0.0.1:12345 + + NNoottee + If you specify a remote IP address, information will be sent as plaintext + over the network. + +EEnnaabblliinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +Regardless of the SASL implementation type, enabling SMTP authentication in the +Postfix SMTP server always requires setting the smtpd_sasl_auth_enable option: + + /etc/postfix/main.cf: + smtpd_sasl_auth_enable = yes + +After a "postfix reload", SMTP clients will see the additional capability AUTH +in an SMTP session, followed by a list of authentication mechanisms the server +supports: + + % tteellnneett sseerrvveerr..eexxaammppllee..ccoomm 2255 + ... + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-AUTH DIGEST-MD5 PLAIN CRAM-MD5 + ... + +However not all clients recognize the AUTH capability as defined by the SASL +authentication RFC. Some historical implementations expect the server to send +an "=" as separator between the AUTH verb and the list of mechanisms that +follows it. + +The broken_sasl_auth_clients configuration option lets Postfix repeat the AUTH +statement in a form that these broken clients understand: + + /etc/postfix/main.cf: + broken_sasl_auth_clients = yes + + NNoottee + + Enable this option for Outlook up to and including version 2003 and Outlook + Express up to version 6. This option does not hurt other clients. + +After "postfix reload", the Postfix SMTP server will propagate the AUTH +capability twice - once for compliant and once for broken clients: + + % tteellnneett sseerrvveerr..eexxaammppllee..ccoomm 2255 + ... + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-AUTH DIGEST-MD5 PLAIN CRAM-MD5 + 250-AUTH=DIGEST-MD5 PLAIN CRAM-MD5 + ... + +PPoossttffiixx SSMMTTPP SSeerrvveerr ppoolliiccyy -- SSAASSLL mmeecchhaanniissmm pprrooppeerrttiieess + +The Postfix SMTP server supports policies that limit the SASL mechanisms that +it makes available to clients, based on the properties of those mechanisms. The +next two sections give examples of how these policies are used. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPrrooppeerrttyy |DDeessccrriippttiioonn | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |noanonymous |Don't use mechanisms that permit anonymous | + | |authentication. | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |noplaintext |Don't use mechanisms that transmit unencrypted username | + | |and password information. | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |nodictionary |Don't use mechanisms that are vulnerable to dictionary | + | |attacks. | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |forward_secrecy|Require forward secrecy between sessions (breaking one | + | |session does not break earlier sessions). | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mutual_auth |Use only mechanisms that authenticate both the client and| + | |the server to each other. | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +UUnneennccrryypptteedd SSMMTTPP sseessssiioonn + +The default policy is to allow any mechanism in the Postfix SMTP server except +for those based on anonymous authentication: + + /etc/postfix/main.cf: + # Specify a list of properties separated by comma or whitespace + smtpd_sasl_security_options = noanonymous + + IImmppoorrttaanntt + + Always set at least the noanonymous option. Otherwise, the Postfix SMTP + server can give strangers the same authorization as a properly- + authenticated client. + +EEnnccrryypptteedd SSMMTTPP sseessssiioonn ((TTLLSS)) + +A separate parameter controls Postfix SASL mechanism policy during a TLS- +encrypted SMTP session. The default is to copy the settings from the +unencrypted session: + + /etc/postfix/main.cf: + smtpd_sasl_tls_security_options = $smtpd_sasl_security_options + +A more sophisticated policy allows plaintext mechanisms, but only over a TLS- +encrypted connection: + + /etc/postfix/main.cf: + smtpd_sasl_security_options = noanonymous, noplaintext + smtpd_sasl_tls_security_options = noanonymous + +To offer SASL authentication only after a TLS-encrypted session has been +established specify this: + + /etc/postfix/main.cf: + smtpd_tls_auth_only = yes + +EEnnaabblliinngg SSAASSLL aauutthhoorriizzaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +After the client has authenticated with SASL, the Postfix SMTP server decides +what the remote SMTP client will be authorized for. Examples of possible SMTP +clients authorizations are: + + * Send a message to a remote recipient. + + * Use a specific envelope sender in the MAIL FROM command. + +These permissions are not enabled by default. + +MMaaiill rreellaayy aauutthhoorriizzaattiioonn + +With permit_sasl_authenticated the Postfix SMTP server can allow SASL- +authenticated SMTP clients to send mail to remote destinations. Examples: + + # With Postfix 2.10 and later, the mail relay policy is + # preferably specified under smtpd_relay_restrictions. + /etc/postfix/main.cf: + smtpd_relay_restrictions = + permit_mynetworks + ppeerrmmiitt__ssaassll__aauutthheennttiiccaatteedd + reject_unauth_destination + + # Older configurations combine relay control and spam control under + # smtpd_recipient_restrictions. To use this example with Postfix >= + # 2.10 specify "smtpd_relay_restrictions=". + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + permit_mynetworks + ppeerrmmiitt__ssaassll__aauutthheennttiiccaatteedd + reject_unauth_destination + ...other rules... + +EEnnvveellooppee sseennddeerr aaddddrreessss aauutthhoorriizzaattiioonn + +By default an SMTP client may specify any envelope sender address in the MAIL +FROM command. That is because the Postfix SMTP server only knows the remote +SMTP client hostname and IP address, but not the user who controls the remote +SMTP client. + +This changes the moment an SMTP client uses SASL authentication. Now, the +Postfix SMTP server knows who the sender is. Given a table of envelope sender +addresses and SASL login names, the Postfix SMTP server can decide if the SASL +authenticated client is allowed to use a particular envelope sender address: + + /etc/postfix/main.cf: + ssmmttppdd__sseennddeerr__llooggiinn__mmaappss == hhaasshh:://eettcc//ppoossttffiixx//ccoonnttrroolllleedd__eennvveellooppee__sseennddeerrss + + smtpd_recipient_restrictions = + ... + rreejjeecctt__sseennddeerr__llooggiinn__mmiissmmaattcchh + permit_sasl_authenticated + ... + +The controlled_envelope_senders table specifies the binding between a sender +envelope address and the SASL login names that own that address: + + /etc/postfix/controlled_envelope_senders + # envelope sender owners (SASL login names) + john@example.com john@example.com + helpdesk@example.com john@example.com, mary@example.com + postmaster admin@example.com + @example.net barney, fred, john@example.com, + mary@example.com + +With this, the reject_sender_login_mismatch restriction above will reject the +sender address in the MAIL FROM command if smtpd_sender_login_maps does not +specify the SMTP client's login name as an owner of that address. + +See also reject_authenticated_sender_login_mismatch, +reject_known_sender_login_mismatch, and +reject_unauthenticated_sender_login_mismatch for additional control over the +SASL login name and the envelope sender. + +AAddddiittiioonnaall SSMMTTPP SSeerrvveerr SSAASSLL ooppttiioonnss + +Postfix provides a wide range of SASL authentication configuration options. The +next section lists a few that are discussed frequently. See postconf(5) for a +complete list. + +PPeerr--aaccccoouunntt aacccceessss ccoonnttrrooll + +Postfix can implement policies that depend on the SASL login name (Postfix 2.11 +and later). Typically this is used to HOLD or REJECT mail from accounts whose +credentials have been compromised. + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + permit_mynetworks + check_sasl_access hash:/etc/postfix/sasl_access + permit_sasl_authenticated + ... + + /etc/postfix/sasl_access: + # Use this when smtpd_sasl_local_domain is empty. + username HOLD + # Use this when smtpd_sasl_local_domain=example.com. + username@example.com HOLD + +DDeeffaauulltt aauutthheennttiiccaattiioonn ddoommaaiinn + +Postfix can append a domain name (or any other string) to a SASL login name +that does not have a domain part, e.g. "john" instead of "john@example.com": + + /etc/postfix/main.cf: + smtpd_sasl_local_domain = example.com + +This is useful as a default setting and safety net for misconfigured clients, +or during a migration to an authentication method/backend that requires an +authentication REALM or domain name, before all SMTP clients are configured to +send such information. + +HHiiddiinngg SSAASSLL aauutthheennttiiccaattiioonn ffrroomm cclliieennttss oorr nneettwwoorrkkss + +Some clients insist on using SASL authentication if it is offered, even when +they are not configured to send credentials - and therefore they will always +fail and disconnect. + +Postfix can hide the AUTH capability from these clients/networks: + + /etc/postfix/main.cf: + smtpd_sasl_exceptions_networks = !192.0.2.171/32, 192.0.2.0/24 + +AAddddiinngg tthhee SSAASSLL llooggiinn nnaammee ttoo mmaaiill hheeaaddeerrss + +To report SASL login names in Received: message headers (Postfix version 2.3 +and later): + + /etc/postfix/main.cf: + smtpd_sasl_authenticated_header = yes + + NNoottee + + The SASL login names will be shared with the entire world. + +TTeessttiinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP SSeerrvveerr + +To test the server side, connect (for example, with telnet) to the Postfix SMTP +server port and you should be able to have a conversation as shown below. +Information sent by the client (that is, you) is shown in bboolldd font. + + % tteellnneett sseerrvveerr..eexxaammppllee..ccoomm 2255 + ... + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-ETRN + 250-AUTH DIGEST-MD5 PLAIN CRAM-MD5 + 250 8BITMIME + AAUUTTHH PPLLAAIINN AAHHRRllcc33QQAAddGGVVzzddHHBBhhcc33MM== + 235 Authentication successful + +To test this over a connection that is encrypted with TLS, use openssl s_client +instead of telnet: + + % ooppeennssssll ss__cclliieenntt --ccoonnnneecctt sseerrvveerr..eexxaammppllee..ccoomm::2255 --ssttaarrttttllss ssmmttpp + ... + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + ...see above example for more... + +Instead of AHRlc3QAdGVzdHBhc3M=, specify the base64-encoded form of +\0username\0password (the \0 is a null byte). The example above is for a user +named `test' with password `testpass'. + + CCaauuttiioonn + + When posting logs of the SASL negotiations to public lists, please keep in + mind that username/password information is trivial to recover from the + base64-encoded form. + +You can use one of the following commands to generate base64 encoded +authentication information: + + * Using a recent version of the bbaasshh shell: + + % eecchhoo --nnee ''\\000000uusseerrnnaammee\\000000ppaasssswwoorrdd'' || ooppeennssssll bbaassee6644 + + Some other shells support similar syntax. + + * Using the pprriinnttff command: + + % pprriinnttff ''\\00%%ss\\00%%ss'' ''uusseerrnnaammee'' ''ppaasssswwoorrdd'' || ooppeennssssll bbaassee6644 + % pprriinnttff ''\\00%%ss\\00%%ss'' ''uusseerrnnaammee'' ''ppaasssswwoorrdd'' || mmmmeennccooddee + + The mmmmeennccooddee command is part of the metamail software. + + * Using Perl MMIIMMEE::::BBaassee6644 (from http://www.cpan.org/): + + % ppeerrll --MMMMIIMMEE::::BBaassee6644 --ee \\ + ''pprriinntt eennccooddee__bbaassee6644((""\\00uusseerrnnaammee\\00ppaasssswwoorrdd""));;'' + + If the username or password contain "@", you must specify "\@". + + * Using the ggeenn--aauutthh script: + + % ggeenn--aauutthh ppllaaiinn + username: uusseerrnnaammee + password: + + The ggeenn--aauutthh Perl script was written by John Jetmore and can be found at + http://jetmore.org/john/code/gen-auth. + +CCoonnffiigguurriinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP//LLMMTTPP cclliieenntt + +The Postfix SMTP and the LMTP client can authenticate with a remote SMTP server +via the Cyrus SASL framework. At this time, the Dovecot SASL implementation +does not provide client functionality. + + NNoottee + + The examples in this section discuss only the SMTP client. Replace smtp_ + with lmtp_ to get the corresponding LMTP client configuration. + +You can read more about the following topics: + + * Enabling SASL authentication in the Postfix SMTP/LMTP client + * Configuring sender-dependent SASL authentication + * Postfix SMTP/LMTP client policy - SASL mechanism pprrooppeerrttiieess + * Postfix SMTP/LMTP client policy - SASL mechanism nnaammeess + +EEnnaabblliinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP//LLMMTTPP cclliieenntt + +This section shows a typical scenario where the Postfix SMTP client sends all +messages via a mail gateway server that requires SASL authentication. + + TTrroouubbllee ssoollvviinngg ttiippss:: + + * If your SASL logins fail with "SASL authentication failure: No worthy + mechs found" in the mail logfile, then see the section "Postfix SMTP/ + LMTP client policy - SASL mechanism pprrooppeerrttiieess". + + * For a solution to a more obscure class of SASL authentication failures, + see "Postfix SMTP/LMTP client policy - SASL mechanism nnaammeess". + +To make the example more readable we introduce it in two parts. The first part +takes care of the basic configuration, while the second part sets up the +username/password information. + + /etc/postfix/main.cf: + smtp_sasl_auth_enable = yes + smtp_tls_security_level = encrypt + smtp_sasl_tls_security_options = noanonymous + relayhost = [mail.isp.example] + # Alternative form: + # relayhost = [mail.isp.example]:submission + smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd + + * The smtp_sasl_auth_enable setting enables client-side authentication. We + will configure the client's username and password information in the second + part of the example. + + * The smtp_tls_security_level setting ensures that the connection to the + remote smtp server will be encrypted, and smtp_sasl_tls_security_options + removes the prohibition on plaintext passwords. + + * The relayhost setting forces the Postfix SMTP to send all remote messages + to the specified mail server instead of trying to deliver them directly to + their destination. + + * In the relayhost setting, the "[" and "]" prevent the Postfix SMTP client + from looking up MX (mail exchanger) records for the enclosed name. + + * The relayhost destination may also specify a non-default TCP port. For + example, the alternative form [mail.isp.example]:submission tells Postfix + to connect to TCP network port 587, which is reserved for email client + applications. + + * The Postfix SMTP client is compatible with SMTP servers that use the non- + standard "AUTH=mmeetthhoodd....." syntax in response to the EHLO command; this + requires no additional Postfix client configuration. + + * With the setting "smtp_tls_wrappermode = yes", the Postfix SMTP client + supports the "wrappermode" protocol, which uses TCP port 465 on the SMTP + server (Postfix 3.0 and later). + + * With the smtp_sasl_password_maps parameter, we configure the Postfix SMTP + client to send username and password information to the mail gateway + server. As discussed in the next section, the Postfix SMTP client supports + multiple ISP accounts. For this reason the username and password are stored + in a table that contains one username/password combination for each mail + gateway server. + + /etc/postfix/sasl_passwd: + # destination credentials + [mail.isp.example] username:password + # Alternative form: + # [mail.isp.example]:submission username:password + + IImmppoorrttaanntt + + Keep the SASL client password file in /etc/postfix, and make the file + read+write only for root to protect the username/password combinations + against other users. The Postfix SMTP client will still be able to read the + SASL client passwords. It opens the file as user root before it drops + privileges, and before entering an optional chroot jail. + + * Use the postmap command whenever you change the /etc/postfix/sasl_passwd + file. + + * If you specify the "[" and "]" in the relayhost destination, then you must + use the same form in the smtp_sasl_password_maps file. + + * If you specify a non-default TCP Port (such as ":submission" or ":587") in + the relayhost destination, then you must use the same form in the + smtp_sasl_password_maps file. + +CCoonnffiigguurriinngg SSeennddeerr--DDeeppeennddeenntt SSAASSLL aauutthheennttiiccaattiioonn + +Postfix supports different ISP accounts for different sender addresses (version +2.3 and later). This can be useful when one person uses the same machine for +work and for personal use, or when people with different ISP accounts share the +same Postfix server. + +To make this possible, Postfix supports per-sender SASL passwords and per- +sender relay hosts. In the example below, the Postfix SMTP client will search +the SASL password file by sender address before it searches that same file by +destination. Likewise, the Postfix trivial-rewrite(8) daemon will search the +per-sender relayhost file, and use the default relayhost setting only as a +final resort. + + /etc/postfix/main.cf: + smtp_sender_dependent_authentication = yes + sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay + smtp_sasl_auth_enable = yes + smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd + relayhost = [mail.isp.example] + # Alternative form: + # relayhost = [mail.isp.example]:submission + + /etc/postfix/sasl_passwd: + # Per-sender authentication; see also /etc/postfix/sender_relay. + user1@example.com username1:password1 + user2@example.net username2:password2 + # Login information for the default relayhost. + [mail.isp.example] username:password + # Alternative form: + # [mail.isp.example]:submission username:password + + /etc/postfix/sender_relay: + # Per-sender provider; see also /etc/postfix/sasl_passwd. + user1@example.com [mail.example.com]:submission + user2@example.net [mail.example.net] + + * If you are creative, then you can try to combine the two tables into one + single MySQL database, and configure different Postfix queries to extract + the appropriate information. + + * Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb + files. To find out what lookup tables Postfix supports, use the command + "ppoossttccoonnff --mm". + + * Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ssaassll__ppaasssswwdd" whenever you change + the sasl_passwd table. + + * Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//sseennddeerr__rreellaayy" whenever you change + the sender_relay table. + +PPoossttffiixx SSMMTTPP//LLMMTTPP cclliieenntt ppoolliiccyy -- SSAASSLL mmeecchhaanniissmm pprrooppeerrttiieess + +Just like the Postfix SMTP server, the SMTP client has a policy that determines +which SASL mechanisms are acceptable, based on their properties. The next two +sections give examples of how these policies are used. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |PPrrooppeerrttyy |DDeessccrriippttiioonn | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |noanonymous |Don't use mechanisms that permit anonymous authentication. | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |noplaintext |Don't use mechanisms that transmit unencrypted username and| + | |password information. | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |nodictionary|Don't use mechanisms that are vulnerable to dictionary | + | |attacks. | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |mutual_auth |Use only mechanisms that authenticate both the client and | + | |the server to each other. | + |_ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +UUnneennccrryypptteedd SSMMTTPP sseessssiioonn + +The default policy is stricter than that of the Postfix SMTP server - plaintext +mechanisms are not allowed (nor is any anonymous mechanism): + + /etc/postfix/main.cf: + smtp_sasl_security_options = noplaintext, noanonymous + +This default policy, which allows no plaintext passwords, leads to +authentication failures if the remote server only offers plaintext +authentication mechanisms (the SMTP server announces "AUTH PLAIN LOGIN"). In +such cases the SMTP client will log the following error message: + + SASL authentication failure: No worthy mechs found + + NNoottee + + This same error message will also be logged when the libplain.so or + liblogin.so modules are not installed in the /usr/lib/sasl2 directory. + +The insecure approach is to lower the security standards and permit plaintext +authentication mechanisms: + + /etc/postfix/main.cf: + smtp_sasl_security_options = noanonymous + +The more secure approach is to protect the plaintext username and password with +TLS session encryption. To find out if the remote SMTP server supports TLS, +connect to the server and see if it announces STARTTLS support as shown in the +example. Information sent by the client (that is, you) is shown in bboolldd font. + + % tteellnneett sseerrvveerr..eexxaammppllee..ccoomm 2255 + ... + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-STARTTLS + ... + +Instead of port 25 (smtp), specify port 587 (submission) where appropriate. + +EEnnccrryypptteedd SSMMTTPP sseessssiioonn ((TTLLSS)) + +To turn on TLS in the Postfix SMTP client, see TLS_README for configuration +details. + +The smtp_sasl_tls_security_options parameter controls Postfix SASL mechanism +policy during a TLS-encrypted SMTP session. The default is to copy the settings +from the unencrypted session: + + /etc/postfix/main.cf: + smtp_sasl_tls_security_options = $smtp_sasl_security_options + +A more sophisticated policy allows plaintext mechanisms, but only over a TLS- +encrypted connection: + + /etc/postfix/main.cf: + smtp_sasl_security_options = noanonymous, noplaintext + smtp_sasl_tls_security_options = noanonymous + +PPoossttffiixx SSMMTTPP//LLMMTTPP cclliieenntt ppoolliiccyy -- SSAASSLL mmeecchhaanniissmm nnaammeess + +Given the SASL security options of the previous section, the Cyrus SASL library +will choose the most secure authentication mechanism that both the SMTP client +and server implement. Unfortunately, that authentication mechanism may fail +because the client or server is not configured to use that mechanism. + +To prevent this, the Postfix SMTP client can filter the names of the +authentication mechanisms from the remote SMTP server. Used correctly, the +filter hides unwanted mechanisms from the Cyrus SASL library, forcing the +library to choose from the mechanisms the Postfix SMTP client filter passes +through. + +The following example filters out everything but the mechanisms PLAIN and +LOGIN: + + /etc/postfix/main.cf: + smtp_sasl_mechanism_filter = plain, login + + NNoottee + + If the remote server does not offer any of the mechanisms on the filter + list, authentication will fail. + +We close this section with an example that passes every mechanism except for +GSSAPI and LOGIN: + + /etc/postfix/main.cf: + smtp_sasl_mechanism_filter = !gssapi, !login, static:all + +BBuuiillddiinngg PPoossttffiixx wwiitthh SSAASSLL ssuuppppoorrtt + +As mentioned elsewhere, Postfix supports two SASL implementations: Cyrus SASL +(SMTP client and server) and Dovecot SASL (SMTP server only). Both +implementations can be built into Postfix simultaneously. + + * Building Dovecot SASL support + * Building Cyrus SASL support + +BBuuiillddiinngg DDoovveeccoott SSAASSLL ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +Support for the Dovecot version 1 SASL protocol is available in Postfix 2.3 and +later. At the time of writing, only server-side SASL support is available, so +you can't use it to authenticate the Postfix SMTP client to your network +provider's server. + +Dovecot uses its own daemon process for authentication. This keeps the Postfix +build process simple, because there is no need to link extra libraries into +Postfix. + +To generate the necessary Makefiles, execute the following in the Postfix top- +level directory: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH \\ + --DDDDEEFF__SSEERRVVEERR__SSAASSLL__TTYYPPEE==\\\\\\""ddoovveeccoott\\\\\\"""" + +After this, proceed with "make" as described in the INSTALL document. + +NNoottee + + * The -DDEF_SERVER_SASL_TYPE=\\\"dovecot\\\" is not necessary; it just makes + Postfix configuration a little more convenient because you don't have to + specify the SASL plug-in type in the Postfix main.cf file (but this may + cause surprises when you switch to a later Postfix version that is built + with the default SASL type of cyrus). + + * If you also want support for LDAP or TLS (or for Cyrus SASL), you need to + merge their CCARGS and AUXLIBS options into the above command line; see the + LDAP_README and TLS_README for details. + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH \\ + --DDDDEEFF__SSEERRVVEERR__SSAASSLL__TTYYPPEE==\\\\\\""ddoovveeccoott\\\\\\"" \\ + ......CCCCAARRGGSS ooppttiioonnss ffoorr LLDDAAPP oorr TTLLSS eettcc........"" \\ + AAUUXXLLIIBBSS==""......AAUUXXLLIIBBSS ooppttiioonnss ffoorr LLDDAAPP oorr TTLLSS eettcc........"" + +BBuuiillddiinngg CCyyrruuss SSAASSLL ssuuppppoorrtt + +BBuuiillddiinngg tthhee CCyyrruuss SSAASSLL lliibbrraarryy + +Postfix works with cyrus-sasl-1.5.x or cyrus-sasl-2.1.x, which are available +from https://github.com/cyrusimap/cyrus-sasl/releases. + + IImmppoorrttaanntt + + If you install the Cyrus SASL libraries as per the default, you will have + to create a symlink /usr/lib/sasl -> /usr/local/lib/sasl for version 1.5.x + or /usr/lib/sasl2 -> /usr/local/lib/sasl2 for version 2.1.x. + +Reportedly, Microsoft Outlook (Express) requires the non-standard LOGIN and/or +NTLM authentication mechanism. To enable these authentication mechanisms, build +the Cyrus SASL libraries with: + + % ..//ccoonnffiigguurree ----eennaabbllee--llooggiinn ----eennaabbllee--nnttllmm + +BBuuiillddiinngg PPoossttffiixx wwiitthh CCyyrruuss SSAASSLL ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +The following assumes that the Cyrus SASL include files are in /usr/local/ +include, and that the Cyrus SASL libraries are in /usr/local/lib. + +On some systems this generates the necessary Makefile definitions: + +Cyrus SASL version 2.1.x + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH --DDUUSSEE__CCYYRRUUSS__SSAASSLL \\ + --II//uussrr//llooccaall//iinncclluuddee//ssaassll"" AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb --llssaassll22"" + + If your Cyrus SASL shared library is in a directory that the RUN-TIME + linker does not know about, add a "-Wl,-R,/path/to/directory" option after + "-lsasl2". + +Cyrus SASL version 1.5.x + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH --DDUUSSEE__CCYYRRUUSS__SSAASSLL \\ + --II//uussrr//llooccaall//iinncclluuddee"" AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb --llssaassll"" + +On Solaris 2.x you need to specify run-time link information, otherwise the +ld.so run-time linker will not find the SASL shared library: + +Cyrus SASL version 2.1.x + + % mmaakkee ttiiddyy # remove left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH --DDUUSSEE__CCYYRRUUSS__SSAASSLL \\ + --II//uussrr//llooccaall//iinncclluuddee//ssaassll"" AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb \\ + --RR//uussrr//llooccaall//lliibb --llssaassll22"" + +Cyrus SASL version 1.5.x + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__SSAASSLL__AAUUTTHH --DDUUSSEE__CCYYRRUUSS__SSAASSLL \\ + --II//uussrr//llooccaall//iinncclluuddee"" AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb \\ + --RR//uussrr//llooccaall//lliibb --llssaassll"" + +UUssiinngg CCyyrruuss SSAASSLL vveerrssiioonn 11..55..xx + +Postfix supports Cyrus SASL version 1.x, but you shouldn't use it unless you +are forced to. The makers of Cyrus SASL write: + + This library is being deprecated and applications should transition to + using the SASLv2 library (source: Project Cyrus: Downloads). + +If you still need to set it up, here's a quick rundown: + +Read the regular section on SMTP server configurations for the Cyrus SASL +framework. The differences are: + + * Cyrus SASL version 1.5.x searches for configuration (smtpd.conf) in /usr/ + lib/sasl/ only. You must place the configuration in that directory. Some + systems may have modified Cyrus SASL and put the files into e.g. /var/lib/ + sasl/. + + * Use the saslpasswd command instead of saslpasswd2 to create users in + sasldb. + + * Use the sasldblistusers command instead of sasldblistusers2 to find users + in sasldb. + + * In the smtpd.conf file you can't use mech_list to limit the range of + mechanisms offered. Instead, remove their libraries from /usr/lib/sasl/ + (and remember remove those files again when a system update re-installs new + versions). + +CCrreeddiittss + + * Postfix SASL support was originally implemented by Till Franke of SuSE + Rhein/Main AG. + * Wietse trimmed down the code to only the bare necessities. + * Support for Cyrus SASL version 2 was contributed by Jason Hoos. + * Liviu Daia added smtpd_sasl_application_name, separated + reject_sender_login_mismatch into + reject_authenticated_sender_login_mismatch and + reject_unauthenticated_sender_login_mismatch, and revised the docs. + * Wietse made another iteration through the code to add plug-in support for + multiple SASL implementations, and for reasons that have been lost, also + changed smtpd_sasl_application_name into smtpd_sasl_path. + * The Dovecot SMTP server-only plug-in was originally implemented by Timo + Sirainen of Procontrol, Finland. + * Patrick Ben Koetter revised this document for Postfix 2.4 and made much + needed updates. + * Patrick Ben Koetter revised this document again for Postfix 2.7 and made + much needed updates. + diff --git a/README_FILES/SCHEDULER_README b/README_FILES/SCHEDULER_README new file mode 100644 index 0000000..448ee0a --- /dev/null +++ b/README_FILES/SCHEDULER_README @@ -0,0 +1,1161 @@ +PPoossttffiixx QQuueeuuee SScchheedduulleerr + +------------------------------------------------------------------------------- + +DDiissccllaaiimmeerr + +Many of the transport-specific configuration parameters discussed in this +document will not show up in "postconf" command output before Postfix version +2.9. This limitation applies to many parameters whose name is a combination of +a master.cf service name such as "relay" and a built-in suffix such as +"_destination_concurrency_limit". + +OOvveerrvviieeww + +The queue manager is by far the most complex part of the Postfix mail system. +It schedules delivery of new mail, retries failed deliveries at specific times, +and removes mail from the queue after the last delivery attempt. There are two +major classes of mechanisms that control the operation of the queue manager. + +Topics covered by this document: + + * Concurrency scheduling, concerned with the number of concurrent deliveries + to a specific destination, including decisions on when to suspend + deliveries after persistent failures. + * Preemptive scheduling, concerned with the selection of email messages and + recipients for a given destination. + * Credits, something this document would not be complete without. + +CCoonnccuurrrreennccyy sscchheedduulliinngg + +The following sections document the Postfix 2.5 concurrency scheduler, after a +discussion of the limitations of the earlier concurrency scheduler. This is +followed by results of medium-concurrency experiments, and a discussion of +trade-offs between performance and robustness. + +The material is organized as follows: + + * Drawbacks of the existing concurrency scheduler + * Summary of the Postfix 2.5 concurrency feedback algorithm + * Summary of the Postfix 2.5 "dead destination" detection algorithm + * Pseudocode for the Postfix 2.5 concurrency scheduler + * Results for delivery to concurrency limited servers + * Discussion of concurrency limited server results + * Limitations of less-than-1 per delivery feedback + * Concurrency configuration parameters + +DDrraawwbbaacckkss ooff tthhee eexxiissttiinngg ccoonnccuurrrreennccyy sscchheedduulleerr + +From the start, Postfix has used a simple but robust algorithm where the per- +destination delivery concurrency is decremented by 1 after delivery failed due +to connection or handshake failure, and incremented by 1 otherwise. Of course +the concurrency is never allowed to exceed the maximum per-destination +concurrency limit. And when a destination's concurrency level drops to zero, +the destination is declared "dead" and delivery is suspended. + +Drawbacks of +/-1 concurrency feedback per delivery are: + + * Overshoot due to exponential delivery concurrency growth with each pseudo- + cohort(*). This can be an issue with high-concurrency channels. For + example, with the default initial concurrency of 5, concurrency would + proceed over time as (5-10-20). + + * Throttling down to zero concurrency after a single pseudo-cohort(*) + failure. This was especially an issue with low-concurrency channels where a + single failure could be sufficient to mark a destination as "dead", causing + the suspension of further deliveries to the affected destination. + +(*) A pseudo-cohort is a number of delivery requests equal to a destination's +delivery concurrency. + +The revised concurrency scheduler has a highly modular structure. It uses +separate mechanisms for per-destination concurrency control and for "dead +destination" detection. The concurrency control in turn is built from two +separate mechanisms: it supports less-than-1 feedback per delivery to allow for +more gradual concurrency adjustments, and it uses feedback hysteresis to +suppress concurrency oscillations. And instead of waiting for delivery +concurrency to throttle down to zero, a destination is declared "dead" after a +configurable number of pseudo-cohorts reports connection or handshake failure. + +SSuummmmaarryy ooff tthhee PPoossttffiixx 22..55 ccoonnccuurrrreennccyy ffeeeeddbbaacckk aallggoorriitthhmm + +We want to increment a destination's delivery concurrency when some (not +necessarily consecutive) number of deliveries complete without connection or +handshake failure. This is implemented with positive feedback g(N) where N is +the destination's delivery concurrency. With g(N)=1 feedback per delivery, +concurrency increases by 1 after each positive feedback event; this gives us +the old scheduler's exponential growth in time. With g(N)=1/N feedback per +delivery, concurrency increases by 1 after an entire pseudo-cohort N of +positive feedback reports; this gives us linear growth in time. Less-than- +1 feedback per delivery and integer truncation naturally give us hysteresis, so +that transitions to larger concurrency happen every 1/g(N) positive feedback +events. + +We want to decrement a destination's delivery concurrency when some (not +necessarily consecutive) number of deliveries complete after connection or +handshake failure. This is implemented with negative feedback f(N) where N is +the destination's delivery concurrency. With f(N)=1 feedback per delivery, +concurrency decreases by 1 after each negative feedback event; this gives us +the old scheduler's behavior where concurrency is throttled down dramatically +after a single pseudo-cohort failure. With f(N)=1/N feedback per delivery, +concurrency backs off more gently. Again, less-than-1 feedback per delivery and +integer truncation naturally give us hysteresis, so that transitions to lower +concurrency happen every 1/f(N) negative feedback events. + +However, with negative feedback we introduce a subtle twist. We "reverse" the +negative hysteresis cycle so that the transition to lower concurrency happens +at the bbeeggiinnnniinngg of a sequence of 1/f(N) negative feedback events. Otherwise, a +correction for overload would be made too late. This makes the choice of f(N) +relatively unimportant, as borne out by measurements later in this document. + +In summary, the main ingredients for the Postfix 2.5 concurrency feedback +algorithm are a) the option of less-than-1 positive feedback per delivery to +avoid overwhelming servers, b) the option of less-than-1 negative feedback per +delivery to avoid giving up too fast, c) feedback hysteresis to avoid rapid +oscillation, and d) a "reverse" hysteresis cycle for negative feedback, so that +it can correct for overload quickly. + +SSuummmmaarryy ooff tthhee PPoossttffiixx 22..55 ""ddeeaadd ddeessttiinnaattiioonn"" ddeetteeccttiioonn aallggoorriitthhmm + +We want to suspend deliveries to a specific destination after some number of +deliveries suffers connection or handshake failure. The old scheduler declares +a destination "dead" when negative (-1) feedback throttles the delivery +concurrency down to zero. With less-than-1 feedback per delivery, this +throttling down would obviously take too long. We therefore have to separate +"dead destination" detection from concurrency feedback. This is implemented by +introducing the concept of pseudo-cohort failure. The Postfix 2.5 concurrency +scheduler declares a destination "dead" after a configurable number of pseudo- +cohorts suffers from connection or handshake failures. The old scheduler +corresponds to the special case where the pseudo-cohort failure limit is equal +to 1. + +PPsseeuuddooccooddee ffoorr tthhee PPoossttffiixx 22..55 ccoonnccuurrrreennccyy sscchheedduulleerr + +The pseudo code shows how the ideas behind new concurrency scheduler are +implemented as of November 2007. The actual code can be found in the module +qmgr/qmgr_queue.c. + +Types: + Each destination has one set of the following variables + int concurrency + double success + double failure + double fail_cohorts + +Feedback functions: + N is concurrency; x, y are arbitrary numbers in [0..1] inclusive + positive feedback: g(N) = x/N | x/sqrt(N) | x + negative feedback: f(N) = y/N | y/sqrt(N) | y + +Initialization: + concurrency = initial_concurrency + success = 0 + failure = 0 + fail_cohorts = 0 + +After success: + fail_cohorts = 0 + Be prepared for feedback > hysteresis, or rounding error + success += g(concurrency) + while (success >= 1) Hysteresis 1 + concurrency += 1 Hysteresis 1 + failure = 0 + success -= 1 Hysteresis 1 + Be prepared for overshoot + if (concurrency > concurrency limit) + concurrency = concurrency limit + +Safety: + Don't apply positive feedback unless + concurrency < busy_refcount + init_dest_concurrency + otherwise negative feedback effect could be delayed + +After failure: + if (concurrency > 0) + fail_cohorts += 1.0 / concurrency + if (fail_cohorts > cohort_failure_limit) + concurrency = 0 + if (concurrency > 0) + Be prepared for feedback > hysteresis, rounding errors + failure -= f(concurrency) + while (failure < 0) + concurrency -= 1 Hysteresis 1 + failure += 1 Hysteresis 1 + success = 0 + Be prepared for overshoot + if (concurrency < 1) + concurrency = 1 + +RReessuullttss ffoorr ddeelliivveerryy ttoo ccoonnccuurrrreennccyy--lliimmiitteedd sseerrvveerrss + +Discussions about the concurrency scheduler redesign started early 2004, when +the primary goal was to find alternatives that did not exhibit exponential +growth or rapid concurrency throttling. No code was implemented until late +2007, when the primary concern had shifted towards better handling of server +concurrency limits. For this reason we measure how well the new scheduler does +this job. The table below compares mail delivery performance of the old +/- +1 feedback per delivery with several less-than-1 feedback functions, for +different limited-concurrency server scenarios. Measurements were done with a +FreeBSD 6.2 client and with FreeBSD 6.2 and various Linux servers. + +Server configuration: + + * The mail flow was slowed down with 1 second latency per recipient + ("smtpd_client_restrictions = sleep 1"). The purpose was to make results + less dependent on hardware details, by avoiding slow-downs by queue file I/ + O, logging I/O, and network I/O. + * Concurrency was limited by the server process limit ("default_process_limit + = 5" and "smtpd_client_event_limit_exceptions = static:all"). Postfix was + stopped and started after changing the process limit, because the same + number is also used as the backlog argument to the listen(2) system call, + and "postfix reload" does not re-issue this call. + * Mail was discarded with "local_recipient_maps = static:all" and + "local_transport = discard". The discard action in access maps or header/ + body checks could not be used as it fails to update the in_flow_delay + counters. + +Client configuration: + + * Queue file overhead was minimized by sending one message to a virtual alias + that expanded into 2000 different remote recipients. All recipients were + accounted for according to the maillog file. The + virtual_alias_expansion_limit setting was increased to avoid complaints + from the cleanup(8) server. + * The number of deliveries was maximized with + "smtp_destination_recipient_limit = 2". A smaller limit would cause Postfix + to schedule the concurrency per recipient instead of domain, which is not + what we want. + * Maximum concurrency was limited with "smtp_destination_concurrency_limit = + 20", and initial_destination_concurrency was set to the same value. + * The positive and negative concurrency feedback hysteresis was 1. + Concurrency was incremented by 1 at the END of 1/feedback steps of positive + feedback, and was decremented by 1 at the START of 1/feedback steps of + negative feedback. + * The SMTP client used the default 30s SMTP connect timeout and 300s SMTP + greeting timeout. + +IImmppaacctt ooff tthhee 3300ss SSMMTTPP ccoonnnneecctt ttiimmeeoouutt + +The first results are for a FreeBSD 6.2 server, where our artificially low +listen(2) backlog results in a very short kernel queue for established +connections. The table shows that all deferred deliveries failed due to a 30s +connection timeout, and none failed due to a server greeting timeout. This +measurement simulates what happens when the server's connection queue is +completely full under load, and the TCP engine drops new connections. + + cclliieenntt sseerrvveerr ffeeeeddbbaacckk ccoonnnneeccttiioonn ppeerrcceennttaaggee cclliieenntt ttiimmeedd--oouutt iinn + lliimmiitt lliimmiitt ssttyyllee ccaacchhiinngg ddeeffeerrrreedd ccoonnccuurrrreennccyy ccoonnnneecctt// + aavveerraaggee//ssttddddeevv ggrreeeettiinngg + + ------------------------------------------------------------------------- + 20 5 1/N no 9.9 19.4 0.49 198 - + + 20 5 1/N yes 10.3 19.4 0.49 206 - + + 20 5 1/sqrt(N) no 10.4 19.6 0.59 208 - + + 20 5 1/sqrt(N) yes 10.6 19.6 0.61 212 - + + 20 5 1 no 10.1 19.5 1.29 202 - + + 20 5 1 yes 10.8 19.3 1.57 216 - + + ------------------------------------------------------------------------- + + A busy server with a completely full connection queue. N is the client + delivery concurrency. Failed deliveries time out after 30s without + completing the TCP handshake. See text for a discussion of results. + +IImmppaacctt ooff tthhee 330000ss SSMMTTPP ggrreeeettiinngg ttiimmeeoouutt + +The next table shows results for a Fedora Core 8 server (results for RedHat 7.3 +are identical). In this case, the artificially small listen(2) backlog argument +does not impact our measurement. The table shows that practically all deferred +deliveries fail after the 300s SMTP greeting timeout. As these timeouts were +10x longer than with the first measurement, we increased the recipient count +(and thus the running time) by a factor of 10 to keep the results comparable. +The deferred mail percentages are a factor 10 lower than with the first +measurement, because the 1s per-recipient delay was 1/300th of the greeting +timeout instead of 1/30th of the connection timeout. + + cclliieenntt sseerrvveerr ffeeeeddbbaacckk ccoonnnneeccttiioonn ppeerrcceennttaaggee cclliieenntt ttiimmeedd--oouutt iinn + lliimmiitt lliimmiitt ssttyyllee ccaacchhiinngg ddeeffeerrrreedd ccoonnccuurrrreennccyy ccoonnnneecctt// + aavveerraaggee//ssttddddeevv ggrreeeettiinngg + + ------------------------------------------------------------------------- + 20 5 1/N no 1.16 19.8 0.37 - 230 + + 20 5 1/N yes 1.36 19.8 0.36 - 272 + + 20 5 1/sqrt(N) no 1.21 19.9 0.23 4 238 + + 20 5 1/sqrt(N) yes 1.36 20.0 0.23 - 272 + + 20 5 1 no 1.18 20.0 0.16 - 236 + + 20 5 1 yes 1.39 20.0 0.16 - 278 + + ------------------------------------------------------------------------- + + A busy server with a non-full connection queue. N is the client delivery + concurrency. Failed deliveries complete at the TCP level, but time out + after 300s while waiting for the SMTP greeting. See text for a discussion + of results. + +IImmppaacctt ooff aaccttiivvee sseerrvveerr ccoonnccuurrrreennccyy lliimmiitteerr + +The final concurrency-limited result shows what happens when SMTP connections +don't time out, but are rejected immediately with the Postfix server's +smtpd_client_connection_count_limit feature (the server replies with a 421 +status and disconnects immediately). Similar results can be expected with +concurrency limiting features built into other MTAs or firewalls. For this +measurement we specified a server concurrency limit and a client initial +destination concurrency of 5, and a server process limit of 10; all other +conditions were the same as with the first measurement. The same result would +be obtained with a FreeBSD or Linux server, because the "pushing back" is done +entirely by the receiving side. + + cclliieenntt sseerrvveerr ffeeeeddbbaacckk ccoonnnneeccttiioonn ppeerrcceennttaaggee cclliieenntt tthheeoorreettiiccaall + lliimmiitt lliimmiitt ssttyyllee ccaacchhiinngg ddeeffeerrrreedd ccoonnccuurrrreennccyy ddeeffeerr rraattee + aavveerraaggee//ssttddddeevv + + ------------------------------------------------------------------------- + 20 5 1/N no 16.5 5.17 0.38 1/6 + + 20 5 1/N yes 16.5 5.17 0.38 1/6 + + 20 5 1/sqrt(N) no 24.5 5.28 0.45 1/4 + + 20 5 1/sqrt(N) yes 24.3 5.28 0.46 1/4 + + 20 5 1 no 49.7 5.63 0.67 1/2 + + 20 5 1 yes 49.7 5.68 0.70 1/2 + + ------------------------------------------------------------------------- + + A server with active per-client concurrency limiter that replies with 421 + and disconnects. N is the client delivery concurrency. The theoretical + defer rate is 1/(1+roundup(1/feedback)). This is always 1/2 with the fixed + +/-1 feedback per delivery; with the concurrency-dependent feedback + variants, the defer rate decreases with increasing concurrency. See text + for a discussion of results. + +DDiissccuussssiioonn ooff ccoonnccuurrrreennccyy--lliimmiitteedd sseerrvveerr rreessuullttss + +All results in the previous sections are based on the first delivery runs only; +they do not include any second etc. delivery attempts. It's also worth noting +that the measurements look at steady-state behavior only. They don't show what +happens when the client starts sending at a much higher or lower concurrency. + +The first two examples show that the effect of feedback is negligible when +concurrency is limited due to congestion. This is because the initial +concurrency is already at the client's concurrency maximum, and because there +is 10-100 times more positive than negative feedback. Under these conditions, +it is no surprise that the contribution from SMTP connection caching is also +negligible. + +In the last example, the old +/-1 feedback per delivery will defer 50% of the +mail when confronted with an active (anvil-style) server concurrency limit, +where the server hangs up immediately with a 421 status (a TCP-level RST would +have the same result). Less aggressive feedback mechanisms fare better than +more aggressive ones. Concurrency-dependent feedback fares even better at +higher concurrencies than shown here, but has limitations as discussed in the +next section. + +LLiimmiittaattiioonnss ooff lleessss--tthhaann--11 ppeerr ddeelliivveerryy ffeeeeddbbaacckk + +Less-than-1 feedback is of interest primarily when sending large amounts of +mail to destinations with active concurrency limiters (servers that reply with +421, or firewalls that send RST). When sending small amounts of mail per +destination, less-than-1 per-delivery feedback won't have a noticeable effect +on the per-destination concurrency, because the number of deliveries to the +same destination is too small. You might just as well use zero per-delivery +feedback and stay with the initial per-destination concurrency. And when mail +deliveries fail due to congestion instead of active concurrency limiters, the +measurements above show that per-delivery feedback has no effect. With large +amounts of mail you might just as well use zero per-delivery feedback and start +with the maximal per-destination concurrency. + +The scheduler with less-than-1 concurrency feedback per delivery solves a +problem with servers that have active concurrency limiters. This works only +because feedback is handled in a peculiar manner: positive feedback will +increment the concurrency by 1 at the eenndd of a sequence of events of length 1/ +feedback, while negative feedback will decrement concurrency by 1 at the +bbeeggiinnnniinngg of such a sequence. This is how Postfix adjusts quickly for overshoot +without causing lots of mail to be deferred. Without this difference in +feedback treatment, less-than-1 feedback per delivery would defer 50% of the +mail, and would be no better in this respect than the old +/-1 feedback per +delivery. + +Unfortunately, the same feature that corrects quickly for concurrency overshoot +also makes the scheduler more sensitive for noisy negative feedback. The reason +is that one lonely negative feedback event has the same effect as a complete +sequence of length 1/feedback: in both cases delivery concurrency is dropped by +1 immediately. As a worst-case scenario, consider multiple servers behind a +load balancer on a single IP address, and no backup MX address. When 1 out of K +servers fails to complete the SMTP handshake or drops the connection, a +scheduler with 1/N (N = concurrency) feedback stops increasing its concurrency +once it reaches a concurrency level of about K, even though the good servers +behind the load balancer are perfectly capable of handling more traffic. + +This noise problem gets worse as the amount of positive feedback per delivery +gets smaller. A compromise is to use fixed less-than-1 positive feedback values +instead of concurrency-dependent positive feedback. For example, to tolerate 1 +of 4 bad servers in the above load balancer scenario, use positive feedback of +1/4 per "good" delivery (no connect or handshake error), and use an equal or +smaller amount of negative feedback per "bad" delivery. The downside of using +concurrency-independent feedback is that some of the old +/-1 feedback problems +will return at large concurrencies. Sites that must deliver mail at non-trivial +per-destination concurrencies will require special configuration. + +CCoonnccuurrrreennccyy ccoonnffiigguurraattiioonn ppaarraammeetteerrss + +The Postfix 2.5 concurrency scheduler is controlled with the following +configuration parameters, where "transport_foo" provides a transport-specific +parameter override. All parameter default settings are compatible with earlier +Postfix versions. + + PPaarraammeetteerr nnaammee PPoossttffiixx DDeessccrriippttiioonn + vveerrssiioonn + + --------------------------------------------------------------------------- + Initial per- + initial_destination_concurrency all destination + transport_initial_destination_concurrency 2.5 delivery + concurrency + + Maximum per- + default_destination_concurrency_limit all destination + transport_destination_concurrency_limit all delivery + concurrency + + Per- + destination + positive + feedback + default_destination_concurrency_positive_feedback 2.5 amount, per + transport_destination_concurrency_positive_feedback 2.5 delivery that + does not fail + with + connection or + handshake + failure + + Per- + destination + negative + feedback + default_destination_concurrency_negative_feedback 2.5 amount, per + transport_destination_concurrency_negative_feedback 2.5 delivery that + fails with + connection or + handshake + failure + + Number of + failed + pseudo- + cohorts after + default_destination_concurrency_failed_cohort_limit 2.5 which a + transport_destination_concurrency_failed_cohort_limit 2.5 destination + is declared + "dead" and + delivery is + suspended + + Enable + verbose + destination_concurrency_feedback_debug 2.5 logging of + concurrency + scheduler + activity + + --------------------------------------------------------------------------- + +PPrreeeemmppttiivvee sscchheedduulliinngg + +The following sections describe the new queue manager and its preemptive +scheduler algorithm. Note that the document was originally written to describe +the changes between the new queue manager (in this text referred to as nqmgr, +the name it was known by before it became the default queue manager) and the +old queue manager (referred to as oqmgr). This is why it refers to oqmgr every +so often. + +This document is divided into sections as follows: + + * The structures used by nqmgr + * What happens when nqmgr picks up the message - how it is assigned to + transports, jobs, peers, entries + * How the entry selection works + * How the preemption works - what messages may be preempted and how and what + messages are chosen to preempt them + * How destination concurrency limits affect the scheduling algorithm + * Dealing with memory resource limits + +TThhee ssttrruuccttuurreess uusseedd bbyy nnqqmmggrr + +Let's start by recapitulating the structures and terms used when referring to +the queue manager and how it operates. Many of these are partially described +elsewhere, but it is nice to have a coherent overview in one place: + + * Each message structure represents one mail message which Postfix is to + deliver. The message recipients specify to what destinations is the message + to be delivered and what transports are going to be used for the delivery. + + * Each recipient entry groups a batch of recipients of one message which are + all going to be delivered to the same destination (and over the same + transport). + + * Each transport structure groups everything what is going to be delivered by + delivery agents dedicated for that transport. Each transport maintains a + set of queues (describing the destinations it shall talk to) and jobs + (referencing the messages it shall deliver). + + * Each transport queue (not to be confused with the on-disk "active" queue or + "incoming" queue) groups everything what is going be delivered to given + destination (aka nexthop) by its transport. Each queue belongs to one + transport, so each destination may be referred to by several queues, one + for each transport. Each queue maintains a list of all recipient entries + (batches of message recipients) which shall be delivered to given + destination (the todo list), and a list of recipient entries already being + delivered by the delivery agents (the busy list). + + * Each queue corresponds to multiple peer structures. Each peer structure is + like the queue structure, belonging to one transport and referencing one + destination. The difference is that it lists only the recipient entries + which all originate from the same message, unlike the queue structure, + whose entries may originate from various messages. For messages with few + recipients, there is usually just one recipient entry for each destination, + resulting in one recipient entry per peer. But for large mailing list + messages the recipients may need to be split to multiple recipient entries, + in which case the peer structure may list many entries for single + destination. + + * Each transport job groups everything it takes to deliver one message via + its transport. Each job represents one message within the context of the + transport. The job belongs to one transport and message, so each message + may have multiple jobs, one for each transport. The job groups all the peer + structures, which describe the destinations the job's message has to be + delivered to. + +The first four structures are common to both nqmgr and oqmgr, the latter two +were introduced by nqmgr. + +These terms are used extensively in the text below, feel free to look up the +description above anytime you'll feel you have lost a sense what is what. + +WWhhaatt hhaappppeennss wwhheenn nnqqmmggrr ppiicckkss uupp tthhee mmeessssaaggee + +Whenever nqmgr moves a queue file into the "active" queue, the following +happens: It reads all necessary information from the queue file as oqmgr does, +and also reads as many recipients as possible - more on that later, for now +let's just pretend it always reads all recipients. + +Then it resolves the recipients as oqmgr does, which means obtaining (address, +nexthop, transport) triple for each recipient. For each triple, it finds the +transport; if it does not exist yet, it instantiates it (unless it's dead). +Within the transport, it finds the destination queue for the given nexthop; if +it does not exist yet, it instantiates it (unless it's dead). The triple is +then bound to given destination queue. This happens in qmgr_resolve() and is +basically the same as in oqmgr. + +Then for each triple which was bound to some queue (and thus transport), the +program finds the job which represents the message within that transport's +context; if it does not exist yet, it instantiates it. Within the job, it finds +the peer which represents the bound destination queue within this jobs context; +if it does not exist yet, it instantiates it. Finally, it stores the address +from the resolved triple to the recipient entry which is appended to both the +queue entry list and the peer entry list. The addresses for the same nexthop +are batched in the entries up to the transport_destination_recipient_limit for +that transport. This happens in qmgr_message_assign(), and apart from that it +operates with job and peer structures, it is basically the same as in oqmgr. + +When the job is instantiated, it is enqueued on the transport's job list based +on the time its message was picked up by nqmgr. For first batch of recipients +this means it is appended to the end of the job list, but the ordering of the +job list by the enqueue time is important as we will see shortly. + +[Now you should have a pretty good idea what the state of the nqmgr is after a +couple of messages were picked up, and what the relation is between all those +job, peer, queue and entry structures.] + +HHooww tthhee eennttrryy sseelleeccttiioonn wwoorrkkss + +Having prepared all those above mentioned structures, the task of the nqmgr's +scheduler is to choose the recipient entries one at a time and pass them to the +delivery agent for corresponding transport. Now how does this work? + +The first approximation of the new scheduling algorithm is like this: + + foreach transport (round-robin-by-transport) + do + if transport busy continue + if transport process limit reached continue + foreach transport's job (in the order of the transport's job list) + do + foreach job's peer (round-robin-by-destination) + if peer->queue->concurrency < peer->queue->window + return next peer entry. + done + done + done + +Now what is the "order of the transport's job list"? As we know already, the +job list is by default kept in the order the message was picked up by the +nqmgr. So by default we get the top-level round-robin transport, and within +each transport we get the FIFO message delivery. The round-robin of the peers +by the destination is perhaps of little importance in most real-life cases +(unless the transport_destination_recipient_limit is reached, in one job there +is only one peer structure for each destination), but theoretically it makes +sure that even within single jobs, destinations are treated fairly. + +[By now you should have a feeling you really know how the scheduler works, +except for the preemption, under ideal conditions - that is, no recipient +resource limits and no destination concurrency problems.] + +HHooww tthhee pprreeeemmppttiioonn wwoorrkkss + +As you might perhaps expect by now, the transport's job list does not remain +sorted by the job's message enqueue time all the time. The most cool thing +about nqmgr is not the simple FIFO delivery, but that it is able to slip mail +with little recipients past the mailing-list bulk mail. This is what the job +preemption is about - shuffling the jobs on the transport's job list to get the +best message delivery rates. Now how is it achieved? + +First I have to tell you that there are in fact two job lists in each +transport. One is the scheduler's job list, which the scheduler is free to play +with, while the other one keeps the jobs always listed in the order of the +enqueue time and is used for recipient pool management we will discuss later. +For now, we will deal with the scheduler's job list only. + +So, we have the job list, which is first ordered by the time the jobs' messages +were enqueued, oldest messages first, the most recently picked one at the end. +For now, let's assume that there are no destination concurrency problems. +Without preemption, we pick some entry of the first (oldest) job on the queue, +assign it to delivery agent, pick another one from the same job, assign it +again, and so on, until all the entries are used and the job is delivered. We +would then move onto the next job and so on and on. Now how do we manage to +sneak in some entries from the recently added jobs when the first job on the +job list belongs to a message going to the mailing-list and has thousands of +recipient entries? + +The nqmgr's answer is that we can artificially "inflate" the delivery time of +that first job by some constant for free - it is basically the same trick you +might remember as "accumulation of potential" from the amortized complexity +lessons. For example, instead of delivering the entries of the first job on the +job list every time a delivery agent becomes available, we can do it only every +second time. If you view the moments the delivery agent becomes available on a +timeline as "delivery slots", then instead of using every delivery slot for the +first job, we can use only every other slot, and still the overall delivery +efficiency of the first job remains the same. So the delivery 11112222 becomes +1.1.1.1.2.2.2.2 (1 and 2 are the imaginary job numbers, . denotes the free +slot). Now what do we do with free slots? + +As you might have guessed, we will use them for sneaking the mail with little +recipients in. For example, if we have one four-recipient mail followed by four +one recipients mail, the delivery sequence (that is, the sequence in which the +jobs are assigned to the delivery slots) might look like this: 12131415. Hmm, +fine for sneaking in the single recipient mail, but how do we sneak in the mail +with more than one recipient? Say if we have one four-recipient mail followed +by two two-recipient mails? + +The simple answer would be to use delivery sequence 12121313. But the problem +is that this does not scale well. Imagine you have mail with a thousand +recipients followed by mail with a hundred recipients. It is tempting to +suggest the delivery sequence like 121212...., but alas! Imagine there arrives +another mail with say ten recipients. But there are no free slots anymore, so +it can't slip by, not even if it had only one recipient. It will be stuck until +the hundred-recipient mail is delivered, which really sucks. + +So, it becomes obvious that while inflating the message to get free slots is a +great idea, one has to be really careful of how the free slots are assigned, +otherwise one might corner himself. So, how does nqmgr really use the free +slots? + +The key idea is that one does not have to generate the free slots in a uniform +way. The delivery sequence 111...1 is no worse than 1.1.1.1, in fact, it is +even better as some entries are in the first case selected earlier than in the +second case, and none is selected later! So it is possible to first +"accumulate" the free delivery slots and then use them all at once. It is even +possible to accumulate some, then use them, then accumulate some more and use +them again, as in 11..1.1 . + +Let's get back to the one hundred recipient example. We now know that we could +first accumulate one hundred free slots, and only after then to preempt the +first job and sneak the one hundred recipient mail in. Applying the algorithm +recursively, we see the hundred recipient job can accumulate ten free delivery +slots, and then we could preempt it and sneak in the ten-recipient mail... Wait +wait wait! Could we? Aren't we overinflating the original one thousand +recipient mail? + +Well, despite the fact that it looks so at the first glance, another trick will +allow us to answer "no, we are not!". If we had said that we will inflate the +delivery time twice at maximum, and then we consider every other slot as a free +slot, then we would overinflate in case of the recursive preemption. BUT! The +trick is that if we use only every n-th slot as a free slot for n>2, there is +always some worst inflation factor which we can guarantee not to be breached, +even if we apply the algorithm recursively. To be precise, if for every k>1 +normally used slots we accumulate one free delivery slot, than the inflation +factor is not worse than k/(k-1) no matter how many recursive preemptions +happen. And it's not worse than (k+1)/k if only non-recursive preemption +happens. Now, having got through the theory and the related math, let's see how +nqmgr implements this. + +Each job has so called "available delivery slot" counter. Each transport has a +transport_delivery_slot_cost parameter, which defaults to +default_delivery_slot_cost parameter which is set to 5 by default. This is the +k from the paragraph above. Each time k entries of the job are selected for +delivery, this counter is incremented by one. Once there are some slots +accumulated, a job which requires no more than that number of slots to be fully +delivered can preempt this job. + +[Well, the truth is, the counter is incremented every time an entry is selected +and it is divided by k when it is used. But to understand, it's good enough to +use the above approximation of the truth.] + +OK, so now we know the conditions which must be satisfied so one job can +preempt another one. But what job gets preempted, how do we choose what job +preempts it if there are several valid candidates, and when does all this +exactly happen? + +The answer for the first part is simple. The job whose entry was selected the +last time is the so called current job. Normally, it is the first job on the +scheduler's job list, but destination concurrency limits may change this as we +will see later. It is always only the current job which may get preempted. + +Now for the second part. The current job has a certain amount of recipient +entries, and as such may accumulate at maximum some amount of available +delivery slots. It might have already accumulated some, and perhaps even +already used some when it was preempted before (remember a job can be preempted +several times). In either case, we know how many are accumulated and how many +are left to deliver, so we know how many it may yet accumulate at maximum. +Every other job which may be delivered by less than that number of slots is a +valid candidate for preemption. How do we choose among them? + +The answer is - the one with maximum enqueue_time/recipient_entry_count. That +is, the older the job is, the more we should try to deliver it in order to get +best message delivery rates. These rates are of course subject to how many +recipients the message has, therefore the division by the recipient (entry) +count. No one shall be surprised that a message with n recipients takes n times +longer to deliver than a message with one recipient. + +Now let's recap the previous two paragraphs. Isn't it too complicated? Why +don't the candidates come only among the jobs which can be delivered within the +number of slots the current job already accumulated? Why do we need to estimate +how much it has yet to accumulate? If you found out the answer, congratulate +yourself. If we did it this simple way, we would always choose the candidate +with the fewest recipient entries. If there were enough single recipient mails +coming in, they would always slip by the bulk mail as soon as possible, and the +two or more recipients mail would never get a chance, no matter how long they +have been sitting around in the job list. + +This candidate selection has an interesting implication - that when we choose +the best candidate for preemption (this is done in qmgr_choose_candidate()), it +may happen that we may not use it for preemption immediately. This leads to an +answer to the last part of the original question - when does the preemption +happen? + +The preemption attempt happens every time next transport's recipient entry is +to be chosen for delivery. To avoid needless overhead, the preemption is not +attempted if the current job could never accumulate more than +transport_minimum_delivery_slots (defaults to default_minimum_delivery_slots +which defaults to 3). If there are already enough accumulated slots to preempt +the current job by the chosen best candidate, it is done immediately. This +basically means that the candidate is moved in front of the current job on the +scheduler's job list and decreasing the accumulated slot counter by the amount +used by the candidate. If there are not enough slots... well, I could say that +nothing happens and the another preemption is attempted the next time. But +that's not the complete truth. + +The truth is that it turns out that it is not really necessary to wait until +the jobs counter accumulates all the delivery slots in advance. Say we have +ten-recipient mail followed by two two-recipient mails. If the preemption +happened when enough delivery slots accumulate (assuming slot cost 2), the +delivery sequence becomes 11112211113311. Now what would we get if we would +wait only for 50% of the necessary slots to accumulate and we promise we would +wait for the remaining 50% later, after we get back to the preempted job? If we +use such a slot loan, the delivery sequence becomes 11221111331111. As we can +see, it makes it not considerably worse for the delivery of the ten-recipient +mail, but it allows the small messages to be delivered sooner. + +The concept of these slot loans is where the transport_delivery_slot_discount +and transport_delivery_slot_loan come from (they default to +default_delivery_slot_discount and default_delivery_slot_loan, whose values are +by default 50 and 3, respectively). The discount (resp. loan) specifies how +many percent (resp. how many slots) one "gets in advance", when the number of +slots required to deliver the best candidate is compared with the number of +slots the current slot had accumulated so far. + +And that pretty much concludes this chapter. + +[Now you should have a feeling that you pretty much understand the scheduler +and the preemption, or at least that you will have after you read the last +chapter a couple more times. You shall clearly see the job list and the +preemption happening at its head, in ideal delivery conditions. The feeling of +understanding shall last until you start wondering what happens if some of the +jobs are blocked, which you might eventually figure out correctly from what had +been said already. But I would be surprised if your mental image of the +scheduler's functionality is not completely shattered once you start wondering +how it works when not all recipients may be read in-core. More on that later.] + +HHooww ddeessttiinnaattiioonn ccoonnccuurrrreennccyy lliimmiittss aaffffeecctt tthhee sscchheedduulliinngg aallggoorriitthhmm + +The nqmgr uses the same algorithm for destination concurrency control as oqmgr. +Now what happens when the destination limits are reached and no more entries +for that destination may be selected by the scheduler? + +From the user's point of view it is all simple. If some of the peers of a job +can't be selected, those peers are simply skipped by the entry selection +algorithm (the pseudo-code described before) and only the selectable ones are +used. If none of the peers may be selected, the job is declared a "blocker +job". Blocker jobs are skipped by the entry selection algorithm and they are +also excluded from the candidates for preemption of the current job. Thus the +scheduler effectively behaves as if the blocker jobs didn't exist on the job +list at all. As soon as at least one of the peers of a blocker job becomes +unblocked (that is, the delivery agent handling the delivery of the recipient +entry for the given destination successfully finishes), the job's blocker +status is removed and the job again participates in all further scheduler +actions normally. + +So the summary is that the users don't really have to be concerned about the +interaction of the destination limits and scheduling algorithm. It works well +on its own and there are no knobs they would need to control it. + +From a programmer's point of view, the blocker jobs complicate the scheduler +quite a lot. Without them, the jobs on the job list would be normally delivered +in strict FIFO order. If the current job is preempted, the job preempting it is +completely delivered unless it is preempted itself. Without blockers, the +current job is thus always either the first job on the job list, or the top of +the stack of jobs preempting the first job on the job list. + +The visualization of the job list and the preemption stack without blockers +would be like this: + + first job-> 1--2--3--5--6--8--... <- job list + on job list | + 4 <- preemption stack + | + current job-> 7 + +In the example above we see that job 1 was preempted by job 4 and then job 4 +was preempted by job 7. After job 7 is completed, remaining entries of job 4 +are selected, and once they are all selected, job 1 continues. + +As we see, it's all very clean and straightforward. Now how does this change +because of blockers? + +The answer is: a lot. Any job may become a blocker job at any time, and also +become a normal job again at any time. This has several important implications: + + 1. The jobs may be completed in arbitrary order. For example, in the example + above, if the current job 7 becomes blocked, the next job 4 may complete + before the job 7 becomes unblocked again. Or if both 7 and 4 are blocked, + then 1 is completed, then 7 becomes unblocked and is completed, then 2 is + completed and only after that 4 becomes unblocked and is completed... You + get the idea. + + [Interesting side note: even when jobs are delivered out of order, from a + single destination's point of view the jobs are still delivered in the + expected order (that is, FIFO unless there was some preemption involved). + This is because whenever a destination queue becomes unblocked (the + destination limit allows selection of more recipient entries for that + destination), all jobs which have peers for that destination are unblocked + at once.] + + 2. The idea of the preemption stack at the head of the job list is gone. That + is, it must be possible to preempt any job on the job list. For example, if + the jobs 7, 4, 1 and 2 in the example above become all blocked, job 3 + becomes the current job. And of course we do not want the preemption to be + affected by the fact that there are some blocked jobs or not. Therefore, if + it turns out that job 3 might be preempted by job 6, the implementation + shall make it possible. + + 3. The idea of the linear preemption stack itself is gone. It's no longer true + that one job is always preempted by only one job at one time (that is + directly preempted, not counting the recursively nested jobs). For example, + in the example above, job 1 is directly preempted by only job 4, and job 4 + by job 7. Now assume job 7 becomes blocked, and job 4 is being delivered. + If it accumulates enough delivery slots, it is natural that it might be + preempted for example by job 8. Now job 4 is preempted by both job 7 AND + job 8 at the same time. + +Now combine the points 2) and 3) with point 1) again and you realize that the +relations on the once linear job list became pretty complicated. If we extend +the point 3) example: jobs 7 and 8 preempt job 4, now job 8 becomes blocked +too, then job 4 completes. Tricky, huh? + +If I illustrate the relations after the above mentioned examples (but those in +point 1), the situation would look like this: + + v- parent + + adoptive parent -> 1--2--3--5--... <- "stack" level 0 + | | + parent gone -> ? 6 <- "stack" level 1 + / \ + children -> 7 8 ^- child <- "stack" level 2 + + ^- siblings + +Now how does nqmgr deal with all these complicated relations? + +Well, it maintains them all as described, but fortunately, all these relations +are necessary only for the purpose of proper counting of available delivery +slots. For the purpose of ordering the jobs for entry selection, the original +rule still applies: "the job preempting the current job is moved in front of +the current job on the job list". So for entry selection purposes, the job +relations remain as simple as this: + + 7--8--1--2--6--3--5--.. <- scheduler's job list order + +The job list order and the preemption parent/child/siblings relations are +maintained separately. And because the selection works only with the job list, +you can happily forget about those complicated relations unless you want to +study the nqmgr sources. In that case the text above might provide some helpful +introduction to the problem domain. Otherwise I suggest you just forget about +all this and stick with the user's point of view: the blocker jobs are simply +ignored. + +[By now, you should have a feeling that there are more things going on under +the hood than you ever wanted to know. You decide that forgetting about this +chapter is the best you can do for the sake of your mind's health and you +basically stick with the idea how the scheduler works in ideal conditions, when +there are no blockers, which is good enough.] + +DDeeaalliinngg wwiitthh mmeemmoorryy rreessoouurrccee lliimmiittss + +When discussing the nqmgr scheduler, we have so far assumed that all recipients +of all messages in the "active" queue are completely read into memory. This is +simply not true. There is an upper bound on the amount of memory the nqmgr may +use, and therefore it must impose some limits on the information it may store +in memory at any given time. + +First of all, not all messages may be read in-core at once. At any time, only +qmgr_message_active_limit messages may be read in-core at maximum. When read +into memory, the messages are picked from the "incoming" and "deferred" queues +and moved to the "active" queue (incoming having priority), so if there are +more than qmgr_message_active_limit messages queued in the "active" queue, the +rest will have to wait until (some of) the messages in the "active" queue are +completely delivered (or deferred). + +Even with the limited amount of in-core messages, there is another limit which +must be imposed in order to avoid memory exhaustion. Each message may contain a +huge number of recipients (tens or hundreds of thousands are not uncommon), so +if nqmgr read all recipients of all messages in the "active" queue, it may +easily run out of memory. Therefore there must be some upper bound on the +amount of message recipients which are read into memory at the same time. + +Before discussing how exactly nqmgr implements the recipient limits, let's see +how the sole existence of the limits themselves affects the nqmgr and its +scheduler. + +The message limit is straightforward - it just limits the size of the lookahead +the nqmgr's scheduler has when choosing which message can preempt the current +one. Messages not in the "active" queue are simply not considered at all. + +The recipient limit complicates more things. First of all, the message reading +code must support reading the recipients in batches, which among other things +means accessing the queue file several times and continuing where the last +recipient batch ended. This is invoked by the scheduler whenever the current +job has space for more recipients, subject to transport's refill_limit and +refill_delay parameters. It is also done any time when all in-core recipients +of the message are dealt with (which may also mean they were deferred) but +there are still more in the queue file. + +The second complication is that with some recipients left unread in the queue +file, the scheduler can't operate with exact counts of recipient entries. With +unread recipients, it is not clear how many recipient entries there will be, as +they are subject to per-destination grouping. It is not even clear to what +transports (and thus jobs) the recipients will be assigned. And with messages +coming from the "deferred" queue, it is not even clear how many unread +recipients are still to be delivered. This all means that the scheduler must +use only estimates of how many recipients entries there will be. Fortunately, +it is possible to estimate the minimum and maximum correctly, so the scheduler +can always err on the safe side. Obviously, the better the estimates, the +better the results, so it is best when we are able to read all recipients in- +core and turn the estimates into exact counts, or at least try to read as many +as possible to make the estimates as accurate as possible. + +The third complication is that it is no longer true that the scheduler is done +with a job once all of its in-core recipients are delivered. It is possible +that the job will be revived later, when another batch of recipients is read in +core. It is also possible that some jobs will be created for the first time +long after the first batch of recipients was read in core. The nqmgr code must +be ready to handle all such situations. + +And finally, the fourth complication is that the nqmgr code must somehow impose +the recipient limit itself. Now how does it achieve it? + +Perhaps the easiest solution would be to say that each message may have at +maximum X recipients stored in-core, but such a solution would be poor for +several reasons. With reasonable qmgr_message_active_limit values, the X would +have to be quite low to maintain a reasonable memory footprint. And with low X +lots of things would not work well. The nqmgr would have problems to use the +transport_destination_recipient_limit efficiently. The scheduler's preemption +would be suboptimal as the recipient count estimates would be inaccurate. The +message queue file would have to be accessed many times to read in more +recipients again and again. + +Therefore it seems reasonable to have a solution which does not use a limit +imposed on a per-message basis, but which maintains a pool of available +recipient slots, which can be shared among all messages in the most efficient +manner. And as we do not want separate transports to compete for resources +whenever possible, it seems appropriate to maintain such a recipient pool for +each transport separately. This is the general idea, now how does it work in +practice? + +First we have to solve a little chicken-and-egg problem. If we want to use the +per-transport recipient pools, we first need to know to what transport(s) the +message is assigned. But we will find that out only after we first read in the +recipients. So it is obvious that we first have to read in some recipients, use +them to find out to what transports the message is to be assigned, and only +after that can we use the per-transport recipient pools. + +Now how many recipients shall we read for the first time? This is what +qmgr_message_recipient_minimum and qmgr_message_recipient_limit values control. +The qmgr_message_recipient_minimum value specifies how many recipients of each +message we will read the first time, no matter what. It is necessary to read at +least one recipient before we can assign the message to a transport and create +the first job. However, reading only qmgr_message_recipient_minimum recipients +even if there are only few messages with few recipients in-core would be +wasteful. Therefore if there are fewer than qmgr_message_recipient_limit +recipients in-core so far, the first batch of recipients may be larger than +qmgr_message_recipient_minimum - as large as is required to reach the +qmgr_message_recipient_limit limit. + +Once the first batch of recipients was read in core and the message jobs were +created, the size of the subsequent recipient batches (if any - of course it's +best when all recipients are read in one batch) is based solely on the position +of the message jobs on their corresponding transports' job lists. Each +transport has a pool of transport_recipient_limit recipient slots which it can +distribute among its jobs (how this is done is described later). The subsequent +recipient batch may be as large as the sum of all recipient slots of all jobs +of the message permits (plus the qmgr_message_recipient_minimum amount which +always applies). + +For example, if a message has three jobs, the first with 1 recipient still in- +core and 4 recipient slots, the second with 5 recipients in-core and 5 +recipient slots, and the third with 2 recipients in-core and 0 recipient slots, +it has 1+5+2=8 recipients in-core and 4+5+0=9 jobs' recipients slots in total. +This means that we could immediately read 2+qmgr_message_recipient_minimum more +recipients of that message in core. + +The above example illustrates several things which might be worth mentioning +explicitly: first, note that although the per-transport slots are assigned to +particular jobs, we can't guarantee that once the next batch of recipients is +read in core, that the corresponding amounts of recipients will be assigned to +those jobs. The jobs lend its slots to the message as a whole, so it is +possible that some jobs end up sponsoring other jobs of their message. For +example, if in the example above the 2 newly read recipients were assigned to +the second job, the first job sponsored the second job with 2 slots. The second +notable thing is the third job, which has more recipients in-core than it has +slots. Apart from sponsoring by other job we just saw it can be result of the +first recipient batch, which is sponsored from global recipient pool of +qmgr_message_recipient_limit recipients. It can be also sponsored from the +message recipient pool of qmgr_message_recipient_minimum recipients. + +Now how does each transport distribute the recipient slots among its jobs? The +strategy is quite simple. As most scheduler activity happens on the head of the +job list, it is our intention to make sure that the scheduler has the best +estimates of the recipient counts for those jobs. As we mentioned above, this +means that we want to try to make sure that the messages of those jobs have all +recipients read in-core. Therefore the transport distributes the slots "along" +the job list from start to end. In this case the job list sorted by message +enqueue time is used, because it doesn't change over time as the scheduler's +job list does. + +More specifically, each time a job is created and appended to the job list, it +gets all unused recipient slots from its transport's pool. It keeps them until +all recipients of its message are read. When this happens, all unused recipient +slots are transferred to the next job (which is now in fact the first such job) +on the job list which still has some recipients unread, or eventually back to +the transport pool if there is no such job. Such a transfer then also happens +whenever a recipient entry of that job is delivered. + +There is also a scenario when a job is not appended to the end of the job list +(for example it was created as a result of a second or later recipient batch). +Then it works exactly as above, except that if it was put in front of the first +unread job (that is, the job of a message which still has some unread +recipients in the queue file), that job is first forced to return all of its +unused recipient slots to the transport pool. + +The algorithm just described leads to the following state: The first unread job +on the job list always gets all the remaining recipient slots of that transport +(if there are any). The jobs queued before this job are completely read (that +is, all recipients of their message were already read in core) and have at +maximum as many slots as they still have recipients in-core (the maximum is +there because of the sponsoring mentioned before) and the jobs after this job +get nothing from the transport recipient pool (unless they got something before +and then the first unread job was created and enqueued in front of them later - +in such a case, they also get at maximum as many slots as they have recipients +in-core). + +Things work fine in such a state for most of the time, because the current job +is either completely read in-core or has as many recipient slots as there are, +but there is one situation which we still have to take care of specially. +Imagine if the current job is preempted by some unread job from the job list +and there are no more recipient slots available, so this new current job could +read only batches of qmgr_message_recipient_minimum recipients at a time. This +would really degrade performance. For this reason, each transport has an extra +pool of transport_extra_recipient_limit recipient slots, dedicated exactly for +this situation. Each time an unread job preempts the current job, it gets half +of the remaining recipient slots from the normal pool and this extra pool. + +And that's it. It sure does sound pretty complicated, but fortunately most +people don't really have to care exactly how it works as long as it works. +Perhaps the only important things to know for most people are the following +upper bound formulas: + +Each transport has at maximum + + max( + qmgr_message_recipient_minimum * qmgr_message_active_limit + + *_recipient_limit + *_extra_recipient_limit, + qmgr_message_recipient_limit + ) + +recipients in core. + +The total amount of recipients in core is + + max( + qmgr_message_recipient_minimum * qmgr_message_active_limit + + sum( *_recipient_limit + *_extra_recipient_limit ), + qmgr_message_recipient_limit + ) + +where the sum is over all used transports. + +And this terribly complicated chapter concludes the documentation of the nqmgr +scheduler. + +[By now you should theoretically know the nqmgr scheduler inside out. In +practice, you still hope that you will never have to really understand the last +or last two chapters completely, and fortunately most people really won't. +Understanding how the scheduler works in ideal conditions is more than good +enough for the vast majority of users.] + +CCrreeddiittss + + * Wietse Venema designed and implemented the initial queue manager with per- + domain FIFO scheduling, and per-delivery +/-1 concurrency feedback. + * Patrik Rak designed and implemented preemption where mail with fewer + recipients can slip past mail with more recipients in a controlled manner, + and wrote up its documentation. + * Wietse Venema initiated a discussion with Patrik Rak and Victor Duchovni on + alternatives for the +/-1 feedback scheduler's aggressive behavior. This is + when K/N feedback was reviewed (N = concurrency). The discussion ended + without a good solution for both negative feedback and dead site detection. + * Victor Duchovni resumed work on concurrency feedback in the context of + concurrency-limited servers. + * Wietse Venema then re-designed the concurrency scheduler in terms of the + simplest possible concepts: less-than-1 concurrency feedback per delivery, + forward and reverse concurrency feedback hysteresis, and pseudo-cohort + failure. At this same time, concurrency feedback was separated from dead + site detection. + * These simplifications, and their modular implementation, helped to develop + further insights into the different roles that positive and negative + concurrency feedback play, and helped to identify some worst-case + scenarios. + diff --git a/README_FILES/SMTPD_ACCESS_README b/README_FILES/SMTPD_ACCESS_README new file mode 100644 index 0000000..7477a36 --- /dev/null +++ b/README_FILES/SMTPD_ACCESS_README @@ -0,0 +1,347 @@ +PPoossttffiixx SSMMTTPP rreellaayy aanndd aacccceessss ccoonnttrrooll + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +The Postfix SMTP server receives mail from the network and is exposed to the +big bad world of junk email and viruses. This document introduces the built-in +and external methods that control what SMTP mail Postfix will accept, what +mistakes to avoid, and how to test your configuration. + +Topics covered in this document: + + * Relay control, junk mail control, and per-user policies + * Restrictions that apply to all SMTP mail + * Getting selective with SMTP access restriction lists + * Delayed evaluation of SMTP access restriction lists + * Dangerous use of smtpd_recipient_restrictions + * SMTP access rule testing + +RReellaayy ccoonnttrrooll,, jjuunnkk mmaaiill ccoonnttrrooll,, aanndd ppeerr--uusseerr ppoolliicciieess + +In a distant past, the Internet was a friendly environment. Mail servers +happily forwarded mail on behalf of anyone towards any destination. On today's +Internet, spammers abuse servers that forward mail from arbitrary systems, and +abused systems end up on anti-spammer denylists. See, for example, the +information on http://www.mail-abuse.org/ and other websites. + +By default, Postfix has a moderately restrictive approach to mail relaying. +Postfix forwards mail only from clients in trusted networks, from clients that +have authenticated with SASL, or to domains that are configured as authorized +relay destinations. For a description of the default mail relay policy, see the +smtpd_relay_restrictions parameter in the postconf(5) manual page, and the +information that is referenced from there. + + NOTE: Postfix versions before 2.10 did not have smtpd_relay_restrictions. + They combined the mail relay and spam blocking policies, under + smtpd_recipient_restrictions. This could lead to unexpected results. For + example, a permissive spam blocking policy could unexpectedly result in a + permissive mail relay policy. An example of this is documented under + "Dangerous use of smtpd_recipient_restrictions". + +Most of the Postfix SMTP server access controls are targeted at stopping junk +email. + + * Protocol oriented: some SMTP server access controls block mail by being + very strict with respect to the SMTP protocol; these catch poorly + implemented and/or poorly configured junk email software, as well as email + worms that come with their own non-standard SMTP client implementations. + Protocol-oriented access controls become less useful over time as spammers + and worm writers learn to read RFC documents. + + * Denylist oriented: some SMTP server access controls query denylists with + known to be bad sites such as open mail relays, open web proxies, and home + computers that have been compromised and that are under remote control by + criminals. The effectiveness of these denylists depends on how complete and + how up to date they are. + + * Threshold oriented: some SMTP server access controls attempt to raise the + bar by either making the client do more work (greylisting) or by asking for + a second opinion (SPF and sender/recipient address verification). The + greylisting and SPF policies are implemented externally, and are the + subject of the SMTPD_POLICY_README document. Sender/recipient address + verification is the subject of the ADDRESS_VERIFICATION_README document. + +Unfortunately, all junk mail controls have the possibility of falsely rejecting +legitimate mail. This can be a problem for sites with many different types of +users. For some users it is unacceptable when any junk email slips through, +while for other users the world comes to an end when a single legitimate email +message is blocked. Because there is no single policy that is "right" for all +users, Postfix supports different SMTP access restrictions for different users. +This is described in the RESTRICTION_CLASS_README document. + +RReessttrriiccttiioonnss tthhaatt aappppllyy ttoo aallll SSMMTTPP mmaaiill + +Besides the restrictions that can be made configurable per client or per user +as described in the next section, Postfix implements a few restrictions that +apply to all SMTP mail. + + * The built-in header_checks and body_checks content restrictions, as + described in the BUILTIN_FILTER_README document. This happens while Postfix + receives mail, before it is stored in the incoming queue. + + * The external before-queue content restrictions, as described in the + SMTPD_PROXY_README document. This happens while Postfix receives mail, + before it is stored in the incoming queue. + + * Requiring that the client sends the HELO or EHLO command before sending the + MAIL FROM or ETRN command. This may cause problems with home-grown + applications that send mail. For this reason, the requirement is disabled + by default ("smtpd_helo_required = no"). + + * Disallowing illegal syntax in MAIL FROM or RCPT TO commands. This may cause + problems with home-grown applications that send mail, and with ancient PC + mail clients. For this reason, the requirement is disabled by default + ("strict_rfc821_envelopes = no"). + + o Disallowing RFC 822 address syntax (example: "MAIL FROM: the dude + <dude@example.com>"). + + o Disallowing addresses that are not enclosed with <> (example: "MAIL + FROM: dude@example.com"). + + * Rejecting mail from a non-existent sender address. This form of egress + filtering helps to slow down worms and other malware, but may cause + problems with home-grown software that sends out mail software with an + unreplyable address. For this reason the requirement is disabled by default + ("smtpd_reject_unlisted_sender = no"). + + * Rejecting mail for a non-existent recipient address. This form of ingress + filtering helps to keep the mail queue free of undeliverable MAILER-DAEMON + messages. This requirement is enabled by default + ("smtpd_reject_unlisted_recipient = yes"). + +GGeettttiinngg sseelleeccttiivvee wwiitthh SSMMTTPP aacccceessss rreessttrriiccttiioonn lliissttss + +Postfix allows you to specify lists of access restrictions for each stage of +the SMTP conversation. Individual restrictions are described in the postconf(5) +manual page. + +Examples of simple restriction lists are: + +/etc/postfix/main.cf: + # Allow connections from trusted networks only. + smtpd_client_restrictions = permit_mynetworks, reject + + # Don't talk to mail systems that don't know their own hostname. + # With Postfix < 2.3, specify reject_unknown_hostname. + smtpd_helo_restrictions = reject_unknown_helo_hostname + + # Don't accept mail from domains that don't exist. + smtpd_sender_restrictions = reject_unknown_sender_domain + + # Spam control: exclude local clients and authenticated clients + # from DNSBL lookups. + smtpd_recipient_restrictions = permit_mynetworks, + permit_sasl_authenticated, + # reject_unauth_destination is not needed here if the mail + # relay policy is specified under smtpd_relay_restrictions + # (available with Postfix 2.10 and later). + reject_unauth_destination + reject_rbl_client zen.spamhaus.org, + reject_rhsbl_reverse_client dbl.spamhaus.org, + reject_rhsbl_helo dbl.spamhaus.org, + reject_rhsbl_sender dbl.spamhaus.org + + # Relay control (Postfix 2.10 and later): local clients and + # authenticated clients may specify any destination domain. + smtpd_relay_restrictions = permit_mynetworks, + permit_sasl_authenticated, + reject_unauth_destination + + # Block clients that speak too early. + smtpd_data_restrictions = reject_unauth_pipelining + + # Enforce mail volume quota via policy service callouts. + smtpd_end_of_data_restrictions = check_policy_service unix:private/policy + +Each restriction list is evaluated from left to right until some restriction +produces a result of PERMIT, REJECT or DEFER (try again later). The end of each +list is equivalent to a PERMIT result. By placing a PERMIT restriction before a +REJECT restriction you can make exceptions for specific clients or users. This +is called allowlisting; the smtpd_relay_restrictions example above allows mail +from local networks, and from SASL authenticated clients, but otherwise rejects +mail to arbitrary destinations. + +The table below summarizes the purpose of each SMTP access restriction list. +All lists use the exact same syntax; they differ only in the time of evaluation +and in the effect of a REJECT or DEFER result. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + | | | |EEffffeecctt ooff | + |RReessttrriiccttiioonn lliisstt nnaammee |VVeerrssiioonn|SSttaattuuss |RREEJJEECCTT oorr | + | | | |DDEEFFEERR | + | | | |rreessuulltt | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | | |Reject all | + |smtpd_client_restrictions |All |Optional |client | + | | | |commands | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | | |Reject | + |smtpd_helo_restrictions |All |Optional |HELO/EHLO | + | | | |information| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | | |Reject MAIL| + |smtpd_sender_restrictions |All |Optional |FROM | + | | | |information| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | |Required if | | + | |>= 2.10|smtpd_relay_restrictions | | + | | |does not enforce relay |Reject RCPT| + |smtpd_recipient_restrictions | |policy |TO | + | |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |information| + | | | | | + | |< 2.10 |Required | | + | | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | |Required if | | + | |>= 2.10|smtpd_recipient_restrictions| | + | | |does not enforce relay |Reject RCPT| + |smtpd_relay_restrictions | |policy |TO | + | |_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |information| + | | | | | + | |< 2.10 |Not available | | + | | | | | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + |smtpd_data_restrictions |>= 2.0 |Optional |Reject DATA| + | | | |command | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + | | | |Reject END-| + |smtpd_end_of_data_restrictions|>= 2.2 |Optional |OF-DATA | + | | | |command | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + |smtpd_etrn_restrictions |All |Optional |Reject ETRN| + | | | |command | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ | + +DDeellaayyeedd eevvaalluuaattiioonn ooff SSMMTTPP aacccceessss rreessttrriiccttiioonn lliissttss + +Early Postfix versions evaluated SMTP access restrictions lists as early as +possible. The client restriction list was evaluated before Postfix sent the +"220 $myhostname..." greeting banner to the SMTP client, the helo restriction +list was evaluated before Postfix replied to the HELO (EHLO) command, the +sender restriction list was evaluated before Postfix replied to the MAIL FROM +command, and so on. This approach turned out to be difficult to use. + +Current Postfix versions postpone the evaluation of client, helo and sender +restriction lists until the RCPT TO or ETRN command. This behavior is +controlled by the smtpd_delay_reject parameter. Restriction lists are still +evaluated in the proper order of (client, helo, etrn) or (client, helo, sender, +relay, recipient, data, or end-of-data) restrictions. When a restriction list +(example: client) evaluates to REJECT or DEFER the restriction lists that +follow (example: helo, sender, etc.) are skipped. + +Around the time that smtpd_delay_reject was introduced, Postfix was also +changed to support mixed restriction lists that combine information about the +client, helo, sender and recipient or etrn command. + +Benefits of delayed restriction evaluation, and of restriction mixing: + + * Some SMTP clients do not expect a negative reply early in the SMTP session. + When the bad news is postponed until the RCPT TO reply, the client goes + away as it is supposed to, instead of hanging around until a timeout + happens, or worse, going into an endless connect-reject-connect loop. + + * Postfix can log more useful information. For example, when Postfix rejects + a client name or address and delays the action until the RCPT TO command, + it can log the sender and the recipient address. This is more useful than + logging only the client hostname and IP address and not knowing whose mail + was being blocked. + + * Mixing is needed for complex allowlisting policies. For example, in order + to reject local sender addresses in mail from non-local clients, you need + to be able to mix restrictions on client information with restrictions on + sender information in the same restriction list. Without this ability, many + per-user access restrictions would be impossible to express. + +DDaannggeerroouuss uussee ooff ssmmttppdd__rreecciippiieenntt__rreessttrriiccttiioonnss + +By now the reader may wonder why we need smtpd client, helo or sender +restrictions, when their evaluation is postponed until the RCPT TO or ETRN +command. Some people recommend placing ALL the access restrictions in the +smtpd_recipient_restrictions list. Unfortunately, this can result in too +permissive access. How is this possible? + +The purpose of the smtpd_recipient_restrictions feature is to control how +Postfix replies to the RCPT TO command. If the restriction list evaluates to +REJECT or DEFER, the recipient address is rejected; no surprises here. If the +result is PERMIT, then the recipient address is accepted. And this is where +surprises can happen. + +The problem is that Postfix versions before 2.10 did not have +smtpd_relay_restrictions. They combined the mail relay and spam blocking +policies, under smtpd_recipient_restrictions. The result is that a permissive +spam blocking policy could unexpectedly result in a permissive mail relay +policy. + +Here is an example that shows when a PERMIT result can result in too much +access permission: + +1 /etc/postfix/main.cf: +2 smtpd_recipient_restrictions = +3 permit_mynetworks +4 check_helo_access hash:/etc/postfix/helo_access +5 reject_unknown_helo_hostname +6 rreejjeecctt__uunnaauutthh__ddeessttiinnaattiioonn +7 +8 /etc/postfix/helo_access: +9 localhost.localdomain PERMIT + +Line 5 rejects mail from hosts that don't specify a proper hostname in the HELO +command (with Postfix < 2.3, specify reject_unknown_hostname). Lines 4 and 9 +make an exception to allow mail from some machine that announces itself with +"HELO localhost.localdomain". + +The problem with this configuration is that smtpd_recipient_restrictions +evaluates to PERMIT for EVERY host that announces itself as +"localhost.localdomain", making Postfix an open relay for all such hosts. + +With Postfix before version 2.10 you should place non-recipient restrictions +AFTER the reject_unauth_destination restriction, not before. In the above +example, the HELO based restrictions should be placed AFTER +reject_unauth_destination, or better, the HELO based restrictions should be +placed under smtpd_helo_restrictions where they can do no harm. + +1 /etc/postfix/main.cf: +2 smtpd_recipient_restrictions = +3 permit_mynetworks +4 rreejjeecctt__uunnaauutthh__ddeessttiinnaattiioonn +5 check_helo_access hash:/etc/postfix/helo_access +6 reject_unknown_helo_hostname +7 +8 /etc/postfix/helo_access: +9 localhost.localdomain PERMIT + +The above mistake will not happen with Postfix 2.10 and later, when the relay +policy is specified under smtpd_relay_restrictions, and the spam blocking +policy under smtpd_recipient_restrictions. Then, a permissive spam blocking +policy will not result in a permissive mail relay policy. + +SSMMTTPP aacccceessss rruullee tteessttiinngg + +Postfix has several features that aid in SMTP access rule testing: + +soft_bounce + This is a safety net that changes SMTP server REJECT actions into DEFER + (try again later) actions. This keeps mail queued that would otherwise be + returned to the sender. Specify "soft_bounce = yes" in the main.cf file to + prevent the Postfix SMTP server from rejecting mail permanently, by + changing all 5xx SMTP reply codes into 4xx. + +warn_if_reject + When placed before a reject-type restriction, access table query, or + check_policy_service query, this logs a "reject_warning" message instead of + rejecting a request (when a reject-type restriction fails due to a + temporary error, this logs a "reject_warning" message for any implicit + "defer_if_permit" actions that would normally prevent mail from being + accepted by some later access restriction). This feature has no effect on + defer_if_reject restrictions. + +XCLIENT + With this feature, an authorized SMTP client can impersonate other systems + and perform realistic SMTP access rule tests. Examples of how to + impersonate other systems for access rule testing are given at the end of + the XCLIENT_README document. + This feature is available in Postfix 2.1. + diff --git a/README_FILES/SMTPD_POLICY_README b/README_FILES/SMTPD_POLICY_README new file mode 100644 index 0000000..47a6fa3 --- /dev/null +++ b/README_FILES/SMTPD_POLICY_README @@ -0,0 +1,651 @@ +PPoossttffiixx SSMMTTPP AAcccceessss PPoolliiccyy DDeelleeggaattiioonn + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff PPoossttffiixx SSMMTTPP aacccceessss ppoolliiccyy ddeelleeggaattiioonn + +The Postfix SMTP server has a number of built-in mechanisms to block or accept +mail at specific SMTP protocol stages. In addition, the Postfix SMTP server can +delegate decisions to an external policy server (Postfix 2.1 and later). + +With this policy delegation mechanism, a simple greylist policy can be +implemented with only a dozen lines of Perl, as is shown at the end of this +document. A complete example can be found in the Postfix source code, in the +directory examples/smtpd-policy. + +Another example of policy delegation is the SPF policy server at https:// +web.archive.org/web/20190221142057/http://www.openspf.org/Software. + +Policy delegation is now the preferred method for adding policies to Postfix. +It's much easier to develop a new feature in few lines of Perl, Python, Ruby, +or TCL, than trying to do the same in C code. The difference in performance +will be unnoticeable except in the most demanding environments. On active +systems a policy daemon process is used multiple times, for up to $max_use +incoming SMTP connections. + +This document covers the following topics: + + * Policy protocol description + * Simple policy client/server configuration + * Advanced policy client configuration + * Example: greylist policy server + * Greylisting mail from frequently forged domains + * Greylisting all your mail + * Routine greylist maintenance + * Example Perl greylist server + +PPrroottooccooll ddeessccrriippttiioonn + +The Postfix policy delegation protocol is really simple. The client sends a +request and the server sends a response. Unless there was an error, the server +must not close the connection, so that the same connection can be used multiple +times. + +The client request is a sequence of name=value attributes separated by newline, +and is terminated by an empty line. The server reply is one name=value +attribute and it, too, is terminated by an empty line. + +Here is an example of all the attributes that the Postfix SMTP server sends in +a delegated SMTPD access policy request: + + PPoossttffiixx vveerrssiioonn 22..11 aanndd llaatteerr:: + request=smtpd_access_policy + protocol_state=RCPT + protocol_name=SMTP + helo_name=some.domain.tld + queue_id=8045F2AB23 + sender=foo@bar.tld + recipient=bar@foo.tld + recipient_count=0 + client_address=1.2.3.4 + client_name=another.domain.tld + reverse_client_name=another.domain.tld + instance=123.456.7 + PPoossttffiixx vveerrssiioonn 22..22 aanndd llaatteerr:: + sasl_method=plain + sasl_username=you + sasl_sender= + size=12345 + ccert_subject=solaris9.porcupine.org + ccert_issuer=Wietse+20Venema + ccert_fingerprint=C2:9D:F4:87:71:73:73:D9:18:E7:C2:F3:C1:DA:6E:04 + PPoossttffiixx vveerrssiioonn 22..33 aanndd llaatteerr:: + encryption_protocol=TLSv1/SSLv3 + encryption_cipher=DHE-RSA-AES256-SHA + encryption_keysize=256 + etrn_domain= + PPoossttffiixx vveerrssiioonn 22..55 aanndd llaatteerr:: + stress= + PPoossttffiixx vveerrssiioonn 22..99 aanndd llaatteerr:: + ccert_pubkey_fingerprint=68:B3:29:DA:98:93:E3:40:99:C7:D8:AD:5C:B9:C9:40 + PPoossttffiixx vveerrssiioonn 33..00 aanndd llaatteerr:: + client_port=1234 + PPoossttffiixx vveerrssiioonn 33..11 aanndd llaatteerr:: + policy_context=submission + PPoossttffiixx vveerrssiioonn 33..22 aanndd llaatteerr:: + server_address=10.3.2.1 + server_port=54321 + PPoossttffiixx vveerrssiioonn 33..88 aanndd llaatteerr:: + compatibility_level=major.minor.patch + mail_version=3.8.0 + [empty line] + +Notes: + + * The "request" attribute is required. In this example the request type is + "smtpd_access_policy". + + * The order of the attributes does not matter. The policy server should + ignore any attributes that it does not care about. + + * When the same attribute name is sent more than once, the server may keep + the first value or the last attribute value. + + * When an attribute value is unavailable, the client either does not send the + attribute, sends the attribute with an empty value ("name="), or sends a + zero value ("name=0") in the case of a numerical attribute. + + * The "recipient" attribute is available in the "RCPT TO" stage. It is also + available in the "DATA" and "END-OF-MESSAGE" stages if Postfix accepted + only one recipient for the current message. The DATA protocol state also + applies to email that is received with BDAT commands (Postfix 3.4 and + later). + + * The "recipient_count" attribute (Postfix 2.3 and later) is non-zero only in + the "DATA" and "END-OF-MESSAGE" stages. It specifies the number of + recipients that Postfix accepted for the current message. The DATA protocol + state also applies to email that is received with BDAT commands (Postfix + 3.4 and later). + + * The remote client or local server IP address is an IPv4 dotted quad in the + form 1.2.3.4 or it is an IPv6 address in the form 1:2:3::4:5:6. + + * The remote client or local server port is a decimal number in the range 0- + 65535. + + * For a discussion of the differences between reverse and verified + client_name information, see the reject_unknown_client_hostname discussion + in the postconf(5) document. + + * An attribute name must not contain "=", null or newline, and an attribute + value must not contain null or newline. + + * The "instance" attribute value can be used to correlate different requests + regarding the same message delivery. These requests are sent over the same + policy connection (unless the policy daemon terminates the connection). + Once Postfix sends a query with a different instance attribute over that + same policy connection, the previous message delivery is either completed + or aborted. + + * The "size" attribute value specifies the message size that the client + specified in the MAIL FROM command (zero if none was specified). With + Postfix 2.2 and later, it specifies the actual message size after the + client sends the END-OF-MESSAGE. + + * The "sasl_*" attributes (Postfix 2.2 and later) specify information about + how the client was authenticated via SASL. These attributes are empty in + case of no SASL authentication. + + * The "ccert_*" attributes (Postfix 2.2 and later) specify information about + how the client was authenticated via TLS. These attributes are empty in + case of no certificate authentication. As of Postfix 2.2.11 these attribute + values are encoded as xtext: some characters are represented by +XX, where + XX is the two-digit hexadecimal representation of the character value. With + Postfix 2.6 and later, the decoded string is an UTF-8 string without non- + printable ASCII characters. + + * The "encryption_*" attributes (Postfix 2.3 and later) specify information + about how the connection is encrypted. With plaintext connections the + protocol and cipher attributes are empty and the keysize is zero. + + * The "etrn_domain" attribute is defined only in the context of the ETRN + command, and specifies the ETRN command parameter. + + * The "stress" attribute is either empty or "yes". See the STRESS_README + document for further information. + + * The "policy_context" attribute provides a way to pass information that is + not available via other attributes (Postfix version 3.1 and later). + + * The "compatibility_level" attribute corresponds to the compatibility_level + parameter value. It has the form major.minor.patch where minor and patch + may be absent. + + * The "mail_version" attribute corresponds to the mail_version parameter + value. It has the form major.minor.patch for stable releases, and + major.minor-yyyymmdd for unstable releases. + +The following is specific to SMTPD delegated policy requests: + + * Protocol names are ESMTP or SMTP. + + * Protocol states are CONNECT, EHLO, HELO, MAIL, RCPT, DATA, END-OF-MESSAGE, + VRFY or ETRN; these are the SMTP protocol states where the Postfix SMTP + server makes an OK/REJECT/HOLD/etc. decision. The DATA protocol state also + applies to email that is received with BDAT commands (Postfix 3.4 and + later). + +The policy server replies with any action that is allowed in a Postfix SMTPD +access(5) table. Example: + + action=defer_if_permit Service temporarily unavailable + [empty line] + +This causes the Postfix SMTP server to reject the request with a 450 temporary +error code and with text "Service temporarily unavailable", if the Postfix SMTP +server finds no reason to reject the request permanently. + +In case of trouble the policy server must not send a reply. Instead the server +must log a warning and disconnect. Postfix will retry the request at some later +time. + +SSiimmppllee ppoolliiccyy cclliieenntt//sseerrvveerr ccoonnffiigguurraattiioonn + +The Postfix delegated policy client can connect to a TCP socket or to a UNIX- +domain socket. Examples: + + inet:127.0.0.1:9998 + unix:/some/where/policy + unix:private/policy + +The first example specifies that the policy server listens on a TCP socket at +127.0.0.1 port 9998. The second example specifies an absolute pathname of a +UNIX-domain socket. The third example specifies a pathname relative to the +Postfix queue directory; use this for policy servers that are spawned by the +Postfix master daemon. On many systems, "local" is a synonym for "unix". + +To create a policy service that listens on a UNIX-domain socket called +"policy", and that runs under control of the Postfix spawn(8) daemon, you would +use something like this: + + 1 /etc/postfix/master.cf: + 2 policy unix - n n - 0 spawn + 3 user=nobody argv=/some/where/policy-server + 4 + 5 /etc/postfix/main.cf: + 6 smtpd_recipient_restrictions = + 7 ... + 8 reject_unauth_destination + 9 check_policy_service unix:private/policy + 10 ... + 11 policy_time_limit = 3600 + 12 # smtpd_policy_service_request_limit = 1 + +NOTES: + + * Lines 2-3: this creates the service called "policy" that listens on a UNIX- + domain socket. The service is implemented by the Postfix spawn(8) daemon, + which executes the policy server program that is specified with the aarrggvv + attribute, using the privileges specified with the uusseerr attribute. + + * Line 2: specify a "0" process limit instead of the default "-", to avoid + "connection refused" and other problems when you increase the smtpd process + limit. + + * Line 8: reject_unauth_destination is not needed here if the mail relay + policy is specified with smtpd_relay_restrictions (available with Postfix + 2.10 and later). + + * Lines 8, 9: always specify "check_policy_service" AFTER + "reject_unauth_destination" or else your system could become an open relay. + + * Line 11: this increases the time that a policy server process may run to + 3600 seconds. The default time limit of 1000 seconds is too short; the + policy daemon needs to run as long as the SMTP server process that talks to + it. See the spawn(8) manpage for more information about the + transport_time_limit parameter. + + Note: the "policy_time_limit" parameter will not show up in "postconf" + command output before Postfix version 2.9. This limitation applies to + many parameters whose name is a combination of a master.cf service name + (in the above example, "policy") and a built-in suffix (in the above + example: "_time_limit"). + + * Line 12: specify smtpd_policy_service_request_limit to avoid error-recovery + delays with policy servers that cannot maintain a persistent connection. + + * With Solaris < 9, or Postfix < 2.10 on any Solaris version, use TCP sockets + instead of UNIX-domain sockets: + + 1 /etc/postfix/master.cf: + 2 127.0.0.1:9998 inet n n n - 0 spawn + 3 user=nobody argv=/some/where/policy-server + 4 + 5 /etc/postfix/main.cf: + 6 smtpd_recipient_restrictions = + 7 ... + 8 reject_unauth_destination + 9 check_policy_service inet:127.0.0.1:9998 + 10 ... + 11 127.0.0.1:9998_time_limit = 3600 + 12 # smtpd_policy_service_request_limit = 1 + +Configuration parameters that control the client side of the policy delegation +protocol: + + * smtpd_policy_service_default_action (default: 451 4.3.5 Server + configuration problem): The default action when an SMTPD policy service + request fails. Available with Postfix 3.0 and later. + + * smtpd_policy_service_max_idle (default: 300s): The amount of time before + the Postfix SMTP server closes an unused policy client connection. + + * smtpd_policy_service_max_ttl (default: 1000s): The amount of time before + the Postfix SMTP server closes an active policy client connection. + + * smtpd_policy_service_request_limit (default: 0): The maximal number of + requests per policy connection, or zero (no limit). Available with Postfix + 3.0 and later. + + * smtpd_policy_service_timeout (default: 100s): The time limit to connect to, + send to or receive from a policy server. + + * smtpd_policy_service_try_limit (default: 2): The maximal number of attempts + to send an SMTPD policy service request before giving up. Available with + Postfix 3.0 and later. + + * smtpd_policy_service_retry_delay (default: 1s): The delay between attempts + to resend a failed SMTPD policy service request. Available with Postfix 3.0 + and later. + + * smtpd_policy_service_policy_context (default: empty): Optional information + that is passed in the "policy_context" attribute of an SMTPD policy service + request (originally, to share the same SMTPD service endpoint among + multiple check_policy_service clients). Available with Postfix 3.1 and + later. + +Configuration parameters that control the server side of the policy delegation +protocol: + + * transport_time_limit ($command_time_limit): The maximal amount of time the + policy daemon is allowed to run before it is terminated. The transport is + the service name of the master.cf entry for the policy daemon service. In + the above examples, the service name is "policy" or "127.0.0.1:9998". + +AAddvvaanncceedd ppoolliiccyy cclliieenntt ccoonnffiigguurraattiioonn + +The previous section lists a number of Postfix main.cf parameters that control +time limits and other settings for all policy clients. This is sufficient for +simple configurations. With more complex configurations it becomes desirable to +have different settings per policy client. This is supported with Postfix 3.0 +and later. + +The following example shows a "non-critical" policy service with a short +timeout, and with "DUNNO" as default action when the service is unvailable. The +"DUNNO" action causes Postfix to ignore the result. + + 1 /etc/postfix/main.cf: + 2 mua_recipient_restrictions = + 3 ... + 4 reject_unauth_destination + 5 check_policy_service { inet:host:port, + 6 timeout=10s, default_action=DUNNO + 7 policy_context=submission } + 8 ... + +Instead of a server endpoint, we now have a list enclosed in {}. + + * Line 5: The first item in the list is the server endpoint. This supports + the exact same "inet" and "unix" syntax as described earlier. + + * Line 6-7: The remainder of the list contains per-client settings. These + settings override global main.cf parameters, and have the same name as + those parameters, without the "smtpd_policy_service_" prefix. + +Inside the list, syntax is similar to what we already know from main.cf: items +separated by space or comma. There is one difference: yyoouu mmuusstt eenncclloossee aa +sseettttiinngg iinn ppaarreenntthheesseess,, aass iinn ""{{ nnaammee == vvaalluuee }}"",, iiff yyoouu wwaanntt ttoo hhaavvee ssppaaccee oorr +ccoommmmaa wwiitthhiinn aa vvaalluuee oorr aarroouunndd ""=="". This comes in handy when different policy +servers require different default actions with different SMTP status codes or +text: + + 1 /etc/postfix/main.cf: + 2 smtpd_recipient_restrictions = + 3 ... + 4 reject_unauth_destination + 5 check_policy_service { + 6 inet:host:port1, + 7 { default_action = 451 4.3.5 See http://www.example.com/ + support1 } + 8 } + 9 ... + +EExxaammppllee:: ggrreeyylliisstt ppoolliiccyy sseerrvveerr + +Greylisting is a defense against junk email that is described at http:// +www.greylisting.org/. The idea was discussed on the postfix-users mailing list +one year before it was popularized. + +The file examples/smtpd-policy/greylist.pl in the Postfix source tree +implements a simplified greylist policy server. This server stores a time stamp +for every (client, sender, recipient) triple. By default, mail is not accepted +until a time stamp is more than 60 seconds old. This stops junk mail with +randomly selected sender addresses, and mail that is sent through randomly +selected open proxies. It also stops junk mail from spammers that change their +IP address frequently. + +Copy examples/smtpd-policy/greylist.pl to /usr/libexec/postfix or whatever +location is appropriate for your system. + +In the greylist.pl Perl script you need to specify the location of the greylist +database file, and how long mail will be delayed before it is accepted. The +default settings are: + + $database_name="/var/mta/greylist.db"; + $greylist_delay=60; + +The /var/mta directory (or whatever you choose) should be writable by "nobody", +or by whatever username you configure below in master.cf for the policy +service. + +Example: + + # mkdir /var/mta + # chown nobody /var/mta + +Note: DO NOT create the greylist database in a world-writable directory such as +/tmp or /var/tmp, and DO NOT create the greylist database in a file system that +may run out of space. Postfix can survive "out of space" conditions with the +mail queue and with the mailbox store, but it cannot survive a corrupted +greylist database. If the file becomes corrupted you may not be able to receive +mail at all until you delete the file by hand. + +The greylist.pl Perl script can be run under control by the Postfix master +daemon. For example, to run the script as user "nobody", using a UNIX-domain +socket that is accessible by Postfix processes only: + + 1 /etc/postfix/master.cf: + 2 greylist unix - n n - 0 spawn + 3 user=nobody argv=/usr/bin/perl /usr/libexec/postfix/greylist.pl + 4 + 5 /etc/postfix/main.cf: + 6 greylist_time_limit = 3600 + 7 smtpd_recipient_restrictions = + 8 ... + 9 reject_unauth_destination + 10 check_policy_service unix:private/greylist + 11 ... + 12 # smtpd_policy_service_request_limit = 1 + +Notes: + + * Lines 2-3: this creates the service called "greylist" that listens on a + UNIX-domain socket. The service is implemented by the Postfix spawn(8) + daemon, which executes the greylist.pl script that is specified with the + aarrggvv attribute, using the privileges specified with the uusseerr attribute. + + * Line 2: specify a "0" process limit instead of the default "-", to avoid + "connection refused" and other problems when you increase the smtpd process + limit. + + * Line 3: Specify "greylist.pl -v" for verbose logging of each request and + reply. + + * Line 6: this increases the time that a greylist server process may run to + 3600 seconds. The default time limit of 1000 seconds is too short; the + greylist daemon needs to run as long as the SMTP server process that talks + to it. See the spawn(8) manpage for more information about the + transport_time_limit parameter. + + * Line 9: reject_unauth_destination is not needed here if the mail relay + policy is specified with smtpd_relay_restrictions (available with Postfix + 2.10 and later). + + Note: the "greylist_time_limit" parameter will not show up in + "postconf" command output before Postfix version 2.9. This limitation + applies to many parameters whose name is a combination of a master.cf + service name (in the above example, "greylist") and a built-in suffix + (in the above example: "_time_limit"). + + * Line 12: specify smtpd_policy_service_request_limit to avoid error-recovery + delays with policy servers that cannot maintain a persistent connection. + +With Solaris < 9, or Postfix < 2.10 on any Solaris version, use inet: style +sockets instead of unix: style, as detailed in the "Policy client/server +configuration" section above. + + 1 /etc/postfix/master.cf: + 2 127.0.0.1:9998 inet n n n - 0 spawn + 3 user=nobody argv=/usr/bin/perl /usr/libexec/postfix/greylist.pl + 4 + 5 /etc/postfix/main.cf: + 6 127.0.0.1:9998_time_limit = 3600 + 7 smtpd_recipient_restrictions = + 8 ... + 9 reject_unauth_destination + 10 check_policy_service inet:127.0.0.1:9998 + 11 ... + 12 # smtpd_policy_service_request_limit = 1 + +GGrreeyylliissttiinngg mmaaiill ffrroomm ffrreeqquueennttllyy ffoorrggeedd ddoommaaiinnss + +It is relatively safe to turn on greylisting for specific domains that often +appear in forged email. At some point in cyberspace/time a list of frequently +forged MAIL FROM domains could be found at https://web.archive.org/web/ +20080526153208/http://www.monkeys.com/anti-spam/filtering/sender-domain- +validate.in. + + 1 /etc/postfix/main.cf: + 2 smtpd_recipient_restrictions = + 3 reject_unlisted_recipient + 4 ... + 5 reject_unauth_destination + 6 check_sender_access hash:/etc/postfix/sender_access + 7 ... + 8 smtpd_restriction_classes = greylist + 9 greylist = check_policy_service unix:private/greylist + 10 + 11 /etc/postfix/sender_access: + 12 aol.com greylist + 13 hotmail.com greylist + 14 bigfoot.com greylist + 15 ... etcetera ... + +NOTES: + + * Line 9: On Solaris < 9, or Postfix < 2.10 on any Solaris version, use inet: + style sockets instead of unix: style, as detailed in the "Example: greylist + policy server" section above. + + * Line 5: reject_unauth_destination is not needed here if the mail relay + policy is specified with smtpd_relay_restrictions (available with Postfix + 2.10 and later). + + * Line 6: Be sure to specify "check_sender_access" AFTER + "reject_unauth_destination" or else your system could become an open mail + relay. + + * Line 3: With Postfix 2.0 snapshot releases, "reject_unlisted_recipient" is + called "check_recipient_maps". Postfix 2.1 understands both forms. + + * Line 3: The greylist database gets polluted quickly with bogus addresses. + It helps if you protect greylist lookups with other restrictions that + reject unknown senders and/or recipients. + +GGrreeyylliissttiinngg aallll yyoouurr mmaaiill + +If you turn on greylisting for all mail you may want to make exceptions for +mailing lists that use one-time sender addresses, because each message will be +delayed due to greylisting, and the one-time sender addresses can pollute your +greylist database relatively quickly. Instead of making exceptions, you can +automatically allowlist clients that survive greylisting repeatedly; this +avoids most of the delays and most of the database pollution problem. + + 1 /etc/postfix/main.cf: + 2 smtpd_recipient_restrictions = + 3 reject_unlisted_recipient + 4 ... + 5 reject_unauth_destination + 6 check_sender_access hash:/etc/postfix/sender_access + 7 check_policy_service unix:private/policy + 8 ... + 9 + 10 /etc/postfix/sender_access: + 11 securityfocus.com OK + 12 ... + +NOTES: + + * Line 7: On Solaris < 9, or Postfix < 2.10 on any Solaris version, use inet: + style sockets instead of unix: style, as detailed in the "Example: greylist + policy server" section above. + + * Line 5: reject_unauth_destination is not needed here if the mail relay + policy is specified with smtpd_relay_restrictions (available with Postfix + 2.10 and later). + + * Lines 6-7: Be sure to specify check_sender_access and check_policy_service + AFTER reject_unauth_destination or else your system could become an open + mail relay. + + * Line 3: The greylist database gets polluted quickly with bogus addresses. + It helps if you precede greylist lookups with restrictions that reject + unknown senders and/or recipients. + +RRoouuttiinnee ggrreeyylliisstt mmaaiinntteennaannccee + +The greylist database grows over time, because the greylist server never +removes database entries. If left unattended, the greylist database will +eventually run your file system out of space. + +When the status file size exceeds some threshold you can simply rename or +remove the file without adverse effects; Postfix automatically creates a new +file. In the worst case, new mail will be delayed by an hour or so. To lessen +the impact, rename or remove the file in the middle of the night at the +beginning of a weekend. + +EExxaammppllee PPeerrll ggrreeyylliisstt sseerrvveerr + +This is the Perl subroutine that implements the example greylist policy. It is +part of a general purpose sample policy server that is distributed with the +Postfix source as examples/smtpd-policy/greylist.pl. + +# +# greylist status database and greylist time interval. DO NOT create the +# greylist status database in a world-writable directory such as /tmp +# or /var/tmp. DO NOT create the greylist database in a file system +# that can run out of space. +# +$database_name="/var/mta/greylist.db"; +$greylist_delay=60; + +# +# Auto-allowlist threshold. Specify 0 to disable, or the number of +# successful "come backs" after which a client is no longer subject +# to greylisting. +# +$auto_allowlist_threshold = 10; + +# +# Demo SMTPD access policy routine. The result is an action just like +# it would be specified on the right-hand side of a Postfix access +# table. Request attributes are available via the %attr hash. +# +sub smtpd_access_policy { + my($key, $time_stamp, $now); + + # Open the database on the fly. + open_database() unless $database_obj; + + # Search the auto-allowlist. + if ($auto_allowlist_threshold > 0) { + $count = read_database($attr{"client_address"}); + if ($count > $auto_allowlist_threshold) { + return "dunno"; + } + } + + # Lookup the time stamp for this client/sender/recipient. + $key = + lc $attr{"client_address"}."/".$attr{"sender"}."/".$attr{"recipient"}; + $time_stamp = read_database($key); + $now = time(); + + # If new request, add this client/sender/recipient to the database. + if ($time_stamp == 0) { + $time_stamp = $now; + update_database($key, $time_stamp); + } + + # The result can be any action that is allowed in a Postfix access(5) map. + # + # To label the mail, return ``PREPEND headername: headertext'' + # + # In case of success, return ``DUNNO'' instead of ``OK'', so that the + # check_policy_service restriction can be followed by other restrictions. + # + # In case of failure, return ``DEFER_IF_PERMIT optional text...'', + # so that mail can still be blocked by other access restrictions. + # + syslog $syslog_priority, "request age %d", $now - $time_stamp if $verbose; + if ($now - $time_stamp > $greylist_delay) { + # Update the auto-allowlist. + if ($auto_allowlist_threshold > 0) { + update_database($attr{"client_address"}, $count + 1); + } + return "dunno"; + } else { + return "defer_if_permit Service temporarily unavailable"; + } +} + diff --git a/README_FILES/SMTPD_PROXY_README b/README_FILES/SMTPD_PROXY_README new file mode 100644 index 0000000..2b0dd91 --- /dev/null +++ b/README_FILES/SMTPD_PROXY_README @@ -0,0 +1,263 @@ +PPoossttffiixx BBeeffoorree--QQuueeuuee CCoonntteenntt FFiilltteerr + +------------------------------------------------------------------------------- + +WWAARRNNIINNGG + +The before-queue content filtering feature described in this document limits +the amount of mail that a site can handle. See the "Pros and Cons" section +below for details. + +TThhee PPoossttffiixx bbeeffoorree--qquueeuuee ccoonntteenntt ffiilltteerr ffeeaattuurree + +As of version 2.1, the Postfix SMTP server can forward all incoming mail to a +content filtering proxy server that inspects all mail BEFORE it is stored in +the Postfix mail queue. It is roughly equivalent in capabilities to the +approach described in MILTER_README, except that the latter uses a dedicated +protocol instead of SMTP. + +The before-queue content filter is meant to be used as follows: + + Postfix BBeeffoorree Postfix Postfix Postfix smtp + Internet -> SMTP -> qquueeuuee -> SMTP -> cleanup -> queue -< local + server ffiilltteerr server server virtual + +The before-queue content filter is not to be confused with the approach +described in the FILTER_README document, where mail is filtered AFTER it is +stored in the Postfix mail queue. + +This document describes the following topics: + + * Principles of operation + * Pros and cons of before-queue content filtering + * Configuring the Postfix SMTP pass-through proxy feature + * Configuration parameters + * How Postfix talks to the before-queue content filter + +PPrriinncciipplleess ooff ooppeerraattiioonn + +As shown in the diagram above, the before-queue filter sits between two Postfix +SMTP server processes. + + * The before-filter Postfix SMTP server accepts connections from the Internet + and does the usual relay access control, SASL authentication, TLS + negotiation, RBL lookups, rejecting non-existent sender or recipient + addresses, etc. + + * The before-queue filter receives unfiltered mail content from Postfix and + does one of the following: + + 1. Re-inject the mail back into Postfix via SMTP, perhaps after changing + its content and/or destination. + + 2. Discard or quarantine the mail. + + 3. Reject the mail by sending a suitable SMTP status code back to Postfix. + Postfix passes the status back to the remote SMTP client. This way, + Postfix does not have to send a bounce message. + + * The after-filter Postfix SMTP server receives mail from the content filter. + From then on Postfix processes the mail as usual. + +The before-queue content filter described here works just like the after-queue +content filter described in the FILTER_README document. In many cases you can +use the same software, within the limitations as discussed in the "Pros and +Cons" section below. + +PPrrooss aanndd ccoonnss ooff bbeeffoorree--qquueeuuee ccoonntteenntt ffiilltteerriinngg + + * Pro: Postfix can reject mail before the incoming SMTP mail transfer + completes, so that Postfix does not have to send rejected mail back to the + sender (which is usually forged anyway). Mail that is not accepted remains + the responsibility of the remote SMTP client. + + * Con: The smtpd(8) service before the smtpd_proxy_filter cannot support + features that involve header or body access, or that involve queue file + manipulation (i.e., anything that involves processing by the cleanup(8) + service). + + o No support for HOLD actions in Postfix smtpd access(5) restrictions. + + o No support for smtpd_milters features that involve message header or + body content. + + o No support for receive_override_options. + + Instead, specify those features with the smtpd(8) service behind the + smtpd_proxy_filter. In some cases, it may be possible to combine a before- + filter PREPEND action that emits a unique pattern (for example containing + the MTA domain name), with an after-filter header_checks action that does + what you want, and with an smtp_header_checks IGNORE action that deletes + the prepended header from transit mail. + + * Con: The remote SMTP client expects an SMTP reply within a deadline. As the + system load increases, fewer and fewer CPU cycles remain available to + answer within the deadline, and eventually you either have to stop + accepting mail or you have to stop filtering mail. It is for this reason + that the before-queue content filter limits the amount of mail that a site + can handle. + + * Con: Content filtering software can use lots of memory resources. You have + to reduce the number of simultaneous content filter processes so that a + burst of mail will not drive your system into the ground. + + o With Postfix versions 2.7 and later, SMTP clients will experience an + increase in the delay between the time the client sends "end-of- + message" and the time the Postfix SMTP server replies (here, the number + of before-filter SMTP server processes can be larger than the number of + filter processes). + + o With Postfix versions before 2.7, SMTP clients will experience an + increase in the delay before they can receive service (here, the number + of before-filter SMTP server processes is always equal to the number of + filter processes). + +CCoonnffiigguurriinngg tthhee PPoossttffiixx SSMMTTPP ppaassss--tthhrroouugghh pprrooxxyy ffeeaattuurree + +In the following example, the before-filter Postfix SMTP server gives mail to a +content filter that listens on localhost port 10025. The after-filter Postfix +SMTP server receives mail from the content filter via localhost port 10026. +From then on mail is processed as usual. + +The content filter itself is not described here. You can use any filter that is +SMTP enabled. For non-SMTP capable content filtering software, Bennett Todd's +SMTP proxy implements a nice Perl-based framework. See: https:// +web.archive.org/web/20151022025756/http://bent.latency.net/smtpprox/ or https:/ +/github.com/jnorell/smtpprox/ + + Postfix + Postfix filter on SMTP server Postfix Postfix + Internet -> SMTP server -> localhost -> on -> cleanup -> incoming + on port 25 port 10025 localhost server queue + port 10026 + +This is configured by editing the master.cf file: + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + # + # Before-filter SMTP server. Receive mail from the network and + # pass it to the content filter on localhost port 10025. + # + smtp inet n - n - 20 smtpd + -o smtpd_proxy_filter=127.0.0.1:10025 + -o smtpd_client_connection_count_limit=10 + # Postfix 2.7 and later performance feature. + # -o smtpd_proxy_options=speed_adjust + # + # After-filter SMTP server. Receive mail from the content filter + # on localhost port 10026. + # + 127.0.0.1:10026 inet n - n - - smtpd + -o smtpd_authorized_xforward_hosts=127.0.0.0/8 + -o smtpd_client_restrictions= + -o smtpd_helo_restrictions= + -o smtpd_sender_restrictions= + # Postfix 2.10 and later: specify empty smtpd_relay_restrictions. + -o smtpd_relay_restrictions= + -o smtpd_recipient_restrictions=permit_mynetworks,reject + -o smtpd_data_restrictions= + -o mynetworks=127.0.0.0/8 + -o receive_override_options=no_unknown_recipient_checks + +Note: do not specify spaces around the "=" or "," characters. + +The before-filter SMTP server entry is a modified version of the default +Postfix SMTP server entry that is normally configured at the top of the +master.cf file: + + * The number of SMTP sessions is reduced from the default 100 to only 20. + This prevents a burst of mail from running your system into the ground with + too many content filter processes. + + * The "-o smtpd_client_connection_count_limit=10" prevents one SMTP client + from using up all 20 SMTP server processes. This limit is not necessary if + you receive all mail from a trusted relay host. + + Note: this setting is available in Postfix version 2.2 and later. Earlier + Postfix versions will ignore it. + + * The "-o smtpd_proxy_filter=127.0.0.1:10025" tells the before-filter SMTP + server that it should give incoming mail to the content filter that listens + on localhost TCP port 10025. + + * The "-o smtpd_proxy_options=speed_adjust" tells the before-filter SMTP + server that it should receive an entire email message before it connects to + a content filter. This reduces the number of simultaneous filter processes. + + NOTE 1: When this option is turned on, a content filter must not + selectively reject recipients of a multi-recipient message. Rejecting all + recipients is OK, as is accepting all recipients. + + NOTE 2: This feature increases the minimum amount of free queue space by + $message_size_limit. The extra space is needed to save the message to a + temporary file. + + * Postfix >= 2.3 supports both TCP and UNIX-domain filters. The above filter + could be specified as "inet:127.0.0.1:10025". To specify a UNIX-domain + filter, specify "unix:pathname". A relative pathname is interpreted + relative to the Postfix queue directory. + +The after-filter SMTP server is a new master.cf entry: + + * The "127.0.0.1:10026" makes the after-filter SMTP server listen on the + localhost address only, without exposing it to the network. NEVER expose + the after-filter SMTP server to the Internet :-) + + * The "-o smtpd_authorized_xforward_hosts=127.0.0.0/8" allows the after- + filter SMTP server to receive remote SMTP client information from the + before-filter SMTP server, so that the after-filter Postfix daemons log the + remote SMTP client information instead of logging localhost[127.0.0.1]. + + * The other after-filter SMTP server settings avoid duplication of work that + is already done in the "before filter" SMTP server. + +By default, the filter has 100 seconds to do its work. If it takes longer then +Postfix gives up and reports an error to the remote SMTP client. You can +increase this time limit (see the "Configuration parameters" section below) but +doing so is pointless because you can't control when the remote SMTP client +times out. + +CCoonnffiigguurraattiioonn ppaarraammeetteerrss + +Parameters that control proxying: + + * smtpd_proxy_filter (syntax: host:port): The host and TCP port of the + before-queue content filter. When no host or host: is specified here, + localhost is assumed. + + * smtpd_proxy_timeout (default: 100s): Timeout for connecting to the before- + queue content filter and for sending and receiving commands and data. All + proxy errors are logged to the maillog file. For privacy reasons, all the + remote SMTP client sees is "451 Error: queue file write error". It would + not be right to disclose internal details to strangers. + + * smtpd_proxy_ehlo (default: $myhostname): The hostname to use when sending + an EHLO command to the before-queue content filter. + +HHooww PPoossttffiixx ttaallkkss ttoo tthhee bbeeffoorree--qquueeuuee ccoonntteenntt ffiilltteerr + +The before-filter Postfix SMTP server connects to the content filter, delivers +one message, and disconnects. While sending mail into the content filter, +Postfix speaks ESMTP but uses no command pipelining. Postfix generates its own +EHLO, XFORWARD (for logging the remote client IP address instead of localhost +[127.0.0.1]), DATA and QUIT commands, and forwards unmodified copies of all the +MAIL FROM and RCPT TO commands that the before-filter Postfix SMTP server +didn't reject itself. Postfix sends no other SMTP commands. + +The content filter should accept the same MAIL FROM and RCPT TO command syntax +as the before-filter Postfix SMTP server, and should forward the commands +without modification to the after-filter SMTP server. If the content filter or +after-filter SMTP server does not support all the ESMTP features that the +before-filter Postfix SMTP server supports, then the missing features must be +turned off in the before-filter Postfix SMTP server with the +smtpd_discard_ehlo_keywords parameter. + +When the filter rejects content, it should send a negative SMTP response back +to the before-filter Postfix SMTP server, and it should abort the connection +with the after-filter Postfix SMTP server without completing the SMTP +conversation with the after-filter Postfix SMTP server. + diff --git a/README_FILES/SMTPUTF8_README b/README_FILES/SMTPUTF8_README new file mode 100644 index 0000000..9e7970e --- /dev/null +++ b/README_FILES/SMTPUTF8_README @@ -0,0 +1,294 @@ + PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document describes Postfix support for Email Address Internationalization +(EAI) as defined in RFC 6531 (SMTPUTF8 extension), RFC 6532 (Internationalized +email headers) and RFC 6533 (Internationalized delivery status notifications). +Introduced with Postfix version 3.0, this fully supports UTF-8 email addresses +and UTF-8 message header values. + +Topics covered in this document: + + * Building with/without SMTPUTF8 support + * Enabling Postfix SMTPUTF8 support + * Using Postfix SMTPUTF8 support + * SMTPUTF8 autodetection + * Limitations of the current implementation + * Compatibility with pre-SMTPUTF8 environments + * Compatibility with IDNA2003 + * Credits + +BBuuiillddiinngg PPoossttffiixx wwiitthh//wwiitthhoouutt SSMMTTPPUUTTFF88 ssuuppppoorrtt + +Postfix will build with SMTPUTF8 support if the ICU version >= 46 library and +header files are installed on the system. The package name varies with the OS +distribution. The table shows package names for a number of platforms at the +time this text was written. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |OOSS DDiissttrriibbuuttiioonn |PPaacckkaaggee | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |FreeBSD, NetBSD, etc.|icu | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |Centos, Fedora, RHEL |libicu-devel| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + |Debian, Ubuntu |libicu-dev | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ | + +To force Postfix to build without SMTPUTF8, specify: + + $ mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDNNOO__EEAAII ......"" + +See the INSTALL document for more "make makefiles" options. + +EEnnaabblliinngg PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt + +There is more to SMTPUTF8 than just Postfix itself. The rest of your email +infrastructure also needs to be able to handle UTF-8 email addresses and +message header values. This includes SMTPUTF8 protocol support in SMTP-based +content filters (Amavisd), LMTP servers (Dovecot), and down-stream SMTP +servers. + +Postfix SMTPUTF8 support is enabled by default, but it may be disabled as part +of a backwards-compatibility safety net (see the COMPATIBILITY_README file). + +SMTPUTF8 support is enabled by setting the smtputf8_enable parameter in +main.cf: + + # ppoossttccoonnff ""ssmmttppuuttff88__eennaabbllee == yyeess"" + # ppoossttffiixx rreellooaadd + +(With Postfix <= 3.1, you may also need to specify "ooppttiioonn__ggrroouupp == cclliieenntt" in +Postfix MySQL client files, to enable UTF8 support in MySQL queries. This +setting is the default as of Postfix 3.2.) + +With SMTPUTF8 support enabled, Postfix changes behavior with respect to earlier +Postfix releases: + + * UTF-8 is permitted in the myorigin parameter value. However, the myhostname + and mydomain parameters must currently specify ASCII-only domain names. + This limitation may be removed later. + + * UTF-8 is the only form of non-ASCII text that Postfix supports in access + tables, address rewriting tables, and other tables that are indexed with an + email address, hostname, or domain name. + + * The header_checks-like and body_checks-like features are not UTF-8 enabled, + and therefore they do not enforce UTF-8 syntax rules on inputs and outputs. + The reason is that non-ASCII text may be sent in encodings other than UTF- + 8, and that real email sometimes contains malformed headers. Instead of + skipping non-UTF-8 content, Postfix should be able to filter it. You may + try to enable UTF-8 processing by starting a PCRE pattern with the sequence + (*UTF8), but this is will result in "message not accepted, try again later" + errors when the PCRE pattern matcher encounters non-UTF-8 input. Other + features that are not UTF-8 enabled are smtpd_command_filter, + smtp_reply_filter, the *_delivery_status_filter features, and the + *_dns_reply_filter features (the latter because DNS is by definition an + ASCII protocol). + + * The Postfix SMTP server announces SMTPUTF8 support in the EHLO response. + + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-VRFY + 250-ETRN + 250-STARTTLS + 250-AUTH PLAIN LOGIN + 250-ENHANCEDSTATUSCODES + 250-8BITMIME + 250-DSN + 250 SMTPUTF8 + + * The Postfix SMTP server accepts the SMTPUTF8 request in MAIL FROM and VRFY + commands. + + MMAAIILL FFRROOMM::<<aaddddrreessss>> SSMMTTPPUUTTFF88 ...... + + VVRRFFYY aaddddrreessss SSMMTTPPUUTTFF88 + + * The Postfix SMTP client may issue the SMTPUTF8 request in MAIL FROM + commands. + + * The Postfix SMTP server accepts UTF-8 in email address domains, but only + after the remote SMTP client issues the SMTPUTF8 request in MAIL FROM or + VRFY commands. + +Postfix already permitted UTF-8 in message header values and in address +localparts. This does not change. + +UUssiinngg PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt + +After Postfix SMTPUTF8 support is turned on, Postfix behavior will depend on 1) +whether a remote SMTP client requests SMTPUTF8 support, 2) the presence of UTF- +8 content in the message envelope and headers, and 3) whether a down-stream +SMTP (or LMTP) server announces SMTPUTF8 support. + + * When the Postfix SMTP server receives a message WITHOUT the SMTPUTF8 + request, Postfix handles the message as it has always done (at least that + is the default, see autodetection below). Specifically, the Postfix SMTP + server does not accept UTF-8 in the envelope sender domain name or envelope + recipient domain name, and the Postfix SMTP client does not issue the + SMTPUTF8 request when delivering that message to an SMTP or LMTP server + that announces SMTPUTF8 support (again, that is the default). Postfix will + accept UTF-8 in message header values and in the localpart of envelope + sender and recipient addresses, because it has always done that. + + * When the Postfix SMTP server receives a message WITH the SMTPUTF8 request, + Postfix will issue the SMTPUTF8 request when delivering that message to an + SMTP or LMTP server that announces SMTPUTF8 support. This is not + configurable. + + * When a message is received with the SMTPUTF8 request, Postfix will deliver + the message to a non-SMTPUTF8 SMTP or LMTP server ONLY if: + + o No message header value contains UTF-8. + + o The envelope sender address contains no UTF-8, + + o No envelope recipient address for that specific SMTP/LMTP delivery + transaction contains UTF-8. + + NOTE: Recipients in other email delivery transactions for that same + message may still contain UTF-8. + + Otherwise, Postfix will return the recipient(s) for that email delivery + transaction as undeliverable. The delivery status notification message will + be an SMTPUTF8 message. It will therefore be subject to the same + restrictions as email that is received with the SMTPUTF8 request. + + * When the Postfix SMTP server receives a message with the SMTPUTF8 request, + that request also applies after the message is forwarded via a virtual or + local alias, or $HOME/.forward file. + +SSMMTTPPUUTTFF88 aauuttooddeetteeccttiioonn + +This section applies only to systems that have SMTPUTF8 support turned on +(smtputf8_enable = yes). + +For compatibility with pre-SMTPUTF8 environments, Postfix does not +automatically set the "SMTPUTF8 requested" flag on messages from non-SMTPUTF8 +clients that contain a UTF-8 header value or UTF-8 address localpart. This +would make such messages undeliverable to non-SMTPUTF8 servers, and could be a +barrier to SMTPUTF8 adoption. + +By default, Postfix sets the "SMTPUTF8 requested" flag only on address +verification probes and on Postfix sendmail submissions that contain UTF-8 in +the sender address, UTF-8 in a recipient address, or UTF-8 in a message header +value. + + /etc/postfix/main.cf: + smtputf8_autodetect_classes = sendmail, verify + +However, if you have a non-ASCII myorigin or mydomain setting, or if you have a +configuration that introduces UTF-8 addresses with virtual aliases, canonical +mappings, or BCC mappings, then you may have to apply SMTPUTF8 autodetection to +all email: + + /etc/postfix/main.cf: + smtputf8_autodetect_classes = all + +This will, of course, also flag email that was received without SMTPUTF8 +request, but that contains UTF-8 in a sender address localpart, receiver +address localpart, or message header value. Such email was not standards- +compliant, but Postfix would have delivered it if SMTPUTF8 support was +disabled. + +LLiimmiittaattiioonnss ooff tthhee ccuurrrreenntt iimmpplleemmeennttaattiioonn + +The Postfix implementation is a work in progress; limitations are steadily +being removed. The text below describes the situation at one point in time. + +NNoo aauuttoommaattiicc ccoonnvveerrssiioonnss bbeettwweeeenn AASSCCIIII aanndd UUTTFF--88 ddoommaaiinn nnaammeess.. + +Some background: According to RFC 6530 and related documents, an +internationalized domain name can appear in two forms: the UTF-8 form, and the +ASCII (xn--mumble) form. An internationalized address localpart must be encoded +in UTF-8; the RFCs do not define an ASCII alternative form. + +Postfix currently does not convert internationalized domain names from UTF- +8 into ASCII (or from ASCII into UTF-8) before using domain names in SMTP +commands and responses, before looking up domain names in lists such as +mydestination, relay_domains or in lookup tables such as access tables, etc., +before using domain names in a policy daemon or Milter request, or before +logging events. + +Postfix does, however, casefold domain names and email addresses before +matching them against a Postfix configuration parameter or lookup table. + +In order to use Postfix SMTPUTF8 support: + + * The Postfix parameters myhostname and mydomain must be in ASCII form. One + is a substring of the other, and the myhostname value is used in SMTP + commands and responses that require ASCII. The parameter myorigin (added to + local addresses without domain) supports UTF-8. + + * You need to configure both the ASCII and UTF-8 forms of an + Internationalized domain name in Postfix parameters such as mydestination + and relay_domains, as well as lookup table search keys. + + * Milters, content filters, policy servers and logfile analysis tools need to + be able to handle both the ASCII and UTF-8 forms of Internationalized + domain names. + +CCoommppaattiibbiilliittyy wwiitthh pprree--SSMMTTPPUUTTFF88 eennvviirroonnmmeennttss + +MMaaiilliinngg lliissttss wwiitthh UUTTFF--88 aanndd nnoonn--UUTTFF--88 ssuubbssccrriibbeerrss + +With Postfix, there is no need to split mailing lists into UTF-8 and non-UTF- +8 members. Postfix will try to deliver the non-UTF8 subscribers over +"traditional" non-SMTPUTF8 sessions, as long as the message has an ASCII +envelope sender address and all-ASCII header values. The mailing list manager +may have to apply RFC 2047 encoding to satisfy that last condition. + +PPrree--eexxiissttiinngg nnoonn--AASSCCIIII eemmaaiill fflloowwss + +With "smtputf8_enable = no", Postfix handles email with non-ASCII in address +localparts (and in headers) as before. The vast majority of email software is +perfectly capable of handling such email, even if pre-SMTPUTF8 standards do not +support such practice. + +RReejjeeccttiinngg nnoonn--UUTTFF88 aaddddrreesssseess + +With "smtputf8_enable = yes", Postfix requires that non-ASCII address +information is encoded in UTF-8 and will reject other encodings such as ISO- +8859. It is not practical for Postfix to support multiple encodings at the same +time. There is no problem with RFC 2047 encodings such as "=?ISO-8859- +1?Q?text?=", because those use only characters from the ASCII characterset. + +RReejjeeccttiinngg nnoonn--AASSCCIIII aaddddrreesssseess iinn nnoonn--SSMMTTPPUUTTFF88 ttrraannssaaccttiioonnss + +Setting "strict_smtputf8 = yes" in addition to "smtputf8_enable = yes" will +enable stricter enforcement of the SMTPUTF8 protocol. Specifically, the Postfix +SMTP server will not only reject non-UTF8 sender or recipient addresses, it +will in addition accept UTF-8 sender or recipient addresses only when the +client requests an SMTPUTF8 mail transaction. + +CCoommppaattiibbiilliittyy wwiitthh IIDDNNAA22000033 + +Postfix >= 3.2 by default disables the 'transitional' compatibility between +IDNA2003 and IDNA2008, when converting UTF-8 domain names to/from the ASCII +form that is used in DNS lookups. This makes Postfix behavior consistent with +current versions of the Firefox and Chrome web browsers. Specify +"enable_idna2003_compatibility = yes" to get the historical behavior. + +This affects the conversion of domain names that contain for example the German +sz (ß) and the Greek zeta (ς). See http://unicode.org/cldr/utility/idna.jsp +for more examples. + +CCrreeddiittss + + * May 15, 2014: Arnt Gulbrandsen posted his patch for Unicode email support. + This work was sponsored by CNNIC. + + * July 15, 2014: Wietse integrated Arnt Gulbrandsen's code and released + Postfix with SMTPUTF8 support. + + * January 2015: Wietse added UTF-8 support for casefolding in Postfix lookup + tables and caseless string comparison in Postfix list-based features. + diff --git a/README_FILES/SOHO_README b/README_FILES/SOHO_README new file mode 100644 index 0000000..bbba7c2 --- /dev/null +++ b/README_FILES/SOHO_README @@ -0,0 +1,288 @@ +PPoossttffiixx SSmmaallll//HHoommee OOffffiiccee HHiinnttss aanndd TTiippss + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document combines hints and tips for "small office/home office" +applications into one document so that they are easier to find. The text +describes the mail sending side only. If your machine does not receive mail +directly (i.e. it does not have its own Internet domain name and its own fixed +IP address), then you will need a solution such as "fetchmail", which is +outside the scope of the Postfix documentation. + + * Selected topics from the STANDARD_CONFIGURATION_README document: + + o Postfix on a stand-alone Internet host + o Postfix on hosts without a real Internet hostname + + Selected topics from the SASL_README document: + + o Enabling SASL authentication in the Postfix SMTP client + o Configuring Sender-Dependent SASL authentication + +See the SASL_README and STANDARD_CONFIGURATION_README documents for further +information on these topics. + +PPoossttffiixx oonn aa ssttaanndd--aalloonnee IInntteerrnneett hhoosstt + +Postfix should work out of the box without change on a stand-alone machine that +has direct Internet access. At least, that is how Postfix installs when you +download the Postfix source code via http://www.postfix.org/. + +You can use the command "ppoossttccoonnff --nn" to find out what settings are overruled +by your main.cf. Besides a few pathname settings, few parameters should be set +on a stand-alone box, beyond what is covered in the BASIC_CONFIGURATION_README +document: + + /etc/postfix/main.cf: + # Optional: send mail as user@domainname instead of user@hostname. + #myorigin = $mydomain + + # Optional: specify NAT/proxy external address. + #proxy_interfaces = 1.2.3.4 + + # Alternative 1: don't relay mail from other hosts. + mynetworks_style = host + relay_domains = + + # Alternative 2: relay mail from local clients only. + # mynetworks = 192.168.1.0/28 + # relay_domains = + +See also the section "Postfix on hosts without a real Internet hostname" if +this is applicable to your configuration. + +PPoossttffiixx oonn hhoossttss wwiitthhoouutt aa rreeaall IInntteerrnneett hhoossttnnaammee + +This section is for hosts that don't have their own Internet hostname. +Typically these are systems that get a dynamic IP address via DHCP or via +dialup. Postfix will let you send and receive mail just fine between accounts +on a machine with a fantasy name. However, you cannot use a fantasy hostname in +your email address when sending mail into the Internet, because no-one would be +able to reply to your mail. In fact, more and more sites refuse mail addresses +with non-existent domain names. + +Note: the following information is Postfix version dependent. To find out what +Postfix version you have, execute the command "ppoossttccoonnff mmaaiill__vveerrssiioonn". + +SSoolluuttiioonn 11:: PPoossttffiixx vveerrssiioonn 22..22 aanndd llaatteerr + +Postfix 2.2 uses the generic(5) address mapping to replace local fantasy email +addresses by valid Internet addresses. This mapping happens ONLY when mail +leaves the machine; not when you send mail between users on the same machine. + +The following example presents additional configuration. You need to combine +this with basic configuration information as discussed in the first half of +this document. + + 1 /etc/postfix/main.cf: + 2 smtp_generic_maps = hash:/etc/postfix/generic + 3 + 4 /etc/postfix/generic: + 5 his@localdomain.local hisaccount@hisisp.example + 6 her@localdomain.local heraccount@herisp.example + 7 @localdomain.local hisaccount+local@hisisp.example + +When mail is sent to a remote host via SMTP: + + * Line 5 replaces his@localdomain.local by his ISP mail address, + + * Line 6 replaces her@localdomain.local by her ISP mail address, and + + * Line 7 replaces other local addresses by his ISP account, with an address + extension of +local (this example assumes that the ISP supports "+" style + address extensions). + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ggeenneerriicc" whenever you change the +generic table. + +SSoolluuttiioonn 22:: PPoossttffiixx vveerrssiioonn 22..11 aanndd eeaarrlliieerr + +The solution with older Postfix systems is to use valid Internet addresses +where possible, and to let Postfix map valid Internet addresses to local +fantasy addresses. With this, you can send mail to the Internet and to local +fantasy addresses, including mail to local fantasy addresses that don't have a +valid Internet address of their own. + +The following example presents additional configuration. You need to combine +this with basic configuration information as discussed in the first half of +this document. + + 1 /etc/postfix/main.cf: + 2 myhostname = hostname.localdomain + 3 mydomain = localdomain + 4 + 5 canonical_maps = hash:/etc/postfix/canonical + 6 + 7 virtual_alias_maps = hash:/etc/postfix/virtual + 8 + 9 /etc/postfix/canonical: + 10 your-login-name your-account@your-isp.com + 11 + 12 /etc/postfix/virtual: + 13 your-account@your-isp.com your-login-name + +Translation: + + * Lines 2-3: Substitute your fantasy hostname here. Do not use a domain name + that is already in use by real organizations on the Internet. See RFC 2606 + for examples of domain names that are guaranteed not to be owned by anyone. + + * Lines 5, 9, 10: This provides the mapping from "your-login- + name@hostname.localdomain" to "your-account@your-isp.com". This part is + required. + + * Lines 7, 12, 13: Deliver mail for "your-account@your-isp.com" locally, + instead of sending it to the ISP. This part is not required but is + convenient. + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ccaannoonniiccaall" whenever you change the +canonical table. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" whenever you change the +virtual table. + +EEnnaabblliinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP//LLMMTTPP cclliieenntt + +This section shows a typical scenario where the Postfix SMTP client sends all +messages via a mail gateway server that requires SASL authentication. + + TTrroouubbllee ssoollvviinngg ttiippss:: + + * If your SASL logins fail with "SASL authentication failure: No worthy + mechs found" in the mail logfile, then see the section "Postfix SMTP/ + LMTP client policy - SASL mechanism pprrooppeerrttiieess". + + * For a solution to a more obscure class of SASL authentication failures, + see "Postfix SMTP/LMTP client policy - SASL mechanism nnaammeess". + +To make the example more readable we introduce it in two parts. The first part +takes care of the basic configuration, while the second part sets up the +username/password information. + + /etc/postfix/main.cf: + smtp_sasl_auth_enable = yes + smtp_tls_security_level = encrypt + smtp_sasl_tls_security_options = noanonymous + relayhost = [mail.isp.example] + # Alternative form: + # relayhost = [mail.isp.example]:submission + smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd + + * The smtp_sasl_auth_enable setting enables client-side authentication. We + will configure the client's username and password information in the second + part of the example. + + * The smtp_tls_security_level setting ensures that the connection to the + remote smtp server will be encrypted, and smtp_sasl_tls_security_options + removes the prohibition on plaintext passwords. + + * The relayhost setting forces the Postfix SMTP to send all remote messages + to the specified mail server instead of trying to deliver them directly to + their destination. + + * In the relayhost setting, the "[" and "]" prevent the Postfix SMTP client + from looking up MX (mail exchanger) records for the enclosed name. + + * The relayhost destination may also specify a non-default TCP port. For + example, the alternative form [mail.isp.example]:submission tells Postfix + to connect to TCP network port 587, which is reserved for email client + applications. + + * The Postfix SMTP client is compatible with SMTP servers that use the non- + standard "AUTH=mmeetthhoodd....." syntax in response to the EHLO command; this + requires no additional Postfix client configuration. + + * With the setting "smtp_tls_wrappermode = yes", the Postfix SMTP client + supports the "wrappermode" protocol, which uses TCP port 465 on the SMTP + server (Postfix 3.0 and later). + + * With the smtp_sasl_password_maps parameter, we configure the Postfix SMTP + client to send username and password information to the mail gateway + server. As discussed in the next section, the Postfix SMTP client supports + multiple ISP accounts. For this reason the username and password are stored + in a table that contains one username/password combination for each mail + gateway server. + + /etc/postfix/sasl_passwd: + # destination credentials + [mail.isp.example] username:password + # Alternative form: + # [mail.isp.example]:submission username:password + + IImmppoorrttaanntt + + Keep the SASL client password file in /etc/postfix, and make the file + read+write only for root to protect the username/password combinations + against other users. The Postfix SMTP client will still be able to read the + SASL client passwords. It opens the file as user root before it drops + privileges, and before entering an optional chroot jail. + + * Use the postmap command whenever you change the /etc/postfix/sasl_passwd + file. + + * If you specify the "[" and "]" in the relayhost destination, then you must + use the same form in the smtp_sasl_password_maps file. + + * If you specify a non-default TCP Port (such as ":submission" or ":587") in + the relayhost destination, then you must use the same form in the + smtp_sasl_password_maps file. + +CCoonnffiigguurriinngg SSeennddeerr--DDeeppeennddeenntt SSAASSLL aauutthheennttiiccaattiioonn + +Postfix supports different ISP accounts for different sender addresses (version +2.3 and later). This can be useful when one person uses the same machine for +work and for personal use, or when people with different ISP accounts share the +same Postfix server. + +To make this possible, Postfix supports per-sender SASL passwords and per- +sender relay hosts. In the example below, the Postfix SMTP client will search +the SASL password file by sender address before it searches that same file by +destination. Likewise, the Postfix trivial-rewrite(8) daemon will search the +per-sender relayhost file, and use the default relayhost setting only as a +final resort. + + /etc/postfix/main.cf: + smtp_sender_dependent_authentication = yes + sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay + smtp_sasl_auth_enable = yes + smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd + relayhost = [mail.isp.example] + # Alternative form: + # relayhost = [mail.isp.example]:submission + + /etc/postfix/sasl_passwd: + # Per-sender authentication; see also /etc/postfix/sender_relay. + user1@example.com username1:password1 + user2@example.net username2:password2 + # Login information for the default relayhost. + [mail.isp.example] username:password + # Alternative form: + # [mail.isp.example]:submission username:password + + /etc/postfix/sender_relay: + # Per-sender provider; see also /etc/postfix/sasl_passwd. + user1@example.com [mail.example.com]:submission + user2@example.net [mail.example.net] + + * If you are creative, then you can try to combine the two tables into one + single MySQL database, and configure different Postfix queries to extract + the appropriate information. + + * Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb + files. To find out what lookup tables Postfix supports, use the command + "ppoossttccoonnff --mm". + + * Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ssaassll__ppaasssswwdd" whenever you change + the sasl_passwd table. + + * Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//sseennddeerr__rreellaayy" whenever you change + the sender_relay table. + diff --git a/README_FILES/SQLITE_README b/README_FILES/SQLITE_README new file mode 100644 index 0000000..f4e98f9 --- /dev/null +++ b/README_FILES/SQLITE_README @@ -0,0 +1,74 @@ +PPoossttffiixx SSQQLLiittee HHoowwttoo + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +The Postfix sqlite map type allows you to hook up Postfix to a SQLite database. +This implementation allows for multiple sqlite databases: you can use one for a +virtual(5) table, one for an access(5) table, and one for an aliases(5) table +if you want. + +BBuuiillddiinngg PPoossttffiixx wwiitthh SSQQLLiittee ssuuppppoorrtt + +The Postfix SQLite client utilizes the sqlite3 library, which can be obtained +from: + + http://www.sqlite.org/ + +In order to build Postfix with sqlite map support, you will need to add to +CCARGS the flags -DHAS_SQLITE and -I with the directory containing the sqlite +header files, and you will need to add to AUXLIBS the directory and name of the +sqlite3 library, plus the name of the standard POSIX thread library (pthread). +For example: + + make -f Makefile.init makefiles \ + "CCARGS=-DHAS_SQLITE -I/usr/local/include" \ + "AUXLIBS_SQLITE=-L/usr/local/lib -lsqlite3 -lpthread" + +If your SQLite shared library is in a directory that the RUN-TIME linker does +not know about, add a "-Wl,-R,/path/to/directory" option after "-lsqlite3". + +Postfix versions before 3.0 use AUXLIBS instead of AUXLIBS_SQLITE. With Postfix +3.0 and later, the old AUXLIBS variable still supports building a statically- +loaded SQLite database client, but only the new AUXLIBS_SQLITE variable +supports building a dynamically-loaded or statically-loaded SQLite database +client. + + Failure to use the AUXLIBS_SQLITE variable will defeat the purpose of + dynamic database client loading. Every Postfix executable file will have + SQLITE database library dependencies. And that was exactly what dynamic + database client loading was meant to avoid. + +Then, just run 'make'. + +UUssiinngg SSQQLLiittee ttaabblleess + +Once Postfix is built with sqlite support, you can specify a map type in +main.cf like this: + + alias_maps = sqlite:/etc/postfix/sqlite-aliases.cf + +The file /etc/postfix/sqlite-aliases.cf specifies lots of information telling +Postfix how to reference the sqlite database. For a complete description, see +the sqlite_table(5) manual page. + +EExxaammppllee:: llooccaall aalliiaasseess + +# +# sqlite config file for local(8) aliases(5) lookups +# + +# Path to database +dbpath = /some/path/to/sqlite_database + +# See sqlite_table(5) for details. +query = SELECT forw_addr FROM mxaliases WHERE alias='%s' AND status='paid' + +CCrreeddiittss + +SQLite support was added with Postfix version 2.8. + + * Implementation by Axel Steiner + * Documentation by Jesus Garcia Crespo + diff --git a/README_FILES/STANDARD_CONFIGURATION_README b/README_FILES/STANDARD_CONFIGURATION_README new file mode 100644 index 0000000..f3cdb58 --- /dev/null +++ b/README_FILES/STANDARD_CONFIGURATION_README @@ -0,0 +1,638 @@ +PPoossttffiixx SSttaannddaarrdd CCoonnffiigguurraattiioonn EExxaammpplleess + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +This document presents a number of typical Postfix configurations. This +document should be reviewed after you have followed the basic configuration +steps as described in the BASIC_CONFIGURATION_README document. In particular, +do not proceed here if you don't already have Postfix working for local mail +submission and for local mail delivery. + +The first part of this document presents standard configurations that each +solve one specific problem. + + * Postfix on a stand-alone Internet host + * Postfix on a null client + * Postfix on a local network + * Postfix email firewall/gateway + +The second part of this document presents additional configurations for hosts +in specific environments. + + * Delivering some but not all accounts locally + * Running Postfix behind a firewall + * Configuring Postfix as primary or backup MX host for a remote site + * Postfix on a dialup machine + * Postfix on hosts without a real Internet hostname + +PPoossttffiixx oonn aa ssttaanndd--aalloonnee IInntteerrnneett hhoosstt + +Postfix should work out of the box without change on a stand-alone machine that +has direct Internet access. At least, that is how Postfix installs when you +download the Postfix source code via http://www.postfix.org/. + +You can use the command "ppoossttccoonnff --nn" to find out what settings are overruled +by your main.cf. Besides a few pathname settings, few parameters should be set +on a stand-alone box, beyond what is covered in the BASIC_CONFIGURATION_README +document: + + /etc/postfix/main.cf: + # Optional: send mail as user@domainname instead of user@hostname. + #myorigin = $mydomain + + # Optional: specify NAT/proxy external address. + #proxy_interfaces = 1.2.3.4 + + # Alternative 1: don't relay mail from other hosts. + mynetworks_style = host + relay_domains = + + # Alternative 2: relay mail from local clients only. + # mynetworks = 192.168.1.0/28 + # relay_domains = + +See also the section "Postfix on hosts without a real Internet hostname" if +this is applicable to your configuration. + +PPoossttffiixx oonn aa nnuullll cclliieenntt + +A null client is a machine that can only send mail. It receives no mail from +the network, and it does not deliver any mail locally. A null client typically +uses POP, IMAP or NFS for mailbox access. + +In this example we assume that the Internet domain name is "example.com" and +that the machine is named "hostname.example.com". As usual, the examples show +only parameters that are not left at their default settings. + + 1 /etc/postfix/main.cf: + 2 myhostname = hostname.example.com + 3 myorigin = $mydomain + 4 relayhost = $mydomain + 5 inet_interfaces = loopback-only + 6 mydestination = + +Translation: + + * Line 2: Set myhostname to hostname.example.com, in case the machine name + isn't set to a fully-qualified domain name (use the command "postconf - + d myhostname" to find out what the machine name is). + + * Line 2: The myhostname value also provides the default value for the + mydomain parameter (here, "mydomain = example.com"). + + * Line 3: Send mail as "user@example.com" (instead of + "user@hostname.example.com"), so that nothing ever has a reason to send + mail to "user@hostname.example.com". + + * Line 4: Forward all mail to the mail server that is responsible for the + "example.com" domain. This prevents mail from getting stuck on the null + client if it is turned off while some remote destination is unreachable. + Specify a real hostname here if your "example.com" domain has no MX record. + + * Line 5: Do not accept mail from the network. + + * Line 6: Disable local mail delivery. All mail goes to the mail server as + specified in line 4. + +PPoossttffiixx oonn aa llooccaall nneettwwoorrkk + +This section describes a local area network environment of one main server and +multiple other systems that send and receive email. As usual we assume that the +Internet domain name is "example.com". All systems are configured to send mail +as "user@example.com", and all systems receive mail for +"user@hostname.example.com". The main server also receives mail for +"user@example.com". We call this machine by the name of mailhost.example.com. + +A drawback of sending mail as "user@example.com" is that mail for "root" and +other system accounts is also sent to the central mailhost. See the section +"Delivering some but not all accounts locally" below for possible solutions. + +As usual, the examples show only parameters that are not left at their default +settings. + +First we present the non-mailhost configuration, because it is the simpler one. +This machine sends mail as "user@example.com" and is the final destination for +"user@hostname.example.com". + + 1 /etc/postfix/main.cf: + 2 myorigin = $mydomain + 3 mynetworks = 127.0.0.0/8 10.0.0.0/24 + 4 relay_domains = + 5 # Optional: forward all non-local mail to mailhost + 6 #relayhost = $mydomain + +Translation: + + * Line 2: Send mail as "user@example.com". + + * Line 3: Specify the trusted networks. + + * Line 4: This host does not relay mail from untrusted networks. + + * Line 6: This is needed if no direct Internet access is available. See also + below, "Postfix behind a firewall". + +Next we present the mailhost configuration. This machine sends mail as +"user@example.com" and is the final destination for "user@hostname.example.com" +as well as "user@example.com". + + 1 DNS: + 2 example.com IN MX 10 mailhost.example.com. + 3 + 4 /etc/postfix/main.cf: + 5 myorigin = $mydomain + 6 mydestination = $myhostname localhost.$mydomain localhost $mydomain + 7 mynetworks = 127.0.0.0/8 10.0.0.0/24 + 8 relay_domains = + 9 # Optional: forward all non-local mail to firewall + 10 #relayhost = [firewall.example.com] + +Translation: + + * Line 2: Send mail for the domain "example.com" to the machine + mailhost.example.com. Remember to specify the "." at the end of the line. + + * Line 5: Send mail as "user@example.com". + + * Line 6: This host is the final mail destination for the "example.com" + domain, in addition to the names of the machine itself. + + * Line 7: Specify the trusted networks. + + * Line 8: This host does not relay mail from untrusted networks. + + * Line 10: This is needed only when the mailhost has to forward non-local + mail via a mail server on a firewall. The [] forces Postfix to do no MX + record lookups. + +In an environment like this, users access their mailbox in one or more of the +following ways: + + * Mailbox access via NFS or equivalent. + + * Mailbox access via POP or IMAP. + + * Mailbox on the user's preferred machine. + +In the latter case, each user has an alias on the mailhost that forwards mail +to her preferred machine: + + /etc/aliases: + joe: joe@joes.preferred.machine + jane: jane@janes.preferred.machine + +On some systems the alias database is not in /etc/aliases. To find out the +location for your system, execute the command "ppoossttccoonnff aalliiaass__mmaappss". + +Execute the command "nneewwaalliiaasseess" whenever you change the aliases file. + +PPoossttffiixx eemmaaiill ffiirreewwaallll//ggaatteewwaayy + +The idea is to set up a Postfix email firewall/gateway that forwards mail for +"example.com" to an inside gateway machine but rejects mail for +"anything.example.com". There is only one problem: with "relay_domains = +example.com", the firewall normally also accepts mail for +"anything.example.com". That would not be right. + +Note: this example requires Postfix version 2.0 and later. To find out what +Postfix version you have, execute the command "ppoossttccoonnff mmaaiill__vveerrssiioonn". + +The solution is presented in multiple parts. This first part gets rid of local +mail delivery on the firewall, making the firewall harder to break. + + 1 /etc/postfix/main.cf: + 2 myorigin = example.com + 3 mydestination = + 4 local_recipient_maps = + 5 local_transport = error:local mail delivery is disabled + 6 + 7 /etc/postfix/master.cf: + 8 Comment out the local delivery agent + +Translation: + + * Line 2: Send mail from this machine as "user@example.com", so that no + reason exists to send mail to "user@firewall.example.com". + + * Lines 3-8: Disable local mail delivery on the firewall machine. + +For the sake of technical correctness the firewall must be able to receive mail +for postmaster@[firewall ip address]. Reportedly, some things actually expect +this ability to exist. The second part of the solution therefore adds support +for postmaster@[firewall ip address], and as a bonus we do abuse@[firewall ip +address] as well. All the mail to these two accounts is forwarded to an inside +address. + + 1 /etc/postfix/main.cf: + 2 virtual_alias_maps = hash:/etc/postfix/virtual + 3 + 4 /etc/postfix/virtual: + 5 postmaster postmaster@example.com + 6 abuse abuse@example.com + +Translation: + + * Because mydestination is empty (see the previous example), only address + literals matching $inet_interfaces or $proxy_interfaces are deemed local. + So "localpart@[a.d.d.r]" can be matched as simply "localpart" in canonical + (5) and virtual(5). This avoids the need to specify firewall IP addresses + in Postfix configuration files. + +The last part of the solution does the email forwarding, which is the real +purpose of the firewall email function. + + 1 /etc/postfix/main.cf: + 2 mynetworks = 127.0.0.0/8 12.34.56.0/24 + 3 relay_domains = example.com + 4 parent_domain_matches_subdomains = + 5 debug_peer_list smtpd_access_maps + + 6a # Postfix 2.10 and later support separate relay control and + 7a # spam control. + 8a smtpd_relay_restrictions = + 9a permit_mynetworks reject_unauth_destination + 10a smtpd_recipient_restrictions = ...spam blocking rules.... + + 6b # Older configurations combine relay control and spam control. To + 7b # use this with Postfix >= 2.10 specify "smtpd_relay_restrictions=". + 8b smtpd_recipient_restrictions = + 9b permit_mynetworks reject_unauth_destination + 10b ...spam blocking rules.... + + 11 relay_recipient_maps = hash:/etc/postfix/relay_recipients + 12 transport_maps = hash:/etc/postfix/transport + 13 + 14 /etc/postfix/relay_recipients: + 15 user1@example.com x + 16 user2@example.com x + 17 . . . + 18 + 19 /etc/postfix/transport: + 20 example.com relay:[inside-gateway.example.com] + +Translation: + + * Lines 1-10: Accept mail from local systems in $mynetworks, and accept mail + from outside for "user@example.com" but not for + "user@anything.example.com". The magic is in lines 4-5. + + * Lines 11, 13-16: Define the list of valid addresses in the "example.com" + domain that can receive mail from the Internet. This prevents the mail + queue from filling up with undeliverable MAILER-DAEMON messages. If you + can't maintain a list of valid recipients then you must specify + "relay_recipient_maps =" (that is, an empty value), or you must specify an + "@example.com x" wild-card in the relay_recipients table. + + * Lines 12, 19-20: Route mail for "example.com" to the inside gateway + machine. The [] forces Postfix to do no MX lookup. This uses the "relay" + delivery transport (a copy of the default "smtp" delivery transport) to + forward inbound mail. This can improve performance of deliveries to + internal domains because they will compete for SMTP clients from the + "relay" delivery transport, instead of competing with other SMTP deliveries + for SMTP clients from the default "smtp" delivery transport. + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//rreellaayy__rreecciippiieennttss" whenever you change +the relay_recipients table. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ttrraannssppoorrtt" whenever you change the +transport table. + +In some installations, there may be separate instances of Postfix processing +inbound and outbound mail on a multi-homed firewall. The inbound Postfix +instance has an SMTP server listening on the external firewall interface, and +the outbound Postfix instance has an SMTP server listening on the internal +interface. In such a configuration is it is tempting to configure +$inet_interfaces in each instance with just the corresponding interface +address. + +In most cases, using inet_interfaces in this way will not work, because as +documented in the $inet_interfaces reference manual, the smtp(8) delivery agent +will also use the specified interface address as the source address for +outbound connections and will be unable to reach hosts on "the other side" of +the firewall. The symptoms are that the firewall is unable to connect to hosts +that are in fact up. See the inet_interfaces parameter documentation for +suggested work-arounds. + +DDeelliivveerriinngg ssoommee bbuutt nnoott aallll aaccccoouunnttss llooccaallllyy + +A drawback of sending mail as "user@example.com" (instead of +"user@hostname.example.com") is that mail for "root" and other system accounts +is also sent to the central mailhost. In order to deliver such accounts +locally, you can set up virtual aliases as follows: + + 1 /etc/postfix/main.cf: + 2 virtual_alias_maps = hash:/etc/postfix/virtual + 3 + 4 /etc/postfix/virtual: + 5 root root@localhost + 6 . . . + +Translation: + + * Line 5: As described in the virtual(5) manual page, the bare name "root" + matches "root@site" when "site" is equal to $myorigin, when "site" is + listed in $mydestination, or when it matches $inet_interfaces or + $proxy_interfaces. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" after editing the file. + +RRuunnnniinngg PPoossttffiixx bbeehhiinndd aa ffiirreewwaallll + +The simplest way to set up Postfix on a host behind a firewalled network is to +send all mail to a gateway host, and to let that mail host take care of +internal and external forwarding. Examples of that are shown in the local area +network section above. A more sophisticated approach is to send only external +mail to the gateway host, and to send intranet mail directly. + +Note: this example requires Postfix version 2.0 and later. To find out what +Postfix version you have, execute the command "ppoossttccoonnff mmaaiill__vveerrssiioonn". + +The following example presents additional configuration. You need to combine +this with basic configuration information as discussed in the first half of +this document. + + 1 /etc/postfix/main.cf: + 2 transport_maps = hash:/etc/postfix/transport + 3 relayhost = + 4 # Optional for a machine that isn't "always on" + 5 #fallback_relay = [gateway.example.com] + 6 + 7 /etc/postfix/transport: + 8 # Internal delivery. + 9 example.com : + 10 .example.com : + 11 # External delivery. + 12 * smtp:[gateway.example.com] + +Translation: + + * Lines 2, 7-12: Request that intranet mail is delivered directly, and that + external mail is given to a gateway. Obviously, this example assumes that + the organization uses DNS MX records internally. The [] forces Postfix to + do no MX lookup. + + * Line 3: IMPORTANT: do not specify a relayhost in main.cf. + + * Line 5: This prevents mail from being stuck in the queue when the machine + is turned off. Postfix tries to deliver mail directly, and gives + undeliverable mail to a gateway. + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ttrraannssppoorrtt" whenever you edit the +transport table. + +CCoonnffiigguurriinngg PPoossttffiixx aass pprriimmaarryy oorr bbaacckkuupp MMXX hhoosstt ffoorr aa rreemmoottee ssiittee + +This section presents additional configuration. You need to combine this with +basic configuration information as discussed in the first half of this +document. + +When your system is SECONDARY MX host for a remote site this is all you need: + + 1 DNS: + 2 the.backed-up.domain.tld IN MX 100 your.machine.tld. + 3 + 4 /etc/postfix/main.cf: + 5 relay_domains = . . . the.backed-up.domain.tld + + 6a # Postfix 2.10 and later support separate relay control and + 7a # spam control. + 8a smtpd_relay_restrictions = + 9a permit_mynetworks reject_unauth_destination + 10a smtpd_recipient_restrictions = ...spam blocking rules.... + + 6b # Older configurations combine relay control and spam control. To + 7b # use this with Postfix >= 2.10 specify "smtpd_relay_restrictions=". + 8b smtpd_recipient_restrictions = + 9b permit_mynetworks reject_unauth_destination + 10b ...spam blocking rules.... + + 11 # You must specify your NAT/proxy external address. + 12 #proxy_interfaces = 1.2.3.4 + 13 + 14 relay_recipient_maps = hash:/etc/postfix/relay_recipients + 15 + 16 /etc/postfix/relay_recipients: + 17 user1@the.backed-up.domain.tld x + 18 user2@the.backed-up.domain.tld x + 19 . . . + +When your system is PRIMARY MX host for a remote site you need the above, plus: + + 20 /etc/postfix/main.cf: + 21 transport_maps = hash:/etc/postfix/transport + 22 + 23 /etc/postfix/transport: + 24 the.backed-up.domain.tld relay:[their.mail.host.tld] + +Important notes: + + * Do not list the.backed-up.domain.tld in mydestination. + + * Do not list the.backed-up.domain.tld in virtual_alias_domains. + + * Do not list the.backed-up.domain.tld in virtual_mailbox_domains. + + * Lines 1-9: Forward mail from the Internet for "the.backed-up.domain.tld" to + the primary MX host for that domain. + + * Line 12: This is a must if Postfix receives mail via a NAT relay or proxy + that presents a different IP address to the world than the local machine. + + * Lines 14-18: Define the list of valid addresses in the "the.backed- + up.domain.tld" domain. This prevents your mail queue from filling up with + undeliverable MAILER-DAEMON messages. If you can't maintain a list of valid + recipients then you must specify "relay_recipient_maps =" (that is, an + empty value), or you must specify an "@the.backed-up.domain.tld x" wild- + card in the relay_recipients table. + + * Line 24: The [] forces Postfix to do no MX lookup. + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ttrraannssppoorrtt" whenever you change the +transport table. + +NOTE for Postfix < 2.2: Do not use the fallback_relay feature when relaying +mail for a backup or primary MX domain. Mail would loop between the Postfix MX +host and the fallback_relay host when the final destination is unavailable. + + * In main.cf specify "relay_transport = relay", + * In master.cf specify "-o fallback_relay =" at the end of the relay entry. + * In transport maps, specify "relay:nexthop..." as the right-hand side for + backup or primary MX domain entries. + +These are default settings in Postfix version 2.2 and later. + +PPoossttffiixx oonn aa ddiiaalluupp mmaacchhiinnee + +This section applies to dialup connections that are down most of the time. For +dialup connections that are up 24x7, see the local area network section above. + +This section presents additional configuration. You need to combine this with +basic configuration information as discussed in the first half of this +document. + +If you do not have your own hostname and IP address (usually with dialup, cable +TV or DSL connections) then you should also study the section on "Postfix on +hosts without a real Internet hostname". + + * Route all outgoing mail to your network provider. + If your machine is disconnected most of the time, there isn't a lot of + opportunity for Postfix to deliver mail to hard-to-reach corners of the + Internet. It's better to give the mail to a machine that is connected all + the time. In the example below, the [] prevents Postfix from trying to look + up DNS MX records. + + /etc/postfix/main.cf: + relayhost = [smtprelay.someprovider.com] + + * Disable spontaneous SMTP mail delivery (if using on-demand dialup IP only). + + Normally, Postfix attempts to deliver outbound mail at its convenience. If + your machine uses on-demand dialup IP, this causes your system to place a + telephone call whenever you submit new mail, and whenever Postfix retries + to deliver delayed mail. To prevent such telephone calls from being placed, + disable spontaneous SMTP mail deliveries. + + /etc/postfix/main.cf: + defer_transports = smtp (Only for on-demand dialup IP hosts) + + * Disable SMTP client DNS lookups (dialup LAN only). + + /etc/postfix/main.cf: + disable_dns_lookups = yes (Only for on-demand dialup IP hosts) + + * Flush the mail queue whenever the Internet link is established. + Put the following command into your PPP or SLIP dialup scripts: + + /usr/sbin/sendmail -q (whenever the Internet link is up) + + The exact location of the Postfix sendmail command is system-specific. Use + the command "ppoossttccoonnff sseennddmmaaiill__ppaatthh" to find out where the Postfix sendmail + command is located on your machine. + + In order to find out if the mail queue is flushed, use something like: + + #!/bin/sh + + # Start mail deliveries. + /usr/sbin/sendmail -q + + # Allow deliveries to start. + sleep 10 + + # Loop until all messages have been tried at least once. + while mailq | grep '^[^ ]*\*' >/dev/null + do + sleep 10 + done + + If you have disabled spontaneous SMTP mail delivery, you also need to run + the "sseennddmmaaiill --qq" command every now and then while the dialup link is up, + so that newly-posted mail is flushed from the queue. + +PPoossttffiixx oonn hhoossttss wwiitthhoouutt aa rreeaall IInntteerrnneett hhoossttnnaammee + +This section is for hosts that don't have their own Internet hostname. +Typically these are systems that get a dynamic IP address via DHCP or via +dialup. Postfix will let you send and receive mail just fine between accounts +on a machine with a fantasy name. However, you cannot use a fantasy hostname in +your email address when sending mail into the Internet, because no-one would be +able to reply to your mail. In fact, more and more sites refuse mail addresses +with non-existent domain names. + +Note: the following information is Postfix version dependent. To find out what +Postfix version you have, execute the command "ppoossttccoonnff mmaaiill__vveerrssiioonn". + +SSoolluuttiioonn 11:: PPoossttffiixx vveerrssiioonn 22..22 aanndd llaatteerr + +Postfix 2.2 uses the generic(5) address mapping to replace local fantasy email +addresses by valid Internet addresses. This mapping happens ONLY when mail +leaves the machine; not when you send mail between users on the same machine. + +The following example presents additional configuration. You need to combine +this with basic configuration information as discussed in the first half of +this document. + + 1 /etc/postfix/main.cf: + 2 smtp_generic_maps = hash:/etc/postfix/generic + 3 + 4 /etc/postfix/generic: + 5 his@localdomain.local hisaccount@hisisp.example + 6 her@localdomain.local heraccount@herisp.example + 7 @localdomain.local hisaccount+local@hisisp.example + +When mail is sent to a remote host via SMTP: + + * Line 5 replaces his@localdomain.local by his ISP mail address, + + * Line 6 replaces her@localdomain.local by her ISP mail address, and + + * Line 7 replaces other local addresses by his ISP account, with an address + extension of +local (this example assumes that the ISP supports "+" style + address extensions). + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ggeenneerriicc" whenever you change the +generic table. + +SSoolluuttiioonn 22:: PPoossttffiixx vveerrssiioonn 22..11 aanndd eeaarrlliieerr + +The solution with older Postfix systems is to use valid Internet addresses +where possible, and to let Postfix map valid Internet addresses to local +fantasy addresses. With this, you can send mail to the Internet and to local +fantasy addresses, including mail to local fantasy addresses that don't have a +valid Internet address of their own. + +The following example presents additional configuration. You need to combine +this with basic configuration information as discussed in the first half of +this document. + + 1 /etc/postfix/main.cf: + 2 myhostname = hostname.localdomain + 3 mydomain = localdomain + 4 + 5 canonical_maps = hash:/etc/postfix/canonical + 6 + 7 virtual_alias_maps = hash:/etc/postfix/virtual + 8 + 9 /etc/postfix/canonical: + 10 your-login-name your-account@your-isp.com + 11 + 12 /etc/postfix/virtual: + 13 your-account@your-isp.com your-login-name + +Translation: + + * Lines 2-3: Substitute your fantasy hostname here. Do not use a domain name + that is already in use by real organizations on the Internet. See RFC 2606 + for examples of domain names that are guaranteed not to be owned by anyone. + + * Lines 5, 9, 10: This provides the mapping from "your-login- + name@hostname.localdomain" to "your-account@your-isp.com". This part is + required. + + * Lines 7, 12, 13: Deliver mail for "your-account@your-isp.com" locally, + instead of sending it to the ISP. This part is not required but is + convenient. + +Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb files. +To find out what lookup tables Postfix supports, use the command "ppoossttccoonnff --mm". + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ccaannoonniiccaall" whenever you change the +canonical table. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" whenever you change the +virtual table. + diff --git a/README_FILES/STRESS_README b/README_FILES/STRESS_README new file mode 100644 index 0000000..8edc148 --- /dev/null +++ b/README_FILES/STRESS_README @@ -0,0 +1,426 @@ +PPoossttffiixx SSttrreessss--DDeeppeennddeenntt CCoonnffiigguurraattiioonn + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document describes the symptoms of Postfix SMTP server overload. It +presents permanent main.cf changes to avoid overload during normal operation, +and temporary main.cf changes to cope with an unexpected burst of mail. This +document makes specific suggestions for Postfix 2.5 and later which support +stress-adaptive behavior, and for earlier Postfix versions that don't. + +Topics covered in this document: + + * Symptoms of Postfix SMTP server overload + * Automatic stress-adaptive behavior + * Service more SMTP clients at the same time + * Spend less time per SMTP client + * Disconnect suspicious SMTP clients + * Temporary measures for older Postfix releases + * Detecting support for stress-adaptive behavior + * Forcing stress-adaptive behavior on or off + * Other measures to off-load zombies + * Credits + +SSyymmppttoommss ooff PPoossttffiixx SSMMTTPP sseerrvveerr oovveerrllooaadd + +Under normal conditions, the Postfix SMTP server responds immediately when an +SMTP client connects to it; the time to deliver mail is noticeable only with +large messages. Performance degrades dramatically when the number of SMTP +clients exceeds the number of Postfix SMTP server processes. When an SMTP +client connects while all Postfix SMTP server processes are busy, the client +must wait until a server process becomes available. + +SMTP server overload may be caused by a surge of legitimate mail (example: a +DNS registrar opens a new zone for registrations), by mistake (mail explosion +caused by a forwarding loop) or by malice (worm outbreak, botnet, or other +illegitimate activity). + +Symptoms of Postfix SMTP server overload are: + + * Remote SMTP clients experience a long delay before Postfix sends the "220 + hostname.example.com ESMTP Postfix" greeting. + + o NOTE: Broken DNS configurations can also cause lengthy delays before + Postfix sends "220 hostname.example.com ...". These delays also exist + when Postfix is NOT overloaded. + + o NOTE: To avoid "overload" delays for end-user mail clients, enable the + "submission" service entry in master.cf (present since Postfix 2.1), + and tell users to connect to this instead of the public SMTP service. + + * The Postfix SMTP server logs an increased number of "lost connection after + CONNECT" events. This happens because remote SMTP clients disconnect before + Postfix answers the connection. + + o NOTE: A portscan for open SMTP ports can also result in "lost + connection ..." logfile messages. + + * Postfix 2.3 and later logs a warning that all server ports are busy: + + Oct 3 20:39:27 spike postfix/master[28905]: warning: service "smtp" + (25) has reached its process limit "30": new clients may experience + noticeable delays + Oct 3 20:39:27 spike postfix/master[28905]: warning: to avoid this + condition, increase the process count in master.cf or reduce the + service time per client + Oct 3 20:39:27 spike postfix/master[28905]: warning: see + http://www.postfix.org/STRESS_README.html for examples of + stress-adapting configuration settings + +Legitimate mail that doesn't get through during an episode of Postfix SMTP +server overload is not necessarily lost. It should still arrive once the +situation returns to normal, as long as the overload condition is temporary. + +AAuuttoommaattiicc ssttrreessss--aaddaappttiivvee bbeehhaavviioorr + +Postfix version 2.5 introduces automatic stress-adaptive behavior. It works as +follows. When a "public" network service such as the SMTP server runs into an +"all server ports are busy" condition, the Postfix master(8) daemon logs a +warning, restarts the service (without interrupting existing network sessions), +and runs the service with "-o stress=yes" on the server process command line: + + 80821 ?? S 0:00.24 smtpd -n smtp -t inet -u -c -o stress=yes + +Normally, the Postfix master(8) daemon runs such a service with "-o stress=" on +the command line (i.e. with an empty parameter value): + + 83326 ?? S 0:00.28 smtpd -n smtp -t inet -u -c -o stress= + +You won't see "-o stress" command-line parameters with services that have local +clients only. These include services internal to Postfix such as the queue +manager, and services that listen on a loopback interface only, such as after- +filter SMTP services. + +The "stress" parameter value is the key to making main.cf parameter settings +stress adaptive. The following settings are the default with Postfix 2.6 and +later. + + 1 smtpd_timeout = ${stress?{10}:{300}}s + 2 smtpd_hard_error_limit = ${stress?{1}:{20}} + 3 smtpd_junk_command_limit = ${stress?{1}:{100}} + 4 # Parameters added after Postfix 2.6: + 5 smtpd_per_record_deadline = ${stress?{yes}:{no}} + 6 smtpd_starttls_timeout = ${stress?{10}:{300}}s + 7 address_verify_poll_count = ${stress?{1}:{3}} + +Postfix versions before 3.0 use the older form ${stress?x}${stress:y} instead +of the newer form ${stress?{x}:{y}}. + +The syntax of ${name?{value}:{value}}, ${name?value} and ${name:value} is +explained at the beginning of the postconf(5) manual page. + +Translation: + + * Line 1: under conditions of stress, use an smtpd_timeout value of 10 + seconds instead of the default 300 seconds. Experience on the postfix-users + list from a variety of sysadmins shows that reducing the "normal" + smtpd_timeout to 60s is unlikely to affect legitimate clients. However, it + is unlikely to become the Postfix default because it's not RFC compliant. + Setting smtpd_timeout to 10s or even 5s under stress will still allow most + legitimate clients to connect and send mail, but may delay mail from some + clients. No mail should be lost, as long as this measure is used only + temporarily. + + * Line 2: under conditions of stress, use an smtpd_hard_error_limit of 1 + instead of the default 20. This disconnects clients after a single error, + giving other clients a chance to connect. However, this may cause + significant delays with legitimate mail, such as a mailing list that + contains a few no-longer-active user names that didn't bother to + unsubscribe. No mail should be lost, as long as this measure is used only + temporarily. + + * Line 3: under conditions of stress, use an smtpd_junk_command_limit of 1 + instead of the default 100. This prevents clients from keeping connections + open by repeatedly sending HELO, EHLO, NOOP, RSET, VRFY or ETRN commands. + + * Line 5: under conditions of stress, change the behavior of smtpd_timeout + and smtpd_starttls_timeout, from a time limit per read or write system + call, to a time limit to send or receive a complete record (an SMTP command + line, SMTP response line, SMTP message content line, or TLS protocol + message). + + * Line 6: under conditions of stress, reduce the time limit for TLS protocol + handshake messages to 10 seconds, from the default value of 300 seconds. + See also the smtpd_timeout discussion above. + + * Line 7: under conditions of stress, do not wait up to 6 seconds for the + completion of an address verification probe. If the result is not already + in the address verification cache, reply immediately with + $unverified_recipient_tempfail_action or + $unverified_sender_tempfail_action. No mail should be lost, as long as this + measure is used only temporarily. + +NOTE: Please keep in mind that the stress-adaptive feature is a fairly +desperate measure to keep ssoommee legitimate mail flowing under overload +conditions. If a site is reaching the SMTP server process limit when there +isn't an attack or bot flood occurring, then either the process limit needs to +be raised or more hardware needs to be added. + +SSeerrvviiccee mmoorree SSMMTTPP cclliieennttss aatt tthhee ssaammee ttiimmee + +This section and the ones that follow discuss permanent measures against mail +server overload. + +One measure to avoid the "all server processes busy" condition is to service +more SMTP clients simultaneously. For this you need to increase the number of +Postfix SMTP server processes. This will improve the responsiveness for remote +SMTP clients, as long as the server machine has enough hardware and software +resources to run the additional processes, and as long as the file system can +keep up with the additional load. + + * You increase the number of SMTP server processes either by increasing the + default_process_limit in main.cf (line 3 below), or by increasing the SMTP + server's "maxproc" field in master.cf (line 10 below). Either way, you need + to issue a "postfix reload" command to make the change effective. + + * Process limits above 1000 require Postfix version 2.4 or later, and an + operating system that supports kernel-based event filters (BSD kqueue(2), + Linux epoll(4), or Solaris /dev/poll). + + * More processes use more memory. You can reduce the Postfix memory footprint + by using cdb: lookup tables instead of Berkeley DB's hash: or btree: + tables. + + 1 /etc/postfix/main.cf: + 2 # Raise the global process limit, 100 since Postfix 2.0. + 3 default_process_limit = 200 + 4 + 5 /etc/postfix/master.cf: + 6 # ============================================================= + 7 # service type private unpriv chroot wakeup maxproc command + 8 # ============================================================= + 9 # Raise the SMTP service process limit only. + 10 smtp inet n - n - 200 smtpd + + * NOTE: older versions of the SMTPD_POLICY_README document contain a mistake: + they configure a fixed number of policy daemon processes. When you raise + the SMTP server's "maxproc" field in master.cf, SMTP server processes will + report problems when connecting to policy server processes, because there + aren't enough of them. Examples of errors are "connection refused" or + "operation timed out". + + To fix, edit master.cf and specify a zero "maxproc" field in all policy + server entries; see line 6 in the example below. Issue a "postfix reload" + command to make the change effective. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # Disable the policy service process limit. + 6 policy unix - n n - 0 spawn + 7 user=nobody argv=/some/where/policy-server + +SSppeenndd lleessss ttiimmee ppeerr SSMMTTPP cclliieenntt + +When increasing the number of SMTP server processes is not practical, you can +improve Postfix server responsiveness by eliminating delays. When Postfix +spends less time per SMTP session, the same number of SMTP server processes can +service more clients in a given amount of time. + + * Eliminate non-functional RBL lookups (blocklists that are no longer in + operation). These lookups can degrade performance. Postfix logs a warning + when an RBL server does not respond. + + * Eliminate redundant RBL lookups (people often use multiple Spamhaus RBLs + that include each other). To find out whether RBLs include other RBLs, look + up the websites that document the RBL's policies. + + * Eliminate header_checks and body_checks, and keep just a few emergency + patterns to block the latest worm explosion or backscatter mail. See + BACKSCATTER_README for examples of the latter. + + * Group your header_checks and body_checks patterns to avoid unnecessary + pattern matching operations: + + 1 /etc/postfix/header_checks: + 2 if /^Subject:/ + 3 /^Subject: virus found in mail from you/ reject + 4 /^Subject: ..other../ reject + 5 endif + 6 + 7 if /^Received:/ + 8 /^Received: from (postfix\.org) / reject forged client name in + received header: $1 + 9 /^Received: from ..other../ reject .... + 10 endif + +DDiissccoonnnneecctt ssuussppiicciioouuss SSMMTTPP cclliieennttss + +Under conditions of overload you can improve Postfix SMTP server responsiveness +by hanging up on suspicious clients, so that other clients get a chance to talk +to Postfix. + + * Use "521" SMTP reply codes (Postfix 2.6 and later) or "421" (Postfix 2.3- + 2.5) to hang up on clients that that match botnet-related RBLs (see next + bullet) or that match selected non-RBL restrictions such as SMTP access + maps. The Postfix SMTP server will reject mail and disconnect without + waiting for the remote SMTP client to send a QUIT command. + + * To hang up connections from denylisted zombies, you can set specific + Postfix SMTP server reject codes for specific RBLs, and for individual + responses from specific RBLs. We'll use zen.spamhaus.org as an example; by + the time you read this document, details may have changed. Right now, their + documents say that a response of 127.0.0.10 or 127.0.0.11 indicates a + dynamic client IP address, which means that the machine is probably running + a bot of some kind. To give a 521 response instead of the default 554 + response, use something like: + + 1 /etc/postfix/main.cf: + 2 smtpd_client_restrictions = + 3 permit_mynetworks + 4 reject_rbl_client zen.spamhaus.org=127.0.0.10 + 5 reject_rbl_client zen.spamhaus.org=127.0.0.11 + 6 reject_rbl_client zen.spamhaus.org + 7 + 8 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps + 9 + 10 /etc/postfix/rbl_reply_maps: + 11 # With Postfix 2.3-2.5 use "421" to hang up connections. + 12 zen.spamhaus.org=127.0.0.10 521 4.7.1 Service unavailable; + 13 $rbl_class [$rbl_what] blocked using + 14 $rbl_domain${rbl_reason?; $rbl_reason} + 15 + 16 zen.spamhaus.org=127.0.0.11 521 4.7.1 Service unavailable; + 17 $rbl_class [$rbl_what] blocked using + 18 $rbl_domain${rbl_reason?; $rbl_reason} + + Although the above example shows three RBL lookups (lines 4-6), Postfix + will only do a single DNS query, so it does not affect the performance. + + * With Postfix 2.3-2.5, use reply code 421 (521 will not cause Postfix to + disconnect). The down-side of replying with 421 is that it works only for + zombies and other malware. If the client is running a real MTA, then it may + connect again several times until the mail expires in its queue. When this + is a problem, stick with the default 554 reply, and use + "smtpd_hard_error_limit = 1" as described below. + + * You can automatically turn on the above overload measure with Postfix 2.5 + and later, or with earlier releases that contain the stress-adaptive + behavior source code patch from the mirrors listed at http:// + www.postfix.org/download.html. Simply replace line above 8 with: + + 8 rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps} + +More information about automatic stress-adaptive behavior is in section +"Automatic stress-adaptive behavior". + +TTeemmppoorraarryy mmeeaassuurreess ffoorr oollddeerr PPoossttffiixx rreelleeaasseess + +See the section "Automatic stress-adaptive behavior" if you are running Postfix +version 2.5 or later, or if you have applied the source code patch for stress- +adaptive behavior from the mirrors listed at http://www.postfix.org/ +download.html. + +The following measures can be applied temporarily during overload. They still +allow mmoosstt legitimate clients to connect and send mail, but may affect some +legitimate clients. + + * Reduce smtpd_timeout (default: 300s). Experience on the postfix-users list + from a variety of sysadmins shows that reducing the "normal" smtpd_timeout + to 60s is unlikely to affect legitimate clients. However, it is unlikely to + become the Postfix default because it's not RFC compliant. Setting + smtpd_timeout to 10s (line 2 below) or even 5s under stress will still + allow mmoosstt legitimate clients to connect and send mail, but may delay mail + from some clients. No mail should be lost, as long as this measure is used + only temporarily. + + * Reduce smtpd_hard_error_limit (default: 20). Setting this to 1 under stress + (line 3 below) helps by disconnecting clients after a single error, giving + other clients a chance to connect. However, this may cause significant + delays with legitimate mail, such as a mailing list that contains a few no- + longer-active user names that didn't bother to unsubscribe. No mail should + be lost, as long as this measure is used only temporarily. + + * Use an smtpd_junk_command_limit of 1 instead of the default 100. This + prevents clients from keeping idle connections open by repeatedly sending + NOOP or RSET commands. + + 1 /etc/postfix/main.cf: + 2 smtpd_timeout = 10 + 3 smtpd_hard_error_limit = 1 + 4 smtpd_junk_command_limit = 1 + +With these measures, no mail should be lost, as long as these measures are used +only temporarily. The next section of this document introduces a way to +automate this process. + +DDeetteeccttiinngg ssuuppppoorrtt ffoorr ssttrreessss--aaddaappttiivvee bbeehhaavviioorr + +To find out if your Postfix installation supports stress-adaptive behavior, use +the "ps" command, and look for the smtpd processes. Postfix has stress-adaptive +support when you see "-o stress=" or "-o stress=yes" command-line options. +Remember that Postfix never enables stress-adaptive behavior on servers that +listen on local addresses only. + +The following example is for FreeBSD or Linux. On Solaris, HP-UX and other +System-V flavors, use "ps -ef" instead of "ps ax". + + $ ps ax|grep smtpd + 83326 ?? S 0:00.28 smtpd -n smtp -t inet -u -c -o stress= + 84345 ?? Ss 0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd- + policy.pl + +You can't use postconf(1) to detect stress-adaptive support. The postconf(1) +command ignores the existence of the stress parameter in main.cf, because the +parameter has no effect there. Command-line "-o parameter" settings always take +precedence over main.cf parameter settings. + +If you configure stress-adaptive behavior in main.cf when it isn't supported, +nothing bad will happen. The processes will run as if the stress parameter +always has an empty value. + +FFoorrcciinngg ssttrreessss--aaddaappttiivvee bbeehhaavviioorr oonn oorr ooffff + +You can manually force stress-adaptive behavior on, by adding a "-o stress=yes" +command-line option in master.cf. This can be useful for testing overrides on +the SMTP service. Issue "postfix reload" to make the change effective. + +Note: setting the stress parameter in main.cf has no effect for services that +accept remote connections. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # + 6 smtp inet n - n - - smtpd + 7 -o stress=yes + 8 -o . . . + +To permanently force stress-adaptive behavior off with a specific service, +specify "-o stress=" on its master.cf command line. This may be desirable for +the "submission" service. Issue "postfix reload" to make the change effective. + +Note: setting the stress parameter in main.cf has no effect for services that +accept remote connections. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # + 6 submission inet n - n - - smtpd + 7 -o stress= + 8 -o . . . + +OOtthheerr mmeeaassuurreess ttoo ooffff--llooaadd zzoommbbiieess + +The postscreen(8) daemon, introduced with Postfix 2.8, provides additional +protection against mail server overload. One postscreen(8) process handles +multiple inbound SMTP connections, and decides which clients may talk to a +Postfix SMTP server process. By keeping spambots away, postscreen(8) leaves +more SMTP server processes available for legitimate clients, and delays the +onset of server overload conditions. + +CCrreeddiittss + + * Thanks to the postfix-users mailing list members for sharing early + experiences with the stress-adaptive feature. + * The RBL example and several other paragraphs of text were adapted from + postfix-users postings by Noel Jones. + * Wietse implemented stress-adaptive behavior as the smallest possible patch + while he should be working on other things. + diff --git a/README_FILES/TLS_LEGACY_README b/README_FILES/TLS_LEGACY_README new file mode 100644 index 0000000..f4dae6b --- /dev/null +++ b/README_FILES/TLS_LEGACY_README @@ -0,0 +1,1119 @@ +PPoossttffiixx lleeggaaccyy TTLLSS SSuuppppoorrtt + +------------------------------------------------------------------------------- + +NNOOTTEE + +This document describes an old TLS user interface that is based on a third- +party TLS patch by Lutz Ja"nicke. As of Postfix version 2.3, the old user +interface still exists to allow migration from earlier Postfix releases, but +its functionality is frozen. + +WWhhaatt PPoossttffiixx TTLLSS ssuuppppoorrtt ddooeess ffoorr yyoouu + +Transport Layer Security (TLS, formerly called SSL) provides certificate-based +authentication and encrypted sessions. An encrypted session protects the +information that is transmitted with SMTP mail or with SASL authentication. + +Postfix version 2.2 introduces support for TLS as described in RFC 3207. TLS +Support for older Postfix versions was available as an add-on patch. The +section "Compatibility with Postfix < 2.2 TLS support" below discusses the +differences between these implementations. + +Topics covered in this document: + + * How Postfix TLS support works + * Building Postfix with TLS support + * SMTP Server specific settings + * SMTP Client specific settings + * TLS manager specific settings + * Reporting problems + * Compatibility with Postfix < 2.2 TLS support + * Credits + +And last but not least, for the impatient: + + * Getting started, quick and dirty + +HHooww PPoossttffiixx TTLLSS ssuuppppoorrtt wwoorrkkss + +The diagram below shows the main elements of the Postfix TLS architecture and +their relationships. Colored boxes with numbered names represent Postfix daemon +programs. Other colored boxes represent storage elements. + + * The smtpd(8) server implements the SMTP over TLS server side. + + * The smtp(8) client implements the SMTP over TLS client side. + + * The tlsmgr(8) server maintains the pseudo-random number generator (PRNG) + that seeds the TLS engines in the smtpd(8) server and smtp(8) client + processes, and maintains the TLS session key cache files. + + <---seed--- ---seed---> +Network-> smtpd(8) tlsmgr(8) smtp(8) ->Network + <-session-> <-session-> + + / | \ + | + / \ + + smtpd PRNG smtp + session state session + key cache file key cache + +BBuuiillddiinngg PPoossttffiixx wwiitthh TTLLSS ssuuppppoorrtt + +To build Postfix with TLS support, first we need to generate the make(1) files +with the necessary definitions. This is done by invoking the command "make +makefiles" in the Postfix top-level directory and with arguments as shown next. + +NNOOTTEE:: DDoo nnoott uussee GGnnuu TTLLSS.. IItt wwiillll ssppoonnttaanneeoouussllyy tteerrmmiinnaattee aa PPoossttffiixx ddaaeemmoonn +pprroocceessss wwiitthh eexxiitt ssttaattuuss ccooddee 22,, iinnsstteeaadd ooff aalllloowwiinngg PPoossttffiixx ttoo 11)) rreeppoorrtt tthhee +eerrrroorr ttoo tthhee mmaaiilllloogg ffiillee,, aanndd ttoo 22)) pprroovviiddee ppllaaiinntteexxtt sseerrvviiccee wwhheerree tthhiiss iiss +aapppprroopprriiaattee.. + + * If the OpenSSL include files (such as ssl.h) are in directory /usr/include/ + openssl, and the OpenSSL libraries (such as libssl.so and libcrypto.so) are + in directory /usr/lib: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS"" AAUUXXLLIIBBSS==""--llssssll --llccrryyppttoo"" + + * If the OpenSSL include files (such as ssl.h) are in directory /usr/local/ + include/openssl, and the OpenSSL libraries (such as libssl.so and + libcrypto.so) are in directory /usr/local/lib: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS --II//uussrr//llooccaall//iinncclluuddee"" \\ + AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb --llssssll --llccrryyppttoo"" + + On Solaris, specify the -R option as shown below: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS --II//uussrr//llooccaall//iinncclluuddee"" \\ + AAUUXXLLIIBBSS==""--RR//uussrr//llooccaall//lliibb --LL//uussrr//llooccaall//lliibb --llssssll --llccrryyppttoo"" + +If you need to apply other customizations (such as Berkeley DB databases, +MySQL, PosgreSQL, LDAP or SASL), see the respective Postfix README documents, +and combine their "make makefiles" instructions with the instructions above: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS \\ + ((ootthheerr --DD oorr --II ooppttiioonnss))"" \\ + AAUUXXLLIIBBSS==""--llssssll --llccrryyppttoo \\ + ((ootthheerr --ll ooppttiioonnss ffoorr lliibbrraarriieess iinn //uussrr//lliibb)) \\ + ((--LL//ppaatthh//nnaammee ++ --ll ooppttiioonnss ffoorr ootthheerr lliibbrraarriieess))"" + +To complete the build process, see the Postfix INSTALL instructions. Postfix +has TLS support turned off by default, so you can start using Postfix as soon +as it is installed. + +SSMMTTPP SSeerrvveerr ssppeecciiffiicc sseettttiinnggss + +Topics covered in this section: + + * Server-side certificate and private key configuration + * Server-side TLS activity logging + * Enabling TLS in the Postfix SMTP server + * Client certificate verification + * Supporting AUTH over TLS only + * Server-side TLS session cache + * Server access control + * Server-side cipher controls + * Miscellaneous server controls + +SSeerrvveerr--ssiiddee cceerrttiiffiiccaattee aanndd pprriivvaattee kkeeyy ccoonnffiigguurraattiioonn + +In order to use TLS, the Postfix SMTP server needs a certificate and a private +key. Both must be in "pem" format. The private key must not be encrypted, +meaning: the key must be accessible without a password. Both certificate and +private key may be in the same file. + +Both RSA and DSA certificates are supported. Typically you will only have RSA +certificates issued by a commercial CA. In addition, the tools supplied with +OpenSSL will by default issue RSA certificates. You can have both at the same +time, in which case the cipher used determines which certificate is presented. +For Netscape and OpenSSL clients without special cipher choices, the RSA +certificate is preferred. + +In order for remote SMTP clients to check the Postfix SMTP server certificates, +the CA certificate (in case of a certificate chain, all CA certificates) must +be available. You should add these certificates to the server certificate, the +server certificate first, then the issuing CA(s). + +Example: the certificate for "server.dom.ain" was issued by "intermediate CA" +which itself has a certificate issued by "root CA". Create the server.pem file +with: + + % ccaatt sseerrvveerr__cceerrtt..ppeemm iinntteerrmmeeddiiaattee__CCAA..ppeemm >> sseerrvveerr..ppeemm + +A Postfix SMTP server certificate supplied here must be usable as an SSL server +certificate and hence pass the "openssl verify -purpose sslserver ..." test. + +A client that trusts the root CA has a local copy of the root CA certificate, +so it is not necessary to include the root CA certificate here. Leaving it out +of the "server.pem" file reduces the overhead of the TLS exchange. + +If you want the Postfix SMTP server to accept remote SMTP client certificates +issued by these CAs, append the root certificate to $smtpd_tls_CAfile or +install it in the $smtpd_tls_CApath directory. When you configure trust in a +root CA, it is not necessary to explicitly trust intermediary CAs signed by the +root CA, unless $smtpd_tls_ccert_verifydepth is less than the number of CAs in +the certificate chain for the clients of interest. With a verify depth of 1 you +can only verify certificates directly signed by a trusted CA, and all trusted +intermediary CAs need to be configured explicitly. With a verify depth of 2 you +can verify clients signed by a root CA or a direct intermediary CA (so long as +the client is correctly configured to supply its intermediate CA certificate). + +RSA key and certificate examples: + + /etc/postfix/main.cf: + smtpd_tls_cert_file = /etc/postfix/server.pem + smtpd_tls_key_file = $smtpd_tls_cert_file + +Their DSA counterparts: + + /etc/postfix/main.cf: + smtpd_tls_dcert_file = /etc/postfix/server-dsa.pem + smtpd_tls_dkey_file = $smtpd_tls_dcert_file + +To verify a remote SMTP client certificate, the Postfix SMTP server needs to +trust the certificates of the issuing Certification Authorities. These +certificates in "pem" format can be stored in a single $smtpd_tls_CAfile or in +multiple files, one CA per file in the $smtpd_tls_CApath directory. If you use +a directory, don't forget to create the necessary "hash" links with: + + # $$OOPPEENNSSSSLL__HHOOMMEE//bbiinn//cc__rreehhaasshh //ppaatthh//ttoo//ddiirreeccttoorryy + +The $smtpd_tls_CAfile contains the CA certificates of one or more trusted CAs. +The file is opened (with root privileges) before Postfix enters the optional +chroot jail and so need not be accessible from inside the chroot jail. + +Additional trusted CAs can be specified via the $smtpd_tls_CApath directory, in +which case the certificates are read (with $mail_owner privileges) from the +files in the directory when the information is needed. Thus, the +$smtpd_tls_CApath directory needs to be accessible inside the optional chroot +jail. + +When you configure Postfix to request client certificates (by setting +$smtpd_tls_ask_ccert = yes), any certificates in $smtpd_tls_CAfile are sent to +the client, in order to allow it to choose an identity signed by a CA you +trust. If no $smtpd_tls_CAfile is specified, no preferred CA list is sent, and +the client is free to choose an identity signed by any CA. Many clients use a +fixed identity regardless of the preferred CA list and you may be able to +reduce TLS negotiation overhead by installing client CA certificates mostly or +only in $smtpd_tls_CApath. In the latter case you need not specify a +$smtpd_tls_CAfile. + +Note, that unless client certificates are used to allow greater access to TLS +authenticated clients, it is best to not ask for client certificates at all, as +in addition to increased overhead some clients (notably in some cases qmail) +are unable to complete the TLS handshake when client certificates are +requested. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_CAfile = /etc/postfix/CAcert.pem + smtpd_tls_CApath = /etc/postfix/certs + +SSeerrvveerr--ssiiddee TTLLSS aaccttiivviittyy llooggggiinngg + +To get additional information about Postfix SMTP server TLS activity you can +increase the loglevel from 0..4. Each logging level also includes the +information that is logged at a lower logging level. + + 0 Disable logging of TLS activity. + + 1 Log TLS handshake and certificate information. + + 2 Log levels during TLS negotiation. + + 3 Log hexadecimal and ASCII dump of TLS negotiation process + + 4 Log hexadecimal and ASCII dump of complete transmission after STARTTLS + +Use loglevel 3 only in case of problems. Use of loglevel 4 is strongly +discouraged. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_loglevel = 0 + +To include information about the protocol and cipher used as well as the client +and issuer CommonName into the "Received:" message header, set the +smtpd_tls_received_header variable to true. The default is no, as the +information is not necessarily authentic. Only information recorded at the +final destination is reliable, since the headers may be changed by intermediate +servers. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_received_header = yes + +EEnnaabblliinngg TTLLSS iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +By default, TLS is disabled in the Postfix SMTP server, so no difference to +plain Postfix is visible. Explicitly switch it on using "smtpd_use_tls = yes". + +Example: + + /etc/postfix/main.cf: + smtpd_use_tls = yes + +With this, Postfix SMTP server announces STARTTLS support to SMTP clients, but +does not require that clients use TLS encryption. + +Note: when an unprivileged user invokes "sendmail -bs", STARTTLS is never +offered due to insufficient privileges to access the server private key. This +is intended behavior. + +You can ENFORCE the use of TLS, so that the Postfix SMTP server announces +STARTTLS and accepts no mail without TLS encryption, by setting +"smtpd_enforce_tls = yes". According to RFC 2487 this MUST NOT be applied in +case of a publicly-referenced Postfix SMTP server. This option is off by +default and should only seldom be used. + +Example: + + /etc/postfix/main.cf: + smtpd_enforce_tls = yes + +TLS is sometimes used in the non-standard "wrapper" mode where a server always +uses TLS, instead of announcing STARTTLS support and waiting for clients to +request TLS service. Some clients, namely Outlook [Express] prefer the +"wrapper" mode. This is true for OE (Win32 < 5.0 and Win32 >=5.0 when run on a +port<>25 and OE (5.01 Mac on all ports). + +It is strictly discouraged to use this mode from main.cf. If you want to +support this service, enable a special port in master.cf and specify "- +o smtpd_tls_wrappermode = yes" as an smtpd(8) command line option. Port 465 +(smtps) was once chosen for this feature. + +Example: + + /etc/postfix/master.cf: + smtps inet n - n - - smtpd + -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes + +CClliieenntt cceerrttiiffiiccaattee vveerriiffiiccaattiioonn + +To receive a remote SMTP client certificate, the Postfix SMTP server must +explicitly ask for one (any contents of $smtpd_tls_CAfile are also sent to the +client as a hint for choosing a certificate from a suitable CA). Unfortunately, +Netscape clients will either complain if no matching client certificate is +available or will offer the user client a list of certificates to choose from. +Additionally some MTAs (notably some versions of qmail) are unable to complete +TLS negotiation when client certificates are requested, and abort the SMTP +session. So this option is "off" by default. You will however need the +certificate if you want to use certificate based relaying with, for example, +the permit_tls_clientcerts feature. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_ask_ccert = no + +You may also decide to REQUIRE a remote SMTP client certificate before allowing +TLS connections. This feature is included for completeness, and implies +"smtpd_tls_ask_ccert = yes". + +Please be aware, that this will inhibit TLS connections without a proper client +certificate and that it makes sense only when non-TLS submission is disabled +(smtpd_enforce_tls = yes). Otherwise, clients could bypass the restriction by +simply not using STARTTLS at all. + +When TLS is not enforced, the connection will be handled as if only +"smtpd_tls_ask_ccert = yes" is specified, and a warning is logged. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_req_ccert = no + +A client certificate verification depth of 1 is sufficient if the certificate +is directly issued by a CA listed in the CA file. The default value (5) should +also suffice for longer chains (root CA issues special CA which then issues the +actual certificate...) + +Example: + + /etc/postfix/main.cf: + smtpd_tls_ccert_verifydepth = 5 + +SSuuppppoorrttiinngg AAUUTTHH oovveerr TTLLSS oonnllyy + +Sending AUTH data over an unencrypted channel poses a security risk. When TLS +layer encryption is required (smtpd_enforce_tls = yes), the Postfix SMTP server +will announce and accept AUTH only after the TLS layer has been activated with +STARTTLS. When TLS layer encryption is optional (smtpd_enforce_tls = no), it +may however still be useful to only offer AUTH when TLS is active. To maintain +compatibility with non-TLS clients, the default is to accept AUTH without +encryption. In order to change this behavior, set "smtpd_tls_auth_only = yes". + +Example: + + /etc/postfix/main.cf: + smtpd_tls_auth_only = no + +SSeerrvveerr--ssiiddee TTLLSS sseessssiioonn ccaacchhee + +The Postfix SMTP server and the remote SMTP client negotiate a session, which +takes some computer time and network bandwidth. By default, this session +information is cached only in the smtpd(8) process actually using this session +and is lost when the process terminates. To share the session information +between multiple smtpd(8) processes, a persistent session cache can be used. +You can specify any database type that can store objects of several kbytes and +that supports the sequence operator. DBM databases are not suitable because +they can only store small objects. The cache is maintained by the tlsmgr(8) +process, so there is no problem with concurrent access. Session caching is +highly recommended, because the cost of repeatedly negotiating TLS session keys +is high. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_session_cache_database = btree:/etc/postfix/smtpd_scache + +As of version 2.5, Postfix will no longer maintain this file in a directory +with non-Postfix ownership. As a migration aid, attempts to open such files are +redirected to the Postfix-owned $data_directory, and a warning is logged. + +Cached Postfix SMTP server session information expires after a certain amount +of time. Postfix/TLS does not use the OpenSSL default of 300s, but a longer +time of 3600sec (=1 hour). RFC 2246 recommends a maximum of 24 hours. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_session_cache_timeout = 3600s + +SSeerrvveerr aacccceessss ccoonnttrrooll + +Postfix TLS support introduces three additional features for Postfix SMTP +server access control: + + permit_tls_clientcerts + Allow the remote SMTP client SMTP request if the client certificate + passes verification, and if its fingerprint is listed in the list of + client certificates (see relay_clientcerts discussion below). + + permit_tls_all_clientcerts + Allow the remote client SMTP request if the client certificate passes + verification. + + check_ccert_access type:table + If the client certificate passes verification, use its fingerprint as a + key for the specified access(5) table. + +The permit_tls_all_clientcerts feature must be used with caution, because it +can result in too many access permissions. Use this feature only if a special +CA issues the client certificates, and only if this CA is listed as a trusted +CA. If other CAs are trusted, any owner of a valid client certificate would be +authorized. The permit_tls_all_clientcerts feature can be practical for a +specially created email relay server. + +It is however recommended to stay with the permit_tls_clientcerts feature and +list all certificates via $relay_clientcerts, as permit_tls_all_clientcerts +does not permit any control when a certificate must no longer be used (e.g. an +employee leaving). + +Example: + + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + ... + permit_tls_clientcerts + reject_unauth_destination + ... + +The Postfix list manipulation routines give special treatment to whitespace and +some other characters, making the use of certificate names impractical. Instead +we use the certificate fingerprints as they are difficult to fake but easy to +use for lookup. Postfix lookup tables are in the form of (key, value) pairs. +Since we only need the key, the value can be chosen freely, e.g. the name of +the user or host. + +Example: + + /etc/postfix/main.cf: + relay_clientcerts = hash:/etc/postfix/relay_clientcerts + + /etc/postfix/relay_clientcerts: + D7:04:2F:A7:0B:8C:A5:21:FA:31:77:E1:41:8A:EE:80 lutzpc.at.home + +SSeerrvveerr--ssiiddee cciipphheerr ccoonnttrroollss + +To influence the Postfix SMTP server cipher selection scheme, you can give +cipherlist string. A detailed description would go too far here; please refer +to the OpenSSL documentation. If you don't know what to do with it, simply +don't touch it and leave the (openssl-)compiled in default! + +DO NOT USE " to enclose the string, specify just the string!!! + +Example: + + /etc/postfix/main.cf: + smtpd_tls_cipherlist = DEFAULT + +If you want to take advantage of ciphers with EDH, DH parameters are needed. +Instead of using the built-in DH parameters for both 1024bit and 512bit, it is +better to generate "own" parameters, since otherwise it would "pay" for a +possible attacker to start a brute force attack against parameters that are +used by everybody. For this reason, the parameters chosen are already different +from those distributed with other TLS packages. + +To generate your own set of DH parameters, use: + + % ooppeennssssll ggeennddhh --oouutt //eettcc//ppoossttffiixx//ddhh__11002244..ppeemm --22 --rraanndd //vvaarr//rruunn//eeggdd--ppooooll + 11002244 + % ooppeennssssll ggeennddhh --oouutt //eettcc//ppoossttffiixx//ddhh__551122..ppeemm --22 --rraanndd //vvaarr//rruunn//eeggdd--ppooooll 551122 + +Examples: + + /etc/postfix/main.cf: + smtpd_tls_dh1024_param_file = /etc/postfix/dh_1024.pem + smtpd_tls_dh512_param_file = /etc/postfix/dh_512.pem + +MMiisscceellllaanneeoouuss sseerrvveerr ccoonnttrroollss + +The smtpd_starttls_timeout parameter limits the time of Postfix SMTP server +write and read operations during TLS startup and shutdown handshake procedures. + +Example: + + /etc/postfix/main.cf: + smtpd_starttls_timeout = 300s + +SSMMTTPP CClliieenntt ssppeecciiffiicc sseettttiinnggss + +Topics covered in this section: + + * Client-side certificate and private key configuration + * Client-side TLS activity logging + * Client-side TLS session cache + * Enabling TLS in the Postfix SMTP client + * Requiring TLS encryption + * Disabling server certificate verification + * Per-site TLS policies + * Closing a DNS loophole with per-site TLS policies + * Discovering servers that support TLS + * Server certificate verification depth + * Client-side cipher controls + * Miscellaneous client controls + +CClliieenntt--ssiiddee cceerrttiiffiiccaattee aanndd pprriivvaattee kkeeyy ccoonnffiigguurraattiioonn + +During TLS startup negotiation the Postfix SMTP client may present a +certificate to the remote SMTP server. The Netscape client is rather clever +here and lets the user select between only those certificates that match CA +certificates offered by the remote SMTP server. As the Postfix SMTP client uses +the "SSL_connect()" function from the OpenSSL package, this is not possible and +we have to choose just one certificate. So for now the default is to use _no_ +certificate and key unless one is explicitly specified here. + +Both RSA and DSA certificates are supported. You can have both at the same +time, in which case the cipher used determines which certificate is presented. + +It is possible for the Postfix SMTP client to use the same key/certificate pair +as the Postfix SMTP server. If a certificate is to be presented, it must be in +"pem" format. The private key must not be encrypted, meaning: it must be +accessible without a password. Both parts (certificate and private key) may be +in the same file. + +In order for remote SMTP servers to verify the Postfix SMTP client +certificates, the CA certificate (in case of a certificate chain, all CA +certificates) must be available. You should add these certificates to the +client certificate, the client certificate first, then the issuing CA(s). + +Example: the certificate for "client.example.com" was issued by "intermediate +CA" which itself has a certificate of "root CA". Create the client.pem file +with: + + % ccaatt cclliieenntt__cceerrtt..ppeemm iinntteerrmmeeddiiaattee__CCAA..ppeemm >> cclliieenntt..ppeemm + +A Postfix SMTP client certificate supplied here must be usable as an SSL client +certificate and hence pass the "openssl verify -purpose sslclient ..." test. + +A server that trusts the root CA has a local copy of the root CA certificate, +so it is not necessary to include the root CA certificate here. Leaving it out +of the "client.pem" file reduces the overhead of the TLS exchange. + +If you want the Postfix SMTP client to accept remote SMTP server certificates +issued by these CAs, append the root certificate to $smtp_tls_CAfile or install +it in the $smtp_tls_CApath directory. When you configure trust in a root CA, it +is not necessary to explicitly trust intermediary CAs signed by the root CA, +unless $smtp_tls_scert_verifydepth is less than the number of CAs in the +certificate chain for the servers of interest. With a verify depth of 1 you can +only verify certificates directly signed by a trusted CA, and all trusted +intermediary CAs need to be configured explicitly. With a verify depth of 2 you +can verify servers signed by a root CA or a direct intermediary CA (so long as +the server is correctly configured to supply its intermediate CA certificate). + +RSA key and certificate examples: + + /etc/postfix/main.cf: + smtp_tls_cert_file = /etc/postfix/client.pem + smtp_tls_key_file = $smtp_tls_cert_file + +Their DSA counterparts: + + /etc/postfix/main.cf: + smtp_tls_dcert_file = /etc/postfix/client-dsa.pem + smtp_tls_dkey_file = $smtp_tls_dcert_file + +To verify a remote SMTP server certificate, the Postfix SMTP client needs to +trust the certificates of the issuing Certification Authorities. These +certificates in "pem" format can be stored in a single $smtp_tls_CAfile or in +multiple files, one CA per file in the $smtp_tls_CApath directory. If you use a +directory, don't forget to create the necessary "hash" links with: + + # $$OOPPEENNSSSSLL__HHOOMMEE//bbiinn//cc__rreehhaasshh //ppaatthh//ttoo//ddiirreeccttoorryy + +The $smtp_tls_CAfile contains the CA certificates of one or more trusted CAs. +The file is opened (with root privileges) before Postfix enters the optional +chroot jail and so need not be accessible from inside the chroot jail. + +Additional trusted CAs can be specified via the $smtp_tls_CApath directory, in +which case the certificates are read (with $mail_owner privileges) from the +files in the directory when the information is needed. Thus, the +$smtp_tls_CApath directory needs to be accessible inside the optional chroot +jail. + +The choice between $smtp_tls_CAfile and $smtp_tls_CApath is a space/time +tradeoff. If there are many trusted CAs, the cost of preloading them all into +memory may not pay off in reduced access time when the certificate is needed. + +Example: + + /etc/postfix/main.cf: + smtp_tls_CAfile = /etc/postfix/CAcert.pem + smtp_tls_CApath = /etc/postfix/certs + +CClliieenntt--ssiiddee TTLLSS aaccttiivviittyy llooggggiinngg + +To get additional information about Postfix SMTP client TLS activity you can +increase the loglevel from 0..4. Each logging level also includes the +information that is logged at a lower logging level. + + 0 Disable logging of TLS activity. + + 1 Log TLS handshake and certificate information. + + 2 Log levels during TLS negotiation. + + 3 Log hexadecimal and ASCII dump of TLS negotiation process + + 4 Log hexadecimal and ASCII dump of complete transmission after STARTTLS + +Example: + + /etc/postfix/main.cf: + smtp_tls_loglevel = 0 + +CClliieenntt--ssiiddee TTLLSS sseessssiioonn ccaacchhee + +The remote SMTP server and the Postfix SMTP client negotiate a session, which +takes some computer time and network bandwidth. By default, this session +information is cached only in the smtp(8) process actually using this session +and is lost when the process terminates. To share the session information +between multiple smtp(8) processes, a persistent session cache can be used. You +can specify any database type that can store objects of several kbytes and that +supports the sequence operator. DBM databases are not suitable because they can +only store small objects. The cache is maintained by the tlsmgr(8) process, so +there is no problem with concurrent access. Session caching is highly +recommended, because the cost of repeatedly negotiating TLS session keys is +high. Future Postfix SMTP servers may limit the number of sessions that a +client is allowed to negotiate per unit time. + +Example: + + /etc/postfix/main.cf: + smtp_tls_session_cache_database = btree:/etc/postfix/smtp_scache + +As of version 2.5, Postfix will no longer maintain this file in a directory +with non-Postfix ownership. As a migration aid, attempts to open such files are +redirected to the Postfix-owned $data_directory, and a warning is logged. + +Cached Postfix SMTP client session information expires after a certain amount +of time. Postfix/TLS does not use the OpenSSL default of 300s, but a longer +time of 3600s (=1 hour). RFC 2246 recommends a maximum of 24 hours. + +Example: + + /etc/postfix/main.cf: + smtp_tls_session_cache_timeout = 3600s + +EEnnaabblliinngg TTLLSS iinn tthhee PPoossttffiixx SSMMTTPP cclliieenntt + +By default, TLS is disabled in the Postfix SMTP client, so no difference to +plain Postfix is visible. If you enable TLS, the Postfix SMTP client will send +STARTTLS when TLS support is announced by the remote SMTP server. + +When the server accepts the STARTTLS command, but the subsequent TLS handshake +fails, and no other server is available, the Postfix SMTP client defers the +delivery attempt, and the mail stays in the queue. After a handshake failure, +the communications channel is in an indeterminate state and cannot be used for +non-TLS deliveries. + +Example: + + /etc/postfix/main.cf: + smtp_use_tls = yes + +RReeqquuiirriinngg TTLLSS eennccrryyppttiioonn + +You can ENFORCE the use of TLS, so that the Postfix SMTP client will not +deliver mail over unencrypted connections. In this mode, the remote SMTP server +hostname must match the information in the remote server certificate, and the +server certificate must be issued by a CA that is trusted by the Postfix SMTP +client. If the remote server certificate doesn't verify or the remote SMTP +server hostname doesn't match, and no other server is available, the delivery +attempt is deferred and the mail stays in the queue. + +The remote SMTP server hostname is verified against all names provided as +dNSNames in the SubjectAlternativeName. If no dNSNames are specified, the +CommonName is checked. Verification may be turned off with the +smtp_tls_enforce_peername option which is discussed below. + +Enforcing the use of TLS is useful if you know that you will only connect to +servers that support RFC 2487 _and_ that present server certificates that meet +the above requirements. An example would be a client only sends email to one +specific mailhub that offers the necessary STARTTLS support. + +Example: + + /etc/postfix/main.cf: + smtp_enforce_tls = yes + +DDiissaabblliinngg sseerrvveerr cceerrttiiffiiccaattee vveerriiffiiccaattiioonn + +As of RFC 2487 the requirements for hostname checking for MTA clients are not +set. When TLS is required (smtp_enforce_tls = yes), the option +smtp_tls_enforce_peername can be set to "no" to disable strict remote SMTP +server hostname checking. In this case, the mail delivery will proceed +regardless of the CommonName etc. listed in the certificate. + +Despite the potential for eliminating "man-in-the-middle" and other attacks, +mandatory certificate/peername verification is not viable as a default Internet +mail delivery policy at this time. A significant fraction of TLS enabled MTAs +uses self-signed certificates, or certificates that are signed by a private +Certification Authority. On a machine that delivers mail to the Internet, if +you set smtp_enforce_tls = yes, you should probably also set +smtp_tls_enforce_peername = no. You can use the per-site TLS policies (see +below) to enable full peer verification for specific destinations that are +known to have verifiable TLS server certificates. + +Example: + + /etc/postfix/main.cf: + smtp_enforce_tls = yes + smtp_tls_enforce_peername = no + +PPeerr--ssiittee TTLLSS ppoolliicciieess + +A small fraction of servers offer STARTTLS but the negotiation consistently +fails, leading to mail aging out of the queue and bouncing back to the sender. +In such cases, you can use the per-site policies to disable TLS for the problem +sites. Alternatively, you can enable TLS for just a few specific sites and not +enable it for all sites. + +The smtp_tls_per_site table is searched for a policy that matches the following +information: + + remote SMTP server hostname + This is simply the DNS name of the server that the Postfix SMTP client + connects to; this name may be obtained from other DNS lookups, such as + MX lookups or CNAME lookups. + next-hop destination + This is normally the domain portion of the recipient address, but it + may be overruled by information from the transport(5) table, from the + relayhost parameter setting, or from the relay_transport setting. When + it's not the recipient domain, the next-hop destination can have the + Postfix-specific form "[name]", [name]:port", "name" or "name:port". + +When both the hostname lookup and the next-hop lookup succeed, the host policy +does not automatically override the next-hop policy. Instead, precedence is +given to either the more specific or the more secure per-site policy as +described below. + +The smtp_tls_per_site table uses a simple "name whitespace value" format. +Specify host names or next-hop destinations on the left-hand side; no wildcards +are allowed. On the right hand side specify one of the following keywords: + + NONE + Don't use TLS at all. This overrides a less specific MMAAYY lookup result + from the alternate host or next-hop lookup key, and overrides the + global smtp_use_tls, smtp_enforce_tls, and smtp_tls_enforce_peername + settings. + MAY + Try to use TLS if the server announces support, otherwise use the + unencrypted connection. This has less precedence than a more specific + result (including NNOONNEE) from the alternate host or next-hop lookup key, + and has less precedence than the more specific global "smtp_enforce_tls + = yes" or "smtp_tls_enforce_peername = yes". + MUST_NOPEERMATCH + Require TLS encryption, but do not require that the remote SMTP server + hostname matches the information in the remote SMTP server certificate, + or that the server certificate was issued by a trusted CA. This + overrides a less secure NNOONNEE or a less specific MMAAYY lookup result from + the alternate host or next-hop lookup key, and overrides the global + smtp_use_tls, smtp_enforce_tls and smtp_tls_enforce_peername settings. + MUST + Require TLS encryption, require that the remote SMTP server hostname + matches the information in the remote SMTP server certificate, and + require that the remote SMTP server certificate was issued by a trusted + CA. This overrides a less secure NNOONNEE and MMUUSSTT__NNOOPPEEEERRMMAATTCCHH or a less + specific MMAAYY lookup result from the alternate host or next-hop lookup + key, and overrides the global smtp_use_tls, smtp_enforce_tls and + smtp_tls_enforce_peername settings. + +The precedences between global (main.cf) and per-site TLS policies can be +summarized as follows: + + * When neither the remote SMTP server hostname nor the next-hop destination + are found in the smtp_tls_per_site table, the policy is based on + smtp_use_tls, smtp_enforce_tls and smtp_tls_enforce_peername. Note: + "smtp_enforce_tls = yes" and "smtp_tls_enforce_peername = yes" imply + "smtp_use_tls = yes". + + * When both hostname and next-hop destination lookups produce a result, the + more specific per-site policy (NONE, MUST, etc.) overrides the less + specific one (MAY), and the more secure per-site policy (MUST, etc.) + overrides the less secure one (NONE). + + * After the per-site policy lookups are combined, the result generally + overrides the global policy. The exception is the less specific MMAAYY per- + site policy, which is overruled by the more specific global + "smtp_enforce_tls = yes" with server certificate verification as specified + with the smtp_tls_enforce_peername parameter. + +CClloossiinngg aa DDNNSS lloooopphhoollee wwiitthh ppeerr--ssiittee TTLLSS ppoolliicciieess + +As long as no secure DNS lookup mechanism is available, false hostnames in MX +or CNAME responses can change the server hostname that Postfix uses for TLS +policy lookup and server certificate verification. Even with a perfect match +between the server hostname and the server certificate, there is no guarantee +that Postfix is connected to the right server. To avoid this loophole take the +following steps: + + * Eliminate MX lookups. Specify local transport(5) table entries for + sensitive domains with explicit smtp:[mailhost] or smtp:[mailhost]:port + destinations (you can assure security of this table unlike DNS); in the + smtp_tls_per_site table specify the value MMUUSSTT for the key [mailhost] or + smtp:[mailhost]:port. This prevents false hostname information in DNS MX + records from changing the server hostname that Postfix uses for TLS policy + lookup and server certificate verification. + + * Disallow CNAME hostname overrides. In main.cf specify + "smtp_cname_overrides_servername = no". This prevents false hostname + information in DNS CNAME records from changing the server hostname that + Postfix uses for TLS policy lookup and server certificate verification. + This feature requires Postfix 2.2.9 or later. + +Example: + + /etc/postfix/main.cf: + smtp_tls_per_site = hash:/etc/postfix/tls_per_site + relayhost = [msa.example.net]:587 + + /etc/postfix/tls_per_site: + # relayhost exact nexthop match + [msa.example.net]:587 MUST + + # TLS should not be used with the example.org MX hosts. + example.org NONE + + # TLS should not be used with the host smtp.example.com. + [smtp.example.com] NONE + +DDiissccoovveerriinngg sseerrvveerrss tthhaatt ssuuppppoorrtt TTLLSS + +As we decide on a "per site" basis whether or not to use TLS, it would be good +to have a list of sites that offered "STARTTLS". We can collect it ourselves +with this option. + +If the smtp_tls_note_starttls_offer feature is enabled and a server offers +STARTTLS while TLS is not already enabled for that server, the Postfix SMTP +client logs a line as follows: + + postfix/smtp[pid]: Host offered STARTTLS: [hostname.example.com] + +Example: + + /etc/postfix/main.cf: + smtp_tls_note_starttls_offer = yes + +SSeerrvveerr cceerrttiiffiiccaattee vveerriiffiiccaattiioonn ddeepptthh + +When verifying a remote SMTP server certificate, a verification depth of 1 is +sufficient if the certificate is directly issued by a CA specified with +smtp_tls_CAfile or smtp_tls_CApath. The default value of 5 should also suffice +for longer chains (root CA issues special CA which then issues the actual +certificate...) + +Example: + + /etc/postfix/main.cf: + smtp_tls_scert_verifydepth = 5 + +CClliieenntt--ssiiddee cciipphheerr ccoonnttrroollss + +To influence the Postfix SMTP client cipher selection scheme, you can give +cipherlist string. A detailed description would go too far here; please refer +to the OpenSSL documentation. If you don't know what to do with it, simply +don't touch it and leave the (openssl-)compiled in default! + +DO NOT USE " to enclose the string, specify just the string!!! + +Example: + + /etc/postfix/main.cf: + smtp_tls_cipherlist = DEFAULT + +MMiisscceellllaanneeoouuss cclliieenntt ccoonnttrroollss + +The smtp_starttls_timeout parameter limits the time of Postfix SMTP client +write and read operations during TLS startup and shutdown handshake procedures. +In case of problems the Postfix SMTP client tries the next network address on +the mail exchanger list, and defers delivery if no alternative server is +available. + +Example: + + /etc/postfix/main.cf: + smtp_starttls_timeout = 300s + +TTLLSS mmaannaaggeerr ssppeecciiffiicc sseettttiinnggss + +The security of cryptographic software such as TLS depends critically on the +ability to generate unpredictable numbers for keys and other information. To +this end, the tlsmgr(8) process maintains a Pseudo Random Number Generator +(PRNG) pool. This is queried by the smtp(8) and smtpd(8) processes when they +initialize. By default, these daemons request 32 bytes, the equivalent to 256 +bits. This is more than sufficient to generate a 128bit (or 168bit) session +key. + +Example: + + /etc/postfix/main.cf: + tls_daemon_random_bytes = 32 + +In order to feed its in-memory PRNG pool, the tlsmgr(8) reads entropy from an +external source, both at startup and during run-time. Specify a good entropy +source, like EGD or /dev/urandom; be sure to only use non-blocking sources (on +OpenBSD, use /dev/arandom when tlsmgr(8) complains about /dev/urandom timeout +errors). If the entropy source is not a regular file, you must prepend the +source type to the source name: "dev:" for a device special file, or "egd:" for +a source with EGD compatible socket interface. + +Examples (specify only one in main.cf): + + /etc/postfix/main.cf: + tls_random_source = dev:/dev/urandom + tls_random_source = egd:/var/run/egd-pool + +By default, tlsmgr(8) reads 32 bytes from the external entropy source at each +seeding event. This amount (256bits) is more than sufficient for generating a +128bit symmetric key. With EGD and device entropy sources, the tlsmgr(8) limits +the amount of data read at each step to 255 bytes. If you specify a regular +file as entropy source, a larger amount of data can be read. + +Example: + + /etc/postfix/main.cf: + tls_random_bytes = 32 + +In order to update its in-memory PRNG pool, the tlsmgr(8) queries the external +entropy source again after a pseudo-random amount of time. The time is +calculated using the PRNG, and is between 0 and the maximal time specified with +tls_random_reseed_period. The default maximal time interval is 1 hour. + +Example: + + /etc/postfix/main.cf: + tls_random_reseed_period = 3600s + +The tlsmgr(8) process saves the PRNG state to a persistent exchange file at +regular times and when the process terminates, so that it can recover the PRNG +state the next time it starts up. This file is created when it does not exist. +Its default location is under the Postfix configuration directory, which is not +the proper place for information that is modified by Postfix. Instead, the file +location should probably be on the /var partition (but nnoott inside the chroot +jail). + +Examples: + + /etc/postfix/main.cf: + tls_random_exchange_name = /etc/postfix/prng_exch + tls_random_prng_update_period = 3600s + +GGeettttiinngg ssttaarrtteedd,, qquuiicckk aanndd ddiirrttyy + +The following steps will get you started quickly. Because you sign your own +Postfix public key certificate, you get TLS encryption but no TLS +authentication. This is sufficient for testing, and for exchanging email with +sites that you have no trust relationship with. For real authentication, your +Postfix public key certificate needs to be signed by a recognized Certification +Authority, and Postfix needs to be configured with a list of public key +certificates of Certification Authorities, so that Postfix can verify the +public key certificates of remote hosts. + +In the examples below, user input is shown in bboolldd font, and a "#" prompt +indicates a super-user shell. + + * Become your own Certification Authority, so that you can sign your own + public keys. This example uses the CA.pl script that ships with OpenSSL. By + default, OpenSSL installs this as /usr/local/ssl/misc/CA.pl, but your + mileage may vary. The script creates a private key in ./demoCA/private/ + cakey.pem and a public key in ./demoCA/cacert.pem. + + % //uussrr//llooccaall//ssssll//mmiisscc//CCAA..ppll --nneewwccaa + CA certificate filename (or enter to create) + + Making CA certificate ... + Using configuration from /etc/ssl/openssl.cnf + Generating a 1024 bit RSA private key + ....................++++++ + .....++++++ + writing new private key to './demoCA/private/cakey.pem' + Enter PEM pass phrase:wwhhaatteevveerr + + * Create an unpassworded private key for host FOO and create an unsigned + public key certificate. + + % ooppeennssssll rreeqq --nneeww --nnooddeess --kkeeyyoouutt FFOOOO--kkeeyy..ppeemm --oouutt FFOOOO--rreeqq..ppeemm --ddaayyss + 336655 + Using configuration from /etc/ssl/openssl.cnf + Generating a 1024 bit RSA private key + ........................................++++++ + ....++++++ + writing new private key to 'FOO-key.pem' + ----- + You are about to be asked to enter information that will be + incorporated + into your certificate request. + What you are about to enter is what is called a Distinguished Name or a + DN. + There are quite a few fields but you can leave some blank + For some fields there will be a default value, + If you enter '.', the field will be left blank. + ----- + Country Name (2 letter code) [AU]:UUSS + State or Province Name (full name) [Some-State]:NNeeww YYoorrkk + Locality Name (eg, city) []:WWeessttcchheesstteerr + Organization Name (eg, company) [Internet Widgits Pty Ltd]:PPoorrccuuppiinnee + Organizational Unit Name (eg, section) []: + Common Name (eg, YOUR name) []:FFOOOO + Email Address []:wwiieettssee@@ppoorrccuuppiinnee..oorrgg + + Please enter the following 'extra' attributes + to be sent with your certificate request + A challenge password []:wwhhaatteevveerr + An optional company name []: + + * Sign the public key certificate for host FOO with the Certification + Authority private key that we created a few steps ago. + + % ooppeennssssll ccaa --oouutt FFOOOO--cceerrtt..ppeemm --iinnffiilleess FFOOOO--rreeqq..ppeemm + Uing configuration from /etc/ssl/openssl.cnf + Enter PEM pass phrase:wwhhaatteevveerr + Check that the request matches the signature + Signature ok + The Subjects Distinguished Name is as follows + countryName :PRINTABLE:'US' + stateOrProvinceName :PRINTABLE:'New York' + localityName :PRINTABLE:'Westchester' + organizationName :PRINTABLE:'Porcupine' + commonName :PRINTABLE:'FOO' + emailAddress :IA5STRING:'wietse@porcupine.org' + Certificate is to be certified until Nov 21 19:40:56 2005 GMT (365 + days) + Sign the certificate? [y/n]:yy + + 1 out of 1 certificate requests certified, commit? [y/n]yy + Write out database with 1 new entries + Data Base Updated + + * Install the host private key, the host public key certificate, and the + Certification Authority certificate files. This requires super-user + privileges. + + # ccpp ddeemmooCCAA//ccaacceerrtt..ppeemm FFOOOO--kkeeyy..ppeemm FFOOOO--cceerrtt..ppeemm //eettcc//ppoossttffiixx + # cchhmmoodd 664444 //eettcc//ppoossttffiixx//FFOOOO--cceerrtt..ppeemm //eettcc//ppoossttffiixx//ccaacceerrtt..ppeemm + # cchhmmoodd 440000 //eettcc//ppoossttffiixx//FFOOOO--kkeeyy..ppeemm + + * Configure Postfix, by adding the following to /etc/postfix/main.cf. + + smtp_tls_CAfile = /etc/postfix/cacert.pem + smtp_tls_cert_file = /etc/postfix/FOO-cert.pem + smtp_tls_key_file = /etc/postfix/FOO-key.pem + smtp_tls_session_cache_database = btree:/var/run/smtp_tls_session_cache + smtp_use_tls = yes + smtpd_tls_CAfile = /etc/postfix/cacert.pem + smtpd_tls_cert_file = /etc/postfix/FOO-cert.pem + smtpd_tls_key_file = /etc/postfix/FOO-key.pem + smtpd_tls_received_header = yes + smtpd_tls_session_cache_database = btree:/var/run/ + smtpd_tls_session_cache + smtpd_use_tls = yes + tls_random_source = dev:/dev/urandom + +RReeppoorrttiinngg pprroobblleemmss + +When reporting a problem, please be thorough in the report. Patches, when +possible, are greatly appreciated too. + +Please differentiate when possible between: + + * Problems in the TLS code: <postfix_tls@aet.tu-cottbus.de> + * Problems in vanilla Postfix: <postfix-users@postfix.org> + +CCoommppaattiibbiilliittyy wwiitthh PPoossttffiixx << 22..22 TTLLSS ssuuppppoorrtt + +Postfix version 2.2 TLS support is based on the Postfix/TLS patch by Lutz +Ja"nicke, but differs in a few minor ways. + + * main.cf: Specify "btree" instead of "sdbm" for TLS session cache databases. + + TLS session cache databases are now accessed only by the tlsmgr(8) process, + so there are no more concurrency issues. Although Postfix has an sdbm + client, the sdbm library (1000 lines of code) is not included with Postfix. + + TLS session caches can use any database that can store objects of several + kbytes or more, and that implements the sequence operation. In most cases, + btree databases should be adequate. + + NOTE: You cannot use dbm databases. TLS session objects are too large. + + * master.cf: Specify "unix" instead of "fifo" as the tlsmgr service type. + + The smtp(8) and smtpd(8) processes now use a client-server protocol in + order to access the tlsmgr(8) pseudo-random number generation (PRNG) pool, + and in order to access the TLS session cache databases. Such a protocol + cannot be run across fifos. + + * smtp_tls_per_site: the MUST_NOPEERMATCH per-site policy cannot override the + global "smtp_tls_enforce_peername = yes" setting. + + * smtp_tls_per_site: a combined (NONE + MAY) lookup result for (hostname and + next-hop destination) produces counter-intuitive results for different + main.cf settings. TLS is enabled with "smtp_tls_enforce_peername = no", but + it is disabled when both "smtp_enforce_tls = yes" and + "smtp_tls_enforce_peername = yes". + +The smtp_tls_per_site limitations were removed by the end of the Postfix 2.2 +support cycle. + +CCrreeddiittss + + * TLS support for Postfix was originally developed by Lutz Ja"nicke at + Cottbus Technical University. + * Wietse Venema adopted the code, did some restructuring, and compiled this + part of the documentation from Lutz's documents. + * Victor Duchovni was instrumental with the re-implementation of the + smtp_tls_per_site code in terms of enforcement levels, which simplified the + implementation greatly. + diff --git a/README_FILES/TLS_README b/README_FILES/TLS_README new file mode 100644 index 0000000..e7fd259 --- /dev/null +++ b/README_FILES/TLS_README @@ -0,0 +1,2491 @@ +PPoossttffiixx TTLLSS SSuuppppoorrtt + +------------------------------------------------------------------------------- + +WWhhaatt PPoossttffiixx TTLLSS ssuuppppoorrtt ddooeess ffoorr yyoouu + +Transport Layer Security (TLS, formerly called SSL) provides certificate-based +authentication and encrypted sessions. An encrypted session protects the +information that is transmitted with SMTP mail or with SASL authentication. + +NOTE: By turning on TLS support in Postfix, you not only get the ability to +encrypt mail and to authenticate remote SMTP clients or servers. You also turn +on hundreds of thousands of lines of OpenSSL library code. Assuming that +OpenSSL is written as carefully as Wietse's own code, every 1000 lines +introduces one additional bug into Postfix. + +Topics covered in this document: + + * How Postfix TLS support works + * SMTP Server specific settings + * SMTP Client specific settings + * TLS manager specific settings + * Building Postfix with TLS support + * Reporting problems + * Credits + +And last but not least, for the impatient: + + * Getting started, quick and dirty + +HHooww PPoossttffiixx TTLLSS ssuuppppoorrtt wwoorrkkss + +The diagram below shows the main elements of the Postfix TLS architecture and +their relationships. Colored boxes with numbered names represent Postfix daemon +programs. Other colored boxes represent storage elements. + + * The smtpd(8) server implements the SMTP over TLS server side. + + * The smtp(8) client implements the SMTP (and LMTP) over TLS client side. + + * The tlsmgr(8) server maintains the pseudo-random number generator (PRNG) + that seeds the TLS engines in the smtpd(8) server and smtp(8) client + processes, and maintains the TLS session key cache files. + +Not shown in the figure are the tlsproxy(8) server and the postscreen(8) +server. These use TLS in the same manner as smtpd(8). + + <---seed---- ----seed---> +Network-> smtpd(8) tlsmgr(8) smtp(8) ->Network + <-key/cert-> <-key/cert-> + + / | \ + | + / \ + + smtpd PRNG smtp + session state session + key cache file key cache + +SSMMTTPP SSeerrvveerr ssppeecciiffiicc sseettttiinnggss + +Topics covered in this section: + + * Server-side certificate and private key configuration + * Server-side forward-secrecy configuration + * Server-side TLS activity logging + * Enabling TLS in the Postfix SMTP server + * Client certificate verification + * Supporting AUTH over TLS only + * Server-side TLS session cache + * Server access control + * Server-side cipher controls + * Miscellaneous server controls + +SSeerrvveerr--ssiiddee cceerrttiiffiiccaattee aanndd pprriivvaattee kkeeyy ccoonnffiigguurraattiioonn + +In order to use TLS, the Postfix SMTP server generally needs a certificate and +a private key. Both must be in "PEM" format. The private key must not be +encrypted, meaning: the key must be accessible without a password. The +certificate and private key may be in the same file, in which case the +certificate file should be owned by "root" and not be readable by any other +user. If the key is stored separately, this access restriction applies to the +key file only, and the certificate file may be "world-readable". + +Public Internet MX hosts without certificates signed by a well-known public CA +must still generate, and be prepared to present to most clients, a self-signed +or private-CA signed certificate. The remote SMTP client will generally not be +able to verify the self-signed certificate, but unless the client is running +Postfix or similar software, it will only negotiate TLS ciphersuites that +require a server certificate. + +For servers that are nnoott public Internet MX hosts, Postfix supports +configurations with no certificates. This entails the use of just the anonymous +TLS ciphers, which are not supported by typical SMTP clients. Since some +clients may not fall back to plain text after a TLS handshake failure, a +certificate-less Postfix SMTP server will be unable to receive email from some +TLS-enabled clients. To avoid accidental configurations with no certificates, +Postfix enables certificate-less operation only when the administrator +explicitly sets "smtpd_tls_cert_file = none". This ensures that new Postfix +SMTP server configurations will not accidentally enable TLS without +certificates. + +Note that server certificates are nnoott optional in TLS 1.3. To run without +certificates you'd have to disable the TLS 1.3 protocol by including +"<=TLSv1.2" (or, for Postfix < 3.6, "!TLSv1.3") in "smtpd_tls_protocols" and +perhaps also "smtpd_tls_mandatory_protocols". It is simpler instead to just +configure a certificate chain. Certificate-less operation is not recommended. + +RSA, DSA and ECDSA (Postfix >= 2.6) certificates are supported. Most sites only +have RSA certificates. You can configure all three at the same time, in which +case the ciphersuite negotiated with the remote SMTP client determines which +certificate is used. If your DNS zone is signed, and you want to publish DANE +TLSA (RFC 6698, RFC 7671, RFC 7672) records, these must match all of the +configured certificate chains. Since the best practice is to publish "3 1 1" +certificate associations, create a separate TLSA record to match each public- +key certificate digest. + +CCrreeaattiinngg tthhee sseerrvveerr cceerrttiiffiiccaattee ffiillee + +To verify the Postfix SMTP server certificate, the remote SMTP client must +receive the issuing CA certificates via the TLS handshake or via public-key +infrastructure. This means that the Postfix server public-key certificate file +must include the server certificate first, then the issuing CA(s) (bottom-up +order). The Postfix SMTP server certificate must be usable as an SSL server +certificate and hence pass the "openssl verify -purpose sslserver ..." test. + +The examples that follow show how to create a server certificate file. We +assume that the certificate for "server.example.com" was issued by +"intermediate CA" which itself has a certificate issued by "root CA". + + * With legacy public CA trust verification, you can omit the root certificate + from the "server.pem" certificate file. If the client trusts the root CA, + it will already have a local copy of the root CA certificate. Omitting the + root CA certificate reduces the size of the server TLS handshake. + + % ccaatt sseerrvveerr__cceerrtt..ppeemm iinntteerrmmeeddiiaattee__CCAA..ppeemm >> sseerrvveerr..ppeemm + + * If you publish DANE TLSA (RFC 6698, RFC 7671, RFC 7672) "2 0 1" or "2 1 1" + records to specify root CA certificate digests, you must include the + corresponding root CA certificates in the "server.pem" certificate file. + + % ccaatt sseerrvveerr__cceerrtt..ppeemm iinntteerrmmeeddiiaattee__CCAA..ppeemm rroooott..ppeemm >> sseerrvveerr..ppeemm + + Remote SMTP clients will be able to use the TLSA record you publish (which + only contains the certificate digest) only if they have access to the + corresponding certificate. Failure to verify certificates per the server's + published TLSA records will typically cause the SMTP client to defer mail + delivery. The foregoing also applies to "2 0 2" and "2 1 2" TLSA records or + any other digest of a CA certificate, but it is expected that SHA256 will + be by far the most common digest for TLSA. + + As a best practice, publish "3 1 1" TLSA associations that specify the + SHA256 digest of the server's public key. These continue to work unmodified + when a certificate is renewed with the same public/private key pair. + +For instructions on how to compute the digest of a certificate or its public +key for use in TLSA records, see the documentation of the +smtpd_tls_fingerprint_digest main.cf parameter. + +When a new key or certificate is generated, an additional TLSA record with the +new digest must be published in advance of the actual deployment of the new key +or certificate on the server. You must allow sufficient time for any TLSA +RRsets with only the old digest to expire from DNS caches. The safest practice +is to wait until the DNSSEC signature on the previous TLSA RRset expires, and +only then switch the server to use new keys published in the updated TLSA +RRset. Once the new certificate trust chain and private key are in effect, the +DNS should be updated once again to remove the old digest from the TLSA RRset. + +If you want the Postfix SMTP server to accept remote SMTP client certificates +issued by one or more root CAs, append the root certificate to +$smtpd_tls_CAfile or install it in the $smtpd_tls_CApath directory. + +CCoonnffiigguurriinngg tthhee sseerrvveerr cceerrttiiffiiccaattee aanndd kkeeyy ffiilleess + +Example: Postfix >= 3.4 all-in-one chain file(s). One or more chain files that +start with a key that is immediately followed by the corresponding certificate +and any additional issuer certificates. A single file can hold multiple (key, +cert, [chain]) sequences, one per algorithm. It is typically simpler to keep +the chain for each algorithm in its own file. Most users are likely to deploy +just a single RSA chain, but with OpenSSL 1.1.1, it is possible to deploy up to +five chains, one each for RSA, ECDSA, ED25519, ED448, and even the obsolete +DSA. + + # Postfix >= 3.4. Preferred configuration interface. Each file + # starts with the private key, followed by the corresponding + # certificate, and any intermediate issuer certificates. The root CA + # cert may also be needed when published as a DANE trust anchor. + # + smtpd_tls_chain_files = + /etc/postfix/rsa.pem, + /etc/postfix/ecdsa.pem, + /etc/postfix/ed25519.pem, + /etc/postfix/ed448.pem + +You can also store the keys separately from their certificates, again provided +each is listed before the corresponding certificate chain. Storing a key and +its associated certificate chain in separate files is not recommended, because +this is prone to race conditions during key rollover, as there is no way to +update multiple files atomically. + + # Postfix >= 3.4. + # Storing keys separately from the associated certificates is not + # recommended. + smtpd_tls_chain_files = + /etc/postfix/rsakey.pem, + /etc/postfix/rsacerts.pem, + /etc/postfix/ecdsakey.pem, + /etc/postfix/ecdsacerts.pem + +The below examples show the legacy algorithm-specific configurations for +Postfix 3.3 and older. With Postfix <= 3.3, even if the key is stored in the +same file as the certificate, the file is read twice and a (brief) race +condition still exists during key rollover. While Postfix >= 3.4 avoids the +race when the key and certificate are in the same file, you should use the new +"smtpd_tls_chain_files" interface shown above. + +RSA key and certificate examples: + + /etc/postfix/main.cf: + smtpd_tls_cert_file = /etc/postfix/server.pem + smtpd_tls_key_file = $smtpd_tls_cert_file + +Their DSA counterparts: + + /etc/postfix/main.cf: + smtpd_tls_dcert_file = /etc/postfix/server-dsa.pem + smtpd_tls_dkey_file = $smtpd_tls_dcert_file + +Their ECDSA counterparts (Postfix >= 2.6 + OpenSSL >= 1.0.0): + + /etc/postfix/main.cf: + # Some clients will not be ECDSA capable, so you will likely still need + # an RSA certificate and private key. + # + smtpd_tls_eccert_file = /etc/postfix/server-ecdsa.pem + smtpd_tls_eckey_file = $smtpd_tls_eccert_file + +TLS without certificates for servers serving exclusively anonymous-cipher +capable clients: + + /etc/postfix/main.cf: + # Not recommended: breaks TLS 1.3 and clients that don't support + # anonymous cipher suites. + smtpd_tls_cert_file = none + +To verify a remote SMTP client certificate, the Postfix SMTP server needs to +trust the certificates of the issuing Certification Authorities. These +certificates in "PEM" format can be stored in a single $smtpd_tls_CAfile or in +multiple files, one CA per file in the $smtpd_tls_CApath directory. If you use +a directory, don't forget to create the necessary "hash" links with: + + # $$OOPPEENNSSSSLL__HHOOMMEE//bbiinn//cc__rreehhaasshh //ppaatthh//ttoo//ddiirreeccttoorryy + +The $smtpd_tls_CAfile contains the CA certificates of one or more trusted CAs. +The file is opened (with root privileges) before Postfix enters the optional +chroot jail and so need not be accessible from inside the chroot jail. + +Additional trusted CAs can be specified via the $smtpd_tls_CApath directory, in +which case the certificates are read (with $mail_owner privileges) from the +files in the directory when the information is needed. Thus, the +$smtpd_tls_CApath directory needs to be accessible inside the optional chroot +jail. + +When you configure the Postfix SMTP server to request client certificates, the +DNs of Certification Authorities in $smtpd_tls_CAfile are sent to the client, +in order to allow it to choose an identity signed by a CA you trust. If no +$smtpd_tls_CAfile is specified, no preferred CA list is sent, and the client is +free to choose an identity signed by any CA. Many clients use a fixed identity +regardless of the preferred CA list and you may be able to reduce TLS +negotiation overhead by installing client CA certificates mostly or only in +$smtpd_tls_CApath. In the latter case you need not specify a $smtpd_tls_CAfile. + +Note, that unless client certificates are used to allow greater access to TLS +authenticated clients, it is best to not ask for client certificates at all, as +in addition to increased overhead some clients (notably in some cases qmail) +are unable to complete the TLS handshake when client certificates are +requested. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_CAfile = /etc/postfix/CAcert.pem + smtpd_tls_CApath = /etc/postfix/certs + +SSeerrvveerr--ssiiddee ffoorrwwaarrdd--sseeccrreeccyy ccoonnffiigguurraattiioonn + +If you want to take maximal advantage of ciphers that offer forward secrecy see +the Getting started section of FORWARD_SECRECY_README. The full document +conveniently presents all information about Postfix forward secrecy support in +one place: what forward secrecy is, how to tweak settings, and what you can +expect to see when Postfix uses ciphers with forward secrecy. + +SSeerrvveerr--ssiiddee TTLLSS aaccttiivviittyy llooggggiinngg + +To get additional information about Postfix SMTP server TLS activity you can +increase the log level from 0..4. Each logging level also includes the +information that is logged at a lower logging level. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |LLeevveell|PPoossttffiixx 22..99 aanndd llaatteerr |EEaarrlliieerr rreelleeaasseess.. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |0 |Disable logging of TLS activity. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |1 |Log only a summary message on TLS |Log the summary message, peer | + | |handshake completion -- no logging|certificate summary information| + | |of client certificate trust-chain |and unconditionally log trust- | + | |verification errors if client |chain verification errors. | + | |certificate verification is not | | + | |required. | | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |2 |Also log levels during TLS negotiation. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |3 |Also log hexadecimal and ASCII dump of TLS negotiation process. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |4 |Also log hexadecimal and ASCII dump of complete transmission after| + | |STARTTLS. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +Use log level 3 only in case of problems. Use of log level 4 is strongly +discouraged. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_loglevel = 0 + +To include information about the protocol and cipher used as well as the client +and issuer CommonName into the "Received:" message header, set the +smtpd_tls_received_header variable to true. The default is no, as the +information is not necessarily authentic. Only information recorded at the +final destination is reliable, since the headers may be changed by intermediate +servers. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_received_header = yes + +EEnnaabblliinngg TTLLSS iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +By default, TLS is disabled in the Postfix SMTP server, so no difference to +plain Postfix is visible. Explicitly switch it on with +"smtpd_tls_security_level = may". + +Example: + + /etc/postfix/main.cf: + smtpd_tls_security_level = may + +With this, the Postfix SMTP server announces STARTTLS support to remote SMTP +clients, but does not require that clients use TLS encryption. + +Note: when an unprivileged user invokes "sendmail -bs", STARTTLS is never +offered due to insufficient privileges to access the Postfix SMTP server +private key. This is intended behavior. + +You can ENFORCE the use of TLS, so that the Postfix SMTP server announces +STARTTLS and accepts no mail without TLS encryption, by setting +"smtpd_tls_security_level = encrypt". According to RFC 2487 this MUST NOT be +applied in case of a publicly-referenced Postfix SMTP server. This option is +off by default and should only seldom be used. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_security_level = encrypt + +TLS is also used in the "wrapper" mode where a server always uses TLS, instead +of announcing STARTTLS support and waiting for remote SMTP clients to request +TLS service. Some clients, namely Outlook [Express] prefer the "wrapper" mode. +This is true for OE (Win32 < 5.0 and Win32 >=5.0 when run on a port<>25 and OE +(5.01 Mac on all ports). + +It is strictly discouraged to use this mode from main.cf. If you want to +support this service, enable a special port in master.cf and specify "- +o smtpd_tls_wrappermode=yes" (note: no space around the "=") as an smtpd(8) +command line option. Port 465 (smtps) was once chosen for this feature. + +Example: + + /etc/postfix/master.cf: + smtps inet n - n - - smtpd + -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes + +CClliieenntt cceerrttiiffiiccaattee vveerriiffiiccaattiioonn + +To receive a remote SMTP client certificate, the Postfix SMTP server must +explicitly ask for one (any contents of $smtpd_tls_CAfile are also sent to the +client as a hint for choosing a certificate from a suitable CA). Unfortunately, +Netscape clients will either complain if no matching client certificate is +available or will offer the user client a list of certificates to choose from. +Additionally some MTAs (notably some versions of qmail) are unable to complete +TLS negotiation when client certificates are requested, and abort the SMTP +session. So this option is "off" by default. You will however need the +certificate if you want to use certificate based relaying with, for example, +the permit_tls_clientcerts feature. A server that wants client certificates +must first present its own certificate. While Postfix by default offers +anonymous ciphers to remote SMTP clients, these are automatically suppressed +when the Postfix SMTP server is configured to ask for client certificates. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_ask_ccert = yes + smtpd_tls_security_level = may + +When TLS is enforced you may also decide to REQUIRE a remote SMTP client +certificate for all TLS connections, by setting "smtpd_tls_req_ccert = yes". +This feature implies "smtpd_tls_ask_ccert = yes". When TLS is not enforced, +"smtpd_tls_req_ccert = yes" is ignored and a warning is logged. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_req_ccert = yes + smtpd_tls_security_level = encrypt + +The client certificate verification depth is specified with the main.cf +smtpd_tls_ccert_verifydepth parameter. The default verification depth is 9 (the +OpenSSL default), for compatibility with Postfix versions before 2.5 where +smtpd_tls_ccert_verifydepth was ignored. When you configure trust in a root CA, +it is not necessary to explicitly trust intermediary CAs signed by the root CA, +unless $smtpd_tls_ccert_verifydepth is less than the number of CAs in the +certificate chain for the clients of interest. With a verify depth of 1 you can +only verify certificates directly signed by a trusted CA, and all trusted +intermediary CAs need to be configured explicitly. With a verify depth of 2 you +can verify clients signed by a root CA or a direct intermediary CA (so long as +the client is correctly configured to supply its intermediate CA certificate). + +Example: + + /etc/postfix/main.cf: + smtpd_tls_ccert_verifydepth = 2 + +SSuuppppoorrttiinngg AAUUTTHH oovveerr TTLLSS oonnllyy + +Sending AUTH data over an unencrypted channel poses a security risk. When TLS +layer encryption is required ("smtpd_tls_security_level = encrypt"), the +Postfix SMTP server will announce and accept AUTH only after the TLS layer has +been activated with STARTTLS. When TLS layer encryption is optional +("smtpd_tls_security_level = may"), it may however still be useful to only +offer AUTH when TLS is active. To maintain compatibility with non-TLS clients, +the default is to accept AUTH without encryption. In order to change this +behavior, set "smtpd_tls_auth_only = yes". + +Example: + + /etc/postfix/main.cf: + smtpd_tls_auth_only = no + +SSeerrvveerr--ssiiddee TTLLSS sseessssiioonn ccaacchhee + +The Postfix SMTP server and the remote SMTP client negotiate a session, which +takes some computer time and network bandwidth. SSL protocol versions other +than SSLv2 support resumption of cached sessions. Not only is this more CPU and +bandwidth efficient, it also reduces latency as only one network round-trip is +used to resume a session while it takes two round-trips to create a session +from scratch. + +Since Postfix uses multiple smtpd(8) service processes, an in-memory cache is +not sufficient for session re-use. Clients store at most one cached session per +server and are very unlikely to repeatedly connect to the same server process. +Thus session caching in the Postfix SMTP server generally requires a shared +cache (an alternative available with Postfix >= 2.11 is described below). + +To share the session information between multiple smtpd(8) processes, a session +cache database is used. You can specify any database type that can store +objects of several kbytes and that supports the sequence operator. DBM +databases are not suitable because they can only store small objects. The cache +is maintained by the tlsmgr(8) process, so there is no problem with concurrent +access. Session caching is highly recommended, because the cost of repeatedly +negotiating TLS session keys is high. + +Starting with Postfix 2.11, linked with a compatible OpenSSL library (at least +0.9.8h, preferably 1.0.0 or later) the Postfix SMTP server supports RFC 5077 +TLS session resumption without server-side state when the remote SMTP client +also supports RFC 5077. The session is encrypted by the server in a session +ticket returned to client for storage. When a client sends a valid session +ticket, the server decrypts it and resumes the session, provided neither the +ticket nor the session have expired. This makes it possible to resume cached +sessions without allocating space for a shared database on the server. +Consequently, for Postfix >= 2.11 the smtpd_tls_session_cache_database +parameter should generally be left empty. Session caching can be disabled by +setting the session cache timeout to zero, otherwise the timeout must be at +least 2 minutes and at most 100 days. + +Note, session tickets can only be negotiated if the client disables SSLv2 and +does not use the legacy SSLv2 compatible HELLO message. This is true by default +with the Postfix >= 2.6 SMTP client. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_scache + +Note: as of version 2.5, Postfix no longer uses root privileges when opening +this file. The file should now be stored under the Postfix-owned +data_directory. As a migration aid, an attempt to open the file under a non- +Postfix directory is redirected to the Postfix-owned data_directory, and a +warning is logged. + +Cached Postfix SMTP server session information expires after a certain amount +of time. Postfix/TLS does not use the OpenSSL default of 300s, but a longer +time of 3600sec (=1 hour). RFC 2246 recommends a maximum of 24 hours. + +Example: + + /etc/postfix/main.cf: + smtpd_tls_session_cache_timeout = 3600s + +As of Postfix 2.11 this setting cannot exceed 100 days. If set <= 0, session +caching is disabled. If set to a positive value less than 2 minutes, the +minimum value of 2 minutes is used instead. + +When the Postfix SMTP server does not save TLS sessions to an external cache +database, client-side session caching is unlikely to be useful. To reduce waste +of client resources, the Postfix SMTP server can be configured to not issue TLS +session ids. By default the Postfix SMTP server always issues TLS session ids. +This works around known interoperability issues with some MUAs, and prevents +possible interoperability issues with other MTAs. + +Example: + + smtpd_tls_always_issue_session_ids = no + +SSeerrvveerr aacccceessss ccoonnttrrooll + +Postfix TLS support introduces three additional features for Postfix SMTP +server access control: + + permit_tls_clientcerts + Allow the remote SMTP client request if the client certificate + fingerprint or certificate public key fingerprint (Postfix 2.9 and + later) is listed in the client certificate table (see relay_clientcerts + discussion below). + + permit_tls_all_clientcerts + Allow the remote SMTP client request if the client certificate passes + trust chain verification. Useful with private-label CAs that only issue + certificates to trusted clients (and not otherwise). + + check_ccert_access type:table + Use the remote SMTP client certificate fingerprint or public key + fingerprint (Postfix 2.9 and later) as the lookup key for the specified + access(5) table. + +The digest algorithm used to compute the client certificate fingerprints is +specified with the main.cf smtpd_tls_fingerprint_digest parameter. The default +algorithm is sshhaa225566 with Postfix >= 3.6 and the ccoommppaattiibbiilliittyy__lleevveell set to 3.6 +or higher. With Postfix <= 3.5, the default algorithm is mmdd55. The best-practice +algorithm is now sshhaa225566. Recent advances in hash function cryptanalysis have +led to md5 and sha1 being deprecated in favor of sha256. However, as long as +there are no known "second pre-image" attacks against the older algorithms, +their use in this context, though not recommended, is still likely safe. + +The permit_tls_all_clientcerts feature must be used with caution, because it +can result in too many access permissions. Use this feature only if a special +CA issues the client certificates, and only if this CA is listed as a trusted +CA. If other CAs are trusted, any owner of a valid client certificate would be +authorized. The permit_tls_all_clientcerts feature can be practical for a +specially created email relay server. + +It is however recommended to stay with the permit_tls_clientcerts feature and +list all certificates via $relay_clientcerts, as permit_tls_all_clientcerts +does not permit any control when a certificate must no longer be used (e.g. an +employee leaving). + +Example: + + # With Postfix 2.10 and later, the mail relay policy is + # preferably specified under smtpd_relay_restrictions. + /etc/postfix/main.cf: + smtpd_relay_restrictions = + permit_mynetworks + permit_tls_clientcerts + reject_unauth_destination + + # Older configurations combine relay control and spam control under + # smtpd_recipient_restrictions. To use this example with Postfix >= + # 2.10 specify "smtpd_relay_restrictions=". + /etc/postfix/main.cf: + smtpd_recipient_restrictions = + permit_mynetworks + permit_tls_clientcerts + reject_unauth_destination + ...other rules... + +Example: Postfix lookup tables are in the form of (key, value) pairs. Since we +only need the key, the value can be chosen freely, e.g. the name of the user or +host: + + /etc/postfix/main.cf: + relay_clientcerts = hash:/etc/postfix/relay_clientcerts + + /etc/postfix/relay_clientcerts: + D7:04:2F:A7:0B:8C:A5:21:FA:31:77:E1:41:8A:EE:80 lutzpc.at.home + +To extract the public key fingerprint from an X.509 certificate, you need to +extract the public key from the certificate and compute the appropriate digest +of its DER (ASN.1) encoding. With OpenSSL the "-pubkey" option of the "x509" +command extracts the public key always in "PEM" format. We pipe the result to +another OpenSSL command that converts the key to DER and then to the "dgst" +command to compute the fingerprint. + +Example: + + $ openssl x509 -in cert.pem -noout -pubkey | + openssl pkey -pubin -outform DER | + openssl dgst -sha256 -c + (stdin)= 64:3f:1f:f6:e5:1e:d4:2a:...:8b:fc:09:1a:61:98:b5:bc:7c:60:58 + +SSeerrvveerr--ssiiddee cciipphheerr ccoonnttrroollss + +The Postfix SMTP server supports 5 distinct cipher grades as specified by the +smtpd_tls_mandatory_ciphers configuration parameter, which determines the +minimum cipher grade with mandatory TLS encryption. The default minimum cipher +grade for mandatory TLS is "medium" which is essentially 128-bit encryption or +better. The smtpd_tls_ciphers parameter (Postfix >= 2.6) controls the minimum +cipher grade used with opportunistic TLS. Here, the default minimum cipher +grade is "medium" for Postfix releases after the middle of 2015, "export" for +older Postfix releases. With Postfix < 2.6, the minimum opportunistic TLS +cipher grade is always "export". + +By default anonymous ciphers are enabled. They are automatically disabled when +remote SMTP client certificates are requested. If clients are expected to +always verify the Postfix SMTP server certificate you may want to disable +anonymous ciphers by setting "smtpd_tls_mandatory_exclude_ciphers = aNULL" or +"smtpd_tls_exclude_ciphers = aNULL", as appropriate. One can't force a remote +SMTP client to check the server certificate, so excluding anonymous ciphers is +generally unnecessary. + +With mandatory and opportunistic TLS encryption, the Postfix SMTP server by +default disables SSLv2 and SSLv3 with Postfix releases after the middle of +2015; older releases only disable SSLv2 for mandatory TLS. The mandatory TLS +protocol list is specified via the smtpd_tls_mandatory_protocols configuration +parameter. The smtpd_tls_protocols parameter (Postfix >= 2.6) controls the TLS +protocols used with opportunistic TLS. + +Note that the OpenSSL library only supports protocol exclusion (not inclusion). +For this reason, Postfix can exclude only protocols that are known at the time +the Postfix software is written. If new protocols are added to the OpenSSL +library, they cannot be excluded without corresponding changes to the Postfix +source code. + +For a server that is not a public Internet MX host, Postfix supports +configurations with no server certificates that use oonnllyy the anonymous ciphers. +This is enabled by explicitly setting "smtpd_tls_cert_file = none" and not +specifying an smtpd_tls_dcert_file or smtpd_tls_eccert_file. Such +configurations may not interoperate with some clients, and require that TLSv1.3 +be explicitly disabled. Therefore, they are not recommended, it is better and +simpler to just configure a suitable certificate. + +Example, MSA that requires TLSv1.2 or higher, with high grade ciphers: + + /etc/postfix/main.cf: + smtpd_tls_cert_file = /etc/postfix/cert.pem + smtpd_tls_key_file = /etc/postfix/key.pem + smtpd_tls_mandatory_ciphers = high + smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 + smtpd_tls_security_level = encrypt + # Preferred syntax with Postfix >= 3.6: + smtpd_tls_mandatory_protocols = >=TLSv1.2 + # Legacy syntax: + smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 + +With Postfix >= 3.4, specify instead a single file that holds the key followed +by the corresponding certificate and any associated issuing certificates, +leaving the "smtpd_tls_cert_file" and "smtpd_tls_key_file" and related DSA and +ECDSA parameters empty. + + /etc/postfix/main.cf: + smtpd_tls_chain_files = /etc/postfix/rsachain.pem + smtpd_tls_cert_file = + smtpd_tls_key_file = + ... + +If you want to take maximal advantage of ciphers that offer forward secrecy see +the Getting started section of FORWARD_SECRECY_README. The full document +conveniently presents all information about Postfix forward secrecy support in +one place: what forward secrecy is, how to tweak settings, and what you can +expect to see when Postfix uses ciphers with forward secrecy. + +Postfix 2.8 and later, in combination with OpenSSL 0.9.7 and later allows TLS +servers to preempt the TLS client's cipher-suite preference list. This is +possible only with SSLv3 and later, as in SSLv2 the client chooses the cipher- +suite from a list supplied by the server. + +By default, the OpenSSL server selects the client's most preferred cipher-suite +that the server supports. With SSLv3 and later, the server may choose its own +most preferred cipher-suite that is supported (offered) by the client. Setting +"tls_preempt_cipherlist = yes" enables server cipher-suite preferences. The +default OpenSSL behavior applies with "tls_preempt_cipherlist = no". + +While server cipher-suite selection may in some cases lead to a more secure or +performant cipher-suite choice, there is some risk of interoperability issues. +In the past, some SSL clients have listed lower priority ciphers that they did +not implement correctly. If the server chooses a cipher that the client prefers +less, it may select a cipher whose client implementation is flawed. Most +notably Windows 2003 Microsoft Exchange servers have flawed implementations of +DES-CBC3-SHA, which OpenSSL considers stronger than RC4-SHA. Enabling server +cipher-suite selection may create interoperability issues with Windows 2003 +Microsoft Exchange clients. + +MMiisscceellllaanneeoouuss sseerrvveerr ccoonnttrroollss + +The smtpd_starttls_timeout parameter limits the time of Postfix SMTP server +write and read operations during TLS startup and shutdown handshake procedures. + +Example: + + /etc/postfix/main.cf: + smtpd_starttls_timeout = 300s + +With Postfix 2.8 and later, the tls_disable_workarounds parameter specifies a +list or bit-mask of default-enabled OpenSSL bug work-arounds to disable. This +may be necessary if one of the work-arounds enabled by default in OpenSSL +proves to pose a security risk, or introduces an unexpected interoperability +issue. The list of enabled bug work-arounds is OpenSSL-release-specific. See +the tls_disable_workarounds parameter documentation for the list of supported +values. + +Example: + + /etc/postfix/main.cf: + tls_disable_workarounds = 0xFFFFFFFF + tls_disable_workarounds = CVE-2010-4180 + +With Postfix >= 2.11, the tls_ssl_options parameter specifies a list or bit- +mask of OpenSSL options to enable. Specify one or more of the named options +below, or a hexadecimal bitmask of options found in the ssl.h file +corresponding to the run-time OpenSSL library. While it may be reasonable to +turn off all bug workarounds (see above), it is not a good idea to attempt to +turn on all features. See the tls_ssl_options parameter documentation for the +list of supported values. + +Example: + + /etc/postfix/main.cf: + tls_ssl_options = no_ticket, no_compression + +You should only enable features via the hexadecimal mask when the need to +control the feature is critical (to deal with a new vulnerability or a serious +interoperability problem). Postfix DOES NOT promise backwards compatible +behavior with respect to the mask bits. A feature enabled via the mask in one +release may be enabled by other means in a later release, and the mask bit will +then be ignored. Therefore, use of the hexadecimal mask is only a temporary +measure until a new Postfix or OpenSSL release provides a better solution. + +SSMMTTPP CClliieenntt ssppeecciiffiicc sseettttiinnggss + +Topics covered in this section: + + * Configuring TLS in the SMTP/LMTP client + * Client-side TLS activity logging + * Client-side certificate and private key configuration + * Client-side TLS connection reuse + * Client-side TLS session cache + * Client TLS limitations + * Per-destination TLS policy + * Discovering servers that support TLS + * Server certificate verification depth + * Client-side cipher controls + * Client-side SMTPS support + * Miscellaneous client controls + +CCoonnffiigguurriinngg TTLLSS iinn tthhee SSMMTTPP//LLMMTTPP cclliieenntt + +Similar to the Postfix SMTP server, the Postfix SMTP/LMTP client implements +multiple TLS security levels. These levels are described in more detail in the +sections that follow. + +nnoonnee + No TLS. +mmaayy + Opportunistic TLS. +eennccrryypptt + Mandatory TLS encryption. +ddaannee + Opportunistic DANE TLS. +ddaannee--oonnllyy + Mandatory DANE TLS. +ffiinnggeerrpprriinntt + Certificate fingerprint verification. +vveerriiffyy + Mandatory server certificate verification. +sseeccuurree + Secure-channel TLS. + +TTLLSS ssuuppppoorrtt iinn tthhee LLMMTTPP ddeelliivveerryy aaggeenntt + +The smtp(8) and lmtp(8) delivery agents are implemented by a single dual- +purpose program. Specifically, all the TLS features described below apply +equally to SMTP and LMTP, after replacing the "smtp_" prefix of the each +parameter name with "lmtp_". + +The Postfix LMTP delivery agent can communicate with LMTP servers listening on +UNIX-domain sockets. When server certificate verification is enabled and the +server is listening on a UNIX-domain socket, the $myhostname parameter is used +to set the TLS verification nexthop and hostname. + +NOTE: Opportunistic encryption of LMTP traffic over UNIX-domain sockets or +loopback TCP connections is futile. TLS is only useful in this context when it +is mandatory, typically to allow at least one of the server or the client to +authenticate the other. The "null" cipher grade may be appropriate in this +context, when available on both client and server. The "null" ciphers provide +authentication without encryption. + +NNoo TTLLSS eennccrryyppttiioonn + +At the "none" TLS security level, TLS encryption is disabled. This is the +default security level, and can be configured explicitly by setting +"smtp_tls_security_level = none". For LMTP, use the corresponding "lmtp_" +parameter. + +Per-destination settings may override this default setting, in which case TLS +is used selectively, only with destinations explicitly configured for TLS. + +You can disable TLS for a subset of destinations, while leaving it enabled for +the rest. With the Postfix TLS policy table, specify the "none" security level. + +OOppppoorrttuunniissttiicc TTLLSS + +At the "may" TLS security level, TLS encryption is opportunistic. The SMTP +transaction is encrypted if the STARTTLS ESMTP feature is supported by the +server. Otherwise, messages are sent in the clear. Opportunistic TLS can be +configured by setting "smtp_tls_security_level = may". For LMTP, use the +corresponding "lmtp_" parameter. + +The "smtp_tls_ciphers" and "smtp_tls_protocols" configuration parameters +(Postfix >= 2.6) provide control over the cipher grade and protocols used with +opportunistic TLS. With earlier Postfix releases, opportunistic TLS always uses +the cipher grade "export" and enables all protocols. + +With opportunistic TLS, mail delivery continues even if the server certificate +is untrusted or bears the wrong name. When the TLS handshake fails for an +opportunistic TLS session, rather than give up on mail delivery, the Postfix +SMTP client retries the transaction with TLS disabled. Trying an unencrypted +connection makes it possible to deliver mail to sites with non-interoperable +server TLS implementations. + +Opportunistic encryption is never used for LMTP over UNIX-domain sockets. The +communications channel is already confidential without TLS, so the only +potential benefit of TLS is authentication. Do not configure opportunistic TLS +for LMTP deliveries over UNIX-domain sockets. Only configure TLS for LMTP over +UNIX-domain sockets at the encrypt security level or higher. Attempts to +configure opportunistic encryption of LMTP sessions will be ignored with a +warning written to the mail logs. + +You can enable opportunistic TLS just for selected destinations. With the +Postfix TLS policy table, specify the "may" security level. + +This is the most common security level for TLS protected SMTP sessions, +stronger security is not generally available and, if needed, is typically only +configured on a per-destination basis. See the section on TLS limitations +above. + +Example: + + /etc/postfix/main.cf: + smtp_tls_security_level = may + +MMaannddaattoorryy TTLLSS eennccrryyppttiioonn + +At the "encrypt" TLS security level, messages are sent only over TLS encrypted +sessions. The SMTP transaction is aborted unless the STARTTLS ESMTP feature is +supported by the remote SMTP server. If no suitable servers are found, the +message will be deferred. Mandatory TLS encryption can be configured by setting +"smtp_tls_security_level = encrypt". Even though TLS encryption is always used, +mail delivery continues even if the server certificate is untrusted or bears +the wrong name. For LMTP, use the corresponding "lmtp_" parameter. + +At this security level and higher, the smtp_tls_mandatory_protocols and +smtp_tls_mandatory_ciphers configuration parameters determine the list of +sufficiently secure SSL protocol versions and the minimum cipher strength. If +the protocol or cipher requirements are not met, the mail transaction is +aborted. The documentation for these parameters includes useful +interoperability and security guidelines. + +Despite the potential for eliminating passive eavesdropping attacks, mandatory +TLS encryption is not viable as a default security level for mail delivery to +the public Internet. Some MX hosts do not support TLS at all, and some of those +that do have broken implementations. On a host that delivers mail to the +Internet, you should not configure mandatory TLS encryption as the default +security level. + +You can enable mandatory TLS encryption just for specific destinations. With +the Postfix TLS policy table, specify the "encrypt" security level. + +Examples: + +In the example below, traffic to example.com and its sub-domains via the +corresponding MX hosts always uses TLS. The SSLv2 protocol will be disabled +(the default setting of smtp_tls_mandatory_protocols excludes SSLv2+3). Only +high- or medium-strength (i.e. 128 bit or better) ciphers will be used by +default for all "encrypt" security level sessions. + + /etc/postfix/main.cf: + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + + /etc/postfix/tls_policy: + example.com encrypt + .example.com encrypt + +In the next example, secure message submission is configured via the MSA " +[example.net]:587". TLS sessions are encrypted without authentication, because +this MSA does not possess an acceptable certificate. This MSA is known to be +capable of "TLSv1" and "high" grade ciphers, so these are selected via the +policy table. + +NNoottee:: the policy table lookup key is the verbatim next-hop specification from +the recipient domain, transport(5) table or relayhost parameter, with any +enclosing square brackets and optional port. Take care to be consistent: the +suffixes ":smtp" or ":25" or no port suffix result in different policy table +lookup keys, even though they are functionally equivalent nexthop +specifications. Use at most one of these forms for all destinations. Below, the +policy table has multiple keys, just in case the transport table entries are +not specified consistently. + + /etc/postfix/main.cf: + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + + /etc/services: + submission 587/tcp msa # mail message + submission + + /etc/postfix/tls_policy: + # Postfix >= 3.6 "protocols" syntax + [example.net]:587 encrypt protocols=>=TLSv1.2 ciphers=high + # Legacy "protocols" syntax + [example.net]:msa encrypt protocols=!SSLv2:!SSLv3 ciphers=high + +DDAANNEE TTLLSS aauutthheennttiiccaattiioonn.. + +The Postfix SMTP client supports two TLS security levels based on DANE TLSA +(RFC 6698, RFC 7671, RFC 7672) records. The opportunistic "dane" level and the +mandatory "dane-only" level. + +The "dane" level is a stronger form of opportunistic TLS that is resistant to +man in the middle and downgrade attacks when the destination domain uses DNSSEC +to publish DANE TLSA records for its MX hosts. If a remote SMTP server has +"usable" (see section 3 of RFC 7672) DANE TLSA records, the server connection +will be authenticated. When DANE authentication fails, there is no fallback to +unauthenticated or plaintext delivery. + +If TLSA records are published for a given remote SMTP server (implying TLS +support), but are all "unusable" due to unsupported parameters or malformed +data, the Postfix SMTP client will use mandatory unauthenticated TLS. +Otherwise, when no TLSA records are published, the Postfix SMTP client behavior +is the same as with may. + +TLSA records must be published in DNSSEC validated DNS zones. Any TLSA records +in DNS zones not protected via DNSSEC are ignored. The Postfix SMTP client will +not look for TLSA records associated with MX hosts whose "A" or "AAAA" records +lie in an "insecure" DNS zone. Such lookups have been observed to cause +interoperability issues with poorly implemented DNS servers, and are in any +case not expected to ever yield "secure" results, since that would require a +very unlikely DLV DNS trust anchor configured between the host record and the +associated "_25._tcp" child TLSA record. + +The "dane-only" level is a form of secure-channel TLS based on the DANE PKI. If +"usable" TLSA records are present these are used to authenticate the remote +SMTP server. Otherwise, or when server certificate verification fails, delivery +via the server in question tempfails. + +At both security levels, the TLS policy for the destination is obtained via +TLSA records validated with DNSSEC. For TLSA policy to be in effect, the +destination domain's containing DNS zone must be signed and the Postfix SMTP +client's operating system must be configured to send its DNS queries to a +recursive DNS nameserver that is able to validate the signed records. Each MX +host's DNS zone needs to also be signed, and needs to publish DANE TLSA (see +section 3 of RFC 7672) records that specify how that MX host's TLS certificate +is to be verified. + +TLSA records do not preempt the normal SMTP MX host selection algorithm, if +some MX hosts support TLSA and others do not, TLS security will vary from +delivery to delivery. It is up to the domain owner to configure their MX hosts +and their DNS sensibly. To configure the Postfix SMTP client for DNSSEC lookups +see the documentation for the smtp_dns_support_level main.cf parameter. The +tls_dane_digests parameter controls the list of supported digests. + +As explained in section 3 of RFC 7672, certificate usages "0" and "1", which +are intended to "constrain" existing Web-PKI trust, are not supported with MTA- +to-MTA SMTP. Rather, TLSA records with usages "0" and "1" are treated as +"unusable". + +The Postfix SMTP client supports only certificate usages "2" and "3". +Experimental support for silently mapping certificate usage "1" to "3" has been +withdrawn starting with Postfix 3.2. + +When usable TLSA records are obtained for the remote SMTP server the Postfix +SMTP client sends the SNI TLS extension in its SSL client hello message. This +may help the remote SMTP server live up to its promise to provide a certificate +that matches its TLSA records. + +For purposes of protocol and cipher selection, the "dane" security level is +treated like a "mandatory" TLS security level, and weak ciphers and protocols +are disabled. Since DANE authenticates server certificates the "aNULL" cipher- +suites are transparently excluded at this level, no need to configure this +manually. RFC 7672 (DANE) TLS authentication is available with Postfix 2.11 and +later. + +When a DANE TLSA record specifies a trust-anchor (TA) certificate (that is an +issuing CA), the strategy used to verify the peername of the server certificate +is unconditionally "nexthop, hostname". Both the nexthop domain and the +hostname obtained from the DNSSEC-validated MX lookup are safe from forgery and +the server certificate must contain at least one of these names. + +When a DANE TLSA record specifies an end-entity (EE) certificate, (that is the +actual server certificate), as with the fingerprint security level below, no +name checks or certificate expiration checks are applied. The server +certificate (or its public key) either matches the DANE record or not. Server +administrators should publish such EE records in preference to all other types. + +The pre-requisites for DANE support in the Postfix SMTP client are: + + * A compile-time OpenSSL library that supports the TLS SNI extension and + "SHA-2" message digests. + * A compile-time DNS resolver library that supports DNSSEC. Postfix binaries + built on an older system will not support DNSSEC even if deployed on a + system with an updated resolver library. + * The "smtp_dns_support_level" must be set to "dnssec". + * The "smtp_host_lookup" parameter must include "dns". + * A DNSSEC-validating recursive resolver (see note below). + +The above client pre-requisites do not apply to the Postfix SMTP server. It +will support DANE provided it supports TLSv1 and its TLSA records are published +in a DNSSEC signed zone. To receive DANE secured mail for multiple domains, use +the same hostname to add the server to each domain's MX records. The Postfix +SMTP server supports SNI (Postfix 3.4 and later), configured with +tls_server_sni_maps. + +Note: The Postfix SMTP client's internal stub DNS resolver is DNSSEC-aware, but +it does not itself validate DNSSEC records, rather it delegates DNSSEC +validation to the operating system's configured recursive DNS nameserver. The +Postfix DNS client relies on a secure channel to the resolver's cache for +DNSSEC integrity, but does not support TSIG to protect the transmission channel +between itself and the nameserver. Therefore, it is strongly recommended (DANE +security guarantee void otherwise) that each MTA run a local DNSSEC-validating +recursive resolver ("unbound" from nlnetlabs.nl is a reasonable choice) +listening on the loopback interface, and that the system be configured to use +only this local nameserver. The local nameserver may forward queries to an +upstream recursive resolver on another host if desired. + +Note: When the operating system's recursive nameserver is not local, enabling +EDNS0 expanded DNS packet sizes and turning on the DNSSEC "DO" bit in the DNS +request and/or the new DNSSEC-specific records returned in the nameserver's +replies may cause problems with older or buggy firewall and DNS server +implementations. Therefore, Postfix does not enable DNSSEC by default. Since MX +lookups happen before the security level is determined, DANE support is +disabled for all destinations unless you set "smtp_dns_support_level = dnssec". +To enable DNSSEC lookups selectively, define a new dedicated transport with a +"-o smtp_dns_support_level=dnssec" override in master.cf and route selected +domains to that transport. If DNSSEC proves to be sufficiently reliable for +these domains, you can enable it for all destinations by changing the global +smtp_dns_support_level in main.cf. + +EExxaammppllee: "dane" security for selected destinations, with opportunistic TLS by +default. This is the recommended configuration for early adopters. + + * The "example.com" destination uses DANE, but if TLSA records are not + present or are unusable, mail is deferred. + + * The "example.org" destination uses DANE if possible, but if no TLSA records + are found opportunistic TLS is used. + + main.cf: + indexed = ${default_database_type}:${config_directory}/ + # + # default: Opportunistic TLS with no DNSSEC lookups. + # + smtp_tls_security_level = may + smtp_dns_support_level = enabled + # + # Per-destination TLS policy + # + smtp_tls_policy_maps = ${indexed}tls_policy + # + # default_transport = smtp, but some destinations are special: + # + transport_maps = ${indexed}transport + + transport: + example.com dane + example.org dane + + tls_policy: + example.com dane-only + + master.cf: + dane unix - - n - - smtp + -o smtp_dns_support_level=dnssec + -o smtp_tls_security_level=dane + +CCeerrttiiffiiccaattee ffiinnggeerrpprriinntt vveerriiffiiccaattiioonn + +At the fingerprint security level, no trusted Certification Authorities are +used or required. The certificate trust chain, expiration date, etc., are not +checked. Instead, the smtp_tls_fingerprint_cert_match parameter or the "match" +attribute in the policy table lists the remote SMTP server certificate +fingerprint or public key fingerprint. Certificate fingerprint verification is +available with Postfix 2.5 and later, public-key fingerprint support is +available with Postfix 2.9 and later. + +If certificate fingerprints are exchanged securely, this is the strongest, and +least scalable security level. The administrator needs to securely collect the +fingerprints of the X.509 certificates of each peer server, store them into a +local file, and update this local file whenever the peer server's public +certificate changes. If public key fingerprints are used in place of +fingerprints of the entire certificate, the fingerprints remain valid even +after the certificate is renewed, pprroovviiddeedd that the same public/private keys +are used to obtain the new certificate. + +Fingerprint verification may be feasible for an SMTP "VPN" connecting a small +number of branch offices over the Internet, or for secure connections to a +central mail hub. It works poorly if the remote SMTP server is managed by a +third party, and its public certificate changes periodically without prior +coordination with the verifying site. + +The digest algorithm used to calculate the fingerprint is selected by the +ssmmttpp__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt parameter. In the policy table multiple +fingerprints can be combined with a "|" delimiter in a single match attribute, +or multiple match attributes can be employed. The ":" character is not used as +a delimiter as it occurs between each pair of fingerprint (hexadecimal) digits. + +The default algorithm is sshhaa225566 with Postfix >= 3.6 and the ccoommppaattiibbiilliittyy__lleevveell +set to 3.6 or higher; with Postfix <= 3.5, the default algorithm is mmdd55. The +best-practice algorithm is now sshhaa225566. Recent advances in hash function +cryptanalysis have led to md5 and sha1 being deprecated in favor of sha256. +However, as long as there are no known "second pre-image" attacks against the +older algorithms, their use in this context, though not recommended, is still +likely safe. + +Example: fingerprint TLS security with an internal mailhub. Two matching +fingerprints are listed. The relayhost may be multiple physical hosts behind a +load-balancer, each with its own private/public key and self-signed +certificate. Alternatively, a single relayhost may be in the process of +switching from one set of private/public keys to another, and both keys are +trusted just prior to the transition. + + relayhost = [mailhub.example.com] + smtp_tls_security_level = fingerprint + smtp_tls_fingerprint_digest = sha256 + smtp_tls_fingerprint_cert_match = + 51:e9:af:2e:1e:40:1f:de:64:...:30:35:2d:09:16:31:5a:eb:82:76 + b6:b4:72:34:e2:59:cd:fb:c2:...:63:0d:4d:cc:2c:7d:84:de:e6:2f + +Example: Certificate fingerprint verification with selected destinations. As in +the example above, we show two matching fingerprints: + + /etc/postfix/main.cf: + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + smtp_tls_fingerprint_digest = sha256 + + /etc/postfix/tls_policy: + example.com fingerprint + match=51:e9:af:2e:1e:40:1f:de:...:35:2d:09:16:31:5a:eb:82:76 + match=b6:b4:72:34:e2:59:cd:fb:...:0d:4d:cc:2c:7d:84:de:e6:2f + +To extract the public key fingerprint from an X.509 certificate, you need to +extract the public key from the certificate and compute the appropriate digest +of its DER (ASN.1) encoding. With OpenSSL the "-pubkey" option of the "x509" +command extracts the public key always in "PEM" format. We pipe the result to +another OpenSSL command that converts the key to DER and then to the "dgst" +command to compute the fingerprint. + +Example: + + $ openssl x509 -in cert.pem -noout -pubkey | + openssl pkey -pubin -outform DER | + openssl dgst -sha256 -c + (stdin)= 64:3f:1f:f6:e5:1e:d4:2a:56:...:09:1a:61:98:b5:bc:7c:60:58 + +MMaannddaattoorryy sseerrvveerr cceerrttiiffiiccaattee vveerriiffiiccaattiioonn + +At the verify TLS security level, messages are sent only over TLS encrypted +sessions if the remote SMTP server certificate is valid (not expired or +revoked, and signed by a trusted Certification Authority) and where the server +certificate name matches a known pattern. Mandatory server certificate +verification can be configured by setting "smtp_tls_security_level = verify". +The smtp_tls_verify_cert_match parameter can override the default "hostname" +certificate name matching strategy. Fine-tuning the matching strategy is +generally only appropriate for secure-channel destinations. For LMTP use the +corresponding "lmtp_" parameters. + +If the server certificate chain is trusted (see smtp_tls_CAfile and +smtp_tls_CApath), any DNS names in the SubjectAlternativeName certificate +extension are used to verify the remote SMTP server name. If no DNS names are +specified, the certificate CommonName is checked. If you want mandatory +encryption without server certificate verification, see above. + +With Postfix >= 2.11 the "smtp_tls_trust_anchor_file" parameter or more +typically the corresponding per-destination "tafile" attribute optionally +modifies trust chain verification. If the parameter is not empty the root CAs +in CAfile and CApath are no longer trusted. Rather, the Postfix SMTP client +will only trust certificate-chains signed by one of the trust-anchors contained +in the chosen files. The specified trust-anchor certificates and public keys +are not subject to expiration, and need not be (self-signed) root CAs. They +may, if desired, be intermediate certificates. Therefore, these certificates +also may be found "in the middle" of the trust chain presented by the remote +SMTP server, and any untrusted issuing parent certificates will be ignored. + +Despite the potential for eliminating "man-in-the-middle" and other attacks, +mandatory certificate trust chain and subject name verification is not viable +as a default Internet mail delivery policy. Some MX hosts do not support TLS at +all, and a significant portion of TLS-enabled MTAs use self-signed +certificates, or certificates that are signed by a private Certification +Authority. On a machine that delivers mail to the Internet, you should not +configure mandatory server certificate verification as a default policy. + +Mandatory server certificate verification as a default security level may be +appropriate if you know that you will only connect to servers that support RFC +2487 and that present verifiable server certificates. An example would be a +client that sends all email to a central mailhub that offers the necessary +STARTTLS support. In such cases, you can often use a secure-channel +configuration instead. + +You can enable mandatory server certificate verification just for specific +destinations. With the Postfix TLS policy table, specify the "verify" security +level. + +Example: + +In this example, the Postfix SMTP client encrypts all traffic to the +example.com domain. The peer hostname is verified, but verification is +vulnerable to DNS response forgery. Mail transmission to example.com recipients +uses "high" grade ciphers. + + /etc/postfix/main.cf: + indexed = ${default_database_type}:${config_directory}/ + smtp_tls_CAfile = ${config_directory}/CAfile.pem + smtp_tls_policy_maps = ${indexed}tls_policy + + /etc/postfix/tls_policy: + example.com verify ciphers=high + +SSeeccuurree sseerrvveerr cceerrttiiffiiccaattee vveerriiffiiccaattiioonn + +At the secure TLS security level, messages are sent only over secure-channel +TLS sessions where DNS forgery resistant server certificate verification +succeeds. If no suitable servers are found, the message will be deferred. +Postfix secure-channels can be configured by setting "smtp_tls_security_level = +secure". The smtp_tls_secure_cert_match parameter can override the default +"nexthop, dot-nexthop" certificate match strategy. For LMTP, use the +corresponding "lmtp_" parameters. + +If the server certificate chain is trusted (see smtp_tls_CAfile and +smtp_tls_CApath), any DNS names in the SubjectAlternativeName certificate +extension are used to verify the remote SMTP server name. If no DNS names are +specified, the CommonName is checked. If you want mandatory encryption without +server certificate verification, see above. + +With Postfix >= 2.11 the "smtp_tls_trust_anchor_file" parameter or more +typically the corresponding per-destination "tafile" attribute optionally +modifies trust chain verification. If the parameter is not empty the root CAs +in CAfile and CApath are no longer trusted. Rather, the Postfix SMTP client +will only trust certificate-chains signed by one of the trust-anchors contained +in the chosen files. The specified trust-anchor certificates and public keys +are not subject to expiration, and need not be (self-signed) root CAs. They +may, if desired, be intermediate certificates. Therefore, these certificates +also may be found "in the middle" of the trust chain presented by the remote +SMTP server, and any untrusted issuing parent certificates will be ignored. + +Despite the potential for eliminating "man-in-the-middle" and other attacks, +mandatory secure server certificate verification is not viable as a default +Internet mail delivery policy. Some MX hosts do not support TLS at all, and a +significant portion of TLS-enabled MTAs use self-signed certificates, or +certificates that are signed by a private Certification Authority. On a machine +that delivers mail to the Internet, you should not configure secure TLS +verification as a default policy. + +Mandatory secure server certificate verification as a default security level +may be appropriate if you know that you will only connect to servers that +support RFC 2487 and that present verifiable server certificates. An example +would be a client that sends all email to a central mailhub that offers the +necessary STARTTLS support. + +You can enable secure TLS verification just for specific destinations. With the +Postfix TLS policy table, specify the "secure" security level. + +Examples: + + * Secure-channel TLS without transport(5) table overrides: + + The Postfix SMTP client will encrypt all traffic and verify the destination + name immune from forged DNS responses. MX lookups are still used to find + the hostnames of the SMTP servers for example.com, but these hostnames are + not used when checking the names in the server certificate(s). Rather, the + requirement is that the MX hosts for example.com have trusted certificates + with a subject name of example.com or a sub-domain, see the documentation + for the smtp_tls_secure_cert_match parameter. + + The related domains example.co.uk and example.co.jp are hosted on the same + MX hosts as the primary example.com domain, and traffic to these is secured + by verifying the primary example.com domain in the server certificates. + This frees the server administrator from needing the CA to sign + certificates that list all the secondary domains. The downside is that + clients that want secure channels to the secondary domains need explicit + TLS policy table entries. + + Note, there are two ways to handle related domains. The first is to use the + default routing for each domain, but add policy table entries to override + the expected certificate subject name. The second is to override the next- + hop in the transport table, and use a single policy table entry for the + common nexthop. We choose the first approach, because it works better when + domain ownership changes. With the second approach we securely deliver mail + to the wrong destination, with the first approach, authentication fails and + mail stays in the local queue, the first approach is more appropriate in + most cases. + + /etc/postfix/main.cf: + smtp_tls_CAfile = /etc/postfix/CAfile.pem + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + + /etc/postfix/transport: + + /etc/postfix/tls_policy: + example.com secure + example.co.uk secure match=example.com:.example.com + example.co.jp secure match=example.com:.example.com + + * Secure-channel TLS with transport(5) table overrides: + + In this case traffic to example.com and its related domains is sent to a + single logical gateway (to avoid a single point of failure, its name may + resolve to one or more load-balancer addresses, or to the combined + addresses of multiple physical hosts). All the physical hosts reachable via + the gateway's IP addresses have the logical gateway name listed in their + certificates. + + /etc/postfix/main.cf: + smtp_tls_CAfile = /etc/postfix/CAfile.pem + transport_maps = hash:/etc/postfix/transport + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + + /etc/postfix/transport: + example.com smtp:[tls.example.com] + example.co.uk smtp:[tls.example.com] + example.co.jp smtp:[tls.example.com] + + /etc/postfix/tls_policy: + [tls.example.com] secure match=tls.example.com + +CClliieenntt--ssiiddee TTLLSS aaccttiivviittyy llooggggiinngg + +To get additional information about Postfix SMTP client TLS activity you can +increase the loglevel from 0..4. Each logging level also includes the +information that is logged at a lower logging level. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |LLeevveell|PPoossttffiixx 22..99 aanndd llaatteerr |EEaarrlliieerr rreelleeaasseess.. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |0 |Disable logging of TLS activity. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |1 |Log only a summary message on TLS |Log the summary message and | + | |handshake completion -- no logging|unconditionally log trust-chain| + | |of remote SMTP server certificate |verification errors. | + | |trust-chain verification errors if| | + | |server certificate verification is| | + | |not required. | | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |2 |Also log levels during TLS negotiation. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |3 |Also log hexadecimal and ASCII dump of TLS negotiation process. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |4 |Also log hexadecimal and ASCII dump of complete transmission after| + | |STARTTLS. | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +Example: + + /etc/postfix/main.cf: + smtp_tls_loglevel = 0 + +CClliieenntt--ssiiddee cceerrttiiffiiccaattee aanndd pprriivvaattee kkeeyy ccoonnffiigguurraattiioonn + +Do not configure Postfix SMTP client certificates unless you mmuusstt present +client TLS certificates to one or more servers. Client certificates are not +usually needed, and can cause problems in configurations that work well without +them. The recommended setting is to let the defaults stand: + + smtp_tls_cert_file = + smtp_tls_dcert_file = + smtp_tls_key_file = + smtp_tls_dkey_file = + # Postfix >= 2.6 + smtp_tls_eccert_file = + smtp_tls_eckey_file = + # Postfix >= 3.4 + smtp_tls_chain_files = + +The best way to use the default settings is to comment out the above parameters +in main.cf if present. + +During TLS startup negotiation the Postfix SMTP client may present a +certificate to the remote SMTP server. Browsers typically let the user select +among the certificates that match the CA names indicated by the remote SMTP +server. The Postfix SMTP client does not yet have a mechanism to select from +multiple candidate certificates on the fly, and supports a single set of +certificates (at most one per public key algorithm). + +RSA, DSA and ECDSA (Postfix >= 2.6) certificates are supported. You can +configure all three at the same time, in which case the cipher used determines +which certificate is presented. + +It is possible for the Postfix SMTP client to use the same key/certificate pair +as the Postfix SMTP server. If a certificate is to be presented, it must be in +"PEM" format. The private key must not be encrypted, meaning: it must be +accessible without a password. Both parts (certificate and private key) may be +in the same file. + +With OpenSSL 1.1.1 and Postfix >= 3.4 it is also possible to configure Ed25519 +and Ed448 certificates. Rather than add two more pairs of key and certificate +parameters, Postfix 3.4 introduces a new "smtp_tls_chain_files" parameter which +specifies all the configured certificates at once, and handles files that hold +both the key and the associated certificates in one pass, thereby avoiding +potential race conditions during key rollover. + +To enable remote SMTP servers to verify the Postfix SMTP client certificate, +the issuing CA certificates must be made available to the server. You should +include the required certificates in the client certificate file, the client +certificate first, then the issuing CA(s) (bottom-up order). + +Example: the certificate for "client.example.com" was issued by "intermediate +CA" which itself has a certificate issued by "root CA". As the "root" super- +user create the client.pem file with: + + # uummaasskk 007777 + # ccaatt cclliieenntt__kkeeyy..ppeemm cclliieenntt__cceerrtt..ppeemm iinntteerrmmeeddiiaattee__CCAA..ppeemm >> cchhaaiinn..ppeemm + +A Postfix SMTP client certificate supplied here must be usable as an SSL client +certificate and hence pass the "openssl verify -purpose sslclient ..." test. + +A server that trusts the root CA has a local copy of the root CA certificate, +so it is not necessary to include the root CA certificate here. Leaving it out +of the "chain.pem" file reduces the overhead of the TLS exchange. + +If you want the Postfix SMTP client to accept remote SMTP server certificates +issued by these CAs, append the root certificate to $smtp_tls_CAfile or install +it in the $smtp_tls_CApath directory. + +Example: Postfix >= 3.4 all-in-one chain file(s). One or more chain files that +start with a key that is immediately followed by the corresponding certificate +and any additional issuer certificates. A single file can hold multiple (key, +cert, [chain]) sequences, one per algorithm. It is typically simpler to keep +the chain for each algorithm in its own file. Most users are likely to deploy +at most a single RSA chain, but with OpenSSL 1.1.1, it is possible to deploy up +five chains, one each for RSA, ECDSA, ED25519, ED448, and even the obsolete +DSA. + + # Postfix >= 3.4. Preferred configuration interface. Each file + # starts with the private key, followed by the corresponding + # certificate, and any intermediate issuer certificates. + # + smtp_tls_chain_files = + /etc/postfix/rsa.pem, + /etc/postfix/ecdsa.pem, + /etc/postfix/ed25519.pem, + /etc/postfix/ed448.pem + +You can also store the keys separately from their certificates, again provided +each is listed before the corresponding certificate chain. Storing a key and +its associated certificate chain in separate files is not recommended, because +this is prone to race conditions during key rollover, as there is no way to +update multiple files atomically. + + # Postfix >= 3.4. + # Storing keys separately from the associated certificates is not + # recommended. + smtp_tls_chain_files = + /etc/postfix/rsakey.pem, + /etc/postfix/rsacerts.pem, + /etc/postfix/ecdsakey.pem, + /etc/postfix/ecdsacerts.pem + +The below examples show the legacy algorithm-specific configurations for +Postfix 3.3 and older. With Postfix <= 3.3, even if the key is stored in the +same file as the certificate, the file is read twice and a (brief) race +condition still exists during key rollover. While Postfix >= 3.4 avoids the +race when the key and certificate are in the same file, you should use the new +"smtp_tls_chain_files" interface shown above. + +RSA key and certificate examples: + + /etc/postfix/main.cf: + smtp_tls_cert_file = /etc/postfix/client.pem + smtp_tls_key_file = $smtp_tls_cert_file + +Their DSA counterparts: + + /etc/postfix/main.cf: + smtp_tls_dcert_file = /etc/postfix/client-dsa.pem + smtp_tls_dkey_file = $smtp_tls_dcert_file + +Their ECDSA counterparts (Postfix >= 2.6 + OpenSSL >= 1.0.0): + + /etc/postfix/main.cf: + smtp_tls_eccert_file = /etc/postfix/client-ecdsa.pem + smtp_tls_eckey_file = $smtp_tls_eccert_file + +To verify a remote SMTP server certificate, the Postfix SMTP client needs to +trust the certificates of the issuing Certification Authorities. These +certificates in "pem" format can be stored in a single $smtp_tls_CAfile or in +multiple files, one CA per file in the $smtp_tls_CApath directory. If you use a +directory, don't forget to create the necessary "hash" links with: + + # $$OOPPEENNSSSSLL__HHOOMMEE//bbiinn//cc__rreehhaasshh //ppaatthh//ttoo//ddiirreeccttoorryy + +The $smtp_tls_CAfile contains the CA certificates of one or more trusted CAs. +The file is opened (with root privileges) before Postfix enters the optional +chroot jail and so need not be accessible from inside the chroot jail. + +Additional trusted CAs can be specified via the $smtp_tls_CApath directory, in +which case the certificates are read (with $mail_owner privileges) from the +files in the directory when the information is needed. Thus, the +$smtp_tls_CApath directory needs to be accessible inside the optional chroot +jail. + +The choice between $smtp_tls_CAfile and $smtp_tls_CApath is a space/time +tradeoff. If there are many trusted CAs, the cost of preloading them all into +memory may not pay off in reduced access time when the certificate is needed. + +Example: + + /etc/postfix/main.cf: + smtp_tls_CAfile = /etc/postfix/CAcert.pem + smtp_tls_CApath = /etc/postfix/certs + +CClliieenntt--ssiiddee TTLLSS ccoonnnneeccttiioonn rreeuussee + +Historically, the Postfix SMTP client has supported multiple deliveries per +plaintext connection. Postfix 3.4 introduces support for multiple deliveries +per TLS-encrypted connection. Multiple deliveries per connection improve mail +delivery performance, especially for destinations that throttle clients that +don't combine deliveries. + +To enable multiple deliveries per TLS connection, specify: + + /etc/postfix/main.cf: + smtp_tls_connection_reuse = yes + +Alternatively, specify the attribute "connection_reuse=yes" in an +smtp_tls_policy_maps entry. + +The implementation of TLS connection reuse relies on the same scache(8) service +as used for delivering plaintext SMTP mail, the same tlsproxy(8) daemon as used +by the postscreen(8) service, and relies on the same hints from the qmgr(8) +daemon. See "Postfix Connection Cache" for a description of the underlying +connection reuse infrastructure. + +Initial SMTP handshake: + + smtp(8) -> remote SMTP server + +Reused SMTP/TLS connection, or new SMTP/TLS connection: + + smtp(8) -> tlsproxy(8) -> remote SMTP server + +Cached SMTP/TLS connection: + + scache(8) -> tlsproxy(8) -> remote SMTP server + +As of Postfix 3.4, TLS connection reuse is disabled by default. This may change +once the impact on over-all performance is understood. + +CClliieenntt--ssiiddee TTLLSS sseessssiioonn ccaacchhee + +The remote SMTP server and the Postfix SMTP client negotiate a session, which +takes some computer time and network bandwidth. By default, this session +information is cached only in the smtp(8) process actually using this session +and is lost when the process terminates. To share the session information +between multiple smtp(8) processes, a persistent session cache can be used. You +can specify any database type that can store objects of several kbytes and that +supports the sequence operator. DBM databases are not suitable because they can +only store small objects. The cache is maintained by the tlsmgr(8) process, so +there is no problem with concurrent access. Session caching is highly +recommended, because the cost of repeatedly negotiating TLS session keys is +high. Future Postfix SMTP servers may limit the number of sessions that a +client is allowed to negotiate per unit time. + +Example: + + /etc/postfix/main.cf: + smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_scache + +Note: as of version 2.5, Postfix no longer uses root privileges when opening +this file. The file should now be stored under the Postfix-owned +data_directory. As a migration aid, an attempt to open the file under a non- +Postfix directory is redirected to the Postfix-owned data_directory, and a +warning is logged. + +Cached Postfix SMTP client session information expires after a certain amount +of time. Postfix/TLS does not use the OpenSSL default of 300s, but a longer +time of 3600s (=1 hour). RFC 2246 recommends a maximum of 24 hours. + +Example: + + /etc/postfix/main.cf: + smtp_tls_session_cache_timeout = 3600s + +As of Postfix 2.11 this setting cannot exceed 100 days. If set <= 0, session +caching is disabled. If set to a positive value less than 2 minutes, the +minimum value of 2 minutes is used instead. + +CClliieenntt TTLLSS lliimmiittaattiioonnss + +The security properties of TLS communication channels are application specific. +While the TLS protocol can provide a confidential, tamper-resistant, mutually +authenticated channel between client and server, not all of these security +features are applicable to every communication. + +For example, while mutual TLS authentication between browsers and web servers +is possible, it is not practical, or even useful, for web-servers that serve +the public to verify the identity of every potential user. In practice, most +HTTPS transactions are asymmetric: the browser verifies the HTTPS server's +identity, but the user remains anonymous. Much of the security policy is up to +the client. If the client chooses to not verify the server's name, the server +is not aware of this. There are many interesting browser security topics, but +we shall not dwell on them here. Rather, our goal is to understand the security +features of TLS in conjunction with SMTP. + +An important SMTP-specific observation is that a public MX host is even more at +the mercy of the SMTP client than is an HTTPS server. Not only can it not +enforce due care in the client's use of TLS, but it cannot even enforce the use +of TLS, because TLS support in SMTP clients is still the exception rather than +the rule. One cannot, in practice, limit access to one's MX hosts to just TLS- +enabled clients. Such a policy would result in a vast reduction in one's +ability to communicate by email with the world at large. + +One may be tempted to try enforcing TLS for mail from specific sending +organizations, but this, too, runs into obstacles. One such obstacle is that we +don't know who is (allegedly) sending mail until we see the "MAIL FROM:" SMTP +command, and at that point, if TLS is not already in use, a potentially +sensitive sender address (and with SMTP PIPELINING one or more of the +recipients) has (have) already been leaked in the clear. Another obstacle is +that mail from the sender to the recipient may be forwarded, and the forwarding +organization may not have any security arrangements with the final destination. +Bounces also need to be protected. These can only be identified by the IP +address and HELO name of the connecting client, and it is difficult to keep +track of all the potential IP addresses or HELO names of the outbound email +servers of the sending organization. + +Consequently, TLS security for mail delivery to public MX hosts is almost +entirely the client's responsibility. The server is largely a passive enabler +of TLS security, the rest is up to the client. While the server has a greater +opportunity to mandate client security policy when it is a dedicated MSA that +only handles outbound mail from trusted clients, below we focus on the client +security policy. + +On the SMTP client, there are further complications. When delivering mail to a +given domain, in contrast to HTTPS, one rarely uses the domain name directly as +the target host of the SMTP session. More typically, one uses MX lookups - +- these are usually unauthenticated -- to obtain the domain's SMTP server +hostname(s). When, as is current practice, the client verifies the insecurely +obtained MX hostname, it is subject to a DNS man-in-the-middle attack. + +Adoption of DNSSEC and RFC6698 (DANE) may gradually (as domains implement +DNSSEC and publish TLSA records for their MX hosts) address the DNS man-in-the- +middle risk and provide scalable key management for SMTP with TLS. Postfix >= +2.11 supports the new dane and dane-only security levels that take advantage of +these standards. + +If clients instead attempted to verify the recipient domain name, an SMTP +server for multiple domains would need to list all its email domain names in +its certificate, and generate a new certificate each time a new domain were +added. At least some CAs set fairly low limits (20 for one prominent CA) on the +number of names that server certificates can contain. This approach is not +consistent with current practice and does not scale. + +It is regrettably the case that TLS secure-channels (fully authenticated and +immune to man-in-the-middle attacks) impose constraints on the sending and +receiving sites that preclude ubiquitous deployment. One needs to manually +configure this type of security for each destination domain, and in many cases +implement non-default TLS policy table entries for additional domains hosted at +a common secured destination. For these reasons secure-channel configurations +will never be the norm. For the generic domain with which you have made no +specific security arrangements, this security level is not a good fit. + +Given that strong authentication is not generally possible, and that verifiable +certificates cost time and money, many servers that implement TLS use self- +signed certificates or private CAs. This further limits the applicability of +verified TLS on the public Internet. + +Historical note: while the documentation of these issues and many of the +related features were new with Postfix 2.3, the issue was well understood +before Postfix 1.0, when Lutz Ja"nicke was designing the first unofficial +Postfix TLS patch. See his original post http://www.imc.org/ietf-apps-tls/mail- +archive/msg00304.html and the first response http://www.imc.org/ietf-apps-tls/ +mail-archive/msg00305.html. The problem is not even unique to SMTP or even TLS, +similar issues exist for secure connections via aliases for HTTPS and Kerberos. +SMTP merely uses indirect naming (via MX records) more frequently. + +TTLLSS ppoolliiccyy ttaabbllee + +A small fraction of servers offer STARTTLS but the negotiation consistently +fails. As long as encryption is not mandatory, the Postfix SMTP client retries +the delivery immediately with TLS disabled, without any need to explicitly +disable TLS for the problem destinations. + +The policy table is specified via the smtp_tls_policy_maps parameter. This +lists optional lookup tables with the Postfix SMTP client TLS security policy +by next-hop destination. + +The TLS policy table is indexed by the full next-hop destination, which is +either the recipient domain, or the verbatim next-hop specified in the +transport table, $local_transport, $virtual_transport, $relay_transport or +$default_transport. This includes any enclosing square brackets and any non- +default destination server port suffix. The LMTP socket type prefix (inet: or +unix:) is not included in the lookup key. + +Only the next-hop domain, or $myhostname with LMTP over UNIX-domain sockets, is +used as the nexthop name for certificate verification. The port and any +enclosing square brackets are used in the table lookup key, but are not used +for server name verification. + +When the lookup key is a domain name without enclosing square brackets or any : +port suffix (typically the recipient domain), and the full domain is not found +in the table, just as with the transport(5) table, the parent domain starting +with a leading "." is matched recursively. This allows one to specify a +security policy for a recipient domain and all its sub-domains. + +The lookup result is a security level, followed by an optional list of +whitespace and/or comma separated name=value attributes that override related +main.cf settings. The TLS security levels are described above. Below, we +describe the corresponding table syntax: + +nnoonnee + No TLS. No additional attributes are supported at this level. +mmaayy + Opportunistic TLS. The optional "ciphers", "exclude" and "protocols" + attributes (available for opportunistic TLS with Postfix >= 2.6) override + the "smtp_tls_ciphers", "smtp_tls_exclude_ciphers" and "smtp_tls_protocols" + configuration parameters. At this level and higher, the optional + "servername" attribute (available with Postfix >= 3.4) overrides the global + "smtp_tls_servername" parameter, enabling per-destination configuration of + the SNI extension sent to the remote SMTP server. +eennccrryypptt + Mandatory encryption. Mail is delivered only if the remote SMTP server + offers STARTTLS and the TLS handshake succeeds. At this level and higher, + the optional "protocols" attribute overrides the main.cf + smtp_tls_mandatory_protocols parameter, the optional "ciphers" attribute + overrides the main.cf smtp_tls_mandatory_ciphers parameter, and the + optional "exclude" attribute (Postfix >= 2.6) overrides the main.cf + smtp_tls_mandatory_exclude_ciphers parameter. +ddaannee + Opportunistic DANE TLS. The TLS policy for the destination is obtained via + TLSA records in DNSSEC. If no TLSA records are found, the effective + security level used is may. If TLSA records are found, but none are usable, + the effective security level is encrypt. When usable TLSA records are + obtained for the remote SMTP server, SSLv2+3 are automatically disabled + (see smtp_tls_mandatory_protocols), and the server certificate must match + the TLSA records. RFC 7672 (DANE) TLS authentication and DNSSEC support is + available with Postfix 2.11 and later. +ddaannee--oonnllyy + Mandatory DANE TLS. The TLS policy for the destination is obtained via TLSA + records in DNSSEC. If no TLSA records are found, or none are usable, no + connection is made to the server. When usable TLSA records are obtained for + the remote SMTP server, SSLv2+3 are automatically disabled (see + smtp_tls_mandatory_protocols), and the server certificate must match the + TLSA records. RFC 7672 (DANE) TLS authentication and DNSSEC support is + available with Postfix 2.11 and later. +ffiinnggeerrpprriinntt + Certificate fingerprint verification. Available with Postfix 2.5 and later. + At this security level, there are no trusted Certification Authorities. The + certificate trust chain, expiration date, ... are not checked. Instead, the + optional mmaattcchh attribute, or else the main.cf + ssmmttpp__ttllss__ffiinnggeerrpprriinntt__cceerrtt__mmaattcchh parameter, lists the server certificate + fingerprints or public key fingerprints (Postfix 2.9 and later). The digest + algorithm used to calculate fingerprints is selected by the + ssmmttpp__ttllss__ffiinnggeerrpprriinntt__ddiiggeesstt parameter. Multiple fingerprints can be + combined with a "|" delimiter in a single match attribute, or multiple + match attributes can be employed. The ":" character is not used as a + delimiter as it occurs between each pair of fingerprint (hexadecimal) + digits. +vveerriiffyy + Mandatory server certificate verification. Mail is delivered only if the + TLS handshake succeeds, if the remote SMTP server certificate can be + validated (not expired or revoked, and signed by a trusted Certification + Authority), and if the server certificate name matches the optional "match" + attribute (or the main.cf smtp_tls_verify_cert_match parameter value when + no optional "match" attribute is specified). With Postfix >= 2.11 the + "tafile" attribute optionally modifies trust chain verification in the same + manner as the "smtp_tls_trust_anchor_file" parameter. The "tafile" + attribute may be specified multiple times to load multiple trust-anchor + files. +sseeccuurree + Secure certificate verification. Mail is delivered only if the TLS + handshake succeeds, and DNS forgery resistant remote SMTP certificate + verification succeeds (not expired or revoked, and signed by a trusted + Certification Authority), and if the server certificate name matches the + optional "match" attribute (or the main.cf smtp_tls_secure_cert_match + parameter value when no optional "match" attribute is specified). With + Postfix >= 2.11 the "tafile" attribute optionally modifies trust chain + verification in the same manner as the "smtp_tls_trust_anchor_file" + parameter. The "tafile" attribute may be specified multiple times to load + multiple trust-anchor files. +Notes: + + * The "match" attribute is especially useful to verify TLS certificates for + domains that are hosted on a shared server. In that case, specify "match" + rules for the shared server's name. While secure verification can also be + achieved with manual routing overrides in Postfix transport(5) tables, that + approach can deliver mail to the wrong host when domains are assigned to + new gateway hosts. The "match" attribute approach avoids the problems of + manual routing overrides; mail is deferred if verification of a new MX host + fails. + + * When a policy table entry specifies multiple match patterns, multiple match + strategies, or multiple protocols, these must be separated by colons. + + * The "exclude" attribute (Postfix >= 2.6) is used to disable ciphers that + cause handshake failures with a specific mandatory TLS destination, without + disabling the ciphers for all mandatory destinations. Alternatively, you + can exclude ciphers that cause issues with multiple remote servers in + main.cf, and selectively enable them on a per-destination basis in the + policy table by setting a shorter or empty exclusion list. The per- + destination "exclude" list preempts both the opportunistic and mandatory + security level exclusions, so that all excluded ciphers can be enabled for + known-good destinations. For non-mandatory TLS destinations that exhibit + cipher-specific problems, Postfix will fall back to plain-text delivery. If + plain-text is not acceptable make TLS mandatory and exclude the problem + ciphers. + +Example: + + /etc/postfix/main.cf: + smtp_tls_policy_maps = hash:/etc/postfix/tls_policy + # Postfix 2.5 and later + smtp_tls_fingerprint_digest = sha256 + /etc/postfix/tls_policy: + example.edu none + example.mil may + example.gov encrypt ciphers=high + example.com verify match=hostname:dot-nexthop ciphers=high + example.net secure + .example.net secure match=.example.net:example.net + [mail.example.org]:587 secure match=nexthop + # Postfix 2.5 and later + [thumb.example.org] fingerprint + match=b6:b4:72:34:e2:59:cd:fb:...:0d:4d:cc:2c:7d:84:de:e6:2f + match=51:e9:af:2e:1e:40:1f:de:...:35:2d:09:16:31:5a:eb:82:76 + # Postfix >= 3.6 "protocols" syntax + example.info may protocols=>=TLSv1 ciphers=medium + exclude=3DES + # Legacy protocols syntax + example.info may protocols=!SSLv2:!SSLv3 ciphers=medium + exclude=3DES + +NNoottee:: The "hostname" strategy if listed in a non-default setting of +smtp_tls_secure_cert_match or in the "match" attribute in the policy table can +render the "secure" level vulnerable to DNS forgery. Do not use the "hostname" +strategy for secure-channel configurations in environments where DNS security +is not assured. + +DDiissccoovveerriinngg sseerrvveerrss tthhaatt ssuuppppoorrtt TTLLSS + +As we decide on a "per site" basis whether or not to use TLS, it would be good +to have a list of sites that offered "STARTTLS". We can collect it ourselves +with this option. + +If the smtp_tls_note_starttls_offer feature is enabled and a server offers +STARTTLS while TLS is not already enabled for that server, the Postfix SMTP +client logs a line as follows: + + postfix/smtp[pid]: Host offered STARTTLS: [hostname.example.com] + +Example: + + /etc/postfix/main.cf: + smtp_tls_note_starttls_offer = yes + +SSeerrvveerr cceerrttiiffiiccaattee vveerriiffiiccaattiioonn ddeepptthh + +The server certificate verification depth is specified with the main.cf +smtp_tls_scert_verifydepth parameter. The default verification depth is 9 (the +OpenSSL default), for compatibility with Postfix versions before 2.5 where +smtp_tls_scert_verifydepth was ignored. When you configure trust in a root CA, +it is not necessary to explicitly trust intermediary CAs signed by the root CA, +unless $smtp_tls_scert_verifydepth is less than the number of CAs in the +certificate chain for the servers of interest. With a verify depth of 1 you can +only verify certificates directly signed by a trusted CA, and all trusted +intermediary CAs need to be configured explicitly. With a verify depth of 2 you +can verify servers signed by a root CA or a direct intermediary CA (so long as +the server is correctly configured to supply its intermediate CA certificate). + +Example: + + /etc/postfix/main.cf: + smtp_tls_scert_verifydepth = 2 + +CClliieenntt--ssiiddee cciipphheerr ccoonnttrroollss + +The Postfix SMTP client supports 5 distinct cipher grades as specified by the +smtp_tls_mandatory_ciphers configuration parameter. This setting controls the +minimum acceptable SMTP client TLS cipher grade for use with mandatory TLS +encryption. The default value "medium" is suitable for most destinations with +which you may want to enforce TLS, and is beyond the reach of today's +cryptanalytic methods. See smtp_tls_policy_maps for information on how to +configure ciphers on a per-destination basis. + +By default anonymous ciphers are allowed, and automatically disabled when +remote SMTP server certificates are verified. If you want to disable anonymous +ciphers even at the "encrypt" security level, set +"smtp_tls_mandatory_exclude_ciphers = aNULL"; and to disable anonymous ciphers +even with opportunistic TLS, set "smtp_tls_exclude_ciphers = aNULL". There is +generally no need to take these measures. Anonymous ciphers save bandwidth and +TLS session cache space, if certificates are ignored, there is little point in +requesting them. + +The "smtp_tls_ciphers" configuration parameter (Postfix >= 2.6) provides +control over the minimum cipher grade for opportunistic TLS. The default +minimum cipher grade for opportunistic TLS is "medium" for Postfix releases +after the middle of 2015, and "export" for older releases. With Postfix < 2.6, +the minimum opportunistic TLS cipher grade is always "export". + +With mandatory and opportunistic TLS encryption, the Postfix SMTP client will +by default disable SSLv2 and SSLv3. The mandatory TLS protocol list is +specified via the smtp_tls_mandatory_protocols configuration parameter. The +corresponding smtp_tls_protocols parameter (Postfix >= 2.6) controls the TLS +protocols used with opportunistic TLS. + +Example: + + /etc/postfix/main.cf: + smtp_tls_mandatory_ciphers = medium + smtp_tls_mandatory_exclude_ciphers = RC4, MD5 + smtp_tls_exclude_ciphers = aNULL + smtp_tls_ciphers = medium + # Preferred form with Postfix >= 3.6: + smtp_tls_mandatory_protocols = >=TLSv1.2 + smtp_tls_protocols = >=TLSv1 + # Legacy form for Postfix < 3.6: + smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 + smtp_tls_protocols = !SSLv2,!SSLv3 + +CClliieenntt--ssiiddee SSMMTTPPSS ssuuppppoorrtt + +These sections show how to send mail to a server that does not support +STARTTLS, but that provides the SMTPS service on TCP port 465. Depending on the +Postfix version, some additional tooling may be required. + +PPoossttffiixx >>== 33..00 + +The Postfix SMTP client has SMTPS support built-in as of version 3.0. Use one +of the following examples, to send all remote mail, or to send only some remote +mail, to an SMTPS server. + +PPoossttffiixx >>== 33..00:: SSeennddiinngg aallll rreemmoottee mmaaiill ttoo aann SSMMTTPPSS sseerrvveerr + +The first example will send all remote mail over SMTPS through a provider's +server called "mail.example.com": + + /etc/postfix/main.cf: + # Client-side SMTPS requires "encrypt" or stronger. + smtp_tls_security_level = encrypt + smtp_tls_wrappermode = yes + # The [] suppress MX lookups. + relayhost = [mail.example.com]:465 + +Use "postfix reload" to make the change effective. + +See SOHO_README for additional information about SASL authentication. + +PPoossttffiixx >>== 33..00:: SSeennddiinngg oonnllyy mmaaiill ffoorr aa ssppeecciiffiicc ddeessttiinnaattiioonn vviiaa SSMMTTPPSS + +The second example will send only mail for "example.com" via SMTPS. This time, +Postfix uses a transport map to deliver only mail for "example.com" via SMTPS: + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + + /etc/postfix/transport: + example.com relay-smtps:example.com:465 + + /etc/postfix/master.cf: + relay-smtps unix - - n - - smtp + # Client-side SMTPS requires "encrypt" or stronger. + -o smtp_tls_security_level=encrypt + -o smtp_tls_wrappermode=yes + +Use "postmap hash:/etc/postfix/transport" and "postfix reload" to make the +change effective. + +See SOHO_README for additional information about SASL authentication. + +PPoossttffiixx << 33..00 + +Although older Postfix SMTP client versions do not support TLS wrapper mode, it +is relatively easy to forward a connection through the stunnel program if +Postfix needs to deliver mail to some legacy system that doesn't support +STARTTLS. + +PPoossttffiixx << 33..00:: SSeennddiinngg aallll rreemmoottee mmaaiill ttoo aann SSMMTTPPSS sseerrvveerr + +The first example uses SMTPS to send all remote mail to a provider's mail +server called "mail.example.com". + +A minimal stunnel.conf file is sufficient to set up a tunnel from local port +11125 to the remote destination "mail.example.com" and port "smtps". Postfix +will later use this tunnel to connect to the remote server. + + /path/to/stunnel.conf: + [smtp-tls-wrapper] + accept = 11125 + client = yes + connect = mail.example.com:smtps + +To test this tunnel, use: + + $ telnet localhost 11125 + +This should produce the greeting from the remote SMTP server at +mail.example.com. + +On the Postfix side, the relayhost feature sends all remote mail through the +local stunnel listener on port 11125: + + /etc/postfix/main.cf: + relayhost = [127.0.0.1]:11125 + +Use "postfix reload" to make the change effective. + +See SOHO_README for additional information about SASL authentication. + +PPoossttffiixx << 33..00:: SSeennddiinngg oonnllyy mmaaiill ffoorr aa ssppeecciiffiicc ddeessttiinnaattiioonn vviiaa SSMMTTPPSS + +The second example will use SMTPS to send only mail for "example.com" via +SMTPS. It uses the same stunnel configuration file as the first example, so it +won't be repeated here. + +This time, the Postfix side uses a transport map to direct only mail for +"example.com" through the tunnel: + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + + /etc/postfix/transport: + example.com relay:[127.0.0.1]:11125 + +Use "postmap hash:/etc/postfix/transport" and "postfix reload" to make the +change effective. + +See SOHO_README for additional information about SASL authentication. + +MMiisscceellllaanneeoouuss cclliieenntt ccoonnttrroollss + +The smtp_starttls_timeout parameter limits the time of Postfix SMTP client +write and read operations during TLS startup and shutdown handshake procedures. +In case of problems the Postfix SMTP client tries the next network address on +the mail exchanger list, and defers delivery if no alternative server is +available. + +Example: + + /etc/postfix/main.cf: + smtp_starttls_timeout = 300s + +With Postfix 2.8 and later, the tls_disable_workarounds parameter specifies a +list or bit-mask of OpenSSL bug work-arounds to disable. This may be necessary +if one of the work-arounds enabled by default in OpenSSL proves to pose a +security risk, or introduces an unexpected interoperability issue. Some bug +work-arounds known to be problematic are disabled in the default value of the +parameter when linked with an OpenSSL library that could be vulnerable. + +Example: + + /etc/postfix/main.cf: + tls_disable_workarounds = 0xFFFFFFFF + tls_disable_workarounds = CVE-2010-4180, LEGACY_SERVER_CONNECT + +Note: Disabling LEGACY_SERVER_CONNECT is not wise at this time, lots of servers +are still unpatched and Postfix is not significantly vulnerable to the +renegotiation issue in the TLS protocol. + +With Postfix >= 2.11, the tls_ssl_options parameter specifies a list or bit- +mask of OpenSSL options to enable. Specify one or more of the named options +below, or a hexadecimal bitmask of options found in the ssl.h file +corresponding to the run-time OpenSSL library. While it may be reasonable to +turn off all bug workarounds (see above), it is not a good idea to attempt to +turn on all features. + +A future version of OpenSSL may by default no longer allow connections to +servers that don't support secure renegotiation. Since the exposure for SMTP is +minimal, and some SMTP servers may remain unpatched, you can add +LEGACY_SERVER_CONNECT to the options to restore the more permissive default of +current OpenSSL releases. + +Example: + + /etc/postfix/main.cf: + tls_ssl_options = NO_TICKET, NO_COMPRESSION, LEGACY_SERVER_CONNECT + +You should only enable features via the hexadecimal mask when the need to +control the feature is critical (to deal with a new vulnerability or a serious +interoperability problem). Postfix DOES NOT promise backwards compatible +behavior with respect to the mask bits. A feature enabled via the mask in one +release may be enabled by other means in a later release, and the mask bit will +then be ignored. Therefore, use of the hexadecimal mask is only a temporary +measure until a new Postfix or OpenSSL release provides a better solution. + +TTLLSS mmaannaaggeerr ssppeecciiffiicc sseettttiinnggss + +The security of cryptographic software such as TLS depends critically on the +ability to generate unpredictable numbers for keys and other information. To +this end, the tlsmgr(8) process maintains a Pseudo Random Number Generator +(PRNG) pool. This is queried by the smtp(8) and smtpd(8) processes when they +initialize. By default, these daemons request 32 bytes, the equivalent to 256 +bits. This is more than sufficient to generate a 128bit (or 168bit) session +key. + +Example: + + /etc/postfix/main.cf: + tls_daemon_random_bytes = 32 + +In order to feed its in-memory PRNG pool, the tlsmgr(8) reads entropy from an +external source, both at startup and during run-time. Specify a good entropy +source, like EGD or /dev/urandom; be sure to only use non-blocking sources (on +OpenBSD, use /dev/arandom when tlsmgr(8) complains about /dev/urandom timeout +errors). If the entropy source is not a regular file, you must prepend the +source type to the source name: "dev:" for a device special file, or "egd:" for +a source with EGD compatible socket interface. + +Examples (specify only one in main.cf): + + /etc/postfix/main.cf: + tls_random_source = dev:/dev/urandom + tls_random_source = egd:/var/run/egd-pool + +By default, tlsmgr(8) reads 32 bytes from the external entropy source at each +seeding event. This amount (256bits) is more than sufficient for generating a +128bit symmetric key. With EGD and device entropy sources, the tlsmgr(8) limits +the amount of data read at each step to 255 bytes. If you specify a regular +file as entropy source, a larger amount of data can be read. + +Example: + + /etc/postfix/main.cf: + tls_random_bytes = 32 + +In order to update its in-memory PRNG pool, the tlsmgr(8) queries the external +entropy source again after a pseudo-random amount of time. The time is +calculated using the PRNG, and is between 0 and the maximal time specified with +tls_random_reseed_period. The default maximal time interval is 1 hour. + +Example: + + /etc/postfix/main.cf: + tls_random_reseed_period = 3600s + +The tlsmgr(8) process saves the PRNG state to a persistent exchange file at +regular times and when the process terminates, so that it can recover the PRNG +state the next time it starts up. This file is created when it does not exist. + +Examples: + + /etc/postfix/main.cf: + tls_random_exchange_name = /var/lib/postfix/prng_exch + tls_random_prng_update_period = 3600s + +As of version 2.5, Postfix no longer uses root privileges when opening this +file. The file should now be stored under the Postfix-owned data_directory. As +a migration aid, an attempt to open the file under a non-Postfix directory is +redirected to the Postfix-owned data_directory, and a warning is logged. If you +wish to continue using a pre-existing PRNG state file, move it to the +data_directory and change the ownership to the account specified with the +mail_owner parameter. + +With earlier Postfix versions the default file location is under the Postfix +configuration directory, which is not the proper place for information that is +modified by Postfix. + +GGeettttiinngg ssttaarrtteedd,, qquuiicckk aanndd ddiirrttyy + +The following steps will get you started quickly. Because you sign your own +Postfix public key certificate, you get TLS encryption but no TLS +authentication. This is sufficient for testing, and for exchanging email with +sites that you have no trust relationship with. For real authentication you +need also enable DNSSEC record signing for your domain and publish TLSA records +and/or your Postfix public key certificate needs to be signed by a recognized +Certification Authority. To authenticate the certificates of a remote host you +need a DNSSEC-validating local resolver and to enable DANE authentication and/ +or configure the Postfix SMTP client with a list of public key certificates of +Certification Authorities, but make sure to read about the limitations of the +latter approach. + +In the examples below, user input is shown in bboolldd font, and a "#" prompt +indicates a super-user shell. + + * Quick-start TLS with Postfix >= 3.1. + + * Self-signed server certificate. + + * Private Certification Authority. + +QQuuiicckk--ssttaarrtt TTLLSS wwiitthh PPoossttffiixx >>== 33..11 + +Postfix 3.1 provides built-in support for enabling TLS in the SMTP client and +server and for ongoing certificate and DANE TLSA record management. + + * Quick-start TLS in the Postfix >= 3.1 SMTP client. + + * Quick-start TLS in the Postfix >= 3.1 SMTP server. + +QQuuiicckk--ssttaarrtt TTLLSS iinn tthhee PPoossttffiixx >>== 33..11 SSMMTTPP cclliieenntt.. + +If you are using Postfix 3.1 or later, and your SMTP client TLS settings are in +their default state, you can enable opportunistic TLS in the SMTP client as +follows: + + # postfix tls enable-client + # postfix reload + +If some of the Postfix SMTP client TLS settings are not in their default state, +this will not make any changes, but will instead suggest the minimal required +settings for SMTP client TLS. The "postfix reload" command is optional, it is +only needed if you want the settings to take effect right away. Note, this does +not enable trust in any public certification authorities, and does not +configure client TLS certificates as these are largely pointless with +opportunistic TLS. + +There is not yet a turn-key command for enabling DANE authentication. This is +because DANE requires changes to your rreessoollvv..ccoonnff file and a corresponding +DNSSEC-validating resolver local to the Postfix host, these changes are +difficult to automate in a portable way. + +If you're willing to revert your settings to the defaults and switch to a +"stock" opportunistic TLS configuration, then you can: erase all the SMTP +client TLS settings and then enable client TLS: + + # postconf -X `postconf -nH | grep -E '^smtp(_|_enforce_|_use_)tls'` + # postfix tls enable-client + # postfix reload + +QQuuiicckk--ssttaarrtt TTLLSS iinn tthhee PPoossttffiixx >>== 33..11 SSMMTTPP sseerrvveerr.. + +If you are using Postfix 3.1 or later, and your SMTP server TLS settings are in +their default state, you can enable opportunistic TLS in the SMTP server as +follows: + + # postfix tls enable-server + # postfix reload + +If some of the Postfix SMTP client TLS settings are not in their default state, +this will not make any changes, but will instead suggest the minimal required +settings for SMTP client TLS. The "postfix reload" command is optional, it is +only needed if you want the settings to take effect right away. This will +generate a self-signed private key and certificate and enable TLS in the +Postfix SMTP server. + +If you're willing to revert your settings to the defaults and switch to a +"stock" server TLS configuration, then you can: erase all the SMTP server TLS +settings and then enable server TLS: + + # postconf -X `postconf -nH | grep -E '^smtpd(_|_enforce_|_use_)tls'` + # postfix tls enable-server + # postfix reload + +Postfix >= 3.1 provides additional built-in support for ongoing management of +TLS in the SMTP server, via additional "postfix tls" sub-commands. These make +it easy to generate certificate signing requests, create and deploy new keys +and certificates, and generate DANE TLSA records. See the postfix-tls(1) +documentation for details. + +SSeellff--ssiiggnneedd sseerrvveerr cceerrttiiffiiccaattee + +The following commands (credits: Viktor Dukhovni) generate and install a 2048- +bit RSA private key and 10-year self-signed certificate for the local Postfix +system. This requires super-user privileges. (By using date-specific filenames +for the certificate and key files, and updating main.cf with new filenames, a +potential race condition in which the key and certificate might not match is +avoided). + + # dir="$(postconf -h config_directory)" + # fqdn=$(postconf -h myhostname) + # case $fqdn in /*) fqdn=$(cat "$fqdn");; esac + # ymd=$(date +%Y-%m-%d) + # key="${dir}/key-${ymd}.pem"; rm -f "${key}" + # cert="${dir}/cert-${ymd}.pem"; rm -f "${cert}" + # (umask 077; openssl genrsa -out "${key}" 2048) && + openssl req -new -key "${key}" \ + -x509 -subj "/CN=${fqdn}" -days 3650 -out "${cert}" && + postconf -e \ + "smtpd_tls_cert_file = ${cert}" \ + "smtpd_tls_key_file = ${key}" \ + 'smtpd_tls_security_level = may' \ + 'smtpd_tls_received_header = yes' \ + 'smtpd_tls_loglevel = 1' \ + 'smtp_tls_security_level = may' \ + 'smtp_tls_loglevel = 1' \ + 'smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache' + \ + 'tls_random_source = dev:/dev/urandom' + +Note: the last command requires both single (') and double (") quotes. + +The postconf(1) command above enables opportunistic TLS for receiving and +sending mail. It also enables logging of TLS connections and recording of TLS +use in the "Received" header. TLS session caching is also enabled in the +Postfix SMTP client. With Postfix >= 2.10, the SMTP server does not need an +explicit session cache since session reuse is better handled via RFC 5077 TLS +session tickets. + +PPrriivvaattee CCeerrttiiffiiccaattiioonn AAuutthhoorriittyy + + * Become your own Certification Authority, so that you can sign your own + certificates, and so that your own systems can authenticate certificates + from your own CA. This example uses the CA.pl script that ships with + OpenSSL. On some systems, OpenSSL installs this as /usr/local/openssl/misc/ + CA.pl. Some systems install this as part of a package named openssl-perl or + something similar. The script creates a private key in ./demoCA/private/ + cakey.pem and a public key in ./demoCA/cacert.pem. + + % //uussrr//llooccaall//ssssll//mmiisscc//CCAA..ppll --nneewwccaa + CA certificate filename (or enter to create) + + Making CA certificate ... + Using configuration from /etc/ssl/openssl.cnf + Generating a 1024 bit RSA private key + ....................++++++ + .....++++++ + writing new private key to './demoCA/private/cakey.pem' + Enter PEM pass phrase:wwhhaatteevveerr + + * Create an unpassworded private key for host foo.porcupine.org and create an + unsigned public key certificate. + + % ((uummaasskk 007777;; ooppeennssssll rreeqq --nneeww --nneewwkkeeyy rrssaa::22004488 --nnooddeess --kkeeyyoouutt ffoooo-- + kkeeyy..ppeemm --oouutt ffoooo--rreeqq..ppeemm)) + Using configuration from /etc/ssl/openssl.cnf + Generating a 2048 bit RSA private key + ........................................++++++ + ....++++++ + writing new private key to 'foo-key.pem' + ----- + You are about to be asked to enter information that will be + incorporated + into your certificate request. + What you are about to enter is what is called a Distinguished Name or a + DN. + There are quite a few fields but you can leave some blank + For some fields there will be a default value, + If you enter '.', the field will be left blank. + ----- + Country Name (2 letter code) [AU]:UUSS + State or Province Name (full name) [Some-State]:NNeeww YYoorrkk + Locality Name (eg, city) []:WWeessttcchheesstteerr + Organization Name (eg, company) [Internet Widgits Pty Ltd]:PPoorrccuuppiinnee + Organizational Unit Name (eg, section) []: + Common Name (eg, YOUR name) []:ffoooo..ppoorrccuuppiinnee..oorrgg + Email Address []:wwiieettssee@@ppoorrccuuppiinnee..oorrgg + + Please enter the following 'extra' attributes + to be sent with your certificate request + A challenge password []:wwhhaatteevveerr + An optional company name []: + + * Sign the public key certificate for host foo.porcupine.org with the + Certification Authority private key that we created a few steps ago. + + % ooppeennssssll ccaa --oouutt ffoooo--cceerrtt..ppeemm --ddaayyss 336655 --iinnffiilleess ffoooo--rreeqq..ppeemm + Using configuration from /etc/ssl/openssl.cnf + Enter PEM pass phrase:wwhhaatteevveerr + Check that the request matches the signature + Signature ok + The Subjects Distinguished Name is as follows + countryName :PRINTABLE:'US' + stateOrProvinceName :PRINTABLE:'New York' + localityName :PRINTABLE:'Westchester' + organizationName :PRINTABLE:'Porcupine' + commonName :PRINTABLE:'foo.porcupine.org' + emailAddress :IA5STRING:'wietse@porcupine.org' + Certificate is to be certified until Nov 21 19:40:56 2005 GMT (365 + days) + Sign the certificate? [y/n]:yy + + 1 out of 1 certificate requests certified, commit? [y/n]yy + Write out database with 1 new entries + Data Base Updated + + * Install the host private key, the host public key certificate, and the + Certification Authority certificate files. This requires super-user + privileges. + + The following commands assume that the key and certificate will be + installed for the local Postfix MTA. You will need to adjust the commands + if the Postfix MTA is on a different host. + + # ccpp ddeemmooCCAA//ccaacceerrtt..ppeemm ffoooo--kkeeyy..ppeemm ffoooo--cceerrtt..ppeemm //eettcc//ppoossttffiixx + # cchhmmoodd 664444 //eettcc//ppoossttffiixx//ffoooo--cceerrtt..ppeemm //eettcc//ppoossttffiixx//ccaacceerrtt..ppeemm + # cchhmmoodd 440000 //eettcc//ppoossttffiixx//ffoooo--kkeeyy..ppeemm + + * Configure Postfix, by adding the following to /etc/postfix/main.cf. It is + generally best to not configure client certificates, unless there are + servers which authenticate your mail submission via client certificates. + Often servers that perform TLS client authentication will issue the + required certificates signed by their own CA. If you configure the client + certificate and key incorrectly, you will be unable to send mail to sites + that request a client certificate, but don't require them from all clients. + + /etc/postfix/main.cf: + smtp_tls_CAfile = /etc/postfix/cacert.pem + smtp_tls_session_cache_database = + btree:/var/lib/postfix/smtp_tls_session_cache + smtp_tls_security_level = may + smtp_tls_loglevel = 1 + smtpd_tls_CAfile = /etc/postfix/cacert.pem + smtpd_tls_cert_file = /etc/postfix/foo-cert.pem + smtpd_tls_key_file = /etc/postfix/foo-key.pem + smtpd_tls_received_header = yes + smtpd_tls_session_cache_database = + btree:/var/lib/postfix/smtpd_tls_session_cache + tls_random_source = dev:/dev/urandom + smtpd_tls_security_level = may + smtpd_tls_loglevel = 1 + +BBuuiillddiinngg PPoossttffiixx wwiitthh TTLLSS ssuuppppoorrtt + +These instructions assume that you build Postfix from source code as described +in the INSTALL document. Some modification may be required if you build Postfix +from a vendor-specific source package. + +To build Postfix with TLS support, first we need to generate the make(1) files +with the necessary definitions. This is done by invoking the command "make +makefiles" in the Postfix top-level directory and with arguments as shown next. + +NNOOTTEE:: DDoo nnoott uussee GGnnuu TTLLSS.. IItt wwiillll ssppoonnttaanneeoouussllyy tteerrmmiinnaattee aa PPoossttffiixx ddaaeemmoonn +pprroocceessss wwiitthh eexxiitt ssttaattuuss ccooddee 22,, iinnsstteeaadd ooff aalllloowwiinngg PPoossttffiixx ttoo 11)) rreeppoorrtt tthhee +eerrrroorr ttoo tthhee mmaaiilllloogg ffiillee,, aanndd ttoo 22)) pprroovviiddee ppllaaiinntteexxtt sseerrvviiccee wwhheerree tthhiiss iiss +aapppprroopprriiaattee.. + + * If the OpenSSL include files (such as ssl.h) are in directory /usr/include/ + openssl, and the OpenSSL libraries (such as libssl.so and libcrypto.so) are + in directory /usr/lib: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS"" AAUUXXLLIIBBSS==""--llssssll --llccrryyppttoo"" + + * If the OpenSSL include files (such as ssl.h) are in directory /usr/local/ + include/openssl, and the OpenSSL libraries (such as libssl.so and + libcrypto.so) are in directory /usr/local/lib: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS --II//uussrr//llooccaall//iinncclluuddee"" \\ + AAUUXXLLIIBBSS==""--LL//uussrr//llooccaall//lliibb --llssssll --llccrryyppttoo"" + + If your OpenSSL shared library is in a directory that the RUN-TIME linker + does not know about, add a "-Wl,-R,/path/to/directory" option after "- + lcrypto". + + On Solaris, specify the -R option as shown below: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS --II//uussrr//llooccaall//iinncclluuddee"" \\ + AAUUXXLLIIBBSS==""--RR//uussrr//llooccaall//lliibb --LL//uussrr//llooccaall//lliibb --llssssll --llccrryyppttoo"" + +If you need to apply other customizations (such as Berkeley DB databases, +MySQL, PostgreSQL, LDAP or SASL), see the respective Postfix README documents, +and combine their "make makefiles" instructions with the instructions above: + + % mmaakkee ttiiddyy # if you have left-over files from a previous build + % mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDUUSSEE__TTLLSS \\ + ((ootthheerr --DD oorr --II ooppttiioonnss))"" \\ + AAUUXXLLIIBBSS==""--llssssll --llccrryyppttoo \\ + ((ootthheerr --ll ooppttiioonnss ffoorr lliibbrraarriieess iinn //uussrr//lliibb)) \\ + ((--LL//ppaatthh//nnaammee ++ --ll ooppttiioonnss ffoorr ootthheerr lliibbrraarriieess))"" + +To complete the build process, see the Postfix INSTALL instructions. Postfix +has TLS support turned off by default, so you can start using Postfix as soon +as it is installed. + +RReeppoorrttiinngg pprroobblleemmss + +Problems are preferably reported via <postfix-users@postfix.org>. See http:// +www.postfix.org/lists.html for subscription information. When reporting a +problem, please be thorough in the report. Patches, when possible, are greatly +appreciated too. + +CCrreeddiittss + + * TLS support for Postfix was originally developed by Lutz Ja"nicke at + Cottbus Technical University. + * Wietse Venema adopted the code, did some restructuring, and compiled this + part of the documentation from Lutz's documents. + * Victor Duchovni was instrumental with the re-implementation of the + smtp_tls_per_site code in terms of enforcement levels, which simplified the + implementation greatly. + * Victor Duchovni implemented the fingerprint security level, added more + sanity checks, and separated TLS connection management from security policy + enforcement. The latter change simplified the code that verifies + certificate signatures, certificate names, and certificate fingerprints. + diff --git a/README_FILES/TUNING_README b/README_FILES/TUNING_README new file mode 100644 index 0000000..1080164 --- /dev/null +++ b/README_FILES/TUNING_README @@ -0,0 +1,494 @@ + PPoossttffiixx PPeerrffoorrmmaannccee TTuunniinngg + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff PPoossttffiixx ppeerrffoorrmmaannccee ttuunniinngg + +The hints and tips in this document help you improve the performance of Postfix +systems that already work. If your Postfix system is unable to receive or +deliver mail, then you need to solve those problems first, using the +DEBUG_README document as guidance. + +For tuning external content filter performance, first read the respective +information in the FILTER_README and SMTPD_PROXY_README documents. Then make +sure to avoid latency in the content filter code. As much as possible avoid +performing queries against external data sources with a high or highly variable +delay. Your content filter will run with a small concurrency to avoid CPU/ +memory starvation, and if any latency creeps in, content filter throughput will +suffer. High volume environments should avoid RBL lookups, complex database +queries and so on. + +Topics on mail receiving performance: + + * General mail receiving performance tips + * Doing more work with your SMTP server processes + * Slowing down SMTP clients that make many errors + * Measures against clients that make too many connections + +Topics on mail delivery performance: + + * General mail delivery performance tips + * Tuning the frequency of deferred mail delivery attempts + * Tuning the number of simultaneous deliveries + * Tuning the number of recipients per delivery + +Other Postfix performance tuning topics: + + * Tuning the number of Postfix processes + * Tuning the number of processes on the system + * Tuning the number of open files or sockets + +The following tools can be used to measure mail system performance under +artificial loads. They are normally not installed with Postfix. + + * smtp-source, SMTP/LMTP message generator + * smtp-sink, SMTP/LMTP message dump + * qmqp-source, QMQP message generator + * qmqp-sink, QMQP message dump + +GGeenneerraall mmaaiill rreecceeiivviinngg ppeerrffoorrmmaannccee ttiippss + + * Read and understand the maildrop queue, incoming queue, and active queue + discussions in the QSHAPE_README document. + + * Run a local name server to reduce slow-down due to DNS lookups. If you run + multiple Postfix systems, point each local name server to a shared + forwarding server to reduce the number of lookups across the upstream + network link. + + * Eliminate unnecessary LDAP lookups, by specifying a domain filter. This + eliminates lookups for addresses in remote domains, and eliminates lookups + of partial addresses. See ldap_table(5) for details. + +When Postfix responds slowly to SMTP clients: + + * Look for obvious signs of trouble as described in the DEBUG_README + document, and eliminate those problems first. + + * Turn off your header_checks and body_checks patterns and see if the problem + goes away. + + * Turn off chroot operation as described in the DEBUG_README document and see + if the problem goes away. + + * If Postfix logs the SMTP client as "unknown" then you have a name service + problem: the name server is bad, or the resolv.conf file contains bad + information, or some packet filter is blocking the DNS requests or replies. + + * If the number of smtpd(8) processes has reached the process limit as + specified in master.cf, new SMTP clients must wait until a process becomes + available. See the STRESS_README and POSTSCREEN_README documents for + measures that help to prevent SMTP server overload. + +DDooiinngg mmoorree wwoorrkk wwiitthh yyoouurr SSMMTTPP sseerrvveerr pprroocceesssseess + +With Postfix versions 2.0 and earlier, the smtpd(8) server pauses before +reporting an error to an SMTP client. The idea is called tar pitting. However, +these delays also slow down Postfix. When the smtpd(8) server replies slowly, +sessions take more time, so that more smtpd(8) server processes are needed to +handle the load. When your Postfix smtpd(8) server process limit is reached, +new clients must wait until a server process becomes available. This means that +all clients experience poor performance. + +You can speed up the handling of smtpd(8) server error replies by turning off +the delay: + + /etc/postfix/main.cf: + # Not needed with Postfix 2.1 + smtpd_error_sleep_time = 0 + +With the above setting, Postfix 2.0 and earlier can serve more SMTP clients +with the same number SMTP server processes. The next section describes how +Postfix deals with clients that make a large number of errors. + +SSlloowwiinngg ddoowwnn SSMMTTPP cclliieennttss tthhaatt mmaakkee mmaannyy eerrrroorrss + +The Postfix smtpd(8) server maintains a per-session error count. The error +count is reset when a message is transferred successfully, and is incremented +when a client request is unrecognized or unimplemented, when a client request +violates access restrictions, or when some other error happens. + +As the per-session error count increases, the smtpd(8) server changes behavior +and begins to insert delays into the responses. The idea is to slow down a run- +away client in order to limit resource usage. The behavior is Postfix version +dependent. + +IMPORTANT: These delays slow down Postfix, too. When too much delay is +configured, the number of simultaneous SMTP sessions will increase until it +reaches the smtpd(8) server process limit, and new SMTP clients must wait until +an smtpd(8) server process becomes available. + +Postfix version 2.1 and later: + + * When the error count reaches $smtpd_soft_error_limit (default: 10), the + Postfix smtpd(8) server delays all non-error and error responses by + $smtpd_error_sleep_time seconds (default: 1 second). + + * When the error count reaches $smtpd_hard_error_limit (default: 20) the + Postfix smtpd(8) server breaks the connection. + +Postfix version 2.0 and earlier: + + * When the error count is less than $smtpd_soft_error_limit (default: 10) the + Postfix smtpd(8) server delays all error replies by $smtpd_error_sleep_time + (1 second with Postfix 2.0, 5 seconds with Postfix 1.1 and earlier). + + * When the error count reaches $smtpd_soft_error_limit, the Postfix smtpd(8) + server delays all responses by "error count" seconds or + $smtpd_error_sleep_time, whichever is more. + + * When the error count reaches $smtpd_hard_error_limit (default: 20) the + Postfix smtpd(8) server breaks the connection. + +MMeeaassuurreess aaggaaiinnsstt cclliieennttss tthhaatt mmaakkee ttoooo mmaannyy ccoonnnneeccttiioonnss + +Note: these features use the Postfix anvil(8) service, introduced with Postfix +version 2.2. + +The Postfix smtpd(8) server can limit the number of simultaneous connections +from the same SMTP client, as well as the connection rate and the rate of +certain SMTP commands from the same client. These statistics are maintained by +the anvil(8) server (translation: if anvil(8) breaks, then connection limits +stop working). + +IMPORTANT: These limits must not be used to regulate legitimate traffic: mail +will suffer grotesque delays if you do so. The limits are designed to protect +the smtpd(8) server against abuse by out-of-control clients. + + smtpd_client_connection_count_limit (default: 50) + The maximum number of connections that an SMTP client may make + simultaneously. + smtpd_client_connection_rate_limit (default: no limit) + The maximum number of connections that an SMTP client may make in the + time interval specified with anvil_rate_time_unit (default: 60s). + smtpd_client_message_rate_limit (default: no limit) + The maximum number of message delivery requests that an SMTP client may + make in the time interval specified with anvil_rate_time_unit (default: + 60s). + smtpd_client_recipient_rate_limit (default: no limit) + The maximum number of recipient addresses that an SMTP client may + specify in the time interval specified with anvil_rate_time_unit + (default: 60s). + smtpd_client_new_tls_session_rate_limit (default: no limit) + The maximum number of new TLS sessions (without using the TLS session + cache) that an SMTP client may negotiate in the time interval specified + with anvil_rate_time_unit (default: 60s). + smtpd_client_auth_rate_limit (default: no limit) + The maximum number of AUTH commands that an SMTP client may send in the + time interval specified with anvil_rate_time_unit (default: 60s). + Available in Postfix 3.1 and later. + smtpd_client_event_limit_exceptions (default: $mynetworks) + SMTP clients that are excluded from connection and rate limits + specified above. + +GGeenneerraall mmaaiill ddeelliivveerryy ppeerrffoorrmmaannccee ttiippss + + * Read and understand the maildrop queue, incoming queue, active queue and + deferred queue discussions in the QSHAPE_README document. + + * In case of slow delivery, run the qshape tool as described in the + QSHAPE_README document. + + * Submit multiple recipients per message instead of submitting messages with + only a few recipients. + + * Submit mail via SMTP instead of /usr/sbin/sendmail. You may have to adjust + the smtpd_recipient_limit parameter setting. + + * Don't overwhelm the disk with mail submissions. Optimize the mail + submission rate by tuning the number of parallel submissions and/or by + tuning the Postfix in_flow_delay parameter setting. + + * Run a local name server to reduce slow-down due to DNS lookups. If you run + multiple Postfix systems, point each local name server to a shared + forwarding server to reduce the number of lookups across the upstream + network link. + + * Reduce the smtp_connect_timeout and smtp_helo_timeout values so that + Postfix does not waste lots of time connecting to non-responding remote + SMTP servers. + + * Use a dedicated mail delivery transport for problematic destinations, with + reduced timeouts and with adjusted concurrency. See "Tuning the number of + simultaneous deliveries" below. + + * Use a fallback_relay host for mail that cannot be delivered upon the first + attempt. This "graveyard" machine can use shorter retry times for difficult + to reach destinations. See "Tuning the frequency of deferred mail delivery + attempts" below. + + * Speed up disk updates with a large (64MB) persistent write cache. This + allows disk updates to be sorted for optimal access speed without + compromising file system integrity when the system crashes. + + * Use a solid-state disk (a persistent RAM disk). This is an expensive + solution that should be used in combination with short SMTP timeouts and a + fallback_relay "graveyard" machine that delivers mail for problem + destinations. + +TTuunniinngg tthhee nnuummbbeerr ooff ssiimmuullttaanneeoouuss ddeelliivveerriieess + +Although Postfix can be configured to run 1000 SMTP client processes at the +same time, it is rarely desirable that it makes 1000 simultaneous connections +to the same remote system. For this reason, Postfix has safety mechanisms in +place to avoid this so-called "thundering herd" problem. + +The Postfix queue manager implements the analog of the TCP slow start flow +control strategy: when delivering to a site, send a small number of messages +first, then increase the concurrency as long as all goes well; reduce +concurrency in the face of congestion. + + * The initial_destination_concurrency parameter (default: 5) controls how + many messages are initially sent to the same destination before adapting + delivery concurrency. Of course, this setting is effective only as long as + it does not exceed the process limit and the destination concurrency limit + for the specific mail transport channel. + + * The default_destination_concurrency_limit parameter (default: 20) controls + how many messages may be sent to the same destination simultaneously. You + can override this setting for specific message delivery transports by + taking the name of the master.cf entry and appending + "_destination_concurrency_limit". + +Examples of transport specific concurrency limits are: + + * The local_destination_concurrency_limit parameter (default: 2) controls how + many messages are delivered simultaneously to the same local recipient. The + recommended limit is low because delivery to the same mailbox must happen + sequentially, so massive parallelism is not useful. Another good reason to + limit delivery concurrency to the same recipient: if the recipient has an + expensive shell command in her .forward file, or if the recipient is a + mailing list manager, you don't want to run too many instances of those + processes at the same time. + + * The default smtp_destination_concurrency_limit of 20 seems enough to + noticeably load a system without bringing it to its knees. Be careful when + changing this to a much larger number. + +The above default values of the concurrency limits work well in a broad range +of situations. Knee-jerk changes to these parameters in the face of congestion +can actually make problems worse. Specifically, large destination concurrencies +should never be the default. They should be used only for transports that +deliver mail to a small number of high volume domains. + +A common situation where high concurrency is called for is on gateways relaying +a high volume of mail between the Internet and an intranet mail environment. +Approximately half the mail (assuming equal volumes inbound and outbound) will +be destined for the internal mail hubs. Since the internal mail hubs will be +receiving all external mail exclusively from the gateway, it is reasonable to +configure the gateway to make greater demands on the capacity of the internal +SMTP servers. + +The tuning of the inbound concurrency limits need not be trial and error. A +high volume capable mailhub should be able to easily handle 50 or 100 (rather +than the default 20) simultaneous connections, especially if the gateway +forwards to multiple MX hosts. When all MX hosts are up and accepting +connections in a timely fashion, throughput will be high. If any MX host is +down and completely unresponsive, the average connection latency rises to at +least 1/N * $smtp_connect_timeout, if there are N MX hosts. This limits +throughput to at most the destination concurrency * N / $smtp_connect_timeout. + +For example, with a destination concurrency of 100 and 2 MX hosts, each host +will handle up to 50 simultaneous connections. If one MX host is down and the +default SMTP connection timeout is 30s, the throughput limit is 100 * 2 / 30 ~= +6 messages per second. This suggests that high volume destinations with good +connectivity and multiple MX hosts need a lower connection timeout, values as +low as 5s or even 1s can be used to prevent congestion when one or more, but +not all MX hosts are down. + +If necessary, set a higher transport_destination_concurrency_limit (in main.cf +since this is a queue manager parameter) and a lower smtp_connect_timeout (with +a "-o" override in master.cf since this parameter has no per-transport name) +for the relay transport and any transports dedicated for specific high volume +destinations. + +TTuunniinngg tthhee nnuummbbeerr ooff rreecciippiieennttss ppeerr ddeelliivveerryy + +The default_destination_recipient_limit parameter (default: 50) controls how +many recipients a Postfix delivery agent will send with each copy of an email +message. You can override this setting for specific Postfix delivery agents. +For example, "uucp_destination_recipient_limit = 100" would limit the number of +recipients per UUCP delivery to 100. + +If an email message exceeds the recipient limit for some destination, the +Postfix queue manager breaks up the list of recipients into smaller lists. +Postfix will attempt to send multiple copies of the message in parallel. + +IMPORTANT: Be careful when increasing the recipient limit per message delivery; +some SMTP servers abort the connection when they run out of memory or when a +hard recipient limit is reached, so that the message will never be delivered. + +The smtpd_recipient_limit parameter (default: 1000) controls how many +recipients the Postfix smtpd(8) server will take per delivery. The default +limit is more than any reasonable SMTP client would send. The limit exists to +protect the local mail system against a run-away client. + +TTuunniinngg tthhee ffrreeqquueennccyy ooff ddeeffeerrrreedd mmaaiill ddeelliivveerryy aatttteemmppttss + +When a Postfix delivery agent (smtp(8), local(8), etc.) is unable to deliver a +message it may blame the message itself, or it may blame the receiving party. + + * When the delivery agent blames the message, the queue manager gives the + queue file a time stamp into the future, so it won't be looked at for a + while. By default, the amount of time to cool down is the amount of time + that has passed since the message arrived. This results in so-called + exponential backoff behavior. + + * When the delivery agent blames the receiving party (for example a local + recipient user, or a remote host), the queue manager not only advances the + queue file time stamp, but also puts the receiving party on a "dead" list + so that it will be skipped for some amount of time. + +This process is governed by a bunch of little parameters. + + queue_run_delay (default: 300 seconds; before Postfix 2.4: 1000s) + How often the queue manager scans the queue for deferred mail. + minimal_backoff_time (default: 300 seconds; before Postfix 2.4: 1000s) + The minimal amount of time a message won't be looked at, and the + minimal amount of time to stay away from a "dead" destination. + maximal_backoff_time (default: 4000 seconds) + The maximal amount of time a message won't be looked at after a + delivery failure. + maximal_queue_lifetime (default: 5 days) + How long a message stays in the queue before it is sent back as + undeliverable. Specify 0 for mail that should be returned immediately + after the first unsuccessful delivery attempt. + bounce_queue_lifetime (default: 5 days, available with Postfix version 2.1 + and later) + How long a MAILER-DAEMON message stays in the queue before it is + considered undeliverable. Specify 0 for mail that should be tried only + once. + qmgr_message_recipient_limit (default: 20000) + The size of many in-memory queue manager data structures. Among others, + this parameter limits the size of the short-term, in-memory list of + "dead" destinations. Destinations that don't fit the list are not + added. + transport_destination_concurrency_failed_cohort_limit + Controls when a destination is considered "dead". This parameter is + critical with a non-zero transport_destination_rate_delay, with a + reduced transport_destination_concurrency_limit, or with a reduced + initial_destination_concurrency. + +IMPORTANT: If you increase the frequency of deferred mail delivery attempts, or +if you flush the deferred mail queue frequently, then you may find that Postfix +mail delivery performance actually becomes worse. The symptoms are as follows: + + * The active queue becomes saturated with mail that has delivery problems. + New mail enters the active queue only when an old message is deferred. This + is a slow process that usually requires timing out one or more SMTP + connections. + + * All available Postfix delivery agents become occupied trying to connect to + unreachable sites etc. New mail has to wait until a delivery agent becomes + available. This is a slow process that usually requires timing out one or + more SMTP connections. + +When mail is being deferred frequently, fixing the problem is always better +than increasing the frequency of delivery attempts. However, if you can control +only the delivery attempt frequency, consider using a dedicated fallback_relay +"graveyard" machine for bad destinations, so that these destinations do not +ruin the performance of normal mail deliveries. + +TTuunniinngg tthhee nnuummbbeerr ooff PPoossttffiixx pprroocceesssseess + +The default_process_limit configuration parameter gives direct control over how +many daemon processes Postfix will run. As of Postfix 2.0 the default limit is +100 SMTP client processes, 100 SMTP server processes, and so on. This may +overwhelm systems with little memory, as well as networks with low bandwidth. + +You can change the global process limit by specifying a non-default +default_process_limit in the main.cf file. For example, to run up to 10 SMTP +client processes, 10 SMTP server processes, and so on: + + /etc/postfix/main.cf: + default_process_limit = 10 + +You need to execute "postfix reload" to make the change effective. This limit +is enforced by the Postfix master(8) daemon which does not automatically read +main.cf when it changes. + +You can override the process limit for specific Postfix daemons by editing the +master.cf file. For example, if you do not wish to receive 100 SMTP messages at +the same time, but do not want to change the process limits for other Postfix +daemons, you could specify: + + /etc/postfix/master.cf: + # ==================================================================== + # service type private unpriv chroot wakeup maxproc command + args + # (yes) (yes) (yes) (never) (100) + # ==================================================================== + . . . + smtp inet n - - - 10 smtpd + . . . + +TTuunniinngg tthhee nnuummbbeerr ooff pprroocceesssseess oonn tthhee ssyysstteemm + + * MacOS X will run out of process slots when you increase Postfix process + limits. The following works with OSX 10.4 and OSX 10.5. + + MacOS X kernel parameters can be specified in /etc/sysctl.conf. + + /etc/sysctl.conf: + kern.maxproc=2048 + kern.maxprocperuid=2048 + + Unfortunately these can't simply be set on the fly with "sysctl -w". You + also have to set the following in /etc/launchd.conf so that the root user + after boot will have the right process limit (2048). Otherwise you have to + always run ulimit -u 2048 as root, then start a user shell, and then start + processes for things to take effect. + + /etc/launchd.conf: + limit maxproc 2048 + + Once these are in place, reboot the system. After that, the limits will + stay in place. + +TTuunniinngg tthhee nnuummbbeerr ooff ooppeenn ffiilleess oorr ssoocckkeettss + +When Postfix opens too many files or sockets, processes will abort with fatal +errors, and the system may log "file table full" errors. + + * Depending on your Postfix and operating system versions you may need to + recompile Postfix if you need more than 1024 file descriptors per process: + + o No recompilation is needed for Postfix version 2.4 and later, when it + was compiled for systems that support BSD kqueue(2) (FreeBSD 4.1, + NetBSD 2.0, OpenBSD 2.9), Solaris 8 /dev/poll, or Linux 2.6 epoll(4). + + o Otherwise, Postfix needs to be recompiled to override the default + FD_SETSIZE value. + + * Reduce the number of processes as described under "Tuning the number of + Postfix processes" above. Fewer processes need fewer open files and + sockets. + + * Configure the kernel for more open files and sockets. The details are + extremely system dependent and change with the operating system version. Be + sure to verify the following information with your system tuning guide: + + o Some FreeBSD kernel parameters can be specified in /boot/loader.conf, + and some can be specified in /etc/sysctl.conf or changed with sysctl + commands. Which is which depends on the version. + + kern.ipc.maxsockets="5000" + kern.ipc.nmbclusters="65536" + kern.maxproc="2048" + kern.maxfiles="16384" + kern.maxfilesperproc="16384" + + o Linux kernel parameters can be specified in /etc/sysctl.conf or changed + with sysctl commands: + + fs.file-max=16384 + kernel.threads-max=2048 + + o Solaris kernel parameters can be specified in /etc/system, as described + in the Solaris FAQ entry titled "How can I increase the number of file + descriptors per process?" + + * set hard limit on file descriptors + set rlim_fd_max = 4096 + * set soft limit on file descriptors + set rlim_fd_cur = 1024 + diff --git a/README_FILES/ULTRIX_README b/README_FILES/ULTRIX_README new file mode 100644 index 0000000..98078d1 --- /dev/null +++ b/README_FILES/ULTRIX_README @@ -0,0 +1,45 @@ +PPoossttffiixx aanndd UUllttrriixx + +------------------------------------------------------------------------------- + +PPoossttffiixx oonn UUllttrriixx + +This document is probably only of historical value, because Ultrix version 4 +dates from the early 1990s. However, as long as Wietse keeps Postfix alive for +SunOS 4, it is likely to run on Ultrix 4 with very little change. Feedback is +welcome if anyone actually still uses Postfix on any version of Ultrix. + +The source of this document is an email message by Christian von Roques that +was sent on Jun 2, 1999. + + I've upgraded the MTA of our DECstation-3100 running Ultrix4.3a to postfix- + 19990317-pl05 and am sending you the patches I needed to get it running + under Ultrix. + + . . . + + One of the bugs of Ultrix's /bin/sh is that shell-variables set in + arguments of `:' expand to garbage if expanded in here-documents. Using a + different shell helps. I needed to replace all calls of ``sh .../makedefs'' + by ``$(SHELL) .../makedefs'' in all the Makefile.in and am now able to use + ``make SHELL=/bin/sh5'' or zsh. + + . . . + + Ultrix's FD_SET_SIZE is 4096, but getdtablesize() returns 64 by default, if + not increased when building a new kernel. getrlimit() doesn't know + RLIMIT_NOFILE. This makes event_init() always log the warning: `could + allocate space for only 64 open files'. + + I just reduced the threshold from 256 to 64, but this is not good. The + initial problem still remains: How to disable this warning on Ultrix + without making the source ugly? + +To work around the first problem, all the Makefile.in files have been updated +to use `$(SHELL)' instead of `sh'. So you only need to supply a non-default +shell in order to eliminate Ultrix shell trouble. + +To work around the latter, util/sys_defs.h was updated for Ultrix, with a +default FD_SETSIZE of 100. This should be sufficient for a workstation. Even in +1999, no-one would run a major mail hub on Ultrix 4. + diff --git a/README_FILES/UUCP_README b/README_FILES/UUCP_README new file mode 100644 index 0000000..1368eb3 --- /dev/null +++ b/README_FILES/UUCP_README @@ -0,0 +1,121 @@ +PPoossttffiixx aanndd UUUUCCPP + +------------------------------------------------------------------------------- + +UUssiinngg UUUUCCPP oovveerr TTCCPP + +Despite a serious lack of sex-appeal, email via UUCP over TCP is a practical +option for sites without permanent Internet connections, and for sites without +a fixed IP address. For first-hand information, see the following guides: + + * Jim Seymour's guide for using UUCP over TCP at http://jimsun.LinxNet.com/ + jdp/uucp_over_tcp/index.html, + * Craig Sanders's guide for SSL-encrypted UUCP over TCP using stunnel at + http://taz.net.au/postfix/uucp/. + +Here's a graphical description of what this document is about: + + LAN to Internet + Local network <---> UUCP <--- UUCP ---> to UUCP <---> Internet + Gateway Gateway + +And here's the table of contents of this document: + + * Setting up a Postfix Internet to UUCP gateway + * Setting up a Postfix LAN to UUCP gateway + +SSeettttiinngg uupp aa PPoossttffiixx IInntteerrnneett ttoo UUUUCCPP ggaatteewwaayy + +Here is how to set up a machine that sits on the Internet and that forwards +mail to a LAN that is connected via UUCP. See the LAN to UUCP gateway section +for the other side of the story. + + * You need an rrmmaaiill program that extracts the sender address from mail that + arrives via UUCP, and that feeds the mail into the Postfix sseennddmmaaiill + command. Most UNIX systems come with an rrmmaaiill utility. If you're in a + pinch, try the one bundled with the Postfix source code in the aauuxxiilliiaarryy// + rrmmaaiill directory. + + * Define a pipe(8) based mail delivery transport for delivery via UUCP: + + /etc/postfix/master.cf: + uucp unix - n n - - pipe + flags=F user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail + ($recipient) + + This runs the uuuuxx command to place outgoing mail into the UUCP queue after + replacing $nexthop by the next-hop hostname (the receiving UUCP host) and + after replacing $recipient by the recipients. The pipe(8) delivery agent + executes the uuuuxx command without assistance from the shell, so there are no + problems with shell meta characters in command-line parameters. + + * Specify that mail for example.com, should be delivered via UUCP, to a host + named uucp-host: + + /etc/postfix/transport: + example.com uucp:uucp-host + .example.com uucp:uucp-host + + See the transport(5) manual page for more details. + + * Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//ttrraannssppoorrtt" whenever you change + the ttrraannssppoorrtt file. + + * Enable ttrraannssppoorrtt table lookups: + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + + Specify ddbbmm instead of hhaasshh if your system uses ddbbmm files instead of ddbb + files. To find out what map types Postfix supports, use the command + "ppoossttccoonnff --mm". + + * Add example.com to the list of domains that your site is willing to relay + mail for. + + /etc/postfix/main.cf: + relay_domains = example.com ...other relay domains... + + See the relay_domains configuration parameter description for details. + + * Execute the command "ppoossttffiixx rreellooaadd" to make the changes effective. + +SSeettttiinngg uupp aa PPoossttffiixx LLAANN ttoo UUUUCCPP ggaatteewwaayy + +Here is how to relay mail from a LAN via UUCP to the Internet. See the Internet +to UUCP gateway section for the other side of the story. + + * You need an rrmmaaiill program that extracts the sender address from mail that + arrives via UUCP, and that feeds the mail into the Postfix sseennddmmaaiill + command. Most UNIX systems come with an rrmmaaiill utility. If you're in a + pinch, try the one bundled with the Postfix source code in the aauuxxiilliiaarryy// + rrmmaaiill directory. + + * Specify that all remote mail must be sent via the uuuuccpp mail transport to + your UUCP gateway host, say, uucp-gateway: + + /etc/postfix/main.cf: + relayhost = uucp-gateway + default_transport = uucp + + Postfix 2.0 and later also allows the following more succinct form: + + /etc/postfix/main.cf: + default_transport = uucp:uucp-gateway + + * Define a pipe(8) based message delivery transport for mail delivery via + UUCP: + + /etc/postfix/master.cf: + uucp unix - n n - - pipe + flags=F user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail + ($recipient) + + This runs the uuuuxx command to place outgoing mail into the UUCP queue. It + substitutes the next-hop hostname (uucp-gateway, or whatever you specified) + and the recipients before executing the command. The uuuuxx command is + executed without assistance from the shell, so there are no problems with + shell meta characters. + + * Execute the command "ppoossttffiixx rreellooaadd" to make the changes effective. + diff --git a/README_FILES/VERP_README b/README_FILES/VERP_README new file mode 100644 index 0000000..a721ece --- /dev/null +++ b/README_FILES/VERP_README @@ -0,0 +1,186 @@ +PPoossttffiixx VVEERRPP HHoowwttoo + +------------------------------------------------------------------------------- + +PPoossttffiixx VVEERRPP ssuuppppoorrtt + +Postfix versions 1.1 and later support variable envelope return path addresses +on request. When VERP style delivery is requested, each recipient of a message +receives a customized copy of the message, with his/her own recipient address +encoded in the envelope sender address. + +For example, when VERP style delivery is requested, Postfix delivers mail from +"owner-listname@origin" for a recipient "user@domain", with a sender address +that encodes the recipient as follows: + + owner-listname+user=domain@origin + +Thus, undeliverable mail can reveal the undeliverable recipient address without +requiring the list owner to parse bounce messages. + +The VERP concept was popularized by the qmail MTA and by the ezmlm mailing list +manager. See http://cr.yp.to/proto/verp.txt for the ideas behind this concept. + +Topics covered in this document: + + * Postfix VERP configuration parameters + * Using VERP with majordomo etc. mailing lists + * VERP support in the Postfix SMTP server + * VERP support in the Postfix sendmail command + * VERP support in the Postfix QMQP server + +PPoossttffiixx VVEERRPP ccoonnffiigguurraattiioonn ppaarraammeetteerrss + +With Postfix, the whole process is controlled by four configuration parameters. + +default_verp_delimiters (default value: +=) + What VERP delimiter characters Postfix uses when VERP style delivery is + requested but no explicit delimiters are specified. + +verp_delimiter_filter (default: -+=) + What characters Postfix accepts as VERP delimiter characters on the + sendmail command line and in SMTP commands. Many characters must not be + used as VERP delimiter characters, either because they already have a + special meaning in email addresses (such as the @ or the %), because they + are used as part of a username or domain name (such as alphanumerics), or + because they are non-ASCII or control characters. And who knows, some + characters may tickle bugs in vulnerable software, and we would not want + that to happen. + +smtpd_authorized_verp_clients (default value: none) + What SMTP clients are allowed to request VERP style delivery. The Postfix + QMQP server uses its own access control mechanism, and local submission + (via /usr/sbin/sendmail etc.) is always authorized. To authorize a host, + list its name, IP address, subnet (net/mask) or parent .domain. + + With Postfix versions 1.1 and 2.0, this parameter is called + authorized_verp_clients (default: $mynetworks). + +disable_verp_bounces (default: no) + Send one bounce report for multi-recipient VERP mail, instead of one bounce + report per recipient. The default, one per recipient, is what ezmlm needs. + +UUssiinngg VVEERRPP wwiitthh mmaajjoorrddoommoo eettcc.. mmaaiilliinngg lliissttss + +In order to make VERP useful with majordomo etc. mailing lists, you would +configure the list manager to submit mail according to one of the following two +forms: + +Postfix 2.3 and later: + + % sendmail -XV -f owner-listname other-arguments... + + % sendmail -XV+= -f owner-listname other-arguments... + +Postfix 2.2 and earlier (Postfix 2.3 understands the old syntax for backwards +compatibility, but will log a warning that reminds you of the new syntax): + + % sendmail -V -f owner-listname other-arguments... + + % sendmail -V+= -f owner-listname other-arguments... + +The first form uses the default main.cf VERP delimiter characters. The second +form allows you to explicitly specify the VERP delimiter characters. The +example shows the recommended values. + +This text assumes that you have set up an owner-listname alias that routes +undeliverable mail to a real person: + + /etc/aliases: + owner-listname: yourname+listname + +In order to process bounces we are going to make extensive use of address +extension tricks. + +You need to tell Postfix that + is the separator between an address and its +optional address extension, that address extensions are appended to .forward +file names, and that address extensions are to be discarded when doing alias +expansions: + + /etc/postfix/main.cf: + recipient_delimiter = + + forward_path = $home/.forward${recipient_delimiter}${extension}, + $home/.forward + propagate_unmatched_extensions = canonical, virtual + +(the last two parameter settings are default settings). + +You need to set up a file named .forward+listname with the commands that +process all the mail that is sent to the owner-listname address: + + ~/.forward+listname: + "|/some/where/command ..." + +With this set up, undeliverable mail for user@domain will be returned to the +following address: + + owner-listname+user=domain@your.domain + +which is processed by the command in your .forward+listname file. The message +should contain, among others, a To: header with the encapsulated recipient +sender address: + + To: owner-listname+user=domain@your.domain + +It is left as an exercise for the reader to parse the To: header line and to +pull out the user=domain part from the recipient address. + +VVEERRPP ssuuppppoorrtt iinn tthhee PPoossttffiixx SSMMTTPP sseerrvveerr + +The Postfix SMTP server implements a command XVERP to enable VERP style +delivery. The syntax allows two forms: + + MAIL FROM:<sender@domain> XVERP + + MAIL FROM:<sender@domain> XVERP=+= + +The first form uses the default main.cf VERP delimiters, the second form +overrides them explicitly. The values shown are the recommended ones. + +You can use the smtpd_command_filter feature to append XVERP to SMTP commands +from legacy software. This requires Postfix 2.7 or later. + + /etc/postfix/main.cf: + smtpd_command_filter = pcre:/etc/postfix/append_verp.pcre + smtpd_authorized_verp_clients = $mynetworks + + /etc/postfix/append_verp.pcre: + /^(MAIL FROM:<listname@example\.com>.*)/ $1 XVERP + +VVEERRPP ssuuppppoorrtt iinn tthhee PPoossttffiixx sseennddmmaaiill ccoommmmaanndd + +The Postfix sendmail command has a -V flag to request VERP style delivery. +Specify one of the following two forms: + +Postfix 2.3 and later: + + % sendmail -XV -f owner-listname .... + + % sendmail -XV+= -f owner-listname .... + +Postfix 2.2 and earlier (Postfix 2.3 understands the old syntax for backwards +compatibility, but will log a warning that reminds you of the new syntax): + + % sendmail -V -f owner-listname .... + + % sendmail -V+= -f owner-listname .... + +The first form uses the default main.cf VERP delimiters, the second form +overrides them explicitly. The values shown are the recommended ones. + +VVEERRPP ssuuppppoorrtt iinn tthhee PPoossttffiixx QQMMQQPP sseerrvveerr + +When the Postfix QMQP server receives mail with an envelope sender address of +the form: + + listname-@your.domain-@[] + +Postfix generates sender addresses "listname-user=domain@your.domain", using "- +=" as the VERP delimiters because qmail/ezmlm expect this. + +More generally, a sender address of "prefix@origin-@[]" requests VERP style +delivery with sender addresses of the form "prefixuser=domain@origin". However, +Postfix allows only VERP delimiters that are specified with the +verp_delimiter_filter parameter. In particular, the "=" delimiter is required +for qmail compatibility (see the qmail addresses(5) manual page for details). + diff --git a/README_FILES/VIRTUAL_README b/README_FILES/VIRTUAL_README new file mode 100644 index 0000000..e693a0c --- /dev/null +++ b/README_FILES/VIRTUAL_README @@ -0,0 +1,483 @@ +PPoossttffiixx VViirrttuuaall DDoommaaiinn HHoossttiinngg HHoowwttoo + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhiiss ddooccuummeenntt + +This document requires Postfix version 2.0 or later. + +This document gives an overview of how Postfix can be used for hosting multiple +Internet domains, both for final delivery on the machine itself and for the +purpose of forwarding to destinations elsewhere. + +The text not only describes delivery mechanisms that are built into Postfix, +but also gives pointers for using non-Postfix mail delivery software. + +The following topics are covered: + + * Canonical versus hosted versus other domains + * Local files versus network databases + * As simple as can be: shared domains, UNIX system accounts + * Postfix virtual ALIAS example: separate domains, UNIX system accounts + * Postfix virtual MAILBOX example: separate domains, non-UNIX accounts + * Non-Postfix mailbox store: separate domains, non-UNIX accounts + * Mail forwarding domains + * Mailing lists + * Autoreplies + +CCaannoonniiccaall vveerrssuuss hhoosstteedd vveerrssuuss ootthheerr ddoommaaiinnss + +Most Postfix systems are the ffiinnaall ddeessttiinnaattiioonn for only a few domain names. +These include the hostnames and [the IP addresses] of the machine that Postfix +runs on, and sometimes also include the parent domain of the hostname. The +remainder of this document will refer to these domains as the canonical +domains. They are usually implemented with the Postfix local domain address +class, as defined in the ADDRESS_CLASS_README file. + +Besides the canonical domains, Postfix can be configured to be the ffiinnaall +ddeessttiinnaattiioonn for any number of additional domains. These domains are called +hosted, because they are not directly associated with the name of the machine +itself. Hosted domains are usually implemented with the virtual alias domain +address class and/or with the virtual mailbox domain address class, as defined +in the ADDRESS_CLASS_README file. + +But wait! There is more. Postfix can be configured as a backup MX host for +other domains. In this case Postfix is nnoott tthhee ffiinnaall ddeessttiinnaattiioonn for those +domains. It merely queues the mail when the primary MX host is down, and +forwards the mail when the primary MX host becomes available. This function is +implemented with the relay domain address class, as defined in the +ADDRESS_CLASS_README file. + +Finally, Postfix can be configured as a transit host for sending mail across +the internet. Obviously, Postfix is not the final destination for such mail. +This function is available only for authorized clients and/or users, and is +implemented by the default domain address class, as defined in the +ADDRESS_CLASS_README file. + +LLooccaall ffiilleess vveerrssuuss nneettwwoorrkk ddaattaabbaasseess + +The examples in this text use table lookups from local files such as DBM or +Berkeley DB. These are easy to debug with the ppoossttmmaapp command: + + Example: postmap -q info@example.com hash:/etc/postfix/virtual + +See the documentation in LDAP_README, MYSQL_README and PGSQL_README for how to +replace local files by databases. The reader is strongly advised to make the +system work with local files before migrating to network databases, and to use +the ppoossttmmaapp command to verify that network database lookups produce the exact +same results as local file lookup. + + Example: postmap -q info@example.com ldap:/etc/postfix/virtual.cf + +AAss ssiimmppllee aass ccaann bbee:: sshhaarreedd ddoommaaiinnss,, UUNNIIXX ssyysstteemm aaccccoouunnttss + +The simplest method to host an additional domain is to add the domain name to +the domains listed in the Postfix mydestination configuration parameter, and to +add the user names to the UNIX password file. + +This approach makes no distinction between canonical and hosted domains. Each +username can receive mail in every domain. + +In the examples we will use "example.com" as the domain that is being hosted on +the local Postfix machine. + + /etc/postfix/main.cf: + mydestination = $myhostname localhost.$mydomain ... example.com + +The limitations of this approach are: + + * A total lack of separation: mail for info@my.host.name is delivered to the + same UNIX system account as mail for info@example.com. + * With users in the UNIX password file, administration of large numbers of + users becomes inconvenient. + +The examples that follow provide solutions for both limitations. + +PPoossttffiixx vviirrttuuaall AALLIIAASS eexxaammppllee:: sseeppaarraattee ddoommaaiinnss,, UUNNIIXX ssyysstteemm aaccccoouunnttss + +With the approach described in this section, every hosted domain can have its +own info etc. email address. However, it still uses UNIX system accounts for +local mailbox deliveries. + +With virtual alias domains, each hosted address is aliased to a local UNIX +system account or to a remote address. The example below shows how to use this +mechanism for the example.com domain. + + 1 /etc/postfix/main.cf: + 2 virtual_alias_domains = example.com ...other hosted domains... + 3 virtual_alias_maps = hash:/etc/postfix/virtual + 4 + 5 /etc/postfix/virtual: + 6 postmaster@example.com postmaster + 7 info@example.com joe + 8 sales@example.com jane + 9 # Uncomment entry below to implement a catch-all address + 10 # @example.com jim + 11 ...virtual aliases for more domains... + +Notes: + + * Line 2: the virtual_alias_domains setting tells Postfix that example.com is + a so-called virtual alias domain. If you omit this setting then Postfix + will reject mail (relay access denied) or will not be able to deliver it + (mail for example.com loops back to myself). + + NEVER list a virtual alias domain name as a mydestination domain! + + * Lines 3-8: the /etc/postfix/virtual file contains the virtual aliases. With + the example above, mail for postmaster@example.com goes to the local + postmaster, while mail for info@example.com goes to the UNIX account joe, + and mail for sales@example.com goes to the UNIX account jane. Mail for all + other addresses in example.com is rejected with the error message "User + unknown". + + * Line 10: the commented out entry (text after #) shows how one would + implement a catch-all virtual alias that receives mail for every + example.com address not listed in the virtual alias file. This is not + without risk. Spammers nowadays try to send mail from (or mail to) every + possible name that they can think of. A catch-all mailbox is likely to + receive many spam messages, and many bounces for spam messages that were + sent in the name of anything@example.com. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" after changing the virtual +file, and execute the command "ppoossttffiixx rreellooaadd" after changing the main.cf file. + +Note: virtual aliases can resolve to a local address or to a remote address, or +both. They don't have to resolve to UNIX system accounts on your machine. + +More details about the virtual alias file are given in the virtual(5) manual +page, including multiple addresses on the right-hand side. + +Virtual aliasing solves one problem: it allows each domain to have its own info +mail address. But there still is one drawback: each virtual address is aliased +to a UNIX system account. As you add more virtual addresses you also add more +UNIX system accounts. The next section eliminates this problem. + +PPoossttffiixx vviirrttuuaall MMAAIILLBBOOXX eexxaammppllee:: sseeppaarraattee ddoommaaiinnss,, nnoonn--UUNNIIXX aaccccoouunnttss + +As a system hosts more and more domains and users, it becomes less desirable to +give every user their own UNIX system account. + +With the Postfix virtual(8) mailbox delivery agent, every recipient address can +have its own virtual mailbox. Unlike virtual alias domains, virtual mailbox +domains do not need the clumsy translation from each recipient addresses into a +different address, and owners of a virtual mailbox address do not need to have +a UNIX system account. + +The Postfix virtual(8) mailbox delivery agent looks up the user mailbox +pathname, uid and gid via separate tables that are searched with the +recipient's mail address. Maildir style delivery is turned on by terminating +the mailbox pathname with "/". + +If you find the idea of multiple tables bothersome, remember that you can +migrate the information (once it works), to an SQL database. If you take that +route, be sure to review the "local files versus databases" section at the top +of this document. + +Here is an example of a virtual mailbox domain "example.com": + + 1 /etc/postfix/main.cf: + 2 virtual_mailbox_domains = example.com ...more domains... + 3 virtual_mailbox_base = /var/mail/vhosts + 4 virtual_mailbox_maps = hash:/etc/postfix/vmailbox + 5 virtual_minimum_uid = 100 + 6 virtual_uid_maps = static:5000 + 7 virtual_gid_maps = static:5000 + 8 virtual_alias_maps = hash:/etc/postfix/virtual + 9 + 10 /etc/postfix/vmailbox: + 11 info@example.com example.com/info + 12 sales@example.com example.com/sales/ + 13 # Comment out the entry below to implement a catch-all. + 14 # @example.com example.com/catchall + 15 ...virtual mailboxes for more domains... + 16 + 17 /etc/postfix/virtual: + 18 postmaster@example.com postmaster + +Notes: + + * Line 2: The virtual_mailbox_domains setting tells Postfix that example.com + is a so-called virtual mailbox domain. If you omit this setting then + Postfix will reject mail (relay access denied) or will not be able to + deliver it (mail for example.com loops back to myself). + + NEVER list a virtual MAILBOX domain name as a mydestination domain! + + NEVER list a virtual MAILBOX domain name as a virtual ALIAS domain! + + * Line 3: The virtual_mailbox_base parameter specifies a prefix for all + virtual mailbox pathnames. This is a safety mechanism in case someone makes + a mistake. It prevents mail from being delivered all over the file system. + + * Lines 4, 10-15: The virtual_mailbox_maps parameter specifies the lookup + table with mailbox (or maildir) pathnames, indexed by the virtual mail + address. In this example, mail for info@example.com goes to the mailbox at + /var/mail/vhosts/example.com/info while mail for sales@example.com goes to + the maildir located at /var/mail/vhosts/example.com/sales/. + + * Line 5: The virtual_minimum_uid specifies a lower bound on the mailbox or + maildir owner's UID. This is a safety mechanism in case someone makes a + mistake. It prevents mail from being written to sensitive files. + + * Lines 6, 7: The virtual_uid_maps and virtual_gid_maps parameters specify + that all the virtual mailboxes are owned by a fixed uid and gid 5000. If + this is not what you want, specify lookup tables that are searched by the + recipient's mail address. + + * Line 14: The commented out entry (text after #) shows how one would + implement a catch-all virtual mailbox address. Be prepared to receive a lot + of spam, as well as bounced spam that was sent in the name of + anything@example.com. + + NEVER put a virtual MAILBOX wild-card in the virtual ALIAS file!! + + * Lines 8, 17, 18: As you see, it is possible to mix virtual aliases with + virtual mailboxes. We use this feature to redirect mail for example.com's + postmaster address to the local postmaster. You can use the same mechanism + to redirect an address to a remote address. + + * Line 18: This example assumes that in main.cf, $myorigin is listed under + the mydestination parameter setting. If that is not the case, specify an + explicit domain name on the right-hand side of the virtual alias table + entries or else mail will go to the wrong domain. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" after changing the virtual +file, execute "ppoossttmmaapp //eettcc//ppoossttffiixx//vvmmaaiillbbooxx" after changing the vmailbox file, +and execute the command "ppoossttffiixx rreellooaadd" after changing the main.cf file. + +Note: mail delivery happens with the recipient's UID/GID privileges specified +with virtual_uid_maps and virtual_gid_maps. Postfix 2.0 and earlier will not +create mailDIRs in world-writable parent directories; you must create them in +advance before you can use them. Postfix may be able to create mailBOX files by +itself, depending on parent directory write permissions, but it is safer to +create mailBOX files ahead of time. + +More details about the virtual mailbox delivery agent are given in the virtual +(8) manual page. + +NNoonn--PPoossttffiixx mmaaiillbbooxx ssttoorree:: sseeppaarraattee ddoommaaiinnss,, nnoonn--UUNNIIXX aaccccoouunnttss + +This is a variation on the Postfix virtual mailbox example. Again, every hosted +address can have its own mailbox. However, most parameters that control the +virtual(8) delivery agent are no longer applicable: only +virtual_mailbox_domains and virtual_mailbox_maps stay in effect. These +parameters are needed to reject mail for unknown recipients. + +While non-Postfix software is being used for final delivery, some Postfix +concepts are still needed in order to glue everything together. For additional +background on this glue you may want to take a look at the virtual mailbox +domain class as defined in the ADDRESS_CLASS_README file. + +The text in this section describes what things should look like from Postfix's +point of view. See CYRUS_README or MAILDROP_README for specific information +about Cyrus or about Courier maildrop. + +Here is an example for a hosted domain example.com that delivers to a non- +Postfix delivery agent: + + 1 /etc/postfix/main.cf: + 2 virtual_transport = ...see below... + 3 virtual_mailbox_domains = example.com ...more domains... + 4 virtual_mailbox_maps = hash:/etc/postfix/vmailbox + 5 virtual_alias_maps = hash:/etc/postfix/virtual + 6 + 7 /etc/postfix/vmailbox: + 8 info@example.com whatever + 9 sales@example.com whatever + 10 # Comment out the entry below to implement a catch-all. + 11 # Configure the mailbox store to accept all addresses. + 12 # @example.com whatever + 13 ...virtual mailboxes for more domains... + 14 + 15 /etc/postfix/virtual: + 16 postmaster@example.com postmaster + +Notes: + + * Line 2: With delivery to a non-Postfix mailbox store for hosted domains, + the virtual_transport parameter usually specifies the Postfix LMTP client, + or the name of a master.cf entry that executes non-Postfix software via the + pipe delivery agent. Typical examples (use only one): + + virtual_transport = lmtp:unix:/path/name (uses UNIX-domain socket) + virtual_transport = lmtp:hostname:port (uses TCP socket) + virtual_transport = maildrop: (uses pipe(8) to command) + + Postfix comes ready with support for LMTP. And an example maildrop delivery + method is already defined in the default Postfix master.cf file. See the + MAILDROP_README document for more details. + + * Line 3: The virtual_mailbox_domains setting tells Postfix that example.com + is delivered via the virtual_transport that was discussed in the previous + paragraph. If you omit this virtual_mailbox_domains setting then Postfix + will either reject mail (relay access denied) or will not be able to + deliver it (mail for example.com loops back to myself). + + NEVER list a virtual MAILBOX domain name as a mydestination domain! + + NEVER list a virtual MAILBOX domain name as a virtual ALIAS domain! + + * Lines 4, 7-13: The virtual_mailbox_maps parameter specifies the lookup + table with all valid recipient addresses. The lookup result value is + ignored by Postfix. In the above example, info@example.com and + sales@example.com are listed as valid addresses; other mail for example.com + is rejected with "User unknown" by the Postfix SMTP server. It's left up to + the non-Postfix delivery agent to reject non-existent recipients from local + submission or from local alias expansion. If you intend to use LDAP, MySQL + or PgSQL instead of local files, be sure to review the "local files versus + databases" section at the top of this document! + + * Line 12: The commented out entry (text after #) shows how one would inform + Postfix of the existence of a catch-all address. Again, the lookup result + is ignored by Postfix. + + NEVER put a virtual MAILBOX wild-card in the virtual ALIAS file!! + + Note: if you specify a wildcard in virtual_mailbox_maps, then you still + need to configure the non-Postfix mailbox store to receive mail for any + address in that domain. + + * Lines 5, 15, 16: As you see above, it is possible to mix virtual aliases + with virtual mailboxes. We use this feature to redirect mail for + example.com's postmaster address to the local postmaster. You can use the + same mechanism to redirect any addresses to a local or remote address. + + * Line 16: This example assumes that in main.cf, $myorigin is listed under + the mydestination parameter setting. If that is not the case, specify an + explicit domain name on the right-hand side of the virtual alias table + entries or else mail will go to the wrong domain. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" after changing the virtual +file, execute "ppoossttmmaapp //eettcc//ppoossttffiixx//vvmmaaiillbbooxx" after changing the vmailbox file, +and execute the command "ppoossttffiixx rreellooaadd" after changing the main.cf file. + +MMaaiill ffoorrwwaarrddiinngg ddoommaaiinnss + +Some providers host domains that have no (or only a few) local mailboxes. The +main purpose of these domains is to forward mail elsewhere. The following +example shows how to set up example.com as a mail forwarding domain: + + 1 /etc/postfix/main.cf: + 2 virtual_alias_domains = example.com ...other hosted domains... + 3 virtual_alias_maps = hash:/etc/postfix/virtual + 4 + 5 /etc/postfix/virtual: + 6 postmaster@example.com postmaster + 7 joe@example.com joe@somewhere + 8 jane@example.com jane@somewhere-else + 9 # Uncomment entry below to implement a catch-all address + 10 # @example.com jim@yet-another-site + 11 ...virtual aliases for more domains... + +Notes: + + * Line 2: The virtual_alias_domains setting tells Postfix that example.com is + a so-called virtual alias domain. If you omit this setting then Postfix + will reject mail (relay access denied) or will not be able to deliver it + (mail for example.com loops back to myself). + + NEVER list a virtual alias domain name as a mydestination domain! + + * Lines 3-11: The /etc/postfix/virtual file contains the virtual aliases. + With the example above, mail for postmaster@example.com goes to the local + postmaster, while mail for joe@example.com goes to the remote address + joe@somewhere, and mail for jane@example.com goes to the remote address + jane@somewhere-else. Mail for all other addresses in example.com is + rejected with the error message "User unknown". + + * Line 10: The commented out entry (text after #) shows how one would + implement a catch-all virtual alias that receives mail for every + example.com address not listed in the virtual alias file. This is not + without risk. Spammers nowadays try to send mail from (or mail to) every + possible name that they can think of. A catch-all mailbox is likely to + receive many spam messages, and many bounces for spam messages that were + sent in the name of anything@example.com. + +Execute the command "ppoossttmmaapp //eettcc//ppoossttffiixx//vviirrttuuaall" after changing the virtual +file, and execute the command "ppoossttffiixx rreellooaadd" after changing the main.cf file. + +More details about the virtual alias file are given in the virtual(5) manual +page, including multiple addresses on the right-hand side. + +MMaaiilliinngg lliissttss + +The examples that were given above already show how to direct mail for virtual +postmaster addresses to a local postmaster. You can use the same method to +direct mail for any address to a local or remote address. + +There is one major limitation: virtual aliases and virtual mailboxes can't +directly deliver to mailing list managers such as majordomo. The solution is to +set up virtual aliases that direct virtual addresses to the local delivery +agent: + + /etc/postfix/main.cf: + virtual_alias_maps = hash:/etc/postfix/virtual + + /etc/postfix/virtual: + listname-request@example.com listname-request + listname@example.com listname + owner-listname@example.com owner-listname + + /etc/aliases: + listname: "|/some/where/majordomo/wrapper ..." + owner-listname: ... + listname-request: ... + +This example assumes that in main.cf, $myorigin is listed under the +mydestination parameter setting. If that is not the case, specify an explicit +domain name on the right-hand side of the virtual alias table entries or else +mail will go to the wrong domain. + +More information about the Postfix local delivery agent can be found in the +local(8) manual page. + +Why does this example use a clumsy virtual alias instead of a more elegant +transport mapping? The reason is that mail for the virtual mailing list would +be rejected with "User unknown". In order to make the transport mapping work +one would still need a bunch of virtual alias or virtual mailbox table entries. + + * In case of a virtual alias domain, there would need to be one identity + mapping from each mailing list address to itself. + * In case of a virtual mailbox domain, there would need to be a dummy mailbox + for each mailing list address. + +AAuuttoorreepplliieess + +In order to set up an autoreply for virtual recipients while still delivering +mail as normal, set up a rule in a virtual alias table: + + /etc/postfix/main.cf: + virtual_alias_maps = hash:/etc/postfix/virtual + + /etc/postfix/virtual: + user@domain.tld user@domain.tld, user@domain.tld@autoreply.mydomain.tld + +This delivers mail to the recipient, and sends a copy of the mail to the +address that produces automatic replies. The address can be serviced on a +different machine, or it can be serviced locally by setting up a transport map +entry that pipes all mail for autoreply.mydomain.tld into some script that +sends an automatic reply back to the sender. + +DO NOT list autoreply.mydomain.tld in mydestination! + + /etc/postfix/main.cf: + transport_maps = hash:/etc/postfix/transport + + /etc/postfix/transport: + autoreply.mydomain.tld autoreply: + + /etc/postfix/master.cf: + # ============================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ============================================================= + autoreply unix - n n - - pipe + flags= user=nobody argv=/path/to/autoreply $sender $mailbox + +This invokes /path/to/autoreply with the sender address and the user@domain.tld +recipient address on the command line. + +For more information, see the pipe(8) manual page, and the comments in the +Postfix master.cf file. + diff --git a/README_FILES/XCLIENT_README b/README_FILES/XCLIENT_README new file mode 100644 index 0000000..89b11bf --- /dev/null +++ b/README_FILES/XCLIENT_README @@ -0,0 +1,199 @@ +PPoossttffiixx XXCCLLIIEENNTT HHoowwttoo + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhee XXCCLLIIEENNTT eexxtteennssiioonn ttoo SSMMTTPP + +When an SMTP server announces support for the XCLIENT command, an SMTP client +may send information that overrides one or more client-related session +attributes. The XCLIENT command targets the following problems: + + 1. Access control tests. SMTP server access rules are difficult to verify when + decisions can be triggered only by remote clients. In order to facilitate + access rule testing, an authorized SMTP client test program needs the + ability to override the SMTP server's idea of the SMTP client hostname, + network address, and other client information, for the entire duration of + an SMTP session. + + 2. Client software that downloads mail from an up-stream mail server and + injects it into a local MTA via SMTP. In order to take advantage of the + local MTA's SMTP server access rules, the client software needs the ability + to override the SMTP server's idea of the remote client name, client + address and other information. Such information can typically be extracted + from the up-stream mail server's Received: message header. + + 3. Post-filter access control and logging. With Internet->filter->MTA style + content filter applications, the filter can be simplified if it can + delegate decisions concerning mail relay and other access control to the + MTA. This is especially useful when the filter acts as a transparent proxy + for SMTP commands. This requires that the filter can override the MTA's + idea of the SMTP client hostname, network address, and other information. + +XXCCLLIIEENNTT CCoommmmaanndd ssyynnttaaxx + +An example client-server conversation is given at the end of this document. + +In SMTP server EHLO replies, the keyword associated with this extension is +XCLIENT. It is followed by the names of the attributes that the XCLIENT +implementation supports. + +The XCLIENT command may be sent at any time, except in the middle of a mail +delivery transaction (i.e. between MAIL and DOT, or MAIL and RSET). The XCLIENT +command may be pipelined when the server supports ESMTP command pipelining. To +avoid triggering spamware detectors, the command should be sent at the end of a +command group. + +The syntax of XCLIENT requests is described below. Upper case and quoted +strings specify terminals, lowercase strings specify meta terminals, and SP is +whitespace. Although command and attribute names are shown in upper case, they +are in fact case insensitive. + + xclient-command = XCLIENT 1*( SP attribute-name"="attribute-value ) + + attribute-name = ( NAME | ADDR | PORT | PROTO | HELO | LOGIN | DESTADDR | + DESTPORT ) + + attribute-value = xtext + + * Attribute values are xtext encoded as per RFC 1891. + + * The NAME attribute specifies a remote SMTP client hostname (not an SMTP + client address), [UNAVAILABLE] when client hostname lookup failed due to a + permanent error, or [TEMPUNAVAIL] when the lookup error condition was + transient. + + * The ADDR attribute specifies a remote SMTP client numerical IPv4 network + address, an IPv6 address prefixed with IPV6:, or [UNAVAILABLE] when the + address information is unavailable. Address information is not enclosed + with []. + + * The PORT attribute specifies a remote SMTP client TCP port number as a + decimal number, or [UNAVAILABLE] when the information is unavailable. + + * The PROTO attribute specifies either SMTP or ESMTP. + + * The DESTADDR attribute specifies a local SMTP server numerical IPv4 network + address, an IPv6 address prefixed with IPV6:, or [UNAVAILABLE] when the + address information is unavailable. Address information is not enclosed + with []. + + * The DESTPORT attribute specifies a local SMTP server TCP port number as a + decimal number, or [UNAVAILABLE] when the information is unavailable. + + * The HELO attribute specifies an SMTP HELO parameter value, or the value + [UNAVAILABLE] when the information is unavailable. + + * The LOGIN attribute specifies a SASL login name, or the value [UNAVAILABLE] + when the information is unavailable. + +Note 1: syntactically valid NAME and HELO attribute-value elements can be up to +255 characters long. The client must not send XCLIENT commands that exceed the +512 character limit for SMTP commands. To avoid exceeding the limit the client +should send the information in multiple XCLIENT commands; for example, send +NAME and ADDR last, after HELO and PROTO. Once ADDR is sent, the client is +usually no longer authorized to send XCLIENT commands. + +Note 2: [UNAVAILABLE], [TEMPUNAVAIL] and IPV6: may be specified in upper case, +lower case or mixed case. + +Note 3: Postfix implementations prior to version 2.3 do not xtext encode +attribute values. Servers that wish to interoperate with these older +implementations should be prepared to receive unencoded information. + +Note 4: Some Postfix implementations do not implement the PORT or LOGIN +attributes. + +XXCCLLIIEENNTT SSeerrvveerr rreessppoonnssee + +Upon receipt of a correctly formatted XCLIENT command, the server resets state +to the initial SMTP greeting protocol stage. Depending on the outcome of +optional access decisions, the server responds with 220 or with a suitable +rejection code. + +For practical reasons it is not always possible to reset the complete server +state to the initial SMTP greeting protocol stage: + + * TLS session information may not be reset, because turning off TLS leaves + the connection in an undefined state. Consequently, the server may not + announce STARTTLS when TLS is already active, and access decisions may be + influenced by client certificate information that was received prior to the + XCLIENT command. + + * The SMTP server must not reset attributes that were received with the last + XCLIENT command. This includes HELO or PROTO attributes. + +NOTE: Postfix implementations prior to version 2.3 do not jump back to the +initial SMTP greeting protocol stage. These older implementations will not +correctly simulate connection-level access decisions under some conditions. + +XXCCLLIIEENNTT sseerrvveerr rreeppllyy ccooddeess + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |CCooddee |MMeeaanniinngg | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |220 |success | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |421 |unable to proceed, disconnecting | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |501 |bad command parameter syntax | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |503 |mail transaction in progress | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |550 |insufficient authorization | + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |other|connection rejected by connection-level access decision| + |_ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +XXCCLLIIEENNTT EExxaammppllee + +In the example, the client impersonates a mail originating system by passing +all SMTP client information via the XCLIENT command. Information sent by the +client is shown in bold font. + + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-VRFY + 250-ETRN + 250-XCLIENT NAME ADDR PROTO HELO + 250 8BITMIME + XXCCLLIIEENNTT NNAAMMEE==ssppiikkee..ppoorrccuuppiinnee..oorrgg AADDDDRR==116688..110000..118899..22 + 220 server.example.com ESMTP Postfix + EEHHLLOO ssppiikkee..ppoorrccuuppiinnee..oorrgg + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-VRFY + 250-ETRN + 250-XCLIENT NAME ADDR PROTO HELO + 250 8BITMIME + MMAAIILL FFRROOMM::<<wwiieettssee@@ppoorrccuuppiinnee..oorrgg>> + 250 Ok + RRCCPPTT TTOO::<<uusseerr@@eexxaammppllee..ccoomm>> + 250 Ok + DDAATTAA + 354 End data with <CR><LF>.<CR><LF> + .. .. ..mmeessssaaggee ccoonntteenntt.. .. .. + .. + 250 Ok: queued as 763402AAE6 + QQUUIITT + 221 Bye + +SSeeccuurriittyy + +The XCLIENT command changes audit trails and/or SMTP client access permissions. +Use of this command must be restricted to authorized SMTP clients. + +SSMMTTPP ccoonnnneeccttiioonn ccaacchhiinngg + +XCLIENT attributes persist until the end of an SMTP session. If one session is +used to deliver mail on behalf of different SMTP clients, the XCLIENT +attributes need to be reset as appropriate before each MAIL FROM command. + +RReeffeerreenncceess + +Moore, K, "SMTP Service Extension for Delivery Status Notifications", RFC 1891, +January 1996. + diff --git a/README_FILES/XFORWARD_README b/README_FILES/XFORWARD_README new file mode 100644 index 0000000..84802ed --- /dev/null +++ b/README_FILES/XFORWARD_README @@ -0,0 +1,179 @@ +PPoossttffiixx XXFFOORRWWAARRDD HHoowwttoo + +------------------------------------------------------------------------------- + +PPuurrppoossee ooff tthhee XXFFOORRWWAARRDD eexxtteennssiioonn ttoo SSMMTTPP + +When an SMTP server announces support for the XFORWARD command, an SMTP client +may send information that overrides one or more client-related logging +attributes. The XFORWARD command targets the following problem: + + * Logging after SMTP-based content filter. With the deployment of Internet- + >MTA1->filter->MTA2 style content filter applications, the logging of + client and message identifying information changes when MTA1 gives the mail + to the content filter. To simplify the interpretation of MTA2 logging, it + would help if MTA1 could forward remote client and/or message identifying + information through the content filter to MTA2, so that the information + could be logged as part of mail handling transactions. + +This extension is implemented as a separate ESMTP command, and can be used to +transmit client or message attributes incrementally. It is not implemented by +passing additional parameters via the MAIL FROM command, because doing so would +require extending the MAIL FROM command length limit by another 600 or more +characters beyond the space that is already needed to support other extensions +such as AUTH and DSN. + +XXFFOORRWWAARRDD CCoommmmaanndd ssyynnttaaxx + +An example of a client-server conversation is given at the end of this +document. + +In SMTP server EHLO replies, the keyword associated with this extension is +XFORWARD. The keyword is followed by the names of the attributes that the +XFORWARD implementation supports. + +After receiving the server's announcement for XFORWARD support, the client may +send XFORWARD requests at any time except in the middle of a mail delivery +transaction (i.e. between MAIL and RSET or DOT). The command may be pipelined +when the server supports ESMTP command pipelining. + +The syntax of XFORWARD requests is described below. Upper case and quoted +strings specify terminals, lowercase strings specify meta terminals, and SP is +whitespace. Although command and attribute names are shown in upper case, they +are in fact case insensitive. + + xforward-command = XFORWARD 1*( SP attribute-name"="attribute-value ) + + attribute-name = ( NAME | ADDR | PORT | PROTO | HELO | IDENT | SOURCE ) + + attribute-value = xtext + + * Attribute values are xtext encoded as per RFC 1891. + + * The NAME attribute specifies the up-stream hostname, or [UNAVAILABLE] when + the information is unavailable. The hostname may be a non-DNS hostname. + + * The ADDR attribute specifies the up-stream network address: a numerical + IPv4 network address, an IPv6 address prefixed with IPV6:, or [UNAVAILABLE] + when the address information is unavailable. Address information is not + enclosed with []. + + * The PORT attribute specifies an up-stream client TCP port number in + decimal, or [UNAVAILABLE] when the information is unavailable. + + * The PROTO attribute specifies the mail protocol for receiving mail from the + up-stream host. This may be an SMTP or non-SMTP protocol name of up to 64 + characters, or [UNAVAILABLE] when the information is unavailable. + + * The HELO attribute specifies the hostname that the up-stream host announced + itself with (not necessarily via the SMTP HELO command), or [UNAVAILABLE] + when the information is unavailable. The hostname may be a non-DNS + hostname. + + * The IDENT attribute specifies a local message identifier on the up-stream + host, or [UNAVAILABLE] when the information is unavailable. The down-stream + MTA may log this information together with its own local message identifier + to facilitate message tracking across MTAs. + + * The SOURCE attribute specifies LOCAL when the message was received from a + source that is local with respect to the up-stream host (for example, the + message originated from the up-stream host itself), REMOTE for all other + mail, or [UNAVAILABLE] when the information is unavailable. The down-stream + MTA may decide to enable features such as header munging or address + qualification with mail from local sources but not other sources. + +Note 1: an attribute-value element must not be longer than 255 characters +(specific attributes may impose shorter lengths). After xtext decoding, +attribute values must not contain control characters, non-ASCII characters, +whitespace, or other characters that are special in message headers. + +Note 2: DNS hostnames can be up to 255 characters long. The XFORWARD client +implementation must not send XFORWARD commands that exceed the 512 character +limit for SMTP commands. + +Note 3: [UNAVAILABLE] may be specified in upper case, lower case or mixed case. + +Note 4: Postfix implementations prior to version 2.3 do not xtext encode +attribute values. Servers that wish to interoperate with these older +implementations should be prepared to receive unencoded information. + +XXFFOORRWWAARRDD SSeerrvveerr ooppeerraattiioonn + +The server maintains a set of XFORWARD attributes with forwarded information, +in addition the current SMTP session attributes. Normally, all XFORWARD +attributes are in the undefined state, and the server uses the current SMTP +session attributes for logging purposes. + +Upon receipt of an initial XFORWARD command, the SMTP server initializes all +XFORWARD attributes to [UNAVAILABLE]. With each valid XFORWARD command, the +server updates XFORWARD attributes with the specified values. + +The server must not mix client attributes from XFORWARD with client attributes +from the current SMTP session. + +At the end of each MAIL FROM transaction (i.e. RSET or DOT), the server resets +all XFORWARD attributes to the undefined state, and is ready to receive another +initial XFORWARD command. + +XXFFOORRWWAARRDD SSeerrvveerr rreeppllyy ccooddeess + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |CCooddee|MMeeaanniinngg | + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |250 |success | + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |421 |unable to proceed, disconnecting| + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |501 |bad command parameter syntax | + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |503 |mail transaction in progress | + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |550 |insufficient authorization | + |_ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +XXFFOORRWWAARRDD EExxaammppllee + +In the following example, information sent by the client is shown in bold font. + + 220 server.example.com ESMTP Postfix + EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm + 250-server.example.com + 250-PIPELINING + 250-SIZE 10240000 + 250-VRFY + 250-ETRN + 250-XFORWARD NAME ADDR PROTO HELO + 250 8BITMIME + XXFFOORRWWAARRDD NNAAMMEE==ssppiikkee..ppoorrccuuppiinnee..oorrgg AADDDDRR==116688..110000..118899..22 PPRROOTTOO==EESSMMTTPP + 250 Ok + XXFFOORRWWAARRDD HHEELLOO==ssppiikkee..ppoorrccuuppiinnee..oorrgg + 250 Ok + MMAAIILL FFRROOMM::<<wwiieettssee@@ppoorrccuuppiinnee..oorrgg>> + 250 Ok + RRCCPPTT TTOO::<<uusseerr@@eexxaammppllee..ccoomm>> + 250 Ok + DDAATTAA + 354 End data with <CR><LF>.<CR><LF> + .. .. ..mmeessssaaggee ccoonntteenntt.. .. .. + .. + 250 Ok: queued as 3CF6B2AAE8 + QQUUIITT + 221 Bye + +SSeeccuurriittyy + +The XFORWARD command changes audit trails. Use of this command must be +restricted to authorized clients. + +SSMMTTPP ccoonnnneeccttiioonn ccaacchhiinngg + +SMTP connection caching makes it possible to deliver multiple messages within +the same SMTP session. The XFORWARD attributes are reset after the MAIL FROM +transaction completes (after RSET or DOT), so there is no risk of information +leakage. + +RReeffeerreenncceess + +Moore, K, "SMTP Service Extension for Delivery Status Notifications", RFC 1891, +January 1996. + |