diff options
Diffstat (limited to 'README_FILES/OVERVIEW')
| -rw-r--r-- | README_FILES/OVERVIEW | 492 |
1 files changed, 492 insertions, 0 deletions
diff --git a/README_FILES/OVERVIEW b/README_FILES/OVERVIEW new file mode 100644 index 0000000..71976d4 --- /dev/null +++ b/README_FILES/OVERVIEW @@ -0,0 +1,492 @@ +PPoossttffiixx AArrcchhiitteeccttuurree OOvveerrvviieeww + +------------------------------------------------------------------------------- + +IInnttrroodduuccttiioonn + +This document presents an overview of the Postfix architecture, and provides +pointers to descriptions of every Postfix command or server program. The text +gives the general context in which each command or server program is used, and +provides pointers to documents with specific usage examples and background +information. + +Topics covered by this document: + + * How Postfix receives mail + * How Postfix delivers mail + * Postfix behind the scenes + * Postfix support commands + +HHooww PPoossttffiixx rreecceeiivveess mmaaiill + +When a message enters the Postfix mail system, the first stop on the inside is +the incoming queue. The figure below shows the main processes that are involved +with new mail. Names followed by a number are Postfix commands or server +programs, while unnumbered names inside shaded areas represent Postfix queues. + + trivial- + rewrite(8) + + Network -> smtpd(8) + + ^ | + \ | v + + Network -> qmqpd(8) -> cleanup(8) -> incoming + + / + + pickup(8) <- maildrop + + ^ + | + + Local -> sendmail(1) -> postdrop(1) + + * Network mail enters Postfix via the smtpd(8) or qmqpd(8) servers. These + servers remove the SMTP or QMQP protocol encapsulation, enforce some sanity + checks to protect Postfix, and give the sender, recipients and message + content to the cleanup(8) server. The smtpd(8) server can be configured to + block unwanted mail, as described in the SMTPD_ACCESS_README document. + + * Local submissions are received with the Postfix sendmail(1) compatibility + command, and are queued in the maildrop queue by the privileged postdrop(1) + command. This arrangement even works while the Postfix mail system is not + running. The local pickup(8) server picks up local submissions, enforces + some sanity checks to protect Postfix, and gives the sender, recipients and + message content to the cleanup(8) server. + + * Mail from internal sources is given directly to the cleanup(8) server. + These sources are not shown in the figure, and include: mail that is + forwarded by the local(8) delivery agent (see next section), messages that + are returned to the sender by the bounce(8) server (see second-next + section), and postmaster notifications about problems with Postfix. + + * The cleanup(8) server implements the final processing stage before mail is + queued. It adds missing From: and other message headers, and transforms + addresses as described in the ADDRESS_REWRITING_README document. + Optionally, the cleanup(8) server can be configured to do light-weight + content inspection with regular expressions as described in the + BUILTIN_FILTER_README document. The cleanup(8) server places the result as + a single file into the incoming queue, and notifies the queue manager (see + next section) of the arrival of new mail. + + * The trivial-rewrite(8) server rewrites addresses to the standard + "user@fully.qualified.domain" form, as described in the + ADDRESS_REWRITING_README document. Postfix currently does not implement a + rewriting language, but a lot can be done via table lookups and, if need + be, regular expressions. + +HHooww PPoossttffiixx ddeelliivveerrss mmaaiill + +Once a message has reached the incoming queue the next step is to deliver it. +The figure shows the main components of the Postfix mail delivery apparatus. +Names followed by a number are Postfix commands or server programs, while +unnumbered names inside shaded areas represent Postfix queues. + + trivial- smtp(8) -> Network + rewrite(8) + / + + - lmtp(8) -> Network + ^ | + | v / + + incoming -> active -> qmgr(8) --- local(8) -> File, command + + \ + ^ | + | v - virtual(8) -> File + + deferred \ + + pipe(8) -> Command + + * The queue manager (the qmgr(8) server process in the figure) is the heart + of Postfix mail delivery. It contacts the smtp(8), lmtp(8), local(8), + virtual(8), pipe(8), discard(8) or error(8) delivery agents, and sends a + delivery request for one or more recipient addresses. The discard(8) and + error(8) delivery agents are special: they discard or bounce all mail, and + are not shown in the figure above. + + The queue manager maintains a small active queue with the messages that it + has opened for delivery. The active queue acts as a limited window on + potentially large incoming or deferred queues. The limited active queue + prevents the queue manager from running out of memory under heavy load. + + The queue manager maintains a separate deferred queue for mail that cannot + be delivered, so that a large mail backlog will not slow down normal queue + accesses. The queue manager's strategy for delayed mail delivery attempts + is described in the QSHAPE_README and TUNING_README documents. + + * The trivial-rewrite(8) server resolves each recipient address according to + its local or remote address class, as defined in the ADDRESS_CLASS_README + document. Additional routing information can be specified with the optional + transport(5) table. The trivial-rewrite(8) server optionally queries the + relocated(5) table for recipients whose address has changed; mail for such + recipients is returned to the sender with an explanation. + + * The smtp(8) client looks up a list of mail exchangers for the destination + host, sorts the list by preference, and tries each server in turn until it + finds a server that responds. It then encapsulates the sender, recipient + and message content as required by the SMTP protocol; this includes + conversion of 8-bit MIME to 7-bit encoding. + + * The lmtp(8) client speaks a protocol similar to SMTP that is optimized for + delivery to mailbox servers such as Cyrus. The advantage of this setup is + that one Postfix machine can feed multiple mailbox servers over LMTP. The + opposite is true as well: one mailbox server can be fed over LMTP by + multiple Postfix machines. + + * The local(8) delivery agent understands UNIX-style mailboxes, qmail- + compatible maildir files, Sendmail-style system-wide aliases(5) databases, + and Sendmail-style per-user .forward files. Multiple local delivery agents + can be run in parallel, but parallel delivery to the same user is usually + limited. + + The local(8) delivery agent has hooks for alternative forms of local + delivery: you can configure it to deliver to mailbox files in user home + directories, you can configure it to delegate mailbox delivery to an + external command such as procmail, or you can delegate delivery to a + different Postfix delivery agent. + + * The virtual(8) delivery agent is a bare-bones delivery agent that delivers + to UNIX-style mailbox or qmail-style maildir files only. This delivery + agent can deliver mail for multiple domains, which makes it especially + suitable for hosting lots of small domains on a single machine. This is + described in the VIRTUAL_README document. + + * The pipe(8) mailer is the outbound interface to other mail processing + systems (the Postfix sendmail(1) command being the inbound interface). The + interface is UNIX compatible: it provides information on the command line + and on the standard input stream, and expects a process exit status code as + defined in <sysexits.h>. Examples of delivery via the pipe(8) mailer are in + the MAILDROP_README and UUCP_README documents. + +PPoossttffiixx bbeehhiinndd tthhee sscceenneess + +The previous sections gave an overview of how Postfix server processes send and +receive mail. These server processes rely on other server processes that do +things behind the scenes. The text below attempts to visualize each service in +its own context. As before, names followed by a number are Postfix commands or +server programs, while unnumbered names inside shaded areas represent Postfix +queues. + + * The resident master(8) server is the supervisor that keeps an eye on the + well-being of the Postfix mail system. It is typically started at system + boot time with the "postfix start" command, and keeps running until the + system goes down. The master(8) server is responsible for starting Postfix + server processes to receive and deliver mail, and for restarting servers + that terminate prematurely because of some problem. The master(8) server is + also responsible for enforcing the server process count limits as specified + in the mmaasstteerr..ccff configuration file. The picture below gives the program + hierarchy when Postfix is started up. Only some of the mail handling daemon + processes are shown. + + postfix(1) + + | + | + + postfix-script(1) + + / | \ + | + / \ + + postsuper(1) master(8) postlog(1) + + / | \ + | + / \ + + smtpd(8) qmgr(8) local(8) + + * The anvil(8) server implements client connection and request rate limiting + for all smtpd(8) servers. The TUNING_README document provides guidance for + dealing with mis-behaving SMTP clients. The anvil(8) service is available + in Postfix version 2.2 and later. + + Network -> smtpd(8) <-> anvil(8) + + * The bounce(8), defer(8) and trace(8) services each maintain their own queue + directory trees with per-message logfiles. Postfix uses this information + when sending "failed", "delayed" or "success" delivery status notifications + to the sender. + + The trace(8) service also implements support for the Postfix "sendmail -bv" + and "sendmail -v" commands which produce reports about how Postfix delivers + mail, and is available with Postfix version 2.1 and later. See DEBUG_README + for examples. + + qmgr(8) Delivery + cleanup(8) -> Postfix -> agents + queue + + ^ | | + | v v + + (Non-) bounce(8) Queue id, + delivery <- defer(8) <- recipient, + notice trace(8) status + + ^ | + | v + + Per- + message + logfiles + + * The flush(8) servers maintain per-destination logs and implement both ETRN + and "sendmail -qRdestination", as described in the ETRN_README document. + This moves selected queue files from the deferred queue back to the + incoming queue and requests their delivery. The flush(8) service is + available with Postfix version 1.0 and later. + + incoming + ^ + deferred + + ^ + | + + smtpd(8) Destination Deferred Delivery + sendmail(1) - to flush -> flush(8) <- destination, - agents, + postqueue(1) queue id qmgr(8) + + ^ | + | v + + Per-dest- + ination + logs + + * The proxymap(8) servers provide read-only and read-write table lookup + service to Postfix processes. This overcomes chroot restrictions, reduces + the number of open lookup tables by sharing one open table among multiple + processes, and implements single-updater tables. + + * The scache(8) server maintains the connection cache for the Postfix smtp(8) + client. When connection caching is enabled for selected destinations, the + smtp(8) client does not disconnect immediately after a mail transaction, + but gives the connection to the connection cache server which keeps the + connection open for a limited amount of time. The smtp(8) client continues + with some other mail delivery request. Meanwhile, any smtp(8) process can + ask the scache(8) server for that cached connection and reuse it for mail + delivery. As a safety measure, Postfix limits the number of times that a + connection may be reused. + + When delivering mail to a destination with multiple mail servers, + connection caching can help to skip over a non-responding server, and thus + dramatically speed up delivery. SMTP connection caching is available in + Postfix version 2.2 and later. More information about this feature is in + the CONNECTION_CACHE_README document. + + /-- smtp(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + + A Postfix smtp(8) client can reuse a TLS-encrypted connection (with + "smtp_tls_connection_reuse = yes"). This can greatly reduce the overhead of + connection setup and improves message delivery rates. After a Postfix smtp + (8) client connects to a remote SMTP server and sends plaintext EHLO and + STARTTLS commands, the smtp(8) client inserts a tlsproxy(8) process into + the connection as shown below. + + After the mail transaction completes, the Postfix smtp(8) client gives the + smtp(8)-to-tlsproxy(8) connection to the scache(8) server, which keeps the + connection open for a limited amount of time. The smtp(8) client continues + with some other mail delivery request. Meanwhile, any Postfix smtp(8) + client can ask the scache(8) server for that cached connection and reuse it + for mail delivery. + + /-- smtp(8) --> tlsproxy(8) --> Internet + + qmgr(8) + | + \-- | smtp(8) + | + | ^ + v | + + scache(8) + + * The showq(8) servers list the Postfix queue status. This is the queue + listing service that does the work for the mailq(1) and postqueue(1) + commands. + + mailq(1) Postfix + Output <- post- <- showq(8) <- queue + queue(1) + + * The spawn(8) servers run non-Postfix commands on request, with the client + connected via socket or FIFO to the command's standard input, output and + error streams. You can find examples of its use in the SMTPD_POLICY_README + document. + + * The tlsmgr(8) server runs when TLS (Transport Layer Security, formerly + known as SSL) is turned on in the Postfix smtp(8) client or smtpd(8) + server. This process has two duties: + + o Maintain the pseudo-random number generator (PRNG) that is used to seed + the TLS engines in Postfix smtp(8) client or smtpd(8) server processes. + The state of this PRNG is periodically saved to a file, and is read + when tlsmgr(8) starts up. + + o Maintain the optional Postfix smtp(8) client or smtpd(8) server caches + with TLS session keys. Saved keys can improve performance by reducing + the amount of computation at the start of a TLS session. + + TLS support is available in Postfix version 2.2 and later. Information + about the Postfix TLS implementation is in the TLS_README document. + + <---seed--- ---seed---> + Network-> smtpd(8) tlsmgr(8) smtp(8) ->Network + <-session-> <-session-> + + / | \ + | + / \ + + smtpd PRNG smtp + session state session + cache file cache + + * The verify(8) server verifies that a sender or recipient address is + deliverable before the smtpd(8) server accepts it. The verify(8) server + queries a cache with address verification results. If a result is not + found, the verify(8) server injects a probe message into the Postfix queue + and processes the status update from a delivery agent or queue manager. + This process is described in the ADDRESS_VERIFICATION_README document. The + verify(8) service is available with Postfix version 2.1 and later. + + probe Postfix + message -> mail + Network -> smtpd(8) <-> verify(8) -> queue + + | + v + + <- probe Postfix -> Local + status <- delivery -> Network + ^ agents + | + v + + Address + verification + cache + + * The postscreen(8) server can be put "in front" of Postfix smtpd(8) + processes. Its purpose is to accept connections from the network and to + decide what SMTP clients are allowed to talk to Postfix. According to the + 2008 MessageLabs annual report, 81% of all email was spam, and 90% of that + was sent by botnets; by 2010, those numbers were 92% and 95%, respectively. + While postscreen(8) keeps the zombies away, more smtpd(8) processes remain + available for legitimate clients. + + postscreen(8) maintains a temporary allowlist for clients that pass its + tests; by allowing allowlisted clients to skip tests, postscreen(8) + minimizes its impact on legitimate email traffic. + + The postscreen(8) server is available with Postfix 2.8 and later. To keep + the implementation simple, postscreen(8) delegates DNS allow/denylist + lookups to dnsblog(8) server processes, and delegates TLS encryption/ + decryption to tlsproxy(8) server processes. This delegation is invisible to + the remote SMTP client. + + zombie + + \ + + zombie - tlsproxy(8) - - smtpd(8) + + \ / + + other --- postscreen(8) + + / \ + + other - - smtpd(8) + + / + + zombie + + * The postlogd(8) server provides an alternative to syslog logging, which + remains the default. This feature is available with Postfix version 3.4 or + later, and supports the following modes: + + o Logging to file, which addresses a usability problem with MacOS, and + eliminates information loss caused by systemd rate limits. + + commands -> postlogd(8) -> /path/to/file + or daemons + + o Logging to stdout, which eliminates a syslog dependency when Postfix + runs inside a container. + + commands -> postlogd(8) -> stdout inherited + or daemons from "postfix start-fg" + + See MAILLOG_README for details and limitations. + +PPoossttffiixx ssuuppppoorrtt ccoommmmaannddss + +The Postfix architecture overview ends with a summary of command-line utilities +for day-to-day use of the Postfix mail system. Besides the Sendmail-compatible +sendmail(1), mailq(1), and newaliases(1) commands, the Postfix system comes +with it own collection of command-line utilities. For consistency, these are +all named postsomething. + + * The postfix(1) command controls the operation of the mail system. It is the + interface for starting, stopping, and restarting the mail system, as well + as for some other administrative operations. This command is reserved to + the super-user. + + * The postalias(1) command maintains Postfix aliases(5) type databases. This + is the program that does the work for the newaliases(1) command. + + * The postcat(1) command displays the contents of Postfix queue files. This + is a limited, preliminary utility. This program is likely to be superseded + by something more powerful that can also edit Postfix queue files. + + * The postconf(1) command displays or updates Postfix main.cf parameters and + displays system dependent information about the supported file locking + methods, and the supported types of lookup tables. + + * The postdrop(1) command is the mail posting utility that is run by the + Postfix sendmail(1) command in order to deposit mail into the maildrop + queue directory. + + * The postkick(1) command makes some Postfix internal communication channels + available for use in, for example, shell scripts. + + * The postlock(1) command provides Postfix-compatible mailbox locking for use + in, for example, shell scripts. + + * The postlog(1) command provides Postfix-compatible logging for shell + scripts. + + * The postmap(1) command maintains Postfix lookup tables such as canonical + (5), virtual(5) and others. It is a cousin of the UNIX makemap command. + + * The postmulti(1) command repeats the "postfix start" etc. command for each + Postfix instance, and supports creation, deletion etc. of Postfix + instances. For a tutorial, see MULTI_INSTANCE_README. + + * The postqueue(1) command is the privileged command that is run by Postfix + sendmail(1) and mailq(1) in order to flush or list the mail queue. + + * The postsuper(1) command maintains the Postfix queue. It removes old + temporary files, and moves queue files into the right directory after a + change in the hashing depth of queue directories. This command is run at + mail system startup time and when Postfix is restarted. + |