summaryrefslogtreecommitdiffstats
path: root/README_FILES/OVERVIEW
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 16:18:56 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 16:18:56 +0000
commitb7c15c31519dc44c1f691e0466badd556ffe9423 (patch)
treef944572f288bab482a615e09af627d9a2b6727d8 /README_FILES/OVERVIEW
parentInitial commit. (diff)
downloadpostfix-b7c15c31519dc44c1f691e0466badd556ffe9423.tar.xz
postfix-b7c15c31519dc44c1f691e0466badd556ffe9423.zip
Adding upstream version 3.7.10.upstream/3.7.10upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'README_FILES/OVERVIEW')
-rw-r--r--README_FILES/OVERVIEW492
1 files changed, 492 insertions, 0 deletions
diff --git a/README_FILES/OVERVIEW b/README_FILES/OVERVIEW
new file mode 100644
index 0000000..71976d4
--- /dev/null
+++ b/README_FILES/OVERVIEW
@@ -0,0 +1,492 @@
+PPoossttffiixx AArrcchhiitteeccttuurree OOvveerrvviieeww
+
+-------------------------------------------------------------------------------
+
+IInnttrroodduuccttiioonn
+
+This document presents an overview of the Postfix architecture, and provides
+pointers to descriptions of every Postfix command or server program. The text
+gives the general context in which each command or server program is used, and
+provides pointers to documents with specific usage examples and background
+information.
+
+Topics covered by this document:
+
+ * How Postfix receives mail
+ * How Postfix delivers mail
+ * Postfix behind the scenes
+ * Postfix support commands
+
+HHooww PPoossttffiixx rreecceeiivveess mmaaiill
+
+When a message enters the Postfix mail system, the first stop on the inside is
+the incoming queue. The figure below shows the main processes that are involved
+with new mail. Names followed by a number are Postfix commands or server
+programs, while unnumbered names inside shaded areas represent Postfix queues.
+
+ trivial-
+ rewrite(8)
+
+ Network -> smtpd(8)
+
+ ^ |
+ \ | v
+
+ Network -> qmqpd(8) -> cleanup(8) -> incoming
+
+ /
+
+ pickup(8) <- maildrop
+
+ ^
+ |
+
+ Local -> sendmail(1) -> postdrop(1)
+
+ * Network mail enters Postfix via the smtpd(8) or qmqpd(8) servers. These
+ servers remove the SMTP or QMQP protocol encapsulation, enforce some sanity
+ checks to protect Postfix, and give the sender, recipients and message
+ content to the cleanup(8) server. The smtpd(8) server can be configured to
+ block unwanted mail, as described in the SMTPD_ACCESS_README document.
+
+ * Local submissions are received with the Postfix sendmail(1) compatibility
+ command, and are queued in the maildrop queue by the privileged postdrop(1)
+ command. This arrangement even works while the Postfix mail system is not
+ running. The local pickup(8) server picks up local submissions, enforces
+ some sanity checks to protect Postfix, and gives the sender, recipients and
+ message content to the cleanup(8) server.
+
+ * Mail from internal sources is given directly to the cleanup(8) server.
+ These sources are not shown in the figure, and include: mail that is
+ forwarded by the local(8) delivery agent (see next section), messages that
+ are returned to the sender by the bounce(8) server (see second-next
+ section), and postmaster notifications about problems with Postfix.
+
+ * The cleanup(8) server implements the final processing stage before mail is
+ queued. It adds missing From: and other message headers, and transforms
+ addresses as described in the ADDRESS_REWRITING_README document.
+ Optionally, the cleanup(8) server can be configured to do light-weight
+ content inspection with regular expressions as described in the
+ BUILTIN_FILTER_README document. The cleanup(8) server places the result as
+ a single file into the incoming queue, and notifies the queue manager (see
+ next section) of the arrival of new mail.
+
+ * The trivial-rewrite(8) server rewrites addresses to the standard
+ "user@fully.qualified.domain" form, as described in the
+ ADDRESS_REWRITING_README document. Postfix currently does not implement a
+ rewriting language, but a lot can be done via table lookups and, if need
+ be, regular expressions.
+
+HHooww PPoossttffiixx ddeelliivveerrss mmaaiill
+
+Once a message has reached the incoming queue the next step is to deliver it.
+The figure shows the main components of the Postfix mail delivery apparatus.
+Names followed by a number are Postfix commands or server programs, while
+unnumbered names inside shaded areas represent Postfix queues.
+
+ trivial- smtp(8) -> Network
+ rewrite(8)
+ /
+
+ - lmtp(8) -> Network
+ ^ |
+ | v /
+
+ incoming -> active -> qmgr(8) --- local(8) -> File, command
+
+ \
+ ^ |
+ | v - virtual(8) -> File
+
+ deferred \
+
+ pipe(8) -> Command
+
+ * The queue manager (the qmgr(8) server process in the figure) is the heart
+ of Postfix mail delivery. It contacts the smtp(8), lmtp(8), local(8),
+ virtual(8), pipe(8), discard(8) or error(8) delivery agents, and sends a
+ delivery request for one or more recipient addresses. The discard(8) and
+ error(8) delivery agents are special: they discard or bounce all mail, and
+ are not shown in the figure above.
+
+ The queue manager maintains a small active queue with the messages that it
+ has opened for delivery. The active queue acts as a limited window on
+ potentially large incoming or deferred queues. The limited active queue
+ prevents the queue manager from running out of memory under heavy load.
+
+ The queue manager maintains a separate deferred queue for mail that cannot
+ be delivered, so that a large mail backlog will not slow down normal queue
+ accesses. The queue manager's strategy for delayed mail delivery attempts
+ is described in the QSHAPE_README and TUNING_README documents.
+
+ * The trivial-rewrite(8) server resolves each recipient address according to
+ its local or remote address class, as defined in the ADDRESS_CLASS_README
+ document. Additional routing information can be specified with the optional
+ transport(5) table. The trivial-rewrite(8) server optionally queries the
+ relocated(5) table for recipients whose address has changed; mail for such
+ recipients is returned to the sender with an explanation.
+
+ * The smtp(8) client looks up a list of mail exchangers for the destination
+ host, sorts the list by preference, and tries each server in turn until it
+ finds a server that responds. It then encapsulates the sender, recipient
+ and message content as required by the SMTP protocol; this includes
+ conversion of 8-bit MIME to 7-bit encoding.
+
+ * The lmtp(8) client speaks a protocol similar to SMTP that is optimized for
+ delivery to mailbox servers such as Cyrus. The advantage of this setup is
+ that one Postfix machine can feed multiple mailbox servers over LMTP. The
+ opposite is true as well: one mailbox server can be fed over LMTP by
+ multiple Postfix machines.
+
+ * The local(8) delivery agent understands UNIX-style mailboxes, qmail-
+ compatible maildir files, Sendmail-style system-wide aliases(5) databases,
+ and Sendmail-style per-user .forward files. Multiple local delivery agents
+ can be run in parallel, but parallel delivery to the same user is usually
+ limited.
+
+ The local(8) delivery agent has hooks for alternative forms of local
+ delivery: you can configure it to deliver to mailbox files in user home
+ directories, you can configure it to delegate mailbox delivery to an
+ external command such as procmail, or you can delegate delivery to a
+ different Postfix delivery agent.
+
+ * The virtual(8) delivery agent is a bare-bones delivery agent that delivers
+ to UNIX-style mailbox or qmail-style maildir files only. This delivery
+ agent can deliver mail for multiple domains, which makes it especially
+ suitable for hosting lots of small domains on a single machine. This is
+ described in the VIRTUAL_README document.
+
+ * The pipe(8) mailer is the outbound interface to other mail processing
+ systems (the Postfix sendmail(1) command being the inbound interface). The
+ interface is UNIX compatible: it provides information on the command line
+ and on the standard input stream, and expects a process exit status code as
+ defined in <sysexits.h>. Examples of delivery via the pipe(8) mailer are in
+ the MAILDROP_README and UUCP_README documents.
+
+PPoossttffiixx bbeehhiinndd tthhee sscceenneess
+
+The previous sections gave an overview of how Postfix server processes send and
+receive mail. These server processes rely on other server processes that do
+things behind the scenes. The text below attempts to visualize each service in
+its own context. As before, names followed by a number are Postfix commands or
+server programs, while unnumbered names inside shaded areas represent Postfix
+queues.
+
+ * The resident master(8) server is the supervisor that keeps an eye on the
+ well-being of the Postfix mail system. It is typically started at system
+ boot time with the "postfix start" command, and keeps running until the
+ system goes down. The master(8) server is responsible for starting Postfix
+ server processes to receive and deliver mail, and for restarting servers
+ that terminate prematurely because of some problem. The master(8) server is
+ also responsible for enforcing the server process count limits as specified
+ in the mmaasstteerr..ccff configuration file. The picture below gives the program
+ hierarchy when Postfix is started up. Only some of the mail handling daemon
+ processes are shown.
+
+ postfix(1)
+
+ |
+ |
+
+ postfix-script(1)
+
+ / | \
+ |
+ / \
+
+ postsuper(1) master(8) postlog(1)
+
+ / | \
+ |
+ / \
+
+ smtpd(8) qmgr(8) local(8)
+
+ * The anvil(8) server implements client connection and request rate limiting
+ for all smtpd(8) servers. The TUNING_README document provides guidance for
+ dealing with mis-behaving SMTP clients. The anvil(8) service is available
+ in Postfix version 2.2 and later.
+
+ Network -> smtpd(8) <-> anvil(8)
+
+ * The bounce(8), defer(8) and trace(8) services each maintain their own queue
+ directory trees with per-message logfiles. Postfix uses this information
+ when sending "failed", "delayed" or "success" delivery status notifications
+ to the sender.
+
+ The trace(8) service also implements support for the Postfix "sendmail -bv"
+ and "sendmail -v" commands which produce reports about how Postfix delivers
+ mail, and is available with Postfix version 2.1 and later. See DEBUG_README
+ for examples.
+
+ qmgr(8) Delivery
+ cleanup(8) -> Postfix -> agents
+ queue
+
+ ^ | |
+ | v v
+
+ (Non-) bounce(8) Queue id,
+ delivery <- defer(8) <- recipient,
+ notice trace(8) status
+
+ ^ |
+ | v
+
+ Per-
+ message
+ logfiles
+
+ * The flush(8) servers maintain per-destination logs and implement both ETRN
+ and "sendmail -qRdestination", as described in the ETRN_README document.
+ This moves selected queue files from the deferred queue back to the
+ incoming queue and requests their delivery. The flush(8) service is
+ available with Postfix version 1.0 and later.
+
+ incoming
+ ^
+ deferred
+
+ ^
+ |
+
+ smtpd(8) Destination Deferred Delivery
+ sendmail(1) - to flush -> flush(8) <- destination, - agents,
+ postqueue(1) queue id qmgr(8)
+
+ ^ |
+ | v
+
+ Per-dest-
+ ination
+ logs
+
+ * The proxymap(8) servers provide read-only and read-write table lookup
+ service to Postfix processes. This overcomes chroot restrictions, reduces
+ the number of open lookup tables by sharing one open table among multiple
+ processes, and implements single-updater tables.
+
+ * The scache(8) server maintains the connection cache for the Postfix smtp(8)
+ client. When connection caching is enabled for selected destinations, the
+ smtp(8) client does not disconnect immediately after a mail transaction,
+ but gives the connection to the connection cache server which keeps the
+ connection open for a limited amount of time. The smtp(8) client continues
+ with some other mail delivery request. Meanwhile, any smtp(8) process can
+ ask the scache(8) server for that cached connection and reuse it for mail
+ delivery. As a safety measure, Postfix limits the number of times that a
+ connection may be reused.
+
+ When delivering mail to a destination with multiple mail servers,
+ connection caching can help to skip over a non-responding server, and thus
+ dramatically speed up delivery. SMTP connection caching is available in
+ Postfix version 2.2 and later. More information about this feature is in
+ the CONNECTION_CACHE_README document.
+
+ /-- smtp(8) --> Internet
+
+ qmgr(8)
+ |
+ \-- | smtp(8)
+ |
+ | ^
+ v |
+
+ scache(8)
+
+ A Postfix smtp(8) client can reuse a TLS-encrypted connection (with
+ "smtp_tls_connection_reuse = yes"). This can greatly reduce the overhead of
+ connection setup and improves message delivery rates. After a Postfix smtp
+ (8) client connects to a remote SMTP server and sends plaintext EHLO and
+ STARTTLS commands, the smtp(8) client inserts a tlsproxy(8) process into
+ the connection as shown below.
+
+ After the mail transaction completes, the Postfix smtp(8) client gives the
+ smtp(8)-to-tlsproxy(8) connection to the scache(8) server, which keeps the
+ connection open for a limited amount of time. The smtp(8) client continues
+ with some other mail delivery request. Meanwhile, any Postfix smtp(8)
+ client can ask the scache(8) server for that cached connection and reuse it
+ for mail delivery.
+
+ /-- smtp(8) --> tlsproxy(8) --> Internet
+
+ qmgr(8)
+ |
+ \-- | smtp(8)
+ |
+ | ^
+ v |
+
+ scache(8)
+
+ * The showq(8) servers list the Postfix queue status. This is the queue
+ listing service that does the work for the mailq(1) and postqueue(1)
+ commands.
+
+ mailq(1) Postfix
+ Output <- post- <- showq(8) <- queue
+ queue(1)
+
+ * The spawn(8) servers run non-Postfix commands on request, with the client
+ connected via socket or FIFO to the command's standard input, output and
+ error streams. You can find examples of its use in the SMTPD_POLICY_README
+ document.
+
+ * The tlsmgr(8) server runs when TLS (Transport Layer Security, formerly
+ known as SSL) is turned on in the Postfix smtp(8) client or smtpd(8)
+ server. This process has two duties:
+
+ o Maintain the pseudo-random number generator (PRNG) that is used to seed
+ the TLS engines in Postfix smtp(8) client or smtpd(8) server processes.
+ The state of this PRNG is periodically saved to a file, and is read
+ when tlsmgr(8) starts up.
+
+ o Maintain the optional Postfix smtp(8) client or smtpd(8) server caches
+ with TLS session keys. Saved keys can improve performance by reducing
+ the amount of computation at the start of a TLS session.
+
+ TLS support is available in Postfix version 2.2 and later. Information
+ about the Postfix TLS implementation is in the TLS_README document.
+
+ <---seed--- ---seed--->
+ Network-> smtpd(8) tlsmgr(8) smtp(8) ->Network
+ <-session-> <-session->
+
+ / | \
+ |
+ / \
+
+ smtpd PRNG smtp
+ session state session
+ cache file cache
+
+ * The verify(8) server verifies that a sender or recipient address is
+ deliverable before the smtpd(8) server accepts it. The verify(8) server
+ queries a cache with address verification results. If a result is not
+ found, the verify(8) server injects a probe message into the Postfix queue
+ and processes the status update from a delivery agent or queue manager.
+ This process is described in the ADDRESS_VERIFICATION_README document. The
+ verify(8) service is available with Postfix version 2.1 and later.
+
+ probe Postfix
+ message -> mail
+ Network -> smtpd(8) <-> verify(8) -> queue
+
+ |
+ v
+
+ <- probe Postfix -> Local
+ status <- delivery -> Network
+ ^ agents
+ |
+ v
+
+ Address
+ verification
+ cache
+
+ * The postscreen(8) server can be put "in front" of Postfix smtpd(8)
+ processes. Its purpose is to accept connections from the network and to
+ decide what SMTP clients are allowed to talk to Postfix. According to the
+ 2008 MessageLabs annual report, 81% of all email was spam, and 90% of that
+ was sent by botnets; by 2010, those numbers were 92% and 95%, respectively.
+ While postscreen(8) keeps the zombies away, more smtpd(8) processes remain
+ available for legitimate clients.
+
+ postscreen(8) maintains a temporary allowlist for clients that pass its
+ tests; by allowing allowlisted clients to skip tests, postscreen(8)
+ minimizes its impact on legitimate email traffic.
+
+ The postscreen(8) server is available with Postfix 2.8 and later. To keep
+ the implementation simple, postscreen(8) delegates DNS allow/denylist
+ lookups to dnsblog(8) server processes, and delegates TLS encryption/
+ decryption to tlsproxy(8) server processes. This delegation is invisible to
+ the remote SMTP client.
+
+ zombie
+
+ \
+
+ zombie - tlsproxy(8) - - smtpd(8)
+
+ \ /
+
+ other --- postscreen(8)
+
+ / \
+
+ other - - smtpd(8)
+
+ /
+
+ zombie
+
+ * The postlogd(8) server provides an alternative to syslog logging, which
+ remains the default. This feature is available with Postfix version 3.4 or
+ later, and supports the following modes:
+
+ o Logging to file, which addresses a usability problem with MacOS, and
+ eliminates information loss caused by systemd rate limits.
+
+ commands -> postlogd(8) -> /path/to/file
+ or daemons
+
+ o Logging to stdout, which eliminates a syslog dependency when Postfix
+ runs inside a container.
+
+ commands -> postlogd(8) -> stdout inherited
+ or daemons from "postfix start-fg"
+
+ See MAILLOG_README for details and limitations.
+
+PPoossttffiixx ssuuppppoorrtt ccoommmmaannddss
+
+The Postfix architecture overview ends with a summary of command-line utilities
+for day-to-day use of the Postfix mail system. Besides the Sendmail-compatible
+sendmail(1), mailq(1), and newaliases(1) commands, the Postfix system comes
+with it own collection of command-line utilities. For consistency, these are
+all named postsomething.
+
+ * The postfix(1) command controls the operation of the mail system. It is the
+ interface for starting, stopping, and restarting the mail system, as well
+ as for some other administrative operations. This command is reserved to
+ the super-user.
+
+ * The postalias(1) command maintains Postfix aliases(5) type databases. This
+ is the program that does the work for the newaliases(1) command.
+
+ * The postcat(1) command displays the contents of Postfix queue files. This
+ is a limited, preliminary utility. This program is likely to be superseded
+ by something more powerful that can also edit Postfix queue files.
+
+ * The postconf(1) command displays or updates Postfix main.cf parameters and
+ displays system dependent information about the supported file locking
+ methods, and the supported types of lookup tables.
+
+ * The postdrop(1) command is the mail posting utility that is run by the
+ Postfix sendmail(1) command in order to deposit mail into the maildrop
+ queue directory.
+
+ * The postkick(1) command makes some Postfix internal communication channels
+ available for use in, for example, shell scripts.
+
+ * The postlock(1) command provides Postfix-compatible mailbox locking for use
+ in, for example, shell scripts.
+
+ * The postlog(1) command provides Postfix-compatible logging for shell
+ scripts.
+
+ * The postmap(1) command maintains Postfix lookup tables such as canonical
+ (5), virtual(5) and others. It is a cousin of the UNIX makemap command.
+
+ * The postmulti(1) command repeats the "postfix start" etc. command for each
+ Postfix instance, and supports creation, deletion etc. of Postfix
+ instances. For a tutorial, see MULTI_INSTANCE_README.
+
+ * The postqueue(1) command is the privileged command that is run by Postfix
+ sendmail(1) and mailq(1) in order to flush or list the mail queue.
+
+ * The postsuper(1) command maintains the Postfix queue. It removes old
+ temporary files, and moves queue files into the right directory after a
+ change in the hashing depth of queue directories. This command is run at
+ mail system startup time and when Postfix is restarted.
+