From a848231ae0f346dc7cc000973fbeb65b0894ee92 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Wed, 10 Apr 2024 21:59:03 +0200 Subject: Adding upstream version 3.8.5. Signed-off-by: Daniel Baumann --- html/OVERVIEW.html | 937 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 937 insertions(+) create mode 100644 html/OVERVIEW.html (limited to 'html/OVERVIEW.html') diff --git a/html/OVERVIEW.html b/html/OVERVIEW.html new file mode 100644 index 0000000..767e2f0 --- /dev/null +++ b/html/OVERVIEW.html @@ -0,0 +1,937 @@ + + + + + + +Postfix Architecture Overview + + + + + + + + +

Postfix +Architecture Overview

+ +

Introduction

+ +

This document presents an overview of the Postfix architecture, +and provides pointers to descriptions of every Postfix command +or server program. The text gives the general context in which +each command or server program is used, and provides pointers to +documents with specific usage examples and background information. +

+ +

Topics covered by this document:

+ +

How Postfix receives mail + +
How Postfix delivers mail + +
Postfix behind the scenes + +
Postfix support commands + +

+ +

How Postfix receives mail

+ +

When a message enters the Postfix mail system, the first stop +on the inside is the incoming queue. The figure below shows the +main processes that are involved with new mail. Names followed by +a number are Postfix commands or server programs, while unnumbered +names inside shaded areas represent Postfix queues.

+ +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
trivial-
rewrite(8)
Network -> smtpd(8) +

+^
| | v +

\
Network -> qmqpd(8) + -> cleanup(8) + -> incoming
/
pickup(8) + <- maildrop
^
|
Local -> sendmail(1) + -> postdrop(1) +
+ +

+ +

Network mail enters Postfix via the smtpd(8) or qmqpd(8) +servers. These servers remove the SMTP or QMQP protocol encapsulation, +enforce some sanity checks to protect Postfix, and give the sender, +recipients and message content to the cleanup(8) server. The +smtpd(8) server can be configured to block unwanted mail, as +described in the SMTPD_ACCESS_README document.
+ +
Local submissions are received with the Postfix sendmail(1) +compatibility command, and are queued in the maildrop queue by +the privileged postdrop(1) command. This arrangement even works +while the Postfix mail system is not running. The local pickup(8) +server picks up local submissions, enforces some sanity checks to +protect Postfix, and gives the sender, recipients and message +content to the cleanup(8) server.
+ +
Mail from internal sources is given directly to the +cleanup(8) server. These sources are not shown in the figure, and +include: mail that is forwarded by the local(8) delivery agent (see +next section), messages that are returned to the sender by the +bounce(8) server (see second-next section), and postmaster +notifications about problems with Postfix.
+ +
The cleanup(8) server implements the final processing +stage before mail is queued. It adds missing From: and other message +headers, and transforms addresses as described in the +ADDRESS_REWRITING_README +document. Optionally, the cleanup(8) server can be configured to +do light-weight content inspection with regular expressions as +described in the BUILTIN_FILTER_README document. The cleanup(8) +server places the result as a single file into the incoming queue, +and notifies the queue manager (see next section) of the arrival +of new mail.
+ +
The trivial-rewrite(8) server rewrites addresses to the +standard "user@fully.qualified.domain" form, as described in the +ADDRESS_REWRITING_README document. Postfix currently does not +implement a rewriting language, but a lot can be done via table +lookups and, if need be, regular expressions.
+ +

+ +

How Postfix delivers mail

+ +

Once a message has reached the incoming queue the next step is +to deliver it. The figure shows the main components of the Postfix +mail delivery apparatus. Names followed by a number are Postfix +commands or server programs, while unnumbered names inside shaded +areas represent Postfix queues.

+ +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
trivial-
rewrite(8) + smtp(8) -> Network
/

+^
| | v +

- lmtp(8) -> Network
/
incoming -> active -> qmgr(8) --- local(8) -> File, command

+^
| | v +

\
- virtual(8) -> File
deferred \
+ + pipe(8) -> Command
+ +

+ +

The queue manager (the qmgr(8) server process in the +figure) is the heart of Postfix mail delivery. It contacts the +smtp(8), lmtp(8), local(8), virtual(8), pipe(8), discard(8) or +error(8) delivery agents, and sends a delivery request for one +or more recipient addresses. The discard(8) and error(8) delivery +agents are special: they discard or bounce all mail, and are not +shown in the figure above.
+ +
The queue manager maintains a small active queue with the +messages that it has opened for delivery. The active queue acts as +a limited window on potentially large incoming or deferred queues. +The limited active queue prevents the queue manager from running +out of memory under heavy load.
+ +
The queue manager maintains a separate deferred queue for mail +that cannot be delivered, so that a large mail backlog will not +slow down normal queue accesses. The queue manager's strategy for +delayed mail delivery attempts is described in the QSHAPE_README +and TUNING_README documents.
+ +
The trivial-rewrite(8) server resolves each recipient +address according to its local or remote address class, as defined +in the ADDRESS_CLASS_README document. Additional routing information +can be specified with the optional transport(5) table. The +trivial-rewrite(8) server optionally queries the relocated(5) table +for recipients whose address has changed; mail for such recipients is +returned to the sender with an explanation.
+ +
The smtp(8) client looks up a list of mail exchangers for +the destination host, sorts the list by preference, and tries each +server in turn until it finds a server that responds. It then +encapsulates the sender, recipient and message content as required +by the SMTP protocol; this includes conversion of 8-bit MIME to +7-bit encoding.
+ +
The lmtp(8) client speaks a protocol similar to SMTP that +is optimized for delivery to mailbox servers such as Cyrus. The +advantage of this setup is that one Postfix machine can feed multiple +mailbox servers over LMTP. The opposite is true as well: one +mailbox server can be fed over LMTP by multiple Postfix machines. +
+ +
The local(8) delivery agent understands UNIX-style mailboxes, +qmail-compatible maildir files, Sendmail-style system-wide aliases(5) +databases, and Sendmail-style per-user .forward files. Multiple +local delivery agents can be run in parallel, but parallel delivery +to the same user is usually limited.
+ +
The local(8) delivery agent has hooks for alternative forms of +local delivery: you can configure it to deliver to mailbox files +in user home directories, you can configure it to delegate mailbox +delivery to an external command such as procmail, or you can delegate +delivery to a different Postfix delivery agent.
+ +
The virtual(8) delivery agent is a bare-bones delivery +agent that delivers to UNIX-style mailbox or qmail-style maildir +files only. This delivery agent can deliver mail for multiple +domains, which makes it especially suitable for hosting lots of +small domains on a single machine. This is described in the +VIRTUAL_README document.
+ +
The pipe(8) mailer is the outbound interface to other mail +processing systems (the Postfix sendmail(1) command being the +inbound interface). The interface is UNIX compatible: it provides +information on the command line and on the standard input stream, +and expects a process exit status code as defined in <sysexits.h>. +Examples of delivery via the pipe(8) mailer are in the MAILDROP_README +and UUCP_README documents. + +

+ +

Postfix behind the scenes

+ +

The previous sections gave an overview of how Postfix server +processes send and receive mail. These server processes rely on +other server processes that do things behind the scenes. The text +below attempts to visualize each service in its own context. As +before, names followed by a number are Postfix commands or server +programs, while unnumbered names inside shaded areas represent +Postfix queues.

+ +

The resident master(8) server is the supervisor that keeps +an eye on the well-being of the Postfix mail system. It is typically +started at system boot time with the "postfix start" command, and +keeps running until the system goes down. The master(8) server is +responsible for starting Postfix server processes to receive and +deliver mail, and for restarting servers that terminate prematurely +because of some problem. The master(8) server is also responsible +for enforcing the server process count limits as specified in the +master.cf configuration file. The picture below gives the +program hierarchy when Postfix is started up. Only some of the mail +handling daemon processes are shown.

+ + + + + + + + + + + + + + + + + +

+postfix(1)

|
|

+postfix-script(1)

	/
/

+|
|

\
	\

postsuper(1)

master(8)

postlog(1)

	/
/

+|
|

\
	\

smtpd(8)

qmgr(8)

local(8)

+ +

The anvil(8) server implements client connection and +request rate +limiting for all smtpd(8) servers. The TUNING_README document +provides guidance for dealing with mis-behaving SMTP clients. The +anvil(8) service is available in Postfix version 2.2 and later. +
+ + + + + +
Network ->
smtpd(8)

<-> +
anvil(8)

+
+ +

The bounce(8), defer(8) and trace(8) services each maintain +their own queue directory trees with per-message logfiles. Postfix +uses this information when sending "failed", "delayed" or "success" +delivery status notifications to the sender.

+ +

The trace(8) service also implements support for the Postfix +"sendmail +-bv" and "sendmail -v" commands which produce reports about how +Postfix delivers mail, and is available with Postfix version 2.1 +and later. See DEBUG_README + for examples.

+ + + + + + + + + + + + + +

cleanup(8) -> qmgr(8)
Postfix
queue -> Delivery
agents

^
| | v +| v

(Non-)
delivery
notice <- bounce(8)
defer(8)
trace(8) <- Queue +id,
recipient,
status

^
| +| v

+Per-
message
logfiles

+ +

The flush(8) servers maintain per-destination logs and +implement both ETRN and "sendmail -qRdestination", as described +in the ETRN_README document. This moves selected queue files from +the deferred queue back to the incoming queue and requests their +delivery. The flush(8) service is available with Postfix version +1.0 and later.

+ + + + + + + + + + + + + +

+ incoming
^ +
deferred +

^
|

smtpd(8)
sendmail(1)
+postqueue(1) - +Destination
to flush -> flush(8) <- + Deferred
destination,
queue +id - +Delivery
agents,
qmgr(8)

^
| +| v

Per-dest-
+ination
logs

+ +

The proxymap(8) servers provide read-only and read-write +table lookup +service to Postfix processes. This overcomes chroot restrictions, +reduces the number of open lookup tables by sharing one open +table among multiple processes, and implements single-updater +tables.
+ +
The scache(8) server maintains the connection cache for +the Postfix smtp(8) client. When connection caching is enabled for +selected destinations, the smtp(8) client does not disconnect +immediately after a mail transaction, but gives the connection to +the connection cache server which keeps the connection open for a +limited amount of time. The smtp(8) client continues with some +other mail delivery request. Meanwhile, any smtp(8) process can +ask the scache(8) server for that cached connection and reuse it +for mail delivery. As a safety measure, Postfix limits the number +of times that a connection may be reused.
+ +
When delivering mail to a destination with multiple mail servers, +connection caching can help to skip over a non-responding server, +and thus dramatically speed up delivery. SMTP connection caching +is available in Postfix version 2.2 and later. More information +about this feature is in the CONNECTION_CACHE_README document.
+ + + + + + + + + + + + + + + +
/-- smtp(8) +--> Internet
qmgr(8) | | | | v
\-- smtp(8) +
^ |
scache(8)
+ +
A Postfix smtp(8) client can reuse a TLS-encrypted connection +(with "smtp_tls_connection_reuse = yes"). This can greatly reduce +the overhead of connection setup and improves message delivery +rates. After a Postfix smtp(8) client connects to a remote SMTP +server and sends plaintext EHLO and STARTTLS commands, the smtp(8) +client inserts a tlsproxy(8) process into the connection as shown +below.
+ +
After the mail transaction completes, the Postfix smtp(8) client +gives the smtp(8)-to-tlsproxy(8) connection to the scache(8) +server, which keeps the connection open for a limited amount of +time. The smtp(8) client continues with some other mail delivery +request. Meanwhile, any Postfix smtp(8) client can ask the scache(8) +server for that cached connection and reuse it for mail delivery. +
+ + + + + + + + + + + + + + + +
/-- smtp(8) +--> tlsproxy(8) + --> Internet
qmgr(8) | | | | v
\-- smtp(8) +
^ |
scache(8)
+ +
The showq(8) servers list the Postfix queue status. This +is the queue listing service that does the work for the mailq(1) +and postqueue(1) commands.
+ + + + + +
Output <- mailq(1)
+ + post-
queue(1)
+ <- showq(8) <- Postfix
queue +
+ +
The spawn(8) servers run non-Postfix commands on request, +with the client connected via socket or FIFO to the command's +standard input, output and error streams. You can find examples of +its use in the SMTPD_POLICY_README document.
+ +

The tlsmgr(8) server runs when TLS (Transport Layer +Security, formerly known as SSL) is turned on in the Postfix smtp(8) +client or smtpd(8) server. This process has two duties:

+ +

Maintain the pseudo-random number generator (PRNG) that +is used to seed the TLS engines in Postfix smtp(8) client or smtpd(8) +server processes. The state of this PRNG is periodically saved to +a file, and is read when tlsmgr(8) starts up.
+ +
Maintain the optional Postfix smtp(8) client or smtpd(8) +server caches with TLS session keys. Saved keys can improve +performance by reducing the amount of computation at the start of +a TLS session.
+ +

+ +

TLS support is available in Postfix version 2.2 and later. +Information about the Postfix TLS implementation is in the TLS_README +document.

+ + + + + + + + + + + +

Network-> smtpd(8) +<---seed--- <-session-> tlsmgr(8) ---seed---> <-session-> + smtp(8)
+ ->Network

+	/
/

|
|

+ +

\
	\

+smtpd
session
cache PRNG
state
file smtp
session
cache

+ + +

The verify(8) server verifies that a sender or recipient +address is deliverable before the smtpd(8) server accepts it. The +verify(8) server queries a cache with address verification results. +If a result is not found, the verify(8) server injects a probe +message into the Postfix queue and processes the status update from +a delivery agent or queue manager. +This process is described in the ADDRESS_VERIFICATION_README +document. The verify(8) service is available with Postfix version +2.1 and later.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+					+ `->`	probe message	`->`	+ Postfix mail queue

Network	`->` +	smtpd(8)	`<->`	verify(8) +
								`\|` `v`
					`<-` +	probe + status	+ `<-`	Postfix delivery agents +	`->` + Local `->` Network
+				+ `^` `\|` `v`


				Address + verification cache

+ +

The postscreen(8) server can be put "in front" of Postfix +smtpd(8) processes. Its purpose is to accept connections from the +network and to decide what SMTP clients are allowed to talk to +Postfix. According to the 2008 MessageLabs annual report, 81% of +all email was spam, and 90% of that was sent by botnets; by 2010, +those numbers were 92% and 95%, respectively. While postscreen(8) +keeps the zombies away, more smtpd(8) processes remain available +for legitimate clients.

+ +

postscreen(8) maintains a temporary allowlist for clients that +pass its tests; by allowing allowlisted clients to skip tests, +postscreen(8) minimizes its impact on legitimate email traffic. +

+ +

The postscreen(8) server is available with Postfix 2.8 and +later. To keep the implementation simple, postscreen(8) delegates +DNS allow/denylist lookups to dnsblog(8) server processes, and +delegates TLS encryption/decryption to tlsproxy(8) server processes. +This delegation is invisible to the remote SMTP client.

+ + + + + + + + + + + + + + + + + + + + + + + + +

		zombie
			`\`
zombie	`-`	tlsproxy(8)	`-`	+		`-`	smtpd(8)
			`\`		`/`
		other	`+---`	+postscreen(8)
			`/`		`\`
		other	+ `-`			+ `-`	smtpd(8) +
			`/`
		zombie

+ +

The postlogd(8) server provides an alternative to syslog +logging, which remains the default. This feature is available with +Postfix version 3.4 or later, and supports the following modes: +
+ + +
- Logging to file, which addresses a usability problem with +MacOS, and eliminates information loss caused by systemd rate limits. +
  + + + + + + + + + +
  +commands
  or daemons
  -> +postlogd(8) -> /path/to/file +
  
  + +
- Logging to stdout, which eliminates a syslog dependency +when Postfix runs inside a container.
  + + + + + + + + + +
  +commands
  or daemons stdout inherited
  from "postfix +start-fg"
  -> +postlogd(8) ->
  
  + +
+ +
See MAILLOG_README for details and limitations.
+ +

+ +

Postfix support commands

+ +

The Postfix architecture overview ends with a summary of +command-line utilities for day-to-day use of the Postfix mail +system. Besides the Sendmail-compatible sendmail(1), mailq(1), and +newaliases(1) commands, the Postfix system comes with it own +collection of command-line utilities. For consistency, these are +all named postsomething.

+ +

The postfix(1) command controls the operation of the mail +system. It is the interface for starting, stopping, and restarting +the mail system, as well as for some other administrative operations. +This command is reserved to the super-user.
+ +
The postalias(1) command maintains Postfix aliases(5) type +databases. This is the program that does the work for the +newaliases(1) command.
+ +
The postcat(1) command displays the contents of Postfix +queue files. This is a limited, preliminary utility. This program +is likely to be superseded by something more powerful that can also +edit Postfix queue files.
+ +
The postconf(1) command displays or updates Postfix main.cf +parameters and displays system dependent information about the +supported file locking methods, and the supported types of lookup +tables.
+ +
The postdrop(1) command is the mail posting utility that +is run by the Postfix sendmail(1) command in order to deposit mail +into the maildrop queue directory.
+ +
The postkick(1) command makes some Postfix internal +communication channels available for use in, for example, shell +scripts.
+ +
The postlock(1) command provides Postfix-compatible mailbox +locking for use in, for example, shell scripts.
+ +
The postlog(1) command provides Postfix-compatible logging +for shell scripts.
+ +
The postmap(1) command maintains Postfix lookup tables +such as canonical(5), virtual(5) and others. It is a cousin of the +UNIX makemap command.
+ +
The postmulti(1) command repeats the "postfix start" etc. +command for each Postfix instance, and supports creation, deletion +etc. of Postfix instances. For a tutorial, see MULTI_INSTANCE_README. +
+ +
The postqueue(1) command is the privileged command that +is run by Postfix sendmail(1) and mailq(1) in order to flush or +list the +mail queue.
+ +
The postsuper(1) command maintains the Postfix queue. It +removes old temporary files, and moves queue files into the right +directory after a change in the hashing depth of queue directories. +This command is run at mail system startup time and when Postfix +is restarted.
+ +

+ + + + -- cgit v1.2.3

	`/--`	smtp(8)	`+-->`		Internet
qmgr(8)		`\| \| \| \| v`
	`\--`		smtp(8)	+
				`^ \|`
		scache(8)

+commands or daemons				stdout inherited from "postfix +start-fg"
	`->`	+postlogd(8)	`->`