path: root/README_FILES/OVERVIEW

PPoossttffiixx AArrcchhiitteeccttuurree OOvveerrvviieeww

-------------------------------------------------------------------------------

IInnttrroodduuccttiioonn

This document presents an overview of the Postfix architecture, and provides
pointers to descriptions of every Postfix command or server program. The text
gives the general context in which each command or server program is used, and
provides pointers to documents with specific usage examples and background
information.

Topics covered by this document:

  * How Postfix receives mail
  * How Postfix delivers mail
  * Postfix behind the scenes
  * Postfix support commands

HHooww PPoossttffiixx rreecceeiivveess mmaaiill

When a message enters the Postfix mail system, the first stop on the inside is
the incoming queue. The figure below shows the main processes that are involved
with new mail. Names followed by a number are Postfix commands or server
programs, while unnumbered names inside shaded areas represent Postfix queues.

                               trivial-
                              rewrite(8)

    Network ->   smtpd(8)

                                    ^ |
                           \        | v

    Network ->   qmqpd(8)  -> cleanup(8)  -> incoming

                           /

                pickup(8)  <-   maildrop

                                     ^
                                     |

    Local   -> sendmail(1) -> postdrop(1)

  * Network mail enters Postfix via the smtpd(8) or qmqpd(8) servers. These
    servers remove the SMTP or QMQP protocol encapsulation, enforce some sanity
    checks to protect Postfix, and give the sender, recipients and message
    content to the cleanup(8) server. The smtpd(8) server can be configured to
    block unwanted mail, as described in the SMTPD_ACCESS_README document.

  * Local submissions are received with the Postfix sendmail(1) compatibility
    command, and are queued in the maildrop queue by the privileged postdrop(1)
    command. This arrangement even works while the Postfix mail system is not
    running. The local pickup(8) server picks up local submissions, enforces
    some sanity checks to protect Postfix, and gives the sender, recipients and
    message content to the cleanup(8) server.

  * Mail from internal sources is given directly to the cleanup(8) server.
    These sources are not shown in the figure, and include: mail that is
    forwarded by the local(8) delivery agent (see next section), messages that
    are returned to the sender by the bounce(8) server (see second-next
    section), and postmaster notifications about problems with Postfix.

  * The cleanup(8) server implements the final processing stage before mail is
    queued. It adds missing From: and other message headers, and transforms
    addresses as described in the ADDRESS_REWRITING_README document.
    Optionally, the cleanup(8) server can be configured to do light-weight
    content inspection with regular expressions as described in the
    BUILTIN_FILTER_README document. The cleanup(8) server places the result as
    a single file into the incoming queue, and notifies the queue manager (see
    next section) of the arrival of new mail.

  * The trivial-rewrite(8) server rewrites addresses to the standard
    "user@fully.qualified.domain" form, as described in the
    ADDRESS_REWRITING_README document. Postfix currently does not implement a
    rewriting language, but a lot can be done via table lookups and, if need
    be, regular expressions.

HHooww PPoossttffiixx ddeelliivveerrss mmaaiill

Once a message has reached the incoming queue the next step is to deliver it.
The figure shows the main components of the Postfix mail delivery apparatus.
Names followed by a number are Postfix commands or server programs, while
unnumbered names inside shaded areas represent Postfix queues.

                             trivial-        smtp(8)  -> Network
                            rewrite(8)
                                         /

                                         -   lmtp(8)  -> Network
                                 ^ |
                                 | v   /

    incoming ->  active  ->   qmgr(8)  ---  local(8)  -> File, command

                                       \
                   ^ |
                   | v                   - virtual(8) -> File

                deferred                 \

                                             pipe(8)  -> Command

  * The queue manager (the qmgr(8) server process in the figure) is the heart
    of Postfix mail delivery. It contacts the smtp(8), lmtp(8), local(8),
    virtual(8), pipe(8), discard(8) or error(8) delivery agents, and sends a
    delivery request for one or more recipient addresses. The discard(8) and
    error(8) delivery agents are special: they discard or bounce all mail, and
    are not shown in the figure above.

    The queue manager maintains a small active queue with the messages that it
    has opened for delivery. The active queue acts as a limited window on
    potentially large incoming or deferred queues. The limited active queue
    prevents the queue manager from running out of memory under heavy load.

    The queue manager maintains a separate deferred queue for mail that cannot
    be delivered, so that a large mail backlog will not slow down normal queue
    accesses. The queue manager's strategy for delayed mail delivery attempts
    is described in the QSHAPE_README and TUNING_README documents.

  * The trivial-rewrite(8) server resolves each recipient address according to
    its local or remote address class, as defined in the ADDRESS_CLASS_README
    document. Additional routing information can be specified with the optional
    transport(5) table. The trivial-rewrite(8) server optionally queries the
    relocated(5) table for recipients whose address has changed; mail for such
    recipients is returned to the sender with an explanation.

  * The smtp(8) client looks up a list of mail exchangers for the destination
    host, sorts the list by preference, and tries each server in turn until it
    finds a server that responds. It then encapsulates the sender, recipient
    and message content as required by the SMTP protocol; this includes
    conversion of 8-bit MIME to 7-bit encoding.

  * The lmtp(8) client speaks a protocol similar to SMTP that is optimized for
    delivery to mailbox servers such as Cyrus. The advantage of this setup is
    that one Postfix machine can feed multiple mailbox servers over LMTP. The
    opposite is true as well: one mailbox server can be fed over LMTP by
    multiple Postfix machines.

  * The local(8) delivery agent understands UNIX-style mailboxes, qmail-
    compatible maildir files, Sendmail-style system-wide aliases(5) databases,
    and Sendmail-style per-user .forward files. Multiple local delivery agents
    can be run in parallel, but parallel delivery to the same user is usually
    limited.

    The local(8) delivery agent has hooks for alternative forms of local
    delivery: you can configure it to deliver to mailbox files in user home
    directories, you can configure it to delegate mailbox delivery to an
    external command such as procmail, or you can delegate delivery to a
    different Postfix delivery agent.

  * The virtual(8) delivery agent is a bare-bones delivery agent that delivers
    to UNIX-style mailbox or qmail-style maildir files only. This delivery
    agent can deliver mail for multiple domains, which makes it especially
    suitable for hosting lots of small domains on a single machine. This is
    described in the VIRTUAL_README document.

  * The pipe(8) mailer is the outbound interface to other mail processing
    systems (the Postfix sendmail(1) command being the inbound interface). The
    interface is UNIX compatible: it provides information on the command line
    and on the standard input stream, and expects a process exit status code as
    defined in <sysexits.h>. Examples of delivery via the pipe(8) mailer are in
    the MAILDROP_README and UUCP_README documents.

PPoossttffiixx bbeehhiinndd tthhee sscceenneess

The previous sections gave an overview of how Postfix server processes send and
receive mail. These server processes rely on other server processes that do
things behind the scenes. The text below attempts to visualize each service in
its own context. As before, names followed by a number are Postfix commands or
server programs, while unnumbered names inside shaded areas represent Postfix
queues.

  * The resident master(8) server is the supervisor that keeps an eye on the
    well-being of the Postfix mail system. It is typically started at system
    boot time with the "postfix start" command, and keeps running until the
    system goes down. The master(8) server is responsible for starting Postfix
    server processes to receive and deliver mail, and for restarting servers
    that terminate prematurely because of some problem. The master(8) server is
    also responsible for enforcing the server process count limits as specified
    in the mmaasstteerr..ccff configuration file. The picture below gives the program
    hierarchy when Postfix is started up. Only some of the mail handling daemon
    processes are shown.

                          postfix(1)

                                 |
                                 |

                     postfix-script(1)

                   /             |     \
                                 |
                 /                       \

    postsuper(1)           master(8)       postlog(1)

                   /             |     \
                                 |
                 /                       \

       smtpd(8)             qmgr(8)         local(8)

  * The anvil(8) server implements client connection and request rate limiting
    for all smtpd(8) servers. The TUNING_README document provides guidance for
    dealing with mis-behaving SMTP clients. The anvil(8) service is available
    in Postfix version 2.2 and later.

    Network -> smtpd(8) <-> anvil(8)

  * The bounce(8), defer(8) and trace(8) services each maintain their own queue
    directory trees with per-message logfiles. Postfix uses this information
    when sending "failed", "delayed" or "success" delivery status notifications
    to the sender.

    The trace(8) service also implements support for the Postfix "sendmail -bv"
    and "sendmail -v" commands which produce reports about how Postfix delivers
    mail, and is available with Postfix version 2.1 and later. See DEBUG_README
    for examples.

                   qmgr(8)      Delivery
    cleanup(8) ->  Postfix  ->   agents
                    queue

          ^             |            |
          |             v            v

      (Non-)      bounce(8)    Queue id,
     delivery  <- defer(8)  <- recipient,
      notice      trace(8)       status

                      ^ |
                      | v

                    Per-
                   message
                  logfiles

  * The flush(8) servers maintain per-destination logs and implement both ETRN
    and "sendmail -qRdestination", as described in the ETRN_README document.
    This moves selected queue files from the deferred queue back to the
    incoming queue and requests their delivery. The flush(8) service is
    available with Postfix version 1.0 and later.

                                  incoming
                                     ^
                                  deferred

                                        ^
                                        |

      smtpd(8)     Destination                   Deferred     Delivery
    sendmail(1)  -  to flush   -> flush(8)  <- destination, - agents,
    postqueue(1)                                 queue id     qmgr(8)

                                      ^ |
                                      | v

                                  Per-dest-
                                   ination
                                    logs

  * The proxymap(8) servers provide read-only and read-write table lookup
    service to Postfix processes. This overcomes chroot restrictions, reduces
    the number of open lookup tables by sharing one open table among multiple
    processes, and implements single-updater tables.

  * The scache(8) server maintains the connection cache for the Postfix smtp(8)
    client. When connection caching is enabled for selected destinations, the
    smtp(8) client does not disconnect immediately after a mail transaction,
    but gives the connection to the connection cache server which keeps the
    connection open for a limited amount of time. The smtp(8) client continues
    with some other mail delivery request. Meanwhile, any smtp(8) process can
    ask the scache(8) server for that cached connection and reuse it for mail
    delivery. As a safety measure, Postfix limits the number of times that a
    connection may be reused.

    When delivering mail to a destination with multiple mail servers,
    connection caching can help to skip over a non-responding server, and thus
    dramatically speed up delivery. SMTP connection caching is available in
    Postfix version 2.2 and later. More information about this feature is in
    the CONNECTION_CACHE_README document.

            /-- smtp(8) --> Internet

    qmgr(8)       |  
                  |
            \--   | smtp(8) --> Internet
                  |
                  ^          
                  |

                  scache(8)

  * The showq(8) servers list the Postfix queue status. This is the queue
    listing service that does the work for the mailq(1) and postqueue(1)
    commands.

              mailq(1)                 Postfix
    Output <-   post-   <- showq(8) <-  queue
              queue(1)

  * The spawn(8) servers run non-Postfix commands on request, with the client
    connected via socket or FIFO to the command's standard input, output and
    error streams. You can find examples of its use in the SMTPD_POLICY_README
    document.

  * The tlsmgr(8) server runs when TLS (Transport Layer Security, formerly
    known as SSL) is turned on in the Postfix smtp(8) client or smtpd(8)
    server. This process has two duties:

      o Maintain the pseudo-random number generator (PRNG) that is used to seed
        the TLS engines in Postfix smtp(8) client or smtpd(8) server processes.
        The state of this PRNG is periodically saved to a file, and is read
        when tlsmgr(8) starts up.

      o Maintain the optional Postfix smtp(8) client or smtpd(8) server caches
        with TLS session keys. Saved keys can improve performance by reducing
        the amount of computation at the start of a TLS session.

    TLS support is available in Postfix version 2.2 and later. Information
    about the Postfix TLS implementation is in the TLS_README document.

                        <---seed---            ---seed--->
    Network-> smtpd(8)              tlsmgr(8)               smtp(8)  ->Network
                        <-session->            <-session->      

                                  /       |    \
                                          |
                                /                \

                         smtpd          PRNG        smtp
                        session        state       session
                         cache          file        cache

  * The verify(8) server verifies that a sender or recipient address is
    deliverable before the smtpd(8) server accepts it. The verify(8) server
    queries a cache with address verification results. If a result is not
    found, the verify(8) server injects a probe message into the Postfix queue
    and processes the status update from a delivery agent or queue manager.
    This process is described in the ADDRESS_VERIFICATION_README document. The
    verify(8) service is available with Postfix version 2.1 and later.

                               
                                             probe     Postfix
                                            message ->   mail
                                                        queue
    Network -> smtpd(8) <->   verify(8)  ->

                                                            |
                                                            v

                                         <-  probe     Postfix  -> Local
                                            status  <- delivery -> Network
                                    ^                   agents
                                    |
                                    v
                                          

                              Address
                            verification
                               cache

  * The postscreen(8) server can be put "in front" of Postfix smtpd(8)
    processes. Its purpose is to accept connections from the network and to
    decide what SMTP clients are allowed to talk to Postfix. According to the
    2008 MessageLabs annual report, 81% of all email was spam, and 90% of that
    was sent by botnets; by 2010, those numbers were 92% and 95%, respectively.
    While postscreen(8) keeps the zombies away, more smtpd(8) processes remain
    available for legitimate clients.

    postscreen(8) maintains a temporary whitelist for clients that pass its
    tests; by allowing whitelisted clients to skip tests, postscreen(8)
    minimizes its impact on legitimate email traffic.

    The postscreen(8) server is available with Postfix 2.8 and later. To keep
    the implementation simple, postscreen(8) delegates DNS white/blacklist
    lookups to dnsblog(8) server processes, and delegates TLS encryption/
    decryption to tlsproxy(8) server processes. This delegation is invisible to
    the remote SMTP client, and is not shown in the diagram below.

    zombie

           \

    zombie -                    - smtpd(8)

             \               /

    other  --- postscreen(8)

             /               \

    other  -                    - smtpd(8)

           /

    zombie

PPoossttffiixx ssuuppppoorrtt ccoommmmaannddss

The Postfix architecture overview ends with a summary of command-line utilities
for day-to-day use of the Postfix mail system. Besides the Sendmail-compatible
sendmail(1), mailq(1), and newaliases(1) commands, the Postfix system comes
with it own collection of command-line utilities. For consistency, these are
all named postsomething.

  * The postfix(1) command controls the operation of the mail system. It is the
    interface for starting, stopping, and restarting the mail system, as well
    as for some other administrative operations. This command is reserved to
    the super-user.

  * The postalias(1) command maintains Postfix aliases(5) type databases. This
    is the program that does the work for the newaliases(1) command.

  * The postcat(1) command displays the contents of Postfix queue files. This
    is a limited, preliminary utility. This program is likely to be superseded
    by something more powerful that can also edit Postfix queue files.

  * The postconf(1) command displays or updates Postfix main.cf parameters and
    displays system dependent information about the supported file locking
    methods, and the supported types of lookup tables.

  * The postdrop(1) command is the mail posting utility that is run by the
    Postfix sendmail(1) command in order to deposit mail into the maildrop
    queue directory.

  * The postkick(1) command makes some Postfix internal communication channels
    available for use in, for example, shell scripts.

  * The postlock(1) command provides Postfix-compatible mailbox locking for use
    in, for example, shell scripts.

  * The postlog(1) command provides Postfix-compatible logging for shell
    scripts.

  * The postmap(1) command maintains Postfix lookup tables such as canonical
    (5), virtual(5) and others. It is a cousin of the UNIX makemap command.

  * The postmulti(1) command repeats the "postfix start" etc. command for each
    Postfix instance, and supports creation, deletion etc. of Postfix
    instances. For a tutorial, see MULTI_INSTANCE_README.

  * The postqueue(1) command is the privileged command that is run by Postfix
    sendmail(1) and mailq(1) in order to flush or list the mail queue.

  * The postsuper(1) command maintains the Postfix queue. It removes old
    temporary files, and moves queue files into the right directory after a
    change in the hashing depth of queue directories. This command is run at
    mail system startup time and when Postfix is restarted.