path: root/README_FILES/ADDRESS_VERIFICATION_README

PPoossttffiixx AAddddrreessss VVeerriiffiiccaattiioonn HHoowwttoo

-------------------------------------------------------------------------------

WWAARRNNIINNGG

Recipient address verification may cause an increased load on down-stream
servers in the case of a dictionary attack or a flood of backscatter bounces.
Sender address verification may cause your site to be blacklisted by some
providers. See also the "Limitations" section below for more.

WWhhaatt PPoossttffiixx aaddddrreessss vveerriiffiiccaattiioonn ccaann ddoo ffoorr yyoouu

Address verification is a feature that allows the Postfix SMTP server to block
a sender (MAIL FROM) or recipient (RCPT TO) address until the address has been
verified to be deliverable.

The technique has obvious uses to reject junk mail with an unreplyable sender
address.

The technique is also useful to block mail for undeliverable recipients, for
example on a mail relay host that does not have a list of all the valid
recipient addresses. This prevents undeliverable junk mail from entering the
queue, so that Postfix doesn't have to waste resources trying to send MAILER-
DAEMON messages back.

This feature is available in Postfix version 2.1 and later.

Topics covered in this document:

  * How address verification works
  * Limitations of address verification
  * Recipient address verification
  * Sender address verification for mail from frequently forged domains
  * Sender address verification for all email
  * Address verification database
  * Managing the address verification database
  * Controlling the routing of address verification probes
  * Forced probe routing examples
  * Limitations of forced probe routing

HHooww aaddddrreessss vveerriiffiiccaattiioonn wwoorrkkss

A Postfix MTA verifies a sender or recipient address by probing the preferred
MTAs for that address, without actually delivering mail. The preferred MTAs
could include the Postfix MTA itself, or some remote MTAs (SMTP interruptus).
Probe messages are like normal mail, except that they are never delivered,
deferred or bounced; probe messages are always discarded.

                                             probe     Postfix
                                            message ->   mail
                Postfix        Postfix   ->             queue
    Internet ->  SMTP   <->     verify
                server          server                      |
                                                            v

                                         <-            Postfix
                                             probe  <- delivery -> Local
                                            status      agents  -> Remote
                                    ^
                                    |
                                    v

                              Address
                            verification
                              database

With Postfix address verification turned on, normal mail will suffer only a
short delay of up to 6 seconds while an address is being verified for the first
time. Once an address status is known, the status is cached and Postfix replies
immediately.

When verification takes too long the Postfix SMTP server defers the sender or
recipient address with a 450 reply. Normal mail clients will connect again
after some delay. The address verification delay is configurable with the
main.cf address_verify_poll_count and address_verify_poll_delay parameters. See
postconf(5) for details.

LLiimmiittaattiioonnss ooff aaddddrreessss vveerriiffiiccaattiioonn

  * Postfix assumes that a remote SMTP server will reject unknown addresses in
    reply to the RCPT TO command. However, some sites report this in reply to
    the DATA command. For such sites you may configure a workaround with the
    smtp_address_verify_target parameter (Postfix 3.0 and later).

  * When verifying a remote address, Postfix probes the preferred MTAs for that
    address, without actually delivering mail. If a preferred MTA accepts the
    address, then Postfix assumes that the address is deliverable. In reality,
    mail for a remote address can bounce AFTER a preferred MTA accepts the
    recipient address, or AFTER a preferred MTA accepts the message content.

  * Some sites may blacklist you when you are probing them too often (a probe
    is an SMTP session that does not deliver mail), or when you are probing
    them too often for a non-existent address. This is one reason why you
    should use sender address verification sparingly, if at all, when your site
    receives lots of email.

  * Normally, address verification probe messages follow the same path as
    regular mail. However, some sites send mail to the Internet via an
    intermediate relayhost; this breaks address verification. See below,
    section "Controlling the routing of address verification probes", for how
    to override mail routing and for possible limitations when you have to do
    this.

  * Postfix assumes that an address is undeliverable when a preferred MTA for
    the address rejects the probe, regardless of the reason for rejection
    (client rejected, HELO rejected, MAIL FROM rejected, etc.). Thus, Postfix
    rejects an address when a preferred MTA for that address rejects mail from
    your machine for any reason. This is not a limitation, but it is mentioned
    here just in case people believe that it is a limitation.

  * Unfortunately, some sites do not reject unknown addresses in reply to the
    RCPT TO or DATA command, but instead report a delivery failure in response
    to end of DATA after a message is transferred. Postfix address verification
    does not work with such sites.

  * By default, Postfix probe messages have a sender address "double-
    bounce@$myorigin" (with Postfix versions before 2.5, the default is
    "postmaster@$myorigin"). This is SAFE because the Postfix SMTP server does
    not reject mail for this address.

    You can change the probe sender address into the null address
    ("address_verify_sender ="). This is UNSAFE because address probes will
    fail with mis-configured sites that reject MAIL FROM: <>, while probes from
    "double-bounce@$myorigin" would succeed.

  * The downside of using a non-empty sender address is that the address may
    end op on spammer mailing lists. Although Postfix always discards mail to
    the double-bounce address, this still results in wasted network bandwidth
    and server capacity. To defeat address harvesting, Postfix 2.9 and later
    support time-dependent sender addresses when you specify a non-zero
    address_verify_sender_ttl value.

RReecciippiieenntt aaddddrreessss vveerriiffiiccaattiioonn

As mentioned earlier, recipient address verification is useful to block mail
for undeliverable recipients on a mail relay host that does not have a list of
all valid recipient addresses. This can help to prevent the mail queue from
filling up with MAILER-DAEMON messages.

Recipient address verification is relatively straightforward and there are no
surprises. If a recipient probe fails, then Postfix rejects mail for the
recipient address. If a recipient probe succeeds, then Postfix accepts mail for
the recipient address. However, recipient address verification probes can
increase the load on down-stream MTAs when you're being flooded by backscatter
bounces, or when some spammer is mounting a dictionary attack.

By default, address verification results are saved in a persistent database
(Postfix version 2.7 and later; with earlier versions, specify the database in
main.cf as described later). The persistent database helps to avoid probing the
same address repeatedly.

    /etc/postfix/main.cf:
        smtpd_recipient_restrictions =
            permit_mynetworks
            # reject_unauth_destination is not needed here if the mail
            # relay policy is specified under smtpd_relay_restrictions
            # (available with Postfix 2.10 and later).
            reject_unauth_destination
            ...
            reject_unknown_recipient_domain
            reject_unverified_recipient
            ...
        # Postfix 2.6 and later privacy feature.
        # unverified_recipient_reject_reason = Address lookup failed

        # Postfix 3.2 and earlier workaround.
        # Do not set enable_original_recipient=no. This prevents Postfix
        # from saving the recipient address verification result under
        # the original address, when the address verification probe
        # message goes through address aliasing or canonical mapping.

The "reject_unknown_recipient_domain" restriction blocks mail for non-existent
domains. Putting this before "reject_unverified_recipient" avoids the overhead
of generating unnecessary probe messages.

The unverified_recipient_reject_code parameter (default 450) specifies the
numerical Postfix SMTP server reply code when a recipient address is known to
bounce. Change this setting into 550 when you trust Postfix's judgments.

The following features are available in Postfix 2.6 and later.

The unverified_recipient_defer_code parameter (default 450) specifies the
numerical Postfix SMTP server reply code when a recipient address probe fails
with some temporary error. Some sites insist on changing this into 250. NOTE:
This change turns MX servers into backscatter sources when the load is high.

The unverified_recipient_reject_reason parameter (default: empty) specifies
fixed text that Postfix will send to remote SMTP clients, instead of sending
actual address verification details. Do not specify the SMTP status code or
enhanced status code.

The unverified_recipient_tempfail_action parameter (default: defer_if_permit)
specifies the Postfix SMTP server action when a recipient address verification
probe fails with some temporary error.

SSeennddeerr aaddddrreessss vveerriiffiiccaattiioonn ffoorr mmaaiill ffrroomm ffrreeqquueennttllyy ffoorrggeedd ddoommaaiinnss

Only for very small sites, it is relatively safe to turn on sender address
verification for specific domains that often appear in forged email.

    /etc/postfix/main.cf:
        smtpd_sender_restrictions = hash:/etc/postfix/sender_access
        unverified_sender_reject_code = 550
        # Postfix 2.6 and later.
        # unverified_sender_defer_code = 250

        # Default setting for Postfix 2.7 and later.
        # Note 1: Be sure to read the "Caching" section below!
        # Note 2: Avoid hash files here. Use btree or lmdb instead.
        address_verify_map = btree:/var/lib/postfix/verify

        # Postfix 3.2 and earlier workaround.
        # Do not set enable_original_recipient=no. This prevents Postfix
        # from saving the sender address verification result under the
        # original address, when the address verification probe message
        # goes through address aliasing or canonical mapping.

    /etc/postfix/sender_access:
        # Don't do this when you handle lots of email.
        aol.com     reject_unverified_sender
        hotmail.com reject_unverified_sender
        bigfoot.com reject_unverified_sender
        ... etcetera ...

At some point in cyberspace/time, a list of frequently forged MAIL FROM domains
could be found at http://www.monkeys.com/anti-spam/filtering/sender-domain-
validate.in.

NOTE: One of the first things you might want to do is to turn on sender address
verification for all your own domains.

SSeennddeerr aaddddrreessss vveerriiffiiccaattiioonn ffoorr aallll eemmaaiill

Unfortunately, sender address verification cannot simply be turned on for all
email - you are likely to lose legitimate mail from mis-configured systems. You
almost certainly will have to set up white lists for specific addresses, or
even for entire domains.

To find out how sender address verification would affect your mail, specify
"warn_if_reject reject_unverified_sender" so that you can see what mail would
be blocked:

    /etc/postfix/main.cf:
        smtpd_sender_restrictions =
            permit_mynetworks
            ...
            check_sender_access hash:/etc/postfix/sender_access
            reject_unknown_sender_domain
            warn_if_reject reject_unverified_sender
            ...
        # Postfix 2.6 and later.
        # unverified_sender_reject_reason = Address verification failed

        # Default setting for Postfix 2.7 and later.
        # Note 1: Be sure to read the "Caching" section below!
        # Note 2: Avoid hash files here. Use btree or lmdb instead.
        address_verify_map = btree:/var/lib/postfix/verify

This is also a good way to populate your cache with address verification
results before you start to actually reject mail.

The sender_access restriction is needed to whitelist domains or addresses that
are known to be OK. Although Postfix will not mark a known-to-be-good address
as bad after a probe fails, it is better to be safe than sorry.

NOTE: You will have to whitelist sites such as securityfocus.com and other
sites that operate mailing lists that use a different sender address for each
posting (VERP). Such addresses pollute the address verification cache quickly,
and generate unnecessary sender verification probes.

    /etc/postfix/sender_access
        securityfocus.com OK
        ...

The "reject_unknown_sender_domain" restriction blocks mail from non-existent
domains. Putting this before "reject_unverified_sender" avoids the overhead of
generating unnecessary probe messages.

The unverified_sender_reject_code parameter (default 450) specifies the
numerical Postfix server reply code when a sender address is known to bounce.
Change this setting into 550 when you trust Postfix's judgments.

The following features are available in Postfix 2.6 and later.

The unverified_sender_defer_code parameter (default 450) specifies the
numerical Postfix SMTP server reply code when a sender address verification
probe fails with some temporary error. Specify a valid 2xx or 4xx code.

The unverified_sender_reject_reason parameter (default: empty) specifies fixed
text that Postfix will send to remote SMTP clients, instead of sending actual
address verification details. Do not specify the SMTP status code or enhanced
status code.

The unverified_sender_tempfail_action parameter (default: defer_if_permit)
specifies the Postfix SMTP server action when a sender address verification
probe fails with some temporary error.

AAddddrreessss vveerriiffiiccaattiioonn ddaattaabbaassee

To improve performance, the Postfix verify(8) daemon can save address
verification results to a persistent database. This is enabled by default with
Postfix 2.7 and later. The address_verify_map (NOTE: singular) configuration
parameter specifies persistent storage for sender or recipient address
verification results. If you specify an empty value, all address verification
results are lost after "postfix reload" or "postfix stop".

    # Example 1: Default setting for Postfix 2.7 and later.
    # Note: avoid hash files here. Use btree or lmdb instead.
    /etc/postfix/main.cf:
        address_verify_map = btree:$data_directory/verify_cache

    # Example 2: Shared persistent lmdb: cache (Postfix 2.11 or later).
    # Disable automatic cache cleanup in all Postfix instances except
    # for one instance that will be responsible for cache cleanup.
    /etc/postfix/main.cf:
        address_verify_map = lmdb:$data_directory/verify_cache
        # address_verify_cache_cleanup_interval = 0

    # Example 3: Shared persistent btree: cache (Postfix 2.9 or later).
    # Disable automatic cache cleanup in all Postfix instances except
    # for one instance that will be responsible for cache cleanup.
    /etc/postfix/main.cf:
        address_verify_map = proxy:btree:$data_directory/verify_cache
        # address_verify_cache_cleanup_interval = 0

    # Example 4: Shared memory cache (requires Postfix 2.9 or later).
    # Disable automatic cache cleanup in all Postfix instances.
    # See memcache_table(5) for details.
    /etc/postfix/main.cf:
        address_verify_map = memcache:/etc/postfix/verify-memcache.cf
        address_verify_cache_cleanup_interval = 0

    # Example 5: Default setting for Postfix 2.6 and earlier.
    # This uses non-persistent storage only.
    /etc/postfix/main.cf:
        address_verify_map =

NOTE 1: The database file should be stored under a Postfix-owned directory,
such as $data_directory.

    As of version 2.5, Postfix no longer uses root privileges when opening this
    file. To maintain backwards compatibility, an attempt to open the file
    under a non-Postfix directory is redirected to the Postfix-owned
    data_directory, and a warning is logged. If you wish to continue using a
    pre-existing database file, change its file ownership to the account
    specified with the mail_owner parameter, and either move the file to the
    data_directory, or move it to some other Postfix-owned directory.

NOTE 2: Do not put this file in a file system that may run out of space. When
the address verification table gets corrupted the world comes to an end and YOU
will have to MANUALLY fix things as described in the next section. Meanwhile,
you will not receive mail via SMTP.

NOTE 3: The verify(8) daemon will create a new database when none exists. It
will open or create the file before entering the chroot jail.

MMaannaaggiinngg tthhee aaddddrreessss vveerriiffiiccaattiioonn ddaattaabbaassee

The verify(8) manual page describes parameters that control how long address
verification results are cached before they need to be refreshed, and how long
results can remain "unrefreshed" before they expire. Postfix uses different
controls for positive results (address was accepted) and for negative results
(address was rejected, or address verification failed for some other reason).

The verify(8) daemon will periodically remove expired entries from the address
verification database, and log the number of entries retained and dropped
(Postfix versions 2.7 and later). A cleanup run is logged as "partial" when the
daemon terminates early because of "postfix reload, "postfix stop", or because
the daemon received no requests for $max_idle seconds. Postfix versions 2.6 and
earlier do not implement automatic address verification database cleanup.
There, the database is managed manually as described next.

When the address verification database file becomes too big, or when it becomes
corrupted, the solution is to manually rename or delete (NOT: truncate) the
file and run "postfix reload". The verify(8) daemon will then create a new
database file.

CCoonnttrroolllliinngg tthhee rroouuttiinngg ooff aaddddrreessss vveerriiffiiccaattiioonn pprroobbeess

By default, Postfix sends address verification probe messages via the same
route as regular mail, because that normally produces the most accurate result.
It's no good to verify a local address by connecting to your own SMTP port;
that just triggers all kinds of mailer loop alarms. The same is true for any
destination that your machine is best MX host for: hidden domains, virtual
domains, etc.

However, some sites have a complex infrastructure where mail is not sent
directly to the Internet, but is instead given to an intermediate relayhost.
This is a problem for address verification, because remote Internet addresses
can be verified only when Postfix can access remote destinations directly.

For this reason, Postfix allows you to override the routing parameters when it
delivers an address verification probe message.

First, the address_verify_relayhost parameter allows you to override the
relayhost setting, and the address_verify_transport_maps parameter allows you
to override the transport_maps setting. The
address_verify_sender_dependent_relayhost_maps parameter does the same for
sender-dependent relayhost selection.

Second, each address class is given its own address verification version of the
message delivery transport, as shown in the table below. Address classes are
defined in the ADDRESS_CLASS_README file.

     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    |DDoommaaiinn lliisstt            |RReegguullaarr ttrraannssppoorrtt|VVeerriiffyy ttrraannssppoorrtt                |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
    |mydestination          |local_transport  |address_verify_local_transport  |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
    |virtual_alias_domains  |(not applicable) |(not applicable)                |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
    |virtual_mailbox_domains|virtual_transport|address_verify_virtual_transport|
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
    |relay_domains          |relay_transport  |address_verify_relay_transport  |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
    |(not applicable)       |default_transport|address_verify_default_transport|
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |

By default, the parameters that control delivery of address probes have the
same value as the parameters that control normal mail delivery.

FFoorrcceedd pprroobbee rroouuttiinngg eexxaammpplleess

In a typical scenario one would override the relayhost setting for address
verification probes and leave everything else alone:

    /etc/postfix/main.cf:
        relayhost = $mydomain
        address_verify_relayhost =
        ...

Sites behind a network address translation box might have to use a different
SMTP client that sends the correct hostname information:

    /etc/postfix/main.cf:
        relayhost = $mydomain
        address_verify_relayhost =
        address_verify_default_transport = direct_smtp

    /etc/postfix/master.cf:
        direct_smtp .. .. .. ..  .. .. .. .. .. smtp
            -o smtp_helo_name=nat.box.tld

LLiimmiittaattiioonnss ooff ffoorrcceedd pprroobbee rroouuttiinngg

Inconsistencies can happen when probe messages don't follow the same path as
regular mail. For example, a message can be accepted when it follows the
regular route while an otherwise identical probe message is rejected when it
follows the forced route. The opposite can happen, too, but is less likely.