path: root/README_FILES/SMTPUTF8_README

 PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt

-------------------------------------------------------------------------------

OOvveerrvviieeww

This document describes Postfix support for Email Address Internationalization
(EAI) as defined in RFC 6531 (SMTPUTF8 extension), RFC 6532 (Internationalized
email headers) and RFC 6533 (Internationalized delivery status notifications).
Introduced with Postfix version 3.0, this fully supports UTF-8 email addresses
and UTF-8 message header values.

Topics covered in this document:

  * Building with/without SMTPUTF8 support
  * Enabling Postfix SMTPUTF8 support
  * Using Postfix SMTPUTF8 support
  * SMTPUTF8 autodetection
  * Limitations of the current implementation
  * Compatibility with pre-SMTPUTF8 environments
  * Compatibility with IDNA2003
  * Credits

BBuuiillddiinngg PPoossttffiixx wwiitthh//wwiitthhoouutt SSMMTTPPUUTTFF88 ssuuppppoorrtt

Postfix will build with SMTPUTF8 support if the ICU version >= 46 library and
header files are installed on the system. The package name varies with the OS
distribution. The table shows package names for a number of platforms at the
time this text was written.

     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    |OOSS DDiissttrriibbuuttiioonn      |PPaacckkaaggee     |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ |
    |FreeBSD, NetBSD, etc.|icu         |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ |
    |Centos, Fedora, RHEL |libicu-devel|
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ |
    |Debian, Ubuntu       |libicu-dev  |
    |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ |

To force Postfix to build without SMTPUTF8, specify:

    $ mmaakkee mmaakkeeffiilleess CCCCAARRGGSS==""--DDNNOO__EEAAII ......""

See the INSTALL document for more "make makefiles" options.

EEnnaabblliinngg PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt

There is more to SMTPUTF8 than just Postfix itself. The rest of your email
infrastructure also needs to be able to handle UTF-8 email addresses and
message header values. This includes SMTPUTF8 protocol support in SMTP-based
content filters (Amavisd), LMTP servers (Dovecot), and down-stream SMTP
servers.

Postfix SMTPUTF8 support is enabled by default, but it may be disabled as part
of a backwards-compatibility safety net (see the COMPATIBILITY_README file).

SMTPUTF8 support is enabled by setting the smtputf8_enable parameter in
main.cf:

    # ppoossttccoonnff ""ssmmttppuuttff88__eennaabbllee == yyeess""
    # ppoossttffiixx rreellooaadd

(With Postfix <= 3.1, you may also need to specify "ooppttiioonn__ggrroouupp == cclliieenntt" in
Postfix MySQL client files, to enable UTF8 support in MySQL queries. This
setting is the default as of Postfix 3.2.)

With SMTPUTF8 support enabled, Postfix changes behavior with respect to earlier
Postfix releases:

  * UTF-8 is permitted in the myorigin parameter value. However, the myhostname
    and mydomain parameters must currently specify ASCII-only domain names.
    This limitation may be removed later.

  * UTF-8 is the only form of non-ASCII text that Postfix supports in access
    tables, address rewriting tables, and other tables that are indexed with an
    email address, hostname, or domain name.

  * The header_checks-like and body_checks-like features are not UTF-8 enabled,
    and therefore they do not enforce UTF-8 syntax rules on inputs and outputs.
    The reason is that non-ASCII text may be sent in encodings other than UTF-
    8, and that real email sometimes contains malformed headers. Instead of
    skipping non-UTF-8 content, Postfix should be able to filter it. You may
    try to enable UTF-8 processing by starting a PCRE pattern with the sequence
    (*UTF8), but this is will result in "message not accepted, try again later"
    errors when the PCRE pattern matcher encounters non-UTF-8 input. Other
    features that are not UTF-8 enabled are smtpd_command_filter,
    smtp_reply_filter, the *_delivery_status_filter features, and the
    *_dns_reply_filter features (the latter because DNS is by definition an
    ASCII protocol).

  * The Postfix SMTP server announces SMTPUTF8 support in the EHLO response.

    220 server.example.com ESMTP Postfix
    EEHHLLOO cclliieenntt..eexxaammppllee..ccoomm
    250-server.example.com
    250-PIPELINING
    250-SIZE 10240000
    250-VRFY
    250-ETRN
    250-STARTTLS
    250-AUTH PLAIN LOGIN
    250-ENHANCEDSTATUSCODES
    250-8BITMIME
    250-DSN
    250 SMTPUTF8

  * The Postfix SMTP server accepts the SMTPUTF8 request in MAIL FROM and VRFY
    commands.

    MMAAIILL FFRROOMM::<<aaddddrreessss>> SSMMTTPPUUTTFF88 ......

    VVRRFFYY aaddddrreessss SSMMTTPPUUTTFF88

  * The Postfix SMTP client may issue the SMTPUTF8 request in MAIL FROM
    commands.

  * The Postfix SMTP server accepts UTF-8 in email address domains, but only
    after the remote SMTP client issues the SMTPUTF8 request in MAIL FROM or
    VRFY commands.

Postfix already permitted UTF-8 in message header values and in address
localparts. This does not change.

UUssiinngg PPoossttffiixx SSMMTTPPUUTTFF88 ssuuppppoorrtt

After Postfix SMTPUTF8 support is turned on, Postfix behavior will depend on 1)
whether a remote SMTP client requests SMTPUTF8 support, 2) the presence of UTF-
8 content in the message envelope and headers, and 3) whether a down-stream
SMTP (or LMTP) server announces SMTPUTF8 support.

  * When the Postfix SMTP server receives a message WITHOUT the SMTPUTF8
    request, Postfix handles the message as it has always done (at least that
    is the default, see autodetection below). Specifically, the Postfix SMTP
    server does not accept UTF-8 in the envelope sender domain name or envelope
    recipient domain name, and the Postfix SMTP client does not issue the
    SMTPUTF8 request when delivering that message to an SMTP or LMTP server
    that announces SMTPUTF8 support (again, that is the default). Postfix will
    accept UTF-8 in message header values and in the localpart of envelope
    sender and recipient addresses, because it has always done that.

  * When the Postfix SMTP server receives a message WITH the SMTPUTF8 request,
    Postfix will issue the SMTPUTF8 request when delivering that message to an
    SMTP or LMTP server that announces SMTPUTF8 support. This is not
    configurable.

  * When a message is received with the SMTPUTF8 request, Postfix will deliver
    the message to a non-SMTPUTF8 SMTP or LMTP server ONLY if:

      o No message header value contains UTF-8.

      o The envelope sender address contains no UTF-8,

      o No envelope recipient address for that specific SMTP/LMTP delivery
        transaction contains UTF-8.

            NOTE: Recipients in other email delivery transactions for that same
            message may still contain UTF-8.

    Otherwise, Postfix will return the recipient(s) for that email delivery
    transaction as undeliverable. The delivery status notification message will
    be an SMTPUTF8 message. It will therefore be subject to the same
    restrictions as email that is received with the SMTPUTF8 request.

  * When the Postfix SMTP server receives a message with the SMTPUTF8 request,
    that request also applies after the message is forwarded via a virtual or
    local alias, or $HOME/.forward file.

SSMMTTPPUUTTFF88 aauuttooddeetteeccttiioonn

This section applies only to systems that have SMTPUTF8 support turned on
(smtputf8_enable = yes).

For compatibility with pre-SMTPUTF8 environments, Postfix does not
automatically set the "SMTPUTF8 requested" flag on messages from non-SMTPUTF8
clients that contain a UTF-8 header value or UTF-8 address localpart. This
would make such messages undeliverable to non-SMTPUTF8 servers, and could be a
barrier to SMTPUTF8 adoption.

By default, Postfix sets the "SMTPUTF8 requested" flag only on address
verification probes and on Postfix sendmail submissions that contain UTF-8 in
the sender address, UTF-8 in a recipient address, or UTF-8 in a message header
value.

    /etc/postfix/main.cf:
        smtputf8_autodetect_classes = sendmail, verify

However, if you have a non-ASCII myorigin or mydomain setting, or if you have a
configuration that introduces UTF-8 addresses with virtual aliases, canonical
mappings, or BCC mappings, then you may have to apply SMTPUTF8 autodetection to
all email:

    /etc/postfix/main.cf:
        smtputf8_autodetect_classes = all

This will, of course, also flag email that was received without SMTPUTF8
request, but that contains UTF-8 in a sender address localpart, receiver
address localpart, or message header value. Such email was not standards-
compliant, but Postfix would have delivered it if SMTPUTF8 support was
disabled.

LLiimmiittaattiioonnss ooff tthhee ccuurrrreenntt iimmpplleemmeennttaattiioonn

The Postfix implementation is a work in progress; limitations are steadily
being removed. The text below describes the situation at one point in time.

NNoo aauuttoommaattiicc ccoonnvveerrssiioonnss bbeettwweeeenn AASSCCIIII aanndd UUTTFF--88 ddoommaaiinn nnaammeess..

Some background: According to RFC 6530 and related documents, an
internationalized domain name can appear in two forms: the UTF-8 form, and the
ASCII (xn--mumble) form. An internationalized address localpart must be encoded
in UTF-8; the RFCs do not define an ASCII alternative form.

Postfix currently does not convert internationalized domain names from UTF-
8 into ASCII (or from ASCII into UTF-8) before using domain names in SMTP
commands and responses, before looking up domain names in lists such as
mydestination, relay_domains or in lookup tables such as access tables, etc.,
before using domain names in a policy daemon or Milter request, or before
logging events.

Postfix does, however, casefold domain names and email addresses before
matching them against a Postfix configuration parameter or lookup table.

In order to use Postfix SMTPUTF8 support:

  * The Postfix parameters myhostname and mydomain must be in ASCII form. One
    is a substring of the other, and the myhostname value is used in SMTP
    commands and responses that require ASCII. The parameter myorigin (added to
    local addresses without domain) supports UTF-8.

  * You need to configure both the ASCII and UTF-8 forms of an
    Internationalized domain name in Postfix parameters such as mydestination
    and relay_domains, as well as lookup table search keys.

  * Milters, content filters, policy servers and logfile analysis tools need to
    be able to handle both the ASCII and UTF-8 forms of Internationalized
    domain names.

CCoommppaattiibbiilliittyy wwiitthh pprree--SSMMTTPPUUTTFF88 eennvviirroonnmmeennttss

MMaaiilliinngg lliissttss wwiitthh UUTTFF--88 aanndd nnoonn--UUTTFF--88 ssuubbssccrriibbeerrss

With Postfix, there is no need to split mailing lists into UTF-8 and non-UTF-
8 members. Postfix will try to deliver the non-UTF8 subscribers over
"traditional" non-SMTPUTF8 sessions, as long as the message has an ASCII
envelope sender address and all-ASCII header values. The mailing list manager
may have to apply RFC 2047 encoding to satisfy that last condition.

PPrree--eexxiissttiinngg nnoonn--AASSCCIIII eemmaaiill fflloowwss

With "smtputf8_enable = no", Postfix handles email with non-ASCII in address
localparts (and in headers) as before. The vast majority of email software is
perfectly capable of handling such email, even if pre-SMTPUTF8 standards do not
support such practice.

RReejjeeccttiinngg nnoonn--UUTTFF88 aaddddrreesssseess

With "smtputf8_enable = yes", Postfix requires that non-ASCII address
information is encoded in UTF-8 and will reject other encodings such as ISO-
8859. It is not practical for Postfix to support multiple encodings at the same
time. There is no problem with RFC 2047 encodings such as "=?ISO-8859-
1?Q?text?=", because those use only characters from the ASCII characterset.

RReejjeeccttiinngg nnoonn--AASSCCIIII aaddddrreesssseess iinn nnoonn--SSMMTTPPUUTTFF88 ttrraannssaaccttiioonnss

Setting "strict_smtputf8 = yes" in addition to "smtputf8_enable = yes" will
enable stricter enforcement of the SMTPUTF8 protocol. Specifically, the Postfix
SMTP server will not only reject non-UTF8 sender or recipient addresses, it
will in addition accept UTF-8 sender or recipient addresses only when the
client requests an SMTPUTF8 mail transaction.

CCoommppaattiibbiilliittyy wwiitthh IIDDNNAA22000033

Postfix >= 3.2 by default disables the 'transitional' compatibility between
IDNA2003 and IDNA2008, when converting UTF-8 domain names to/from the ASCII
form that is used in DNS lookups. This makes Postfix behavior consistent with
current versions of the Firefox and Chrome web browsers. Specify
"enable_idna2003_compatibility = yes" to get the historical behavior.

This affects the conversion of domain names that contain for example the German
sz (ß) and the Greek zeta (ς). See http://unicode.org/cldr/utility/idna.jsp
for more examples.

CCrreeddiittss

  * May 15, 2014: Arnt Gulbrandsen posted his patch for Unicode email support.
    This work was sponsored by CNNIC.

  * July 15, 2014: Wietse integrated Arnt Gulbrandsen's code and released
    Postfix with SMTPUTF8 support.

  * January 2015: Wietse added UTF-8 support for casefolding in Postfix lookup
    tables and caseless string comparison in Postfix list-based features.