diff options
Diffstat (limited to 'README_FILES/BUILTIN_FILTER_README')
| -rw-r--r-- | README_FILES/BUILTIN_FILTER_README | 321 |
1 files changed, 321 insertions, 0 deletions
diff --git a/README_FILES/BUILTIN_FILTER_README b/README_FILES/BUILTIN_FILTER_README new file mode 100644 index 0000000..2ce639d --- /dev/null +++ b/README_FILES/BUILTIN_FILTER_README @@ -0,0 +1,321 @@ + PPoossttffiixx BBuuiilltt--iinn CCoonntteenntt IInnssppeeccttiioonn + +------------------------------------------------------------------------------- + +BBuuiilltt--iinn ccoonntteenntt iinnssppeeccttiioonn iinnttrroodduuccttiioonn + +Postfix supports a built-in filter mechanism that examines message header and +message body content, one line at a time, before it is stored in the Postfix +queue. The filter is usually implemented with POSIX or PCRE regular +expressions, as described in the header_checks(5) manual page. + +The original purpose of the built-in filter is to stop an outbreak of specific +email worms or viruses, and it does this job well. The filter has also helped +to block bounced junk email, bounced email from worms or viruses, and +notifications from virus detection systems. Information about this secondary +application is given in the BACKSCATTER_README document. + +Because the built-in filter is optimized for stopping specific worms and virus +outbreaks, it has limitations that make it NOT suitable for general junk email +and virus detection. For that, you should use one of the external content +inspection methods that are described in the FILTER_README, SMTPD_PROXY_README +and MILTER_README documents. + +The following diagram gives an over-all picture of how Postfix built-in content +inspection works: + + Postmaster + notifications + + | + v + + Network or -> BBuuiilltt--iinn -> Postfix -> Delivery -> Network or + local users ffiilltteerr queue agents local mailbox + + ^ | + | v + + Undeliverable mail + Forwarded mail + +The picture makes clear that the filter works while Postfix is receiving new +mail. This means that Postfix can reject mail from the network without having +to return undeliverable mail to the originator address (which is often spoofed +anyway). However, this ability comes at a price: if mail inspection takes too +much time, then the remote client will time out, and the client may send the +same message repeatedly. + +Topics covered by this document: + + * What mail is subjected to header/body checks + * Limitations of Postfix header/body checks + * Preventing daily mail status reports from being blocked + * Configuring header/body checks for mail from outside users only + * Configuring different header/body checks for MX service and submission + service + * Configuring header/body checks for mail to some domains only + +WWhhaatt mmaaiill iiss ssuubbjjeecctteedd ttoo hheeaaddeerr//bbooddyy cchheecckkss + +Postfix header/body checks are implemented by the cleanup(8) server before it +injects mail into the incoming queue. The diagram below zooms in on the cleanup +(8) server, and shows that this server handles mail from many different +sources. In order to keep the diagram readable, the sources of postmaster +notifications are not shown, because they can be produced by many Postfix +daemon processes. + + bounce(8) + (undeliverable) + + ssmmttppdd((88)) | + ((nneettwwoorrkk)) \ v + + qqmmqqppdd((88)) -\ cleanup(8) -> incoming + ((nneettwwoorrkk)) -/ queue + + ppiicckkuupp((88)) / ^ + ((llooccaall)) | + + local(8) + (forwarded) + +For efficiency reasons, only mail that enters from outside of Postfix is +inspected with header/body checks. It would be inefficient to filter already +filtered mail again, and it would be undesirable to block postmaster +notifications. The table below summarizes what mail is and is not subject to +header/body checks. + + _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ + |MMeessssaaggee ttyyppee |SSoouurrccee |HHeeaaddeerr//bbooddyy cchheecckkss??| + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Undeliverable mail|bounce(8)|No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Network mail |smtpd(8) |Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Network mail |qmqpd(8) |Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local submission |pickup(8)|Configurable | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Local forwarding |local(8) |No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + |Postmaster notice |many |No | + |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | + +How does Postfix decide what mail needs to be filtered? It would be clumsy to +make the decision in the cleanup(8) server, as this program receives mail from +so many different sources. Instead, header/body checks are requested by the +source. Examples of how to turn off header/body checks for mail received with +smtpd(8), qmqpd(8) or pickup(8) are given below under "Configuring header/body +checks for mail from outside users only", "Configuring different header/body +checks for MX service and submission service", and "Configuring header/body +checks for mail to some domains only". + +LLiimmiittaattiioonnss ooff PPoossttffiixx hheeaaddeerr//bbooddyy cchheecckkss + + * Header/body checks do not decode message headers or message body content. + For example, if text in the message body is BASE64 encoded (RFC 2045) then + your regular expressions will have to match the BASE64 encoded form. + Likewise, message headers with encoded non-ASCII characters (RFC 2047) need + to be matched in their encoded form. + + * Header/body checks cannot filter on a combination of message headers or + body lines. Header/body checks examine content one message header at a + time, or one message body line at a time, and cannot carry a decision over + to the next message header or body line. + + * Header/body checks cannot depend on the recipient of a message. + + o One message can have multiple recipients, and all recipients of a + message receive the same treatment. Workarounds have been proposed that + involve selectively deferring some recipients of multi-recipient mail, + but that results in poor SMTP performance and does not work for non- + SMTP mail. + + o Some sources of mail send the headers and content ahead of the + recipient information. It would be inefficient to buffer up an entire + message before deciding if it needs to be filtered, and it would be + clumsy to filter mail and to buffer up all the actions until it is + known whether those actions need to be executed. + + * Despite warnings, some people try to use the built-in filter feature for + general junk email and/or virus blocking, using hundreds or even thousands + of regular expressions. This can result in catastrophic performance + failure. The symptoms are as follows: + + o The cleanup(8) processes use up all available CPU time in order to + process the regular expressions, and/or they use up all available + memory so that the system begins to swap. This slows down all incoming + mail deliveries. + + o As Postfix needs more and more time to receive an email message, the + number of simultaneous SMTP sessions increases to the point that the + SMTP server process limit is reached. + + o While all SMTP server processes are waiting for the cleanup(8) servers + to finish, new SMTP clients have to wait until an SMTP server process + becomes available. This causes mail deliveries to time out before they + have even begun. + + The remedy for this type of performance problem is simple: don't use + header/body checks for general junk email and/or virus blocking, and don't + filter mail before it is queued. When performance is a concern, use an + external content filter that runs after mail is queued, as described in the + FILTER_README document. + +PPrreevveennttiinngg ddaaiillyy mmaaiill ssttaattuuss rreeppoorrttss ffrroomm bbeeiinngg bblloocckkeedd + +The following is quoted from Jim Seymour's Pflogsumm FAQ at http:// +jimsun.linxnet.com/downloads/pflogsumm-faq.txt. Pflogsumm is a program that +analyzes Postfix logs, including the logging from rejected mail. If these logs +contain text that was rejected by Postfix body_checks patterns, then the +logging is also likely to be rejected by those same body_checks patterns. This +problem does not exist with header_checks patterns, because those are not +applied to the text that is part of the mail status report. + + You configure Postfix to do body checks, Postfix does its thing, Pflogsumm + reports it and Postfix catches the same string in the Pflogsumm report. + There are several solutions to this. + + Wolfgang Zeikat contributed this: + + #!/usr/bin/perl + use MIME::Lite; + + ### Create a new message: + $msg = MIME::Lite->new( + From => 'your@send.er', + To => 'your@recipie.nt', + # Cc => 'some@other.com, some@more.com', + Subject => 'pflogsumm', + Date => `date`, + Type => 'text/plain', + Encoding => 'base64', + Path => '/tmp/pflogg', + ); + + $msg->send; + + Where "/tmp/pflogg" is the output of Pflogsumm. This puts Pflogsumm's + output in a base64 MIME attachment. + +Note by Wietse: if you run this on a machine that is accessible by untrusted +users, it is safer to store the Pflogsumm report in a directory that is not +world writable. + + In a follow-up to a thread in the postfix-users mailing list, Ralf + Hildebrandt noted: + + "mpack does the same thing." + +And it does. Which tool one should use is a matter of preference. + +Other solutions involve additional body_checks rules that make exceptions for +daily mail status reports, but this is not recommended. Such rules slow down +all mail and complicate Postfix maintenance. + +CCoonnffiigguurriinngg hheeaaddeerr//bbooddyy cchheecckkss ffoorr mmaaiill ffrroomm oouuttssiiddee uusseerrss oonnllyy + +The following information applies to Postfix 2.1 and later. Earlier Postfix +versions do not support the receive_override_options feature. + +The easiest approach is to configure ONE Postfix instance with multiple SMTP +server IP addresses in master.cf: + + * Two SMTP server IP addresses for mail from inside users only, with header/ + body filtering turned off, and a local mail pickup service with header/body + filtering turned off. + + /etc/postfix.master.cf: + # ================================================================== + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================== + 1.2.3.4:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + 127.0.0.1:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + pickup fifo n - n 60 1 pickup + -o receive_override_options=no_header_body_checks + + * Add some firewall rule to prevent access to 1.2.3.4:smtp from the outside + world. + + * One SMTP server address for mail from outside users with header/body + filtering turned on via main.cf. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + 1.2.3.5:smtp inet n - n - - smtpd + +CCoonnffiigguurriinngg ddiiffffeerreenntt hheeaaddeerr//bbooddyy cchheecckkss ffoorr MMXX sseerrvviiccee aanndd ssuubbmmiissssiioonn sseerrvviiccee + +If authorized user submissions require different header/body checks than mail +from remote MTAs, then this is possible as long as you have separate mail +streams for authorized users and for MX service. + +The example below assumes that authorized users connect to TCP port 587 +(submission) or 465 (smtps), and that remote MTAs connect to TCP port 25 +(smtp). + +First, we define a few "user-defined" parameters that will override settings +for the submission and smtps services. + + /etc/postfix/main.cf: + msa_cleanup_service_name = msa_cleanup + msa_header_checks = pcre:/etc/postfix/msa_header_checks + msa_body_checks = pcre:/etc/postfix/msa_body_checks + +Next, we define msa_cleanup as a dedicated cleanup service that will be used +only by the submission and smtps services. This service uses the header_checks +and body_checks overrides that were defined above. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + smtp inet n - n - - smtpd + msa_cleanup unix n - n - 0 cleanup + -o header_checks=$msa_header_checks + -o body_checks=$msa_body_checks + submission inet n - n - - smtpd + -o cleanup_service_name=$msa_cleanup_service_name + -o syslog_name=postfix/submission + ...[see sample master.cf file for more]... + smtps inet n - n - - smtpd + -o cleanup_service_name=$msa_cleanup_service_name + -o syslog_name=postfix/smtps + -o smtpd_tls_wrappermode=yes + ...[see sample master.cf file for more]... + +By keeping the "msa_xxx" parameter settings in main.cf, you keep your master.cf +file simple, and you minimize the amount of duplication. + +CCoonnffiigguurriinngg hheeaaddeerr//bbooddyy cchheecckkss ffoorr mmaaiill ttoo ssoommee ddoommaaiinnss oonnllyy + +The following information applies to Postfix 2.1. Earlier Postfix versions do +not support the receive_override_options feature. + +If you are an MX service provider and want to enable header/body checks only +for some domains, you can configure ONE Postfix instance with multiple SMTP +server IP addresses in master.cf. Each address provides a different service. + + /etc/postfix.master.cf: + # ================================================================= + # service type private unpriv chroot wakeup maxproc command + # (yes) (yes) (yes) (never) (100) + # ================================================================= + # SMTP service for domains with header/body checks turned on. + 1.2.3.4:smtp inet n - n - - smtpd + + # SMTP service for domains with header/body checks turned off. + 1.2.3.5:smtp inet n - n - - smtpd + -o receive_override_options=no_header_body_checks + +Once this is set up you can configure MX records in the DNS that route each +domain to the proper SMTP server instance. + |