From b7c15c31519dc44c1f691e0466badd556ffe9423 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 7 Apr 2024 18:18:56 +0200 Subject: Adding upstream version 3.7.10. Signed-off-by: Daniel Baumann --- html/FILTER_README.html | 980 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 980 insertions(+) create mode 100644 html/FILTER_README.html (limited to 'html/FILTER_README.html') diff --git a/html/FILTER_README.html b/html/FILTER_README.html new file mode 100644 index 0000000..6eec9ab --- /dev/null +++ b/html/FILTER_README.html @@ -0,0 +1,980 @@ + + + + + + +Postfix After-Queue Content Filter + + + + + + + +

Postfix After-Queue Content Filter

+ +
+ +

Introduction

+ +

This document requires Postfix version 2.1 or later.

+ +

Normally, Postfix receives mail, stores it in the mail queue +and then delivers it. With the external content filter described +here, mail is filtered AFTER it is queued. This approach decouples +mail receiving processes from mail filtering processes, and gives +you maximal control over how many filtering processes you are +willing to run in parallel.

+ +

The after-queue content filter is meant to be used as follows:

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + +
+ Network or
local users 		-> + Postfix
queue 		-> + Content
filter 		-> + Postfix
queue 		-> + Network or
local mailbox
+ +
+ +

This document describes implementations that use a single +Postfix instance for everything: receiving, filtering and delivering +mail. Applications that use two separate Postfix instances will +be covered by a later version of this document.

+ +

The after-queue content filter is not to be confused with the +approaches described in the SMTPD_PROXY_README or MILTER_README +documents, +where incoming SMTP mail is filtered BEFORE it is stored into the +Postfix queue.

+ +

This document describes two approaches to content filter +all email, as well as several options to filter mail selectively:

+ + + + +

Principles of operation

+ +

An after-queue content filter receives unfiltered mail from Postfix +(as described further below) and can do one of the following:

+ +
    + +

  1. Re-inject the mail back into Postfix, perhaps after changing + content and/or destination.

    + +

  2. Discard or quarantine the mail.

    + +

  3. Reject the mail (by sending a suitable status code back to + Postfix). Postfix will send the mail back to the sender address.

    + +
+ +

NOTE: in this time of mail worms and forged spam, it is a VERY +BAD IDEA to send viruses back to the sender address, because the +sender address is almost certainly not the originator. It is better +to discard known viruses, and to quarantine material that is +suspect so that a human can decide what to do with it.

+ +

Simple content filter example

+ +

The first example is simple to set up, but has major limitations +that will be addressed in a second example. Postfix receives +unfiltered mail from the network with the smtpd(8) server, and +delivers unfiltered mail to a content filter with the Postfix +pipe(8) delivery agent. The content filter injects filtered mail +back into Postfix with the Postfix sendmail(1) command, so that +Postfix can deliver it to the final destination.

+ +

This means that mail submitted via the Postfix sendmail(1) +command cannot be content filtered.

+ +

In the figure below, names followed by a number represent +Postfix commands or daemon programs. See the OVERVIEW +document for an introduction to the Postfix architecture.

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Unfiltered

 ->

 + smtpd(8)

pickup(8) 		>- + cleanup(8) -> + qmgr(8)
Postfix
queue 		-< + local(8)
smtp(8)
pipe(8) 		->
+ ->
 Filtered
Filtered
+
^
| 		|
v
+ maildrop
+ queue 		<- Postfix
+ postdrop(1) 		<- Postfix
+ sendmail(1) 		<- Content +
filter
+ +
+ +

The content filter can be a simple shell script like this:

+ +
+
+ 1 #!/bin/sh
+ 2 
+ 3 # Simple shell-based filter. It is meant to be invoked as follows:
+ 4 #       /path/to/script -f sender recipients...
+ 5 
+ 6 # Localize these. The -G option does nothing before Postfix 2.3.
+ 7 INSPECT_DIR=/var/spool/filter
+ 8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
+ 9 
+10 # Exit codes from <sysexits.h>
+11 EX_TEMPFAIL=75
+12 EX_UNAVAILABLE=69
+13 
+14 # Clean up when done or when aborting.
+15 trap "rm -f in.$$" 0 1 2 3 15
+16 
+17 # Start processing.
+18 cd $INSPECT_DIR || {
+19     echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
+20 
+21 cat >in.$$ || { 
+22     echo Cannot save mail to file; exit $EX_TEMPFAIL; }
+23 
+24 # Specify your content filter here.
+25 # filter <in.$$ || {
+26 #   echo Message content rejected; exit $EX_UNAVAILABLE; }
+27 
+28 $SENDMAIL "$@" <in.$$
+29 
+30 exit $?
+
+
+ +

Notes:

+ + + +

I suggest that you first run this script by hand until you are +satisfied with the results. Run it with a real message (headers+body) +as input:

+ +
+
+% /path/to/script -f sender -- recipient... <message-file
+
+
+ +

Once you're satisfied with the content filtering script:

+ + + +

Simple content filter performance

+ +

With the shell script as shown above you will lose a factor of +four in Postfix performance for transit mail that arrives and leaves +via SMTP. You will lose another factor in transit performance for +each additional temporary file that is created and deleted in the +process of content filtering. The performance impact is less for +mail that is submitted or delivered locally, because such deliveries +are already slower than SMTP transit mail.

+ +

Simple content filter limitations

+ +

The problem with content filters like the one above is that +they are not very robust. The reason is that the software does not +talk a well-defined protocol with Postfix. If the filter shell +script aborts because the shell runs into some memory allocation +problem, the script will not produce a nice exit status as defined +in the file /usr/include/sysexits.h. Instead of going to the +deferred queue, mail will bounce. The same lack of robustness can +happen when the content filtering software itself runs into a +resource problem.

+ +

The simple content filter method is not suitable for content +filter actions that are invoked via header_checks or body_checks +patterns. These patterns will be applied again after mail is +re-injected with the Postfix sendmail command, resulting in a mail +filtering loop. The advanced content filtering method (see below) +makes it possible to turn off header_checks or body_checks patterns +for filtered mail.

+ +

Turning off the simple content filter

+ +

To turn off "simple" content filtering:

+ + + +

Advanced content filter example

+ +

The second example is more complex, but can give better +performance, and is less likely to bounce mail when the machine +runs into some resource problem. This content filter receives +unfiltered mail with SMTP on localhost port 10025, and sends filtered +mail back into Postfix with SMTP on localhost port 10026.

+ +

For non-SMTP capable content filtering software, Bennett Todd's +SMTP proxy implements a nice PERL/SMTP content filtering framework. +See: https://web.archive.org/web/20151022025756/http://bent.latency.net/smtpprox/.

+ +

In the figure below, names followed by a number represent +Postfix commands or daemon programs. See the OVERVIEW +document for an introduction to the Postfix architecture.

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Unfiltered

+ Unfiltered		 ->

+ -> 		+ smtpd(8)

pickup(8) 		>- + cleanup(8) -> + qmgr(8)
Postfix
queue 		-< + smtp(8)

local(8) 		->

+ -> 		Filtered

+ Filtered
^
| 		|
v
+ smtpd(8)
10026 		+ smtp(8)
^
| 		|
v
content filter 10025
+ +
+ +

The example given here filters all mail, including mail that +arrives via SMTP and mail that is locally submitted via the Postfix +sendmail command (local submissions enter Postfix via the pickup(8) +server; to keep the figure simple we omit local submission details). +See examples near the end of this document for +how to exclude local users from filtering, or how to configure a +destination dependent content filter.

+ +

You can expect to lose about a factor of two in Postfix +performance for mail that arrives and leaves via SMTP, provided +that the content filter creates no temporary files. Each temporary +file created by the content filter adds another factor to the +performance loss.

+ +

Advanced content filter: requesting that all mail is filtered

+ +

To enable the advanced content filter method for all mail, +specify in main.cf:

+ +
+
+/etc/postfix/main.cf:
+    content_filter = scan:localhost:10025
+    receive_override_options = no_address_mappings
+
+
+ + + +

Advanced content filter: sending unfiltered mail to the content +filter

+ +

In this example, "scan" is an instance of the Postfix SMTP +client with slightly different configuration parameters. This is +how one would set up the service in the Postfix master.cf file: +

+ +
+
+/etc/postfix/master.cf:
+    # =============================================================
+    # service type  private unpriv  chroot  wakeup  maxproc command
+    #               (yes)   (yes)   (yes)   (never) (100)
+    # =============================================================
+    scan      unix  -       -       n       -       10      smtp
+        -o smtp_send_xforward_command=yes
+        -o disable_mime_output_conversion=yes
+        -o smtp_generic_maps=
+
+
+ + + +

Advanced content filter: running the content filter

+ +

The content filter can be set up with the Postfix spawn service, +which is the Postfix equivalent of inetd. For example, to instantiate +up to 10 content filtering processes on localhost port 10025:

+ +
+
+/etc/postfix/master.cf:
+    # ===================================================================
+    # service       type  private unpriv  chroot  wakeup  maxproc command
+    #                     (yes)   (yes)   (yes)   (never) (100)
+    # ===================================================================
+    localhost:10025 inet  n       n       n       -       10      spawn
+        user=filter argv=/path/to/filter localhost 10026
+
+
+ + + +

If you want to have your filter listening on port localhost:10025 +instead of Postfix, then you must run your filter as a stand-alone +program, and must not use the Postfix spawn service.

+ +

Advanced filter: injecting mail back into Postfix

+ +

The job of the content filter is to either bounce mail with a +suitable diagnostic, or to feed the mail back into Postfix through +a dedicated listener on port localhost 10026.

+ +

The simplest content filter just copies SMTP commands and data +between its inputs and outputs. If it has a problem, all it has to +do is to reply to an input of `.' from Postfix with `550 content +rejected', and to disconnect without sending `.' on the connection +that injects mail back into Postfix.

+ +
+
+/etc/postfix/master.cf:
+    # ===================================================================
+    # service       type  private unpriv  chroot  wakeup  maxproc command
+    #                     (yes)   (yes)   (yes)   (never) (100)
+    # ===================================================================
+    localhost:10026 inet  n       -       n       -       10      smtpd
+        -o content_filter= 
+        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters
+        -o smtpd_helo_restrictions=
+        -o smtpd_client_restrictions=
+        -o smtpd_sender_restrictions=
+        # Postfix 2.10 and later: specify empty smtpd_relay_restrictions.
+        -o smtpd_relay_restrictions=
+        -o smtpd_recipient_restrictions=permit_mynetworks,reject
+        -o mynetworks=127.0.0.0/8
+        -o smtpd_authorized_xforward_hosts=127.0.0.0/8
+
+
+ + + +

Advanced content filter performance

+ +

With the "sandwich" approach to content filtering described +here, it is important to match the filter concurrency to the +available CPU, memory and I/O resources. Too few content filter +processes and mail accumulates in the active queue even with low +traffic volume; too much concurrency and Postfix ends up deferring +mail destined for the content filter because processes fail due to +insufficient resources.

+ +

Currently, content filter performance tuning is a process of +trial and error; analysis is handicapped because filtered and +unfiltered messages share the same queue. As mentioned in the +introduction of this document, content filtering with multiple +Postfix instances will be covered in a future version.

+ +

Turning off the advanced content filter

+ +

To turn off "advanced" content filtering:

+ + + +

Filtering mail from outside users only

+ +

The easiest approach is to configure ONE Postfix instance with +multiple SMTP server IP addresses in master.cf:

+ + + +

After this, you can follow the same procedure as outlined in +the "advanced" or "simple" content filtering examples above, except +that you must not specify "content_filter" or "receive_override_options" +in the main.cf file.

+ +

Different filters for different +domains

+ +

If you are an MX service provider and want to apply different +content filters for different domains, you can configure ONE Postfix +instance with multiple SMTP server IP addresses in master.cf. Each +address provides a different content filter service.

+ +
+
+/etc/postfix.master.cf:
+    # =================================================================
+    # service     type  private unpriv  chroot  wakeup  maxproc command
+    #                   (yes)   (yes)   (yes)   (never) (100)
+    # =================================================================
+    # SMTP service for domains that are filtered with service1:dest1
+    1.2.3.4:smtp  inet  n       -       n       -       -       smtpd
+        -o content_filter=service1:dest1 
+        -o receive_override_options=no_address_mappings
+
+    # SMTP service for domains that are filtered with service2:dest2
+    1.2.3.5:smtp  inet  n       -       n       -       -       smtpd
+        -o content_filter=service2:dest2
+        -o receive_override_options=no_address_mappings
+
+
+ +

After this, you can follow the same procedure as outlined in +the "advanced" or "simple" content filtering examples above, except +that you must not specify "content_filter" or "receive_override_options" +in the main.cf file.

+ +

Set up MX records in the DNS that route each domain to the +proper SMTP server instance.

+ +

FILTER actions in access or header/body +tables

+ +

The above filtering configurations are static. Mail that follows +a given path is either always filtered or it is never filtered. As +of Postfix 2.0 you can also turn on content filtering on the fly. +

+ +

To turn on content filtering with an access(5) table rule:

+ +
+
+/etc/postfix/access:
+    whatever       FILTER foo:bar
+
+
+ +

To turn on content filtering with a header_checks(5) or +body_checks(5) table pattern:

+ +
+
+/etc/postfix/header_checks:
+    /whatever/     FILTER foo:bar
+
+
+ +

You can do this in smtpd access maps as well as the cleanup +server's header/body_checks. This feature must be used with great +care: you must disable all the UCE features in the after-filter +smtpd and cleanup daemons or else you will have a content filtering +loop.

+ +

Limitations:

+ + + + + + -- cgit v1.2.3