1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
|
<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Postfix Connection Cache </title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel='stylesheet' type='text/css' href='postfix-doc.css'>
</head>
<body>
<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix Connection Cache </h1>
<hr>
<h2>Introduction</h2>
<p> This document describes the Postfix connection cache implementation,
which is available with Postfix version 2.2 and later. </p>
<p> Topics covered in this document: </p>
<ul>
<li><a href="#summary"> What SMTP connection caching can do for you</a>
<li><a href="#implementation"> Connection cache implementation</a>
<li><a href="#configuration"> Connection cache configuration</a>
<li><a href="#safety">Connection cache safety mechanisms </a>
<li><a href="#limitations">Connection cache limitations</a>
<li><a href="#statistics">Connection cache statistics</a>
</ul>
<h2><a name="summary">What SMTP connection caching can do for
you</a></h2>
<p> With SMTP connection caching, Postfix can deliver multiple
messages over the same SMTP connection. By default, Postfix 2.2
reuses a plaintext SMTP connection automatically when a destination has
high volume of mail in the <a href="QSHAPE_README.html#active_queue">active queue</a>. </p>
<p> SMTP Connection caching is a performance feature. Whether or not
it actually improves performance depends on the conditions: </p>
<ul>
<li> <p> SMTP Connection caching can greatly improve performance
when delivering mail to a destination with multiple mail servers,
because it can help Postfix to skip over a non-responding server.
</p>
<li> <p> SMTP Connection caching can also help with receivers that
impose rate limits on new connections. </p>
<li> <p> Otherwise, the benefits of SMTP connection caching are
minor: it eliminates the latency of the TCP handshake (SYN, SYN+ACK,
ACK), plus the latency of the SMTP initial handshake (220 greeting,
EHLO command, EHLO response). With TLS-encrypted connections, this
can save an additional two roundtrips that would otherwise be needed
to send STARTTLS and to resume a TLS session. </p>
<li> <p> SMTP Connection caching gives no gains with respect to
SMTP session tear-down. The Postfix <a href="smtp.8.html">smtp(8)</a> client normally does
not wait for the server's reply to the QUIT command, and it never
waits for the TCP final handshake to complete. </p>
<li> <p> SMTP Connection caching introduces some overhead: the
client needs to send an RSET command to find out if a connection
is still usable, before it can send the next MAIL FROM command.
This introduces one additional round-trip delay. </p>
</ul>
<p> For other potential issues with SMTP connection caching, see
the discussion of <a href="#limitations">limitations</a> at the end
of this document. </p>
<h2><a name="implementation">Connection cache implementation</a></h2>
<p> For an overview of how Postfix delivers mail, see the Postfix
architecture <a href="OVERVIEW.html">OVERVIEW</a> document. </p>
<p> The Postfix connection cache is shared among Postfix mail
delivering processes. This maximizes the opportunity to reuse an
open connection. Some MTAs such as Sendmail have a
non-shared connection cache. Here, a connection can be reused only
by the mail delivering process that creates the connection. To get
the same performance improvement as with a shared connection cache,
non-shared connections need to be kept open for a longer time. </p>
<p> The <a href="scache.8.html">scache(8)</a> server, introduced with Postfix version 2.2,
maintains the shared connection cache. With Postfix version 2.2,
only the <a href="smtp.8.html">smtp(8)</a> client has support to access this cache. </p>
<p> When SMTP connection caching is enabled (see next section), the
<a href="smtp.8.html">smtp(8)</a> client does not disconnect after a mail transaction, but
gives the connection to the <a href="scache.8.html">scache(8)</a> server which keeps the
connection open for a limited amount of time. </p>
<p> After handing over the open connection to the <a href="scache.8.html">scache(8)</a> server,
the <a href="smtp.8.html">smtp(8)</a> client continues with some other mail delivery request.
Meanwhile, any <a href="smtp.8.html">smtp(8)</a> client process can ask the <a href="scache.8.html">scache(8)</a> server
for that cached connection and reuse it for mail delivery. </p>
<blockquote>
<table>
<tr> <td> </td> <td> <tt> /-- </tt> </td> <td align="center"
colspan="3" bgcolor="#f0f0ff"> <a href="smtp.8.html">smtp(8)</a> </td> <td colspan="2"> <tt>
--> </tt> </td> <td> Internet </td> </tr>
<tr> <td align="center" bgcolor="#f0f0ff"> <a href="qmgr.8.html">qmgr(8)</a> </td> <td> </td>
<td align="center" rowspan="3"><tt>|<br>|<br>|<br>|<br>v</tt></td>
</tr>
<tr> <td> </td> <td> <tt> \-- </tt> </td> <td align="center"
colspan="4" bgcolor="#f0f0ff"> <a href="smtp.8.html">smtp(8)</a> </td> <td align="left">
</td> </tr>
<tr> <td colspan="2"> </td> <td> </td> <td
align="center"><tt>^<br>|</tt></td> </tr>
<tr> <td colspan="2"> </td> <td align="center" colspan="3"
bgcolor="#f0f0ff"> <a href="scache.8.html">scache(8)</a> </td> </tr>
</table>
</blockquote>
<p> With TLS connection reuse (Postfix 3.4 and later), the Postfix
<a href="smtp.8.html">smtp(8)</a> client connects to a remote SMTP server and sends plaintext
EHLO and STARTTLS commands, then inserts a <a href="tlsproxy.8.html">tlsproxy(8)</a> process into
the connection as shown below. </p>
<p> After delivering mail, the <a href="smtp.8.html">smtp(8)</a> client hands over the open
<a href="smtp.8.html">smtp(8)</a>-to-<a href="tlsproxy.8.html">tlsproxy(8)</a> connection to the <a href="scache.8.html">scache(8)</a> server, and
continues with some other mail delivery request. Meanwhile, any
<a href="smtp.8.html">smtp(8)</a> client process can ask the <a href="scache.8.html">scache(8)</a> server for that cached
connection and reuse it for mail delivery. </p>
<blockquote>
<table>
<tr> <td> </td> <td> <tt> /-- </tt> </td> <td align="center"
colspan="3" bgcolor="#f0f0ff"> <a href="smtp.8.html">smtp(8)</a> </td> <td colspan="2"> <tt>
--> </tt> </td> <td align="center"bgcolor="#f0f0ff"> <a href="tlsproxy.8.html">tlsproxy(8)</a>
</td> <td> <tt> --> </tt> </td> <td> Internet </td> </tr>
<tr> <td align="center" bgcolor="#f0f0ff"> <a href="qmgr.8.html">qmgr(8)</a> </td> <td> </td>
<td align="center" rowspan="3"><tt>|<br>|<br>|<br>|<br>v</tt></td>
</tr>
<tr> <td> </td> <td> <tt> \-- </tt> </td> <td align="center"
colspan="4" bgcolor="#f0f0ff"> <a href="smtp.8.html">smtp(8)</a> </td> <td align="left">
</td> </tr>
<tr> <td colspan="2"> </td> <td> </td> <td
align="center"><tt>^<br>|</tt></td> </tr>
<tr> <td colspan="2"> </td> <td align="center" colspan="3"
bgcolor="#f0f0ff"> <a href="scache.8.html">scache(8)</a> </td> </tr>
</table>
</blockquote>
<p> The connection cache can be searched by destination domain name
(the right-hand side of the recipient address) and by the IP address
of the host at the other end of the connection. This allows Postfix
to reuse a connection even when the remote host is a mail server for
domains with different names. </p>
<h2><a name="configuration">Connection cache configuration </a></h2>
<p> The Postfix <a href="smtp.8.html">smtp(8)</a> client supports two connection caching
strategies: </p>
<ul>
<li> <p> On-demand connection caching. This is enabled by default,
and is controlled with the <a href="postconf.5.html#smtp_connection_cache_on_demand">smtp_connection_cache_on_demand</a> configuration
parameter. When this feature is enabled, the Postfix <a href="smtp.8.html">smtp(8)</a> client
automatically saves a connection to the connection cache when a
destination has a high volume of mail in the <a href="QSHAPE_README.html#active_queue">active queue</a>. </p>
<p> Example: </p>
<blockquote>
<pre>
/etc/postfix/<a href="postconf.5.html">main.cf</a>:
<a href="postconf.5.html#smtp_connection_cache_on_demand">smtp_connection_cache_on_demand</a> = yes
</pre>
</blockquote>
<li> <p> Per-destination connection caching. This is enabled by
explicitly listing specific destinations with the
<a href="postconf.5.html#smtp_connection_cache_destinations">smtp_connection_cache_destinations</a> configuration parameter. After
completing delivery to a selected destination, the Postfix <a href="smtp.8.html">smtp(8)</a>
client <i>always</i> saves the connection to the connection cache.
</p>
<p> Specify a comma or white space separated list of destinations
or pseudo-destinations: </p>
<ul>
<li> <p> if mail is sent without a <a href="postconf.5.html#relayhost">relay host</a>: a domain name (the
right-hand side of an email address, without the [] around a numeric
IP address), </p>
<li> <p> if mail is sent via a <a href="postconf.5.html#relayhost">relay host</a>: a <a href="postconf.5.html#relayhost">relay host</a> name (without
the [] or non-default TCP port), as specified in <a href="postconf.5.html">main.cf</a> or in the
transport map, </p>
<li> <p> a /file/name with domain names and/or <a href="postconf.5.html#relayhost">relay host</a> names as
defined above, </p>
<li> <p> a "<a href="DATABASE_README.html">type:table</a>" with domain names and/or <a href="postconf.5.html#relayhost">relay host</a> names
on the left-hand side. The right-hand side result from "<a href="DATABASE_README.html">type:table</a>"
lookups is ignored. </p>
</ul>
<p> Examples: </p>
<blockquote>
<pre>
/etc/postfix/<a href="postconf.5.html">main.cf</a>:
<a href="postconf.5.html#smtp_connection_cache_destinations">smtp_connection_cache_destinations</a> = $<a href="postconf.5.html#relayhost">relayhost</a>
<a href="postconf.5.html#smtp_connection_cache_destinations">smtp_connection_cache_destinations</a> = hotmail.com, ...
<a href="postconf.5.html#smtp_connection_cache_destinations">smtp_connection_cache_destinations</a> = <a href="DATABASE_README.html#types">static</a>:all (<i>not recommended</i>)
</pre>
</blockquote>
<p> See <a href="TLS_README.html#client_tls_reuse">Client-side TLS
connection reuse</a> to enable multiple deliveries over a TLS-encrypted
connection (Postfix version 3.4 and later). </p>
</ul>
<h2><a name="safety">Connection cache safety mechanisms </a></h2>
<p> Connection caching must be used wisely. It is anti-social to
keep an unused SMTP connection open for a significant amount of
time, and it is unwise to send huge numbers of messages through
the same connection. In order to avoid problems with SMTP connection
caching, Postfix implements the following safety mechanisms: </p>
<ul>
<li> <p> The Postfix <a href="scache.8.html">scache(8)</a> server keeps a connection open for
only a limited time. The time limit is specified with the
<a href="postconf.5.html#smtp_connection_cache_time_limit">smtp_connection_cache_time_limit</a> and with the <a href="postconf.5.html#connection_cache_ttl_limit">connection_cache_ttl_limit</a>
configuration parameters. This prevents anti-social behavior. </p>
<li> <p> The Postfix <a href="smtp.8.html">smtp(8)</a> client reuses a session for only a
limited number of times. This avoids triggering bugs in implementations
that do not correctly handle multiple deliveries per session. </p>
<p> As of Postfix 2.3 connection reuse is preferably limited with
the <a href="postconf.5.html#smtp_connection_reuse_time_limit">smtp_connection_reuse_time_limit</a> parameter. In addition, Postfix
2.11 provides <a href="postconf.5.html#smtp_connection_reuse_count_limit">smtp_connection_reuse_count_limit</a> to limit how many
times a connection may be reused, but this feature is unsafe as it
introduces a "fatal attractor" failure mode (when a destination has
multiple inbound MTAs, the slowest inbound MTA will attract most
connections from Postfix to that destination). </p>
<p> Postfix 2.3 logs the use count of multiply-used connections,
as shown in the following example: </p>
<blockquote>
<pre>
Nov 3 16:04:31 myname postfix/smtp[30840]: 19B6B2900FE:
to=<wietse@test.example.com>, orig_to=<wietse@test>,
relay=mail.example.com[1.2.3.4], <b>conn_use=2</b>, delay=0.22,
delays=0.04/0.01/0.05/0.1, dsn=2.0.0, status=sent (250 2.0.0 Ok)
</pre>
</blockquote>
<li> <p> The connection cache explicitly labels each cached connection
with destination domain and IP address information. A connection
cache lookup succeeds only when the correct information is specified.
This prevents mis-delivery of mail. </p>
</ul>
<h2><a name="limitations">Connection cache limitations</a></h2>
<p> Postfix SMTP connection caching conflicts with certain applications:
</p>
<ul>
<li> <p> With Postfix versions < 3.4, the Postfix shared connection
cache cannot be used with TLS, because an open TLS connection can
be reused only in the process that creates it. For this reason,
the Postfix <a href="smtp.8.html">smtp(8)</a> client historically always closed the connection
after completing an attempt to deliver mail over TLS.</p>
<li> <p> Postfix connection caching currently does not support
multiple SASL accounts per mail server. Specifically, Postfix
connection caching assumes that a SASL credential is valid for all
hostnames or domain names that deliver via the same mail server IP
address and TCP port, and assumes that the SASL credential does not
depend on the message originator. </p>
</ul>
<h2><a name="statistics">Connection cache statistics </a></h2>
<p> The <a href="scache.8.html">scache(8)</a> connection cache server logs statistics about the
peak cache size and the cache hit rates. This information is logged
every <a href="postconf.5.html#connection_cache_status_update_time">connection_cache_status_update_time</a> seconds, when the process
terminates after the maximal idle time is exceeded, or when Postfix
is reloaded. </p>
<ul>
<li> <p> Hit rates for connection cache lookups by domain will tell
you how useful connection caching is. </p>
<li> <p> Connection cache lookups by network address will always
fail, unless you're sending mail to different domains that share
the same MX hosts. </p>
<li> <p> No statistics are logged when no attempts are made to
access the connection cache. </p>
</ul>
</body>
</html>
|
