1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
|
<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Postfix Lookup Table Overview</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel='stylesheet' type='text/css' href='postfix-doc.css'>
</head>
<body>
<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix
Lookup Table Overview</h1>
<hr>
<h2>Overview </h2>
This document covers the following topics:
<ul>
<li><a href="#intro">The Postfix lookup table model</a>
<li><a href="#lists">Postfix lists versus tables </a>
<li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>
<li><a href="#detect">Maintaining Postfix lookup table files</a>
<li><a href="#safe_db">Updating Berkeley DB files safely</a>
<li><a href="#types">Postfix lookup table types</a>
</ul>
<h2><a name="intro">The Postfix lookup table model</a></h2>
<p> Postfix uses lookup tables to store and look up information
for access control, address rewriting and even for content filtering.
All Postfix lookup tables are specified as "type:table", where
"type" is one of the database types described under "<a
href="#types">Postfix lookup table types</a>" at the end of this
document, and where "table" is the lookup table name. The Postfix
documentation uses the terms "database" and "lookup table" for the
same thing. </p>
<p> Examples of lookup tables that appear often in the Postfix
documentation: </p>
<blockquote>
<pre>
/etc/postfix/main.cf:
alias_maps = hash:/etc/postfix/aliases (local aliasing)
header_checks = regexp:/etc/postfix/header_checks (content filtering)
transport_maps = hash:/etc/postfix/transport (routing table)
virtual_alias_maps = hash:/etc/postfix/virtual (address rewriting)
</pre>
</blockquote>
<p> All Postfix lookup tables store information as (key, value)
pairs. This interface may seem simplistic at first, but it turns
out to be very powerful. The (key, value) query interface completely
hides the complexities of LDAP or SQL from Postfix. This is a good
example of connecting complex systems with simple interfaces. </p>
<p> Benefits of the Postfix (key, value) query interface:</p>
<ul>
<li> You can implement Postfix lookup tables first with local
Berkeley DB files and then switch to LDAP or MySQL without any
impact on the Postfix configuration itself, as described under "<a
href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>"
below.
<li> You can use Berkeley DB files with fixed lookup strings for
simple address rewriting operations and you can use regular expression
tables for the more complicated work. In other words, you don't
have to put everything into the same table.
</ul>
<h2><a name="lists">Postfix lists versus tables </a></h2>
<p> Most Postfix lookup tables are used to look up information.
Examples are address rewriting (the lookup string is the old address,
and the result is the new address) or access control (the lookup
string is the client, sender or recipient, and the result is an
action such as "reject"). </p>
<p> With some tables, however, Postfix needs to know only if the
lookup key exists. Any non-empty lookup result value may be used
here: the lookup result is not used. Examples
are the local_recipient_maps that determine what local recipients
Postfix accepts in mail from the network, the mydestination parameter
that specifies what domains Postfix delivers locally for, or the
mynetworks parameter that specifies the IP addresses of trusted
clients or client networks. Technically, these are lists, not
tables. Despite the difference, Postfix lists are described here
because they use the same underlying infrastructure as Postfix
lookup tables. </p>
<h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a>
</h2>
<p> LDAP and SQL are complex systems. Trying to set up both Postfix
and LDAP or SQL at the same time is definitely not a good idea.
You can save yourself a lot of time by implementing Postfix first
with local files such as Berkeley DB. Local files have few surprises,
and are easy to debug with the postmap(1) command: </p>
<blockquote>
<pre>
% <b>postmap -q info@example.com hash:/etc/postfix/virtual </b>
</pre>
</blockquote>
<p> Once you have local files working properly you can follow the
instructions in ldap_table(5), mysql_table(5), pgsql_table(5)
or sqlite_table(5)
and replace local file lookups with LDAP or SQL lookups. When you
do this, you should use the postmap(1) command again, to verify
that database lookups still produce the exact same results as local
file lookup: </p>
<blockquote>
<pre>
% <b>postmap -q info@example.com ldap:/etc/postfix/virtual.cf </b>
</pre>
</blockquote>
<p> Be sure to exercise all the partial address or parent domain
queries that are documented under "table search order" in the
relevant manual page: access(5), canonical(5), virtual(5),
transport(5), or under the relevant configuration parameter:
mynetworks, relay_domains, parent_domain_matches_subdomains. </p>
<h2><a name="detect">Maintaining Postfix lookup table files</a></h2>
<p> When you make changes to a database while the mail system is
running, it would be desirable if Postfix avoids reading information
while that information is being changed. It would also be nice if
you can change a database without having to execute "postfix reload",
in order to force Postfix to use the new information. Each time
you do "postfix reload" Postfix loses a lot of performance.
</p>
<ul>
<li> <p> If you change a network database such as LDAP, NIS or
SQL, there is no need to execute "postfix reload". The LDAP, NIS
or SQL server takes care of read/write access conflicts and gives
the new data to Postfix once that data is available. </p>
<li> <p> If you change a regexp:, pcre:, cidr: or texthash: file
then Postfix
may not pick up the file changes immediately. This is because a
Postfix process reads the entire file into memory once and never
examines the file again. </p>
<ul>
<li> <p> If the file is used by a short-running process such as
smtpd(8), cleanup(8) or local(8), there is no need to execute
"postfix reload" after making a change. </p>
<li> <p> If the file is being used by a long-running process such
as trivial-rewrite(8) on a busy server it may be necessary to
execute "postfix reload". </p>
</ul>
<li> <p> If you change a local file based database such as DBM or
Berkeley DB, there is no need to execute "postfix reload". Postfix
uses file locking to avoid read/write access conflicts, and whenever
a Postfix daemon process notices that a file has changed it will
terminate before handling the next client request, so that a new
process can initialize with the new database. </p>
</ul>
<h2><a name="safe_db">Updating Berkeley DB files safely</a></h2>
<p> Postfix uses file locking to avoid access conflicts while
updating Berkeley DB or other local database files. This used to
be safe, but as Berkeley DB has evolved to use more aggressive
caching, file locking may no longer be sufficient. </p>
<p> Furthermore, file locking would not prevent problems when the
update fails because the disk is full or something else causes a
database update to fail. In particular, commands such as postmap(1)
or postalias(1) overwrite existing files. If the overwrite
fails in the middle then you have no usable database, and Postfix
will stop working. This is not an issue with the CDB database type
available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a>
creates a new file, and renames the file upon successful completion.
</p>
<p> With Berkeley DB and other "one file" databases, it is
possible to add some extra robustness by using "mv" to REPLACE an
existing database file instead of overwriting it: </p>
<blockquote>
<pre>
# <b>postmap access.in && mv access.in.db access.db</b>
</pre>
</blockquote>
<p> This converts the input file "access.in" into the output file
"access.in.db", and replaces the file "access.db" only when the
postmap(1) command was successful. Of course typing such commands
becomes boring quickly, and this is why people use "make" instead,
as shown below. User input is shown in bold font. </p>
<blockquote>
<pre>
# <b>cat Makefile</b>
all: aliases.db access.db virtual.db ...etcetera...
# Note 1: commands are specified after a TAB character.
# Note 2: use postalias(1) for local aliases, postmap(1) for the rest.
aliases.db: aliases.in
postalias aliases.in
mv aliases.in.db aliases.db
access.db: access.in
postmap access.in
mv access.in.db access.db
virtual.db: virtual.in
postmap virtual.in
mv virtual.in.db virtual.db
...etcetera...
# <b>vi access.in</b>
...editing session not shown...
# <b>make</b>
postmap access.in
mv access.in.db access.db
#
</pre>
</blockquote>
<p> The "make" command updates only the files that have changed.
In case of error, the "make" command will stop and will not invoke
the "mv" command, so that Postfix will keep using the existing
database file as if nothing happened. </p>
<h2><a name="types">Postfix lookup table types</a> </h2>
<p> To find out what database types your Postfix system supports,
use the "<b>postconf -m</b>" command. Here is a list of database types
that are often supported: </p>
<blockquote>
<dl>
<dt> <b>btree</b> </dt>
<dd> A sorted, balanced tree structure. This is available only on
systems with support for Berkeley DB databases. Database files are
created with the postmap(1) or postalias(1) command. The lookup
table name as used in "btree:table" is the database file name
without the ".db" suffix. </dd>
<dt> <b>cdb</b> </dt>
<dd> A read-optimized structure with no support for incremental updates.
Database files are created with the postmap(1) or postalias(1) command.
The lookup table name as used in "cdb:table" is the database file name
without the ".cdb" suffix. This feature is available with Postfix 2.2
and later. </dd>
<dt> <b>cidr</b> </dt>
<dd> A table that associates values with Classless Inter-Domain
Routing (CIDR) patterns. The table format is described in cidr_table(5).
</dd>
<dt> <b>dbm</b> </dt>
<dd> An indexed file type based on hashing. This is available only
on systems with support for DBM databases. Public database files
are created with the postmap(1) or postalias(1) command, and private
databases are maintained by Postfix daemons. The lookup table name
as used in "dbm:table" is the database file name without the ".dir"
or ".pag" suffix. </dd>
<dt> <b>environ</b> </dt>
<dd> The UNIX process environment array. The lookup key is the
variable name. The lookup table name in "environ:table" is ignored.
</dd>
<dt> <b>fail</b> </dt>
<dd> A table that reliably fails all requests. The lookup table
name is used for logging only. This table exists to simplify Postfix
error tests. </dd>
<dt> <b>hash</b> </dt>
<dd> An indexed file type based on hashing. This is available only
on systems with support for Berkeley DB databases. Public database
files are created with the postmap(1) or postalias(1) command, and
private databases are maintained by Postfix daemons. The database
name as used in "hash:table" is the database file name without the
".db" suffix. </dd>
<dt> <b>inline</b> (read-only) </dt>
<dd> A non-shared, in-memory lookup table. Example: "inline:{
<i>key=value</i>, { <i>key = text with whitespace or comma</i> }}".
Key-value pairs are separated by whitespace or comma; with a key-value
pair inside "{}", whitespace is ignored after the opening "{",
around the "=" between key and value, and before the closing "}".
Inline tables eliminate the
need to create a database file for just a few fixed elements. See
also the static: map type. </dd>
<dt> <b>internal</b> </dt>
<dd> A non-shared, in-memory hash table. Its contents are lost when
a process terminates. </dd>
<dt> <b>lmdb</b> </dt>
<dd> OpenLDAP LMDB database. This is available only on systems
with support for LMDB databases. Public database files are created
with the postmap(1) or postalias(1) command, and private databases
are maintained by Postfix daemons. The database name as used in
"lmdb:table" is the database file name without the ".lmdb" suffix.
See lmdb_table(5) for details. </dd>
<dt> <b>ldap</b> (read-only) </dt>
<dd> LDAP database client. Configuration details are given in the
ldap_table(5). </dd>
<dt> <b>memcache</b> </dt>
<dd> Memcache database client. Configuration details are given in
memcache_table(5). </dd>
<dt> <b>mysql</b> (read-only) </dt>
<dd> MySQL database client. Configuration details are given in
mysql_table(5). </dd>
<dt> <b>netinfo</b> (read-only) </dt>
<dd> Netinfo database client. </dd>
<dt> <b>nis</b> (read-only) </dt>
<dd> NIS database client. </dd>
<dt> <b>nisplus</b> (read-only) </dt>
<dd> NIS+ database client. Configuration details are given in
nisplus_table(5). </dd>
<dt> <b>pcre</b> (read-only) </dt>
<dd> A lookup table based on Perl Compatible Regular Expressions.
The file format is described in pcre_table(5). The lookup table
name as used in "pcre:table" is the name of the regular expression
file. </dd>
<dt> <b>pipemap</b> (read-only) </dt>
<dd> A pipeline of lookup tables. Example:
"pipemap:{<i>type<sub>1</sub>:name<sub>1</sub>, ...,
type<sub>n</sub>:name<sub>n</sub></i>}". Each "pipemap:" query is
given to the first table. Each lookup result becomes the query for
the next table in the pipeline, and the last table produces the
final result. When any table lookup produces no result, the pipeline
produces no result. The first and last characters of the "pipemap:"
table name must be "{" and "}". Within these, individual maps are
separated with comma or whitespace. </dd>
<dt> <b>pgsql</b> (read-only) </dt>
<dd> PostgreSQL database client. Configuration details are given
in pgsql_table(5). </dd>
<dt> <b>proxy</b> </dt>
<dd> Postfix proxymap(8) client for shared access to Postfix
databases. The lookup table name syntax is "proxy:type:table".
</dd>
<dt> <b>randmap</b> (read-only) </dt>
<dd> An in-memory table that performs random selection. Example:
"randmap:{<i>result<sub>1</sub>. ..., result<sub>n</sub></i>}".
Each table query returns a random choice from the specified results.
The first and last characters of the "randmap:" table name must be
"{" and "}". Within these, individual maps are separated with comma
or whitespace. To give a specific result more weight, specify it
multiple times. </dd>
<dt> <b>regexp</b> (read-only) </dt>
<dd> A lookup table based on regular expressions. The file format
is described in regexp_table(5). The lookup table name as used in
"regexp:table" is the name of the regular expression file. </dd>
<dt> <b>sdbm</b> </dt>
<dd> An indexed file type based on hashing. This is available only
on systems with support for SDBM databases. Public database files
are created with the postmap(1) or postalias(1) command, and private
databases are maintained by Postfix daemons. The lookup table name
as used in "sdbm:table" is the database file name without the ".dir"
or ".pag" suffix. </dd>
<dt> <b>socketmap</b> (read-only) </dt>
<dd> Sendmail-style socketmap client. The name of the table is
either <b>inet</b>:<i>host</i>:<i>port</i>:<i>name</i> for a TCP/IP
server, or <b>unix</b>:<i>pathname</i>:<i>name</i> for a UNIX-domain
server. See socketmap_table(5) for details. </dd>
<dt> <b>sqlite</b> (read-only) </dt>
<dd> SQLite database. Configuration details are given in sqlite_table(5).
</dd>
<dt> <b>static</b> (read-only) </dt>
<dd> A table that always returns its name as the lookup result.
For example, "static:foobar" always returns the string "foobar" as
lookup result. Specify "static:{ <i>text with whitespace</i> }"
when the result contains whitespace; this form ignores whitespace
after the opening "{" and before the closing "}". See also the
inline: map type. </dd>
<dt> <b>tcp</b> </dt>
<dd> TCP/IP client. The protocol is described in tcp_table(5). The
lookup table name is "tcp:host:port" where "host" specifies a
symbolic hostname or a numeric IP address, and "port" specifies a
symbolic service name or a numeric port number. </dd>
<dt> <b>texthash</b> (read-only) </dt>
<dd> A table that produces similar results as hash: files, except
that you don't have to run the postmap(1) command before you can
use the file, and that texthash: does not detect changes after the
file is read. The lookup table name is "texthash:filename", where
the file name is taken literally; no suffix is appended. </dd>
<dt> <b>unionmap</b> (read-only) </dt>
<dd> A table that sends each query to multiple lookup tables and
that concatenates all found results, separated by comma. The table
name syntax is the same as for pipemap tables. </dd>
<dt> <b>unix</b> (read-only) </dt>
<dd> A limited view of the UNIX authentication database. The following
tables are implemented:
<dl>
<dt> <b>unix:passwd.byname</b> </dt>
<dd>The table is the UNIX password database. The key is a login
name. The result is a password file entry in passwd(5) format.
</dd>
<dt> <b>unix:group.byname</b> </dt>
<dd> The table is the UNIX group database. The key is a group name.
The result is a group file entry in group(5) format. </dd>
</dl> </dd>
</dl>
</blockquote>
<p> Other lookup table types may be available depending on how
Postfix was built. With some Postfix distributions the list is
dynamically extensible as support for lookup tables is dynamically
linked into Postfix. </p>
</body>
</html>
|
