diff options
Diffstat (limited to 'html/DATABASE_README.html')
| -rw-r--r-- | html/DATABASE_README.html | 497 |
1 files changed, 497 insertions, 0 deletions
diff --git a/html/DATABASE_README.html b/html/DATABASE_README.html new file mode 100644 index 0000000..1305fdb --- /dev/null +++ b/html/DATABASE_README.html @@ -0,0 +1,497 @@ +<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" + "http://www.w3.org/TR/html4/loose.dtd"> + +<html> + +<head> + +<title>Postfix Lookup Table Overview</title> + +<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> + +</head> + +<body> + +<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix +Lookup Table Overview</h1> + +<hr> + +<h2>Overview </h2> + +This document covers the following topics: + +<ul> + +<li><a href="#intro">The Postfix lookup table model</a> + +<li><a href="#lists">Postfix lists versus tables </a> + +<li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a> + +<li><a href="#detect">Maintaining Postfix lookup table files</a> + +<li><a href="#safe_db">Updating Berkeley DB files safely</a> + +<li><a href="#types">Postfix lookup table types</a> + +</ul> + +<h2><a name="intro">The Postfix lookup table model</a></h2> + +<p> Postfix uses lookup tables to store and look up information +for access control, address rewriting and even for content filtering. +All Postfix lookup tables are specified as "<a href="DATABASE_README.html">type:table</a>", where +"type" is one of the database types described under "<a +href="#types">Postfix lookup table types</a>" at the end of this +document, and where "table" is the lookup table name. The Postfix +documentation uses the terms "database" and "lookup table" for the +same thing. </p> + +<p> Examples of lookup tables that appear often in the Postfix +documentation: </p> + +<blockquote> +<pre> +/etc/postfix/<a href="postconf.5.html">main.cf</a>: + <a href="postconf.5.html#alias_maps">alias_maps</a> = <a href="DATABASE_README.html#types">hash</a>:/etc/postfix/aliases (local aliasing) + <a href="postconf.5.html#header_checks">header_checks</a> = <a href="regexp_table.5.html">regexp</a>:/etc/postfix/header_checks (content filtering) + <a href="postconf.5.html#transport_maps">transport_maps</a> = <a href="DATABASE_README.html#types">hash</a>:/etc/postfix/transport (routing table) + <a href="postconf.5.html#virtual_alias_maps">virtual_alias_maps</a> = <a href="DATABASE_README.html#types">hash</a>:/etc/postfix/virtual (address rewriting) +</pre> +</blockquote> + +<p> All Postfix lookup tables store information as (key, value) +pairs. This interface may seem simplistic at first, but it turns +out to be very powerful. The (key, value) query interface completely +hides the complexities of LDAP or SQL from Postfix. This is a good +example of connecting complex systems with simple interfaces. </p> + +<p> Benefits of the Postfix (key, value) query interface:</p> + +<ul> + +<li> You can implement Postfix lookup tables first with local +Berkeley DB files and then switch to LDAP or MySQL without any +impact on the Postfix configuration itself, as described under "<a +href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>" +below. + +<li> You can use Berkeley DB files with fixed lookup strings for +simple address rewriting operations and you can use regular expression +tables for the more complicated work. In other words, you don't +have to put everything into the same table. + +</ul> + +<h2><a name="lists">Postfix lists versus tables </a></h2> + +<p> Most Postfix lookup tables are used to look up information. +Examples are address rewriting (the lookup string is the old address, +and the result is the new address) or access control (the lookup +string is the client, sender or recipient, and the result is an +action such as "reject"). </p> + +<p> With some tables, however, Postfix needs to know only if the +lookup key exists. Any non-empty lookup result value may be used +here: the lookup result is not used. Examples +are the <a href="postconf.5.html#local_recipient_maps">local_recipient_maps</a> that determine what local recipients +Postfix accepts in mail from the network, the <a href="postconf.5.html#mydestination">mydestination</a> parameter +that specifies what domains Postfix delivers locally for, or the +<a href="postconf.5.html#mynetworks">mynetworks</a> parameter that specifies the IP addresses of trusted +clients or client networks. Technically, these are lists, not +tables. Despite the difference, Postfix lists are described here +because they use the same underlying infrastructure as Postfix +lookup tables. </p> + +<h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a> +</h2> + +<p> LDAP and SQL are complex systems. Trying to set up both Postfix +and LDAP or SQL at the same time is definitely not a good idea. +You can save yourself a lot of time by implementing Postfix first +with local files such as Berkeley DB. Local files have few surprises, +and are easy to debug with the <a href="postmap.1.html">postmap(1)</a> command: </p> + +<blockquote> +<pre> +% <b>postmap -q info@example.com <a href="DATABASE_README.html#types">hash</a>:/etc/postfix/virtual </b> +</pre> +</blockquote> + +<p> Once you have local files working properly you can follow the +instructions in <a href="ldap_table.5.html">ldap_table(5)</a>, <a href="mysql_table.5.html">mysql_table(5)</a>, <a href="pgsql_table.5.html">pgsql_table(5)</a> +or <a href="sqlite_table.5.html">sqlite_table(5)</a> +and replace local file lookups with LDAP or SQL lookups. When you +do this, you should use the <a href="postmap.1.html">postmap(1)</a> command again, to verify +that database lookups still produce the exact same results as local +file lookup: </p> + +<blockquote> +<pre> +% <b>postmap -q info@example.com <a href="ldap_table.5.html">ldap</a>:/etc/postfix/virtual.cf </b> +</pre> +</blockquote> + +<p> Be sure to exercise all the partial address or parent domain +queries that are documented under "table search order" in the +relevant manual page: <a href="access.5.html">access(5)</a>, <a href="canonical.5.html">canonical(5)</a>, <a href="virtual.5.html">virtual(5)</a>, +<a href="transport.5.html">transport(5)</a>, or under the relevant configuration parameter: +<a href="postconf.5.html#mynetworks">mynetworks</a>, <a href="postconf.5.html#relay_domains">relay_domains</a>, <a href="postconf.5.html#parent_domain_matches_subdomains">parent_domain_matches_subdomains</a>. </p> + +<h2><a name="detect">Maintaining Postfix lookup table files</a></h2> + +<p> When you make changes to a database while the mail system is +running, it would be desirable if Postfix avoids reading information +while that information is being changed. It would also be nice if +you can change a database without having to execute "postfix reload", +in order to force Postfix to use the new information. Each time +you do "postfix reload" Postfix loses a lot of performance. +</p> + +<ul> + +<li> <p> If you change a network database such as LDAP, NIS or +SQL, there is no need to execute "postfix reload". The LDAP, NIS +or SQL server takes care of read/write access conflicts and gives +the new data to Postfix once that data is available. </p> + +<li> <p> If you change a <a href="regexp_table.5.html">regexp</a>:, <a href="pcre_table.5.html">pcre</a>:, <a href="cidr_table.5.html">cidr</a>: or <a href="DATABASE_README.html#types">texthash</a>: file +then Postfix +may not pick up the file changes immediately. This is because a +Postfix process reads the entire file into memory once and never +examines the file again. </p> + +<ul> + +<li> <p> If the file is used by a short-running process such as +<a href="smtpd.8.html">smtpd(8)</a>, <a href="cleanup.8.html">cleanup(8)</a> or <a href="local.8.html">local(8)</a>, there is no need to execute +"postfix reload" after making a change. </p> + +<li> <p> If the file is being used by a long-running process such +as <a href="trivial-rewrite.8.html">trivial-rewrite(8)</a> on a busy server it may be necessary to +execute "postfix reload". </p> + +</ul> + +<li> <p> If you change a local file based database such as DBM or +Berkeley DB, there is no need to execute "postfix reload". Postfix +uses file locking to avoid read/write access conflicts, and whenever +a Postfix daemon process notices that a file has changed it will +terminate before handling the next client request, so that a new +process can initialize with the new database. </p> + +</ul> + +<h2><a name="safe_db">Updating Berkeley DB files safely</a></h2> + +<p> Postfix uses file locking to avoid access conflicts while +updating Berkeley DB or other local database files. This used to +be safe, but as Berkeley DB has evolved to use more aggressive +caching, file locking may no longer be sufficient. </p> + +<p> Furthermore, file locking would not prevent problems when the +update fails because the disk is full or something else causes a +database update to fail. In particular, commands such as <a href="postmap.1.html">postmap(1)</a> +or <a href="postalias.1.html">postalias(1)</a> overwrite existing files. If the overwrite +fails in the middle then you have no usable database, and Postfix +will stop working. This is not an issue with the CDB database type +available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a> +creates a new file, and renames the file upon successful completion. +</p> + +<p> With Berkeley DB and other "one file" databases, it is +possible to add some extra robustness by using "mv" to REPLACE an +existing database file instead of overwriting it: </p> + +<blockquote> +<pre> +# <b>postmap access.in && mv access.in.db access.db</b> +</pre> +</blockquote> + +<p> This converts the input file "access.in" into the output file +"access.in.db", and replaces the file "access.db" only when the +<a href="postmap.1.html">postmap(1)</a> command was successful. Of course typing such commands +becomes boring quickly, and this is why people use "make" instead, +as shown below. User input is shown in bold font. </p> + +<blockquote> +<pre> +# <b>cat Makefile</b> +all: aliases.db access.db virtual.db ...etcetera... + +# Note 1: commands are specified after a TAB character. +# Note 2: use <a href="postalias.1.html">postalias(1)</a> for local aliases, <a href="postmap.1.html">postmap(1)</a> for the rest. +aliases.db: aliases.in + postalias aliases.in + mv aliases.in.db aliases.db + +access.db: access.in + postmap access.in + mv access.in.db access.db + +virtual.db: virtual.in + postmap virtual.in + mv virtual.in.db virtual.db + +...etcetera... +# <b>vi access.in</b> +...editing session not shown... +# <b>make</b> +postmap access.in +mv access.in.db access.db +# +</pre> +</blockquote> + +<p> The "make" command updates only the files that have changed. +In case of error, the "make" command will stop and will not invoke +the "mv" command, so that Postfix will keep using the existing +database file as if nothing happened. </p> + +<h2><a name="types">Postfix lookup table types</a> </h2> + +<p> To find out what database types your Postfix system supports, +use the "<b>postconf -m</b>" command. Here is a list of database types +that are often supported: </p> + +<blockquote> + +<dl> + +<dt> <b>btree</b> </dt> + +<dd> A sorted, balanced tree structure. This is available only on +systems with support for Berkeley DB databases. Database files are +created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup +table name as used in "<a href="DATABASE_README.html#types">btree</a>:table" is the database file name +without the ".db" suffix. </dd> + +<dt> <b>cdb</b> </dt> + +<dd> A read-optimized structure with no support for incremental updates. +Database files are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. +The lookup table name as used in "<a href="CDB_README.html">cdb</a>:table" is the database file name +without the ".cdb" suffix. This feature is available with Postfix 2.2 +and later. </dd> + +<dt> <b>cidr</b> </dt> + +<dd> A table that associates values with Classless Inter-Domain +Routing (CIDR) patterns. The table format is described in <a href="cidr_table.5.html">cidr_table(5)</a>. +</dd> + +<dt> <b>dbm</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for DBM databases. Public database files +are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command, and private +databases are maintained by Postfix daemons. The lookup table name +as used in "<a href="DATABASE_README.html#types">dbm</a>:table" is the database file name without the ".dir" +or ".pag" suffix. </dd> + +<dt> <b>environ</b> </dt> + +<dd> The UNIX process environment array. The lookup key is the +variable name. The lookup table name in "<a href="DATABASE_README.html#types">environ</a>:table" is ignored. +</dd> + +<dt> <b>fail</b> </dt> + +<dd> A table that reliably fails all requests. The lookup table +name is used for logging only. This table exists to simplify Postfix +error tests. </dd> + +<dt> <b>hash</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for Berkeley DB databases. Public database +files are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command, and +private databases are maintained by Postfix daemons. The database +name as used in "<a href="DATABASE_README.html#types">hash</a>:table" is the database file name without the +".db" suffix. </dd> + +<dt> <b>inline</b> (read-only) </dt> + +<dd> A non-shared, in-memory lookup table. Example: "<a href="DATABASE_README.html#types">inline</a>:{ +<i>key=value</i>, { <i>key = text with whitespace or comma</i> }}". +Key-value pairs are separated by whitespace or comma; with a key-value +pair inside "{}", whitespace is ignored after the opening "{", +around the "=" between key and value, and before the closing "}". +Inline tables eliminate the +need to create a database file for just a few fixed elements. See +also the <a href="DATABASE_README.html#types">static</a>: map type. </dd> + +<dt> <b>internal</b> </dt> + +<dd> A non-shared, in-memory hash table. Its contents are lost when +a process terminates. </dd> + +<dt> <b>lmdb</b> </dt> + +<dd> OpenLDAP LMDB database. This is available only on systems +with support for LMDB databases. Public database files are created +with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command, and private databases +are maintained by Postfix daemons. The database name as used in +"<a href="lmdb_table.5.html">lmdb</a>:table" is the database file name without the ".lmdb" suffix. +See <a href="lmdb_table.5.html">lmdb_table(5)</a> for details. </dd> + +<dt> <b>ldap</b> (read-only) </dt> + +<dd> LDAP database client. Configuration details are given in the +<a href="ldap_table.5.html">ldap_table(5)</a>. </dd> + +<dt> <b>memcache</b> </dt> + +<dd> Memcache database client. Configuration details are given in +<a href="memcache_table.5.html">memcache_table(5)</a>. </dd> + +<dt> <b>mysql</b> (read-only) </dt> + +<dd> MySQL database client. Configuration details are given in +<a href="mysql_table.5.html">mysql_table(5)</a>. </dd> + +<dt> <b>netinfo</b> (read-only) </dt> + +<dd> Netinfo database client. </dd> + +<dt> <b>nis</b> (read-only) </dt> + +<dd> NIS database client. </dd> + +<dt> <b>nisplus</b> (read-only) </dt> + +<dd> NIS+ database client. Configuration details are given in +<a href="nisplus_table.5.html">nisplus_table(5)</a>. </dd> + +<dt> <b>pcre</b> (read-only) </dt> + +<dd> A lookup table based on Perl Compatible Regular Expressions. +The file format is described in <a href="pcre_table.5.html">pcre_table(5)</a>. The lookup table +name as used in "<a href="pcre_table.5.html">pcre</a>:table" is the name of the regular expression +file. </dd> + +<dt> <b>pipemap</b> (read-only) </dt> + +<dd> A pipeline of lookup tables. Example: +"<a href="DATABASE_README.html#types">pipemap</a>:{<i>type<sub>1</sub>:name<sub>1</sub>, ..., +type<sub>n</sub>:name<sub>n</sub></i>}". Each "<a href="DATABASE_README.html#types">pipemap</a>:" query is +given to the first table. Each lookup result becomes the query for +the next table in the pipeline, and the last table produces the +final result. When any table lookup produces no result, the pipeline +produces no result. The first and last characters of the "<a href="DATABASE_README.html#types">pipemap</a>:" +table name must be "{" and "}". Within these, individual maps are +separated with comma or whitespace. </dd> + +<dt> <b>pgsql</b> (read-only) </dt> + +<dd> PostgreSQL database client. Configuration details are given +in <a href="pgsql_table.5.html">pgsql_table(5)</a>. </dd> + +<dt> <b>proxy</b> </dt> + +<dd> Postfix <a href="proxymap.8.html">proxymap(8)</a> client for shared access to Postfix +databases. The lookup table name syntax is "<a href="proxymap.8.html">proxy</a>:<a href="DATABASE_README.html">type:table</a>". +</dd> + +<dt> <b>randmap</b> (read-only) </dt> + +<dd> An in-memory table that performs random selection. Example: +"<a href="DATABASE_README.html#types">randmap</a>:{<i>result<sub>1</sub>. ..., result<sub>n</sub></i>}". +Each table query returns a random choice from the specified results. +The first and last characters of the "<a href="DATABASE_README.html#types">randmap</a>:" table name must be +"{" and "}". Within these, individual maps are separated with comma +or whitespace. To give a specific result more weight, specify it +multiple times. </dd> + +<dt> <b>regexp</b> (read-only) </dt> + +<dd> A lookup table based on regular expressions. The file format +is described in <a href="regexp_table.5.html">regexp_table(5)</a>. The lookup table name as used in +"<a href="regexp_table.5.html">regexp</a>:table" is the name of the regular expression file. </dd> + +<dt> <b>sdbm</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for SDBM databases. Public database files +are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command, and private +databases are maintained by Postfix daemons. The lookup table name +as used in "<a href="DATABASE_README.html#types">sdbm</a>:table" is the database file name without the ".dir" +or ".pag" suffix. </dd> + +<dt> <b>socketmap</b> (read-only) </dt> + +<dd> Sendmail-style socketmap client. The name of the table is +either <b>inet</b>:<i>host</i>:<i>port</i>:<i>name</i> for a TCP/IP +server, or <b>unix</b>:<i>pathname</i>:<i>name</i> for a UNIX-domain +server. See <a href="socketmap_table.5.html">socketmap_table(5)</a> for details. </dd> + +<dt> <b>sqlite</b> (read-only) </dt> + +<dd> SQLite database. Configuration details are given in <a href="sqlite_table.5.html">sqlite_table(5)</a>. +</dd> + +<dt> <b>static</b> (read-only) </dt> + +<dd> A table that always returns its name as the lookup result. +For example, "<a href="DATABASE_README.html#types">static</a>:foobar" always returns the string "foobar" as +lookup result. Specify "<a href="DATABASE_README.html#types">static</a>:{ <i>text with whitespace</i> }" +when the result contains whitespace; this form ignores whitespace +after the opening "{" and before the closing "}". See also the +<a href="DATABASE_README.html#types">inline</a>: map type. </dd> + +<dt> <b>tcp</b> </dt> + +<dd> TCP/IP client. The protocol is described in <a href="tcp_table.5.html">tcp_table(5)</a>. The +lookup table name is "<a href="tcp_table.5.html">tcp</a>:host:port" where "host" specifies a +symbolic hostname or a numeric IP address, and "port" specifies a +symbolic service name or a numeric port number. </dd> + +<dt> <b>texthash</b> (read-only) </dt> + +<dd> A table that produces similar results as <a href="DATABASE_README.html#types">hash</a>: files, except +that you don't have to run the <a href="postmap.1.html">postmap(1)</a> command before you can +use the file, and that <a href="DATABASE_README.html#types">texthash</a>: does not detect changes after the +file is read. The lookup table name is "<a href="DATABASE_README.html#types">texthash</a>:filename", where +the file name is taken literally; no suffix is appended. </dd> + +<dt> <b>unionmap</b> (read-only) </dt> + +<dd> A table that sends each query to multiple lookup tables and +that concatenates all found results, separated by comma. The table +name syntax is the same as for pipemap tables. </dd> + +<dt> <b>unix</b> (read-only) </dt> + +<dd> A limited view of the UNIX authentication database. The following +tables are implemented: + +<dl> + +<dt> <b>unix:passwd.byname</b> </dt> + +<dd>The table is the UNIX password database. The key is a login +name. The result is a password file entry in passwd(5) format. +</dd> + +<dt> <b>unix:group.byname</b> </dt> + +<dd> The table is the UNIX group database. The key is a group name. +The result is a group file entry in group(5) format. </dd> + +</dl> </dd> + +</dl> + +</blockquote> + +<p> Other lookup table types may be available depending on how +Postfix was built. With some Postfix distributions the list is +dynamically extensible as support for lookup tables is dynamically +linked into Postfix. </p> + +</body> + +</html> |