diff options
Diffstat (limited to '')
| -rw-r--r-- | proto/DATABASE_README.html | 497 |
1 files changed, 497 insertions, 0 deletions
diff --git a/proto/DATABASE_README.html b/proto/DATABASE_README.html new file mode 100644 index 0000000..1db3372 --- /dev/null +++ b/proto/DATABASE_README.html @@ -0,0 +1,497 @@ +<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" + "http://www.w3.org/TR/html4/loose.dtd"> + +<html> + +<head> + +<title>Postfix Lookup Table Overview</title> + +<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> + +</head> + +<body> + +<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix +Lookup Table Overview</h1> + +<hr> + +<h2>Overview </h2> + +This document covers the following topics: + +<ul> + +<li><a href="#intro">The Postfix lookup table model</a> + +<li><a href="#lists">Postfix lists versus tables </a> + +<li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a> + +<li><a href="#detect">Maintaining Postfix lookup table files</a> + +<li><a href="#safe_db">Updating Berkeley DB files safely</a> + +<li><a href="#types">Postfix lookup table types</a> + +</ul> + +<h2><a name="intro">The Postfix lookup table model</a></h2> + +<p> Postfix uses lookup tables to store and look up information +for access control, address rewriting and even for content filtering. +All Postfix lookup tables are specified as "type:table", where +"type" is one of the database types described under "<a +href="#types">Postfix lookup table types</a>" at the end of this +document, and where "table" is the lookup table name. The Postfix +documentation uses the terms "database" and "lookup table" for the +same thing. </p> + +<p> Examples of lookup tables that appear often in the Postfix +documentation: </p> + +<blockquote> +<pre> +/etc/postfix/main.cf: + alias_maps = hash:/etc/postfix/aliases (local aliasing) + header_checks = regexp:/etc/postfix/header_checks (content filtering) + transport_maps = hash:/etc/postfix/transport (routing table) + virtual_alias_maps = hash:/etc/postfix/virtual (address rewriting) +</pre> +</blockquote> + +<p> All Postfix lookup tables store information as (key, value) +pairs. This interface may seem simplistic at first, but it turns +out to be very powerful. The (key, value) query interface completely +hides the complexities of LDAP or SQL from Postfix. This is a good +example of connecting complex systems with simple interfaces. </p> + +<p> Benefits of the Postfix (key, value) query interface:</p> + +<ul> + +<li> You can implement Postfix lookup tables first with local +Berkeley DB files and then switch to LDAP or MySQL without any +impact on the Postfix configuration itself, as described under "<a +href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>" +below. + +<li> You can use Berkeley DB files with fixed lookup strings for +simple address rewriting operations and you can use regular expression +tables for the more complicated work. In other words, you don't +have to put everything into the same table. + +</ul> + +<h2><a name="lists">Postfix lists versus tables </a></h2> + +<p> Most Postfix lookup tables are used to look up information. +Examples are address rewriting (the lookup string is the old address, +and the result is the new address) or access control (the lookup +string is the client, sender or recipient, and the result is an +action such as "reject"). </p> + +<p> With some tables, however, Postfix needs to know only if the +lookup key exists. Any non-empty lookup result value may be used +here: the lookup result is not used. Examples +are the local_recipient_maps that determine what local recipients +Postfix accepts in mail from the network, the mydestination parameter +that specifies what domains Postfix delivers locally for, or the +mynetworks parameter that specifies the IP addresses of trusted +clients or client networks. Technically, these are lists, not +tables. Despite the difference, Postfix lists are described here +because they use the same underlying infrastructure as Postfix +lookup tables. </p> + +<h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a> +</h2> + +<p> LDAP and SQL are complex systems. Trying to set up both Postfix +and LDAP or SQL at the same time is definitely not a good idea. +You can save yourself a lot of time by implementing Postfix first +with local files such as Berkeley DB. Local files have few surprises, +and are easy to debug with the postmap(1) command: </p> + +<blockquote> +<pre> +% <b>postmap -q info@example.com hash:/etc/postfix/virtual </b> +</pre> +</blockquote> + +<p> Once you have local files working properly you can follow the +instructions in ldap_table(5), mysql_table(5), pgsql_table(5) +or sqlite_table(5) +and replace local file lookups with LDAP or SQL lookups. When you +do this, you should use the postmap(1) command again, to verify +that database lookups still produce the exact same results as local +file lookup: </p> + +<blockquote> +<pre> +% <b>postmap -q info@example.com ldap:/etc/postfix/virtual.cf </b> +</pre> +</blockquote> + +<p> Be sure to exercise all the partial address or parent domain +queries that are documented under "table search order" in the +relevant manual page: access(5), canonical(5), virtual(5), +transport(5), or under the relevant configuration parameter: +mynetworks, relay_domains, parent_domain_matches_subdomains. </p> + +<h2><a name="detect">Maintaining Postfix lookup table files</a></h2> + +<p> When you make changes to a database while the mail system is +running, it would be desirable if Postfix avoids reading information +while that information is being changed. It would also be nice if +you can change a database without having to execute "postfix reload", +in order to force Postfix to use the new information. Each time +you do "postfix reload" Postfix loses a lot of performance. +</p> + +<ul> + +<li> <p> If you change a network database such as LDAP, NIS or +SQL, there is no need to execute "postfix reload". The LDAP, NIS +or SQL server takes care of read/write access conflicts and gives +the new data to Postfix once that data is available. </p> + +<li> <p> If you change a regexp:, pcre:, cidr: or texthash: file +then Postfix +may not pick up the file changes immediately. This is because a +Postfix process reads the entire file into memory once and never +examines the file again. </p> + +<ul> + +<li> <p> If the file is used by a short-running process such as +smtpd(8), cleanup(8) or local(8), there is no need to execute +"postfix reload" after making a change. </p> + +<li> <p> If the file is being used by a long-running process such +as trivial-rewrite(8) on a busy server it may be necessary to +execute "postfix reload". </p> + +</ul> + +<li> <p> If you change a local file based database such as DBM or +Berkeley DB, there is no need to execute "postfix reload". Postfix +uses file locking to avoid read/write access conflicts, and whenever +a Postfix daemon process notices that a file has changed it will +terminate before handling the next client request, so that a new +process can initialize with the new database. </p> + +</ul> + +<h2><a name="safe_db">Updating Berkeley DB files safely</a></h2> + +<p> Postfix uses file locking to avoid access conflicts while +updating Berkeley DB or other local database files. This used to +be safe, but as Berkeley DB has evolved to use more aggressive +caching, file locking may no longer be sufficient. </p> + +<p> Furthermore, file locking would not prevent problems when the +update fails because the disk is full or something else causes a +database update to fail. In particular, commands such as postmap(1) +or postalias(1) overwrite existing files. If the overwrite +fails in the middle then you have no usable database, and Postfix +will stop working. This is not an issue with the CDB database type +available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a> +creates a new file, and renames the file upon successful completion. +</p> + +<p> With Berkeley DB and other "one file" databases, it is +possible to add some extra robustness by using "mv" to REPLACE an +existing database file instead of overwriting it: </p> + +<blockquote> +<pre> +# <b>postmap access.in && mv access.in.db access.db</b> +</pre> +</blockquote> + +<p> This converts the input file "access.in" into the output file +"access.in.db", and replaces the file "access.db" only when the +postmap(1) command was successful. Of course typing such commands +becomes boring quickly, and this is why people use "make" instead, +as shown below. User input is shown in bold font. </p> + +<blockquote> +<pre> +# <b>cat Makefile</b> +all: aliases.db access.db virtual.db ...etcetera... + +# Note 1: commands are specified after a TAB character. +# Note 2: use postalias(1) for local aliases, postmap(1) for the rest. +aliases.db: aliases.in + postalias aliases.in + mv aliases.in.db aliases.db + +access.db: access.in + postmap access.in + mv access.in.db access.db + +virtual.db: virtual.in + postmap virtual.in + mv virtual.in.db virtual.db + +...etcetera... +# <b>vi access.in</b> +...editing session not shown... +# <b>make</b> +postmap access.in +mv access.in.db access.db +# +</pre> +</blockquote> + +<p> The "make" command updates only the files that have changed. +In case of error, the "make" command will stop and will not invoke +the "mv" command, so that Postfix will keep using the existing +database file as if nothing happened. </p> + +<h2><a name="types">Postfix lookup table types</a> </h2> + +<p> To find out what database types your Postfix system supports, +use the "<b>postconf -m</b>" command. Here is a list of database types +that are often supported: </p> + +<blockquote> + +<dl> + +<dt> <b>btree</b> </dt> + +<dd> A sorted, balanced tree structure. This is available only on +systems with support for Berkeley DB databases. Database files are +created with the postmap(1) or postalias(1) command. The lookup +table name as used in "btree:table" is the database file name +without the ".db" suffix. </dd> + +<dt> <b>cdb</b> </dt> + +<dd> A read-optimized structure with no support for incremental updates. +Database files are created with the postmap(1) or postalias(1) command. +The lookup table name as used in "cdb:table" is the database file name +without the ".cdb" suffix. This feature is available with Postfix 2.2 +and later. </dd> + +<dt> <b>cidr</b> </dt> + +<dd> A table that associates values with Classless Inter-Domain +Routing (CIDR) patterns. The table format is described in cidr_table(5). +</dd> + +<dt> <b>dbm</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for DBM databases. Public database files +are created with the postmap(1) or postalias(1) command, and private +databases are maintained by Postfix daemons. The lookup table name +as used in "dbm:table" is the database file name without the ".dir" +or ".pag" suffix. </dd> + +<dt> <b>environ</b> </dt> + +<dd> The UNIX process environment array. The lookup key is the +variable name. The lookup table name in "environ:table" is ignored. +</dd> + +<dt> <b>fail</b> </dt> + +<dd> A table that reliably fails all requests. The lookup table +name is used for logging only. This table exists to simplify Postfix +error tests. </dd> + +<dt> <b>hash</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for Berkeley DB databases. Public database +files are created with the postmap(1) or postalias(1) command, and +private databases are maintained by Postfix daemons. The database +name as used in "hash:table" is the database file name without the +".db" suffix. </dd> + +<dt> <b>inline</b> (read-only) </dt> + +<dd> A non-shared, in-memory lookup table. Example: "inline:{ +<i>key=value</i>, { <i>key = text with whitespace or comma</i> }}". +Key-value pairs are separated by whitespace or comma; with a key-value +pair inside "{}", whitespace is ignored after the opening "{", +around the "=" between key and value, and before the closing "}". +Inline tables eliminate the +need to create a database file for just a few fixed elements. See +also the static: map type. </dd> + +<dt> <b>internal</b> </dt> + +<dd> A non-shared, in-memory hash table. Its contents are lost when +a process terminates. </dd> + +<dt> <b>lmdb</b> </dt> + +<dd> OpenLDAP LMDB database. This is available only on systems +with support for LMDB databases. Public database files are created +with the postmap(1) or postalias(1) command, and private databases +are maintained by Postfix daemons. The database name as used in +"lmdb:table" is the database file name without the ".lmdb" suffix. +See lmdb_table(5) for details. </dd> + +<dt> <b>ldap</b> (read-only) </dt> + +<dd> LDAP database client. Configuration details are given in the +ldap_table(5). </dd> + +<dt> <b>memcache</b> </dt> + +<dd> Memcache database client. Configuration details are given in +memcache_table(5). </dd> + +<dt> <b>mysql</b> (read-only) </dt> + +<dd> MySQL database client. Configuration details are given in +mysql_table(5). </dd> + +<dt> <b>netinfo</b> (read-only) </dt> + +<dd> Netinfo database client. </dd> + +<dt> <b>nis</b> (read-only) </dt> + +<dd> NIS database client. </dd> + +<dt> <b>nisplus</b> (read-only) </dt> + +<dd> NIS+ database client. Configuration details are given in +nisplus_table(5). </dd> + +<dt> <b>pcre</b> (read-only) </dt> + +<dd> A lookup table based on Perl Compatible Regular Expressions. +The file format is described in pcre_table(5). The lookup table +name as used in "pcre:table" is the name of the regular expression +file. </dd> + +<dt> <b>pipemap</b> (read-only) </dt> + +<dd> A pipeline of lookup tables. Example: +"pipemap:{<i>type<sub>1</sub>:name<sub>1</sub>, ..., +type<sub>n</sub>:name<sub>n</sub></i>}". Each "pipemap:" query is +given to the first table. Each lookup result becomes the query for +the next table in the pipeline, and the last table produces the +final result. When any table lookup produces no result, the pipeline +produces no result. The first and last characters of the "pipemap:" +table name must be "{" and "}". Within these, individual maps are +separated with comma or whitespace. </dd> + +<dt> <b>pgsql</b> (read-only) </dt> + +<dd> PostgreSQL database client. Configuration details are given +in pgsql_table(5). </dd> + +<dt> <b>proxy</b> </dt> + +<dd> Postfix proxymap(8) client for shared access to Postfix +databases. The lookup table name syntax is "proxy:type:table". +</dd> + +<dt> <b>randmap</b> (read-only) </dt> + +<dd> An in-memory table that performs random selection. Example: +"randmap:{<i>result<sub>1</sub>. ..., result<sub>n</sub></i>}". +Each table query returns a random choice from the specified results. +The first and last characters of the "randmap:" table name must be +"{" and "}". Within these, individual maps are separated with comma +or whitespace. To give a specific result more weight, specify it +multiple times. </dd> + +<dt> <b>regexp</b> (read-only) </dt> + +<dd> A lookup table based on regular expressions. The file format +is described in regexp_table(5). The lookup table name as used in +"regexp:table" is the name of the regular expression file. </dd> + +<dt> <b>sdbm</b> </dt> + +<dd> An indexed file type based on hashing. This is available only +on systems with support for SDBM databases. Public database files +are created with the postmap(1) or postalias(1) command, and private +databases are maintained by Postfix daemons. The lookup table name +as used in "sdbm:table" is the database file name without the ".dir" +or ".pag" suffix. </dd> + +<dt> <b>socketmap</b> (read-only) </dt> + +<dd> Sendmail-style socketmap client. The name of the table is +either <b>inet</b>:<i>host</i>:<i>port</i>:<i>name</i> for a TCP/IP +server, or <b>unix</b>:<i>pathname</i>:<i>name</i> for a UNIX-domain +server. See socketmap_table(5) for details. </dd> + +<dt> <b>sqlite</b> (read-only) </dt> + +<dd> SQLite database. Configuration details are given in sqlite_table(5). +</dd> + +<dt> <b>static</b> (read-only) </dt> + +<dd> A table that always returns its name as the lookup result. +For example, "static:foobar" always returns the string "foobar" as +lookup result. Specify "static:{ <i>text with whitespace</i> }" +when the result contains whitespace; this form ignores whitespace +after the opening "{" and before the closing "}". See also the +inline: map type. </dd> + +<dt> <b>tcp</b> </dt> + +<dd> TCP/IP client. The protocol is described in tcp_table(5). The +lookup table name is "tcp:host:port" where "host" specifies a +symbolic hostname or a numeric IP address, and "port" specifies a +symbolic service name or a numeric port number. </dd> + +<dt> <b>texthash</b> (read-only) </dt> + +<dd> A table that produces similar results as hash: files, except +that you don't have to run the postmap(1) command before you can +use the file, and that texthash: does not detect changes after the +file is read. The lookup table name is "texthash:filename", where +the file name is taken literally; no suffix is appended. </dd> + +<dt> <b>unionmap</b> (read-only) </dt> + +<dd> A table that sends each query to multiple lookup tables and +that concatenates all found results, separated by comma. The table +name syntax is the same as for pipemap tables. </dd> + +<dt> <b>unix</b> (read-only) </dt> + +<dd> A limited view of the UNIX authentication database. The following +tables are implemented: + +<dl> + +<dt> <b>unix:passwd.byname</b> </dt> + +<dd>The table is the UNIX password database. The key is a login +name. The result is a password file entry in passwd(5) format. +</dd> + +<dt> <b>unix:group.byname</b> </dt> + +<dd> The table is the UNIX group database. The key is a group name. +The result is a group file entry in group(5) format. </dd> + +</dl> </dd> + +</dl> + +</blockquote> + +<p> Other lookup table types may be available depending on how +Postfix was built. With some Postfix distributions the list is +dynamically extensible as support for lookup tables is dynamically +linked into Postfix. </p> + +</body> + +</html> |