diff options
Diffstat (limited to 'doc/man/doveadm-sync.1.in')
| -rw-r--r-- | doc/man/doveadm-sync.1.in | 458 |
1 files changed, 458 insertions, 0 deletions
diff --git a/doc/man/doveadm-sync.1.in b/doc/man/doveadm-sync.1.in new file mode 100644 index 0000000..9f926e3 --- /dev/null +++ b/doc/man/doveadm-sync.1.in @@ -0,0 +1,458 @@ +.\" Copyright (c) 2014-2018 Dovecot authors, see the included COPYING file +.TH DOVEADM\-SYNC 1 "2015-08-31" "Dovecot v2.3" "Dovecot" +.SH NAME +doveadm\-sync \- Dovecot\(aqs two\-way mailbox synchronization utility +.br +doveadm\-backup \- Dovecot\(aqs one\-way mailbox synchronization utility +.\"------------------------------------------------------------------------ +.SH SYNOPSIS +.BR doveadm " [" \-Dv "] " sync +[\fB\-u\fP \fIuser\fP|\fB\-A\fP|\fB\-F\fP \fIfile\fP] +[\fB\-S\fP \fIsocket_path\fP] +.RB [ \-1fPRU ] +[\fB\-l\fP \fIsecs\fP] +[\fB\-r\fP \fIrawlog path\fP] +[\fB\-m\fP \fImailbox\fP] +[\fB\-g\fP \fImailbox guid\fP] +[\fB\-n\fP \fInamespace\fP|\fB\-N\fP] +[\fB\-x\fP \fIexclude\fP] +[\fB\-a\fP \fIall mailbox\fP] +[\fB\-s\fP \fIstate\fP] +[\fB\-T\fP \fIsecs\fP] +[\fB\-t\fP \fIstart date\fP] +[\fB\-e\fP \fIend date\fP] +[\fB\-O\fP \fIsync flag\fP] +[\fB\-I\fP \fImax size\fP] +\fB\-d\fP|\fIdestination\fP +.\"------------------------------------- +.PP +.BR doveadm " [" \-Dv "] " backup +[\fB\-u\fP \fIuser\fP|\fB\-A\fP|\fB\-F\fP \fIfile\fP] +[\fB\-S\fP \fIsocket_path\fP] +.RB [ \-fPRU ] +[\fB\-l\fP \fIsecs\fP] +[\fB\-r\fP \fIrawlog path\fP] +[\fB\-m\fP \fImailbox\fP] +[\fB\-g\fP \fImailbox guid\fP] +[\fB\-n\fP \fInamespace\fP|\fB\-N\fP] +[\fB\-x\fP \fIexclude\fP] +[\fB\-a\fP \fIall mailbox\fP] +[\fB\-s\fP \fIstate\fP] +[\fB\-T\fP \fIsecs\fP] +[\fB\-t\fP \fIstart date\fP] +[\fB\-e\fP \fIend date\fP] +[\fB\-O\fP \fIsync flag\fP] +[\fB\-I\fP \fImax size\fP] +\fB\-d\fP|\fIdestination\fP +.\"------------------------------------------------------------------------ +.SH DESCRIPTION +dsync is Dovecot\(aqs mailbox synchronization utility. +It can be used for several different use cases: Two\-way synchronization of +mailboxes, creating backups of mails, and convert mailboxes from/to +different mailbox formats. +All of these can be used within the same server or between different +servers (via +.BR ssh (1) +or tcp connections). +Remote mailboxes can be accessed also via IMAP protocol, which allows using +dsync for mailbox migration purposes. +.PP +You can run dsync in one of three modes: +.RS +.\"------------------------------------- +.IP \(bu +.B doveadm backup +performs one\-way synchronization. +If there are any changes in the destination they will be deleted, so the +destination will look exactly like the source. +.\"------------------------------------- +.IP \(bu +.B doveadm sync +performs two\-way synchronization. +It merges all changes without losing anything. +Both the mailboxes will end up looking identical after the synchronization +is finished. +.\"------------------------------------- +.IP \(bu +.B doveadm sync \-1 +performs one\-way synchronization, but it merges the changes in destination +without deleting anything. +This doesn\(aqt currently work perfectly, so its use should be limited. +Its main purpose is that during mailbox migration you can run +.B doveadm backup +multiple times, then switch mails to be delivered to the new mailbox and +run +.B doveadm sync \-1 +once more to transfer any last new mails from the old mailbox. +.IP +The one\-way algorithm is the same as two-way dsync algorithm except the +source account is not modified. It fetches the message's GUID (Global UID), +which is used to identify any conflicting UIDs in messages. As long as the +source and destination side has matching UID<\->GUID mapping, those emails +are assumed to be synced correctly. Only after the first mismatch will +changes begin. +.IP +Example: Source mailbox has messages UID 1..5; source mailbox is sync'd +using +.B doveadm backup +to the destination. Subsequently, UID 6 is delivered to the source mailbox +and UID 1 is expunged from the destination mailbox. In this example, UID 1 +is kept removed (in destination) because UID 1..5 have identical +Date+Message\-ID headers. UID 6 is not seen in destination so it's copied. +.IP +If both source and destination have UID 6, but the messages are different, +the headers don't match and both the messages are kept in the destination but +they're given new UIDs 7 and 8 just to be sure any client didn't get confused +about what UID 6 actually was. Thus, one\-way sync begins to quickly diverge +from the source mailbox once changes start to occur on either side; one\-way +sync should therefore normally only be used within a short period of time +after a +.B doveadm backup +or +.B doveadm sync +command was used to synchronize the mailboxes. +.\"------------------------------------- +.RE +.PP +There are also three different synchronization algorithms: +.RS +.\"------------------------------------- +.IP \(bu +Full synchronization (\-f parameter) scans through all the messages in all +the mailboxes. +This guarantees that everything will be synchronized, but it\(aqs +unnecessarily slow for incremental synchronization. +.\"------------------------------------- +.IP \(bu +Fast synchronization (default) first attempts to find mailboxes that have +changed, and synchronize only those. +This is done by checking the mailboxes\(aq metadata (NEXTUID and +HIGHESTMODSEQ). +Usually this works fine, especially with one\-way synchronization, but if +both sides do exactly the same number of changes, the metadata may end up +containing the same values even if the changes were different. +.\"------------------------------------- +.IP \(bu +Stateful synchronization (\-s parameter) is the most efficient way to +synchronize mailboxes. +It relies on having the earlier dsync run\(aqs state saved somewhere and +being passed to the next dsync run. +Based on this state dsync can send only the changes that happened after the +previous dsync run. +As long as the state or the mailboxes aren\(aqt corrupted this algorithm +should work perfectly. +The replicator process uses this internally to perform most of the +synchronization. +.\"------------------------------------- +.RE +.PP +The syncing is done as perfectly as possible: an IMAP or a POP3 client +shouldn\(aqt be able to notice any differences between the two mailboxes. +Two\-way syncing means that it\(aqs safe to do any kind of modifications in +both sides, and dsync will merge the changes without losing any changes +done on either side. +This is possible because dsync can access Dovecot\(aqs index logs that keep +track of changes. +It\(aqs of course possible to have conflicts during merging, these are +resolved in a safe way. +See the +.I dsync design +document for more information. +.PP +dsync uses the same configuration files as the rest of Dovecot (via +.BR doveconf (1) +binary). +The entire configuration can be changed by giving \-c parameter to another +configuration file, or using \-o parameter to override specific settings. +When executing a remote dsync program it works the same way: +it uses its own local configuration. +.PP +dsync can be run completely standalone. +It doesn\(aqt require any Dovecot server processes to be running, except +when using \-u parameter to do a +.I userdb +lookup from auth process. +.PP +dsync can sync either one or multiple users using the \-u or \-A +parameters. +For continuous replication you can use the Dovecot replicator process, +which automatically runs dsync whenever messages have changed. +.\"------------------------------------------------------------------------ +@INCLUDE:global-options@ +.\" --- command specific options --- "/. +.PP +Command specific +.IR options : +@INCLUDE:option-A@ +.\"------------------------------------- +@INCLUDE:option-F-file@ +.\"------------------------------------- +.TP +.B \-1 +Do one\-way synchronization instead of two\-way synchronization. +.\"------------------------------------- +.TP +.B \-f +Do full synchronization. +.\"------------------------------------- +.TP +.B \-N +Synchronize all the available namespaces. +By default only namespaces that don\(aqt have explicit location setting +are synchronized. +.\"------------------------------------- +.TP +.B \-P +Run a +.BR doveadm\-purge (1) +for the destination (remote) storage after synchronization. +.\"------------------------------------- +.TP +.B \-R +Do a reverse sync. Normally, messages would be pushed from the local +system to the destination (remote). This option reverses the flow, and +will instead pull messages from the remote to the local storage. +.\"------------------------------------- +@INCLUDE:option-S-socket@ +.\"------------------------------------- +.TP +.BI \-T \ secs +Specify the time in seconds, how long +.BR doveadm (1) +should wait for stalled I/O operations. +The default timeout is 600 seconds. +.\"------------------------------------- +.TP +.B \-U +This is used internally by replicator to have dsync notify it when the +synchronization is finished. +.\"------------------------------------- +.TP +.B \-d +Use the default destination, which is looked up from the +.I mail_replica userdb +extra field. +.\"------------------------------------- +.TP +.BI \-g \ mailbox_guid +Same as \-m, but find the mailbox to be synchronized by its GUID instead +of by name. +.\"------------------------------------- +.TP +.BI \-l \ secs +Lock the dsync for this user. +Wait for maximum +.I secs +before giving up. +This parameter should be used to avoid broken synchronization if it\(aqs +possible that dsync is being run concurrently for the same user. +.\"------------------------------------- +.TP +.BI \-m \ mailbox +Synchronize only this mailbox name. +.\"------------------------------------- +.TP +.BI \-n \ namespace +Synchronize only the specified namespace. +This parameter can be used multiple times. +.\"------------------------------------- +.TP +.BI \-a \ all\ mailbox +Name for the "All mails" virtual mailbox. If specified, mails are attempted to +be copied from this mailbox instead of being saved separately. This may +reduce the total disk space usage as well as disk IO. +.\"------------------------------------- +.TP +.BI \-t \ start\ date +Skip any mails whose received-timestamp is older than the specified time. +.\"------------------------------------- +.TP +.BI \-e \ end\ date +Skip any mails whose received-timestamp is newer than the specified time. +.\"------------------------------------- +.TP +.BI \-O \ sync\ flag +Sync only mails that have the specified flag. +If the flag name begins with \(dq\fB\-\fP\(dq, sync all mails except the ones +with the specified flag. +.\"------------------------------------- +.TP +.BI \-I \ max\ size +Skip any mails larger than the specified size. +.\"------------------------------------- +.TP +.BI \-r \ rawlog_path +Running dsync remotely, write the remote input/output traffic to the +specified log file. +.\"------------------------------------- +.TP +.BI \-s \ previous_state +Use stateful synchronization. +If the previous state is unknown, use an empty string. +The new state is always printed to standard output. +.\"------------------------------------- +@INCLUDE:option-u-user@ +.\"------------------------------------- +.TP +.BI \-x \ mailbox_mask +Exclude the specified mailbox name/mask. +The mask may contain \(dq\fB?\fP\(dq and \(dq\fB*\fP\(dq wildcards. +The mask can also be a special-use name (e.g. \\Trash). +This parameter can be used multiple times. +.\"------------------------------------------------------------------------ +.SH ARGUMENTS +.TP +.I destination +This argument specifies the synchronized destination. +It can be one of: +.RS +.TP +location +Same as +.I mail_location +setting, e.g. maildir:\(ti/Maildir +.TP +.BI remote: login@host +Uses +.I dsync_remote_cmd +setting to connect to the remote host (usually via ssh) +.TP +.I remoteprefix:login@host +This is the same as remote, except \(dquser@domain\(rsn\(dq is sent before +dsync protocol starts. +This allows implementing a trusted wrapper script that runs doveadm +dsync\-server by reading the username from the first line. +.TP +.BI tcp: host[:port] +Connects to remote doveadm server via TCP. +The default port is specified by +.IR doveadm_port " setting." +.TP +.BI tcps: host[:port] +This is the same as tcp, but with SSL. +.TP +.BI command\ [arg1\ [,\ arg2,\ ...]] +Runs a local command that connects its standard input & output +to a dsync server. +.RE +.\"------------------------------------------------------------------------ +.SH "EXIT STATUS" +.B dsync +will exit with one of the following values: +.TP 4 +.B 0 +Synchronization was done perfectly. +.TP +.B 2 +Synchronization was done without errors, but some changes couldn\(aqt be done, +so the mailboxes aren\(aqt perfectly synchronized. Running dsync again +usually fixes this. Typically this occurs for message modification +sequences with newly created mailboxes. It can also occur if one of the +mailboxes change during the syncing. +.TP +.B 1, >2 +Synchronization failed. +.\"------------------------------------------------------------------------ +.SH EXAMPLE +.SS SYNCHRONIZATION +Synchronize mailboxes with a remote server. +Any errors are written to stderr. +.PP +.RS +.nf +.ft B +doveadm sync \-u username@example.com remote:server\-replica.example.com +.ft P +.fi +.RE +.PP +If you need more complex parameters to ssh, you can use e.g.: +.PP +.RS +.nf +.ft B +doveadm sync \-u username@example.com ssh \-i id_dsa.dovecot \(rs +mailuser@example.com doveadm dsync\-server \-u username@example.com +.ft P +.fi +.RE +.\"------------------------------------------------------------------------ +.SS CONVERTING +Assuming that the +.I mail_location +setting in +.I @pkgsysconfdir@/conf.d/10\-mail.conf +is set to: +.BR "mail_location = mdbox:\(ti/mdbox" , +a logged in system user may convert her/his mails from its Maildir in +her/his home directory to the mdbox mailbox format. +The user has to execute the command: +.PP +.RS +.nf +.ft B +doveadm sync maildir:\(ti/Maildir +.ft P +.fi +.RE +.PP +If you want to do this without any downtime, you can do the conversion one +user at a time. +Initially: +.RS 4 +.IP \(bu 4 +Configuration uses +.B mail_location = maildir:\(ti/Maildir +.IP \(bu +Set up the possibility of doing per\-user mail location using +.I userdb +extra fields. +.RE +.PP +Then for each user: +.RS 4 +.IP 1. 4 +Run +.I doveadm sync +once to do the initial conversion. +.IP 2. +Run +.I doveadm sync +again, because the initial conversion could have taken a while and new +changes could have occurred during it. +This second time only applies changes, so it should be fast. +.IP 3. +Update mail extra field in userdb to +.BR mdbox:\(ti/mdbox . +If you\(aqre using auth cache, you need to flush it, e.g. +.BR "doveadm auth cache flush" . +.IP 4. +Wait for a few seconds and then kill (doveadm kick) the user\(aqs all +existing imap and pop3 sessions (that are still using maildir). +.IP 5. +Run +.I doveadm sync +once more to apply final changes that were possibly done. +After this there should be no changes to Maildir, because the user\(aqs +mail location has been changed and all existing processes using it have +been killed. +.RE +.PP +Once all users have been converted, you can set the default +.I mail_location +to mdbox and remove the per\-user mail locations from +.IR userdb . +.\"------------------------------------------------------------------------ +@INCLUDE:reporting-bugs@ +.\"------------------------------------------------------------------------ +.SH SEE ALSO +.BR doveadm (1), +.BR doveadm\-auth (1), +.BR doveadm\-kick (1), +.BR doveadm\-purge (1), +.BR doveconf (1) +.\"------------------------------------- +.PP +Additional resources: +.IP "dsync design" +http://wiki2.dovecot.org/Design/Dsync |