Adding upstream version 5.4.4.upstream/5.4.4

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
author: Daniel Baumann <daniel.baumann@progress-linux.org> 2024-04-15 19:12:14 +0000
committer: Daniel Baumann <daniel.baumann@progress-linux.org> 2024-04-15 19:12:14 +0000
commit: 4b8a0f3f3dcf60dac2ce308ea08d413a535af29f (patch)
tree: 0f09c0ad2a4d0f535d89040a63dc3a866a6606e6 /docs/manual.html
parent: Initial commit. (diff)
download: reprepro-4b8a0f3f3dcf60dac2ce308ea08d413a535af29f.tar.xz
reprepro-4b8a0f3f3dcf60dac2ce308ea08d413a535af29f.zip
1 files changed, 1497 insertions, 0 deletions
diff --git a/docs/manual.html b/docs/manual.html
new file mode 100644
index 0000000..af6a993
--- /dev/null
+++ b/docs/manual.html
@@ -0,0 +1,1497 @@
+<html><head>
+<title>reprepro manual</title>
+<!-- some style elements stolen from or inspired by bugs.debian.org /-->
+<style>
+<!--
+html { color: #000; background: #fefefe; font-family: serif; margin: 1em; border: 0; padding: 0; line-height: 120%; }
+body { margin: 0; border: 0; padding: 0; }
+pre { text-align: left; border: #f0f0f0 1px solid;  padding: 1px;}
+pre.shell { text-align: left; border: none; border-left: #f0f0f0 1px solid;  padding: 2px;}
+h1, h2, h3 { text-align: left; font-family: sans-serif; background-color: #f0f0ff; color: #3c3c3c; border: #a7a7a7 1px solid;  padding: 10px;}
+h1 { font-size: 180%; line-height: 150%; }
+h2 { font-size: 150% }
+h3 { font-size: 100% }
+ul.dir { list-style-type: disc; }
+ul { list-style-type: square; }
+dt.dir, dt.file, dt.symlink { font-weight:bold; font-family: sans-serif; }
+/-->
+</style>
+</head>
+<body>
+<h1>reprepro manual</h1>
+This manual documents reprepro, a tool to generate and administer
+Debian package repositories.
+<br>
+Other useful resources:
+<ul>
+<li> the <a href="http://mirrorer.alioth.debian.org/">homepage</a> of  reprepro.</li>
+<li> <a href="file://localhost/usr/share/doc/reprepro/">local directory</a> with documentation and examples, if you have reprepro installed.</li>
+<li> the <a href="http://git.debian.org/?p=mirrorer/reprepro.git;a=blob_plain;f=docs/FAQ;hb=HEAD">Frequently Asked Questions</a></li>
+</ul>
+<h2>Table of contents</h2>
+Sections of this document:
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#firststeps">First steps</a></li>
+<li><a href="#dirbasics">Repository basics</a></li>
+<li><a href="#config">Config files</a></li>
+<li><a href="#export">Generation of index files</a>
+ <ul>
+ <li><a href="#compression">Compression and file names</a></li>
+ <li><a href="#signing">Signing</a></li>
+ <li><a href="#contents">Contents files</a></li>
+ <li><a href="#exporthook">Additional index files (like .diff)</a></li>
+ </ul></li>
+<li><a href="#localpackages">Local packages</a>
+ <ul>
+ <li><a href="#include">Including via command line</a></li>
+ <li><a href="#incoming">Processing an incoming queue</a></li>
+ </ul></li>
+<li><a href="#mirroring">Mirroring</a></li>
+<li><a href="#propagation">Propagation of packages</a></li>
+<li><a href="#snapshosts">Snapshots</a> (TODO)</li>
+<li><a href="#tracking">Source package tracking</a> (TODO)</li>
+<li><a href="#hooks">Extending reprepro / Hooks and more</a></li>
+<li><a href="#maintenance">Maintenance</a></li>
+<li><a href="#internals">Internals</a></li>
+<li><a href="#recovery">Disaster recovery</a></li>
+<li><a href="#paranoia">Paranoia</a></li>
+<li><a href="#counterindications">What reprepro cannot do</a></li>
+</ul>
+<h2><a name="introduction">Introduction</a></h2>
+<h3>What reprepro does</h3>
+Reprepro is a tool to take care of a repository of Debian packages
+(<tt>.dsc</tt>,<tt>.deb</tt> and <tt>.udeb</tt>).
+It installs them to the proper places, generates indices of packages
+(<tt>Packages</tt> and <tt>Sources</tt> and their compressed variants)
+and of index files (<tt>Release</tt> and optionally <tt>Release.gpg</tt>),
+so tools like <tt>apt</tt> know what is available and where to get it from.
+It will keep track which file belongs to where and remove files no longer
+needed (unless told to not do so).
+It can also make (partial) partial mirrors of remote repositories,
+including merging multiple sources and
+automatically (if explicitly requested) removing packages no longer available
+in the source.
+And many other things (sometimes I fear it got a few features too much).
+<h3>What reprepro needs</h3>
+It needs some libraries (<tt>zlib</tt>, <tt>libgpgme</tt>, <tt>libdb</tt> (Version 3, 4.3 or 4.4)) and can be compiled with some more for additional features (<tt>libarchive</tt>,
+<tt>libbz2</tt>).
+Otherwise it only needs
+<tt>apt</tt>'s methods (only when downloading stuff),
+<tt>gpg</tt> (only when signing or checking signatures),
+and if compiled without <tt>libarchive</tt> it needs <tt>tar</tt> and <tt>ar</tt> installed.
+<br>
+If you tell reprepro to call scripts for you, you will of course need the interpreters for these scripts:
+The included example to generate pdiff files needs python. The example to extract
+changelogs needs dpkg-source.
+<h3>What this manual aims to do</h3>
+This manual aims to give some overview over the most important features,
+so people can use them and so that I do not implement something a second
+time because I forgot support is already there.
+For a full reference of all possible commands and config options take a
+look at the man page, as this manual might miss some of the more obscure
+options.
+<h2><a name="firststeps">First steps</a></h2>
+<h3>generate a repository with local packages</h3>
+<ul>
+<li>Choose a directory (or create it).</li>
+<li>Create a subdirectory called <tt>conf</tt> in there.</li>
+<li>In the <tt>conf/</tt> subdirectory create a file called <tt>distributions</tt>,
+with content like:
+<pre class="file">
+Codename: mystuff
+Components: main bad
+Architectures: sparc i386 source
+</pre>
+or with content like:
+<pre class="file">
+Codename: andy
+Suite: rusty
+Components: main bad
+Architectures: sparc i386 source
+Origin: myorg
+Version: 20.3
+Description: my first little repository
+</pre>
+(Multiple distributions are separated by empty lines, Origin, Version and Description
+are just copied to the generated Release files, more things controlling reprepro can
+appear which are described later).
+</li>
+<li>If your <tt>conf/distributions</tt> file contained a <tt>Suite:</tt> and you
+are too lazy to generate the symbolic links yourself, call:
+<pre class="shell">
+reprepro -b $YOURBASEDIR createsymlinks
+</pre>
+</li>
+<li>Include some package, like:
+<pre class="shell">
+reprepro -b $YOURBASEDIR include mystuff mypackage.changes
+</pre>
+or:
+<pre class="shell">
+reprepro -b $YOURBASEDIR includedeb mystuff mypackage.deb
+</pre>
+</li>
+<li>Take a look at the generated <tt>pool</tt> and <tt>dists</tt>
+directories. They contain everything needed to apt-get from.
+Tell apt to include it by adding the following to your <tt>sources.list</tt>:
+<pre class="file">
+deb file:///$YOURBASEDIR mystuff main bad
+</pre>
+or make it available via http or ftp and do the same <tt>http://</tt> or <tt>ftp://</tt> source.</li>
+</ul>
+<h3>mirroring packages from other repositories</h3>
+This example shows how to generate a mirror of a single architecture with
+all packages of etch plus security updates:
+<ul>
+<li>Choose a directory (or create it).</li>
+<li>Create a subdirectory called <tt>conf</tt> in there (if not already existent).</li>
+<li>In the <tt>conf/</tt> subdirectory create a file called <tt>distributions</tt>,
+with content like (or add to that file after an empty line):
+<pre class="file">
+Origin: Debian
+Label: Debian
+Suite: stable
+Version: 4.0
+Codename: etch
+Architectures: i386
+Components: main
+Description: Debian 4.0 etch + security updates
+Update: - debian security
+Log: logfile
+</pre>
+Actually only <tt>Codename</tt>, <tt>Components</tt>, <tt>Architecture</tt> and <tt>Update</tt> is needed, the rest is just information for clients.
+The <tt>Update</tt> line tells to delete everything no longer available (<tt>-</tt>),
+then add the <tt>debian</tt> and <tt>security</tt> rules, which still have to be defined:
+</li>
+<li>In the <tt>conf/</tt> subdirectory create a file called <tt>updates</tt>,
+with content like (or add to that file after an empty line:):
+or with content like:
+<pre class="file">
+Name: security
+Method: http://security.debian.org/debian-security
+Fallback: ftp://klecker.debian.org/debian-security
+Suite: */updates
+VerifyRelease: A70DAF536070D3A1|B5D0C804ADB11277
+Architectures: i386
+Components: main
+UDebComponents:
+
+Name: debian
+Method: http://ftp2.de.debian.org/debian
+Config: Acquire::Http::Proxy=http://proxy.myorg.de:8080
+VerifyRelease: A70DAF536070D3A1|B5D0C804ADB11277
+</pre>
+(If there are no Architecture, Components or UDebComponents, it will try all the distribution to update has. Fallback means a URL to try when the first cannot offer some file (Has to be the same method)).
+</li>
+<li>Tell reprepro to update:
+<pre class="shell">
+reprepro -b $YOURBASEDIR update etch
+</pre>
+</li>
+<li>Take a look at the generated <tt>pool</tt> and <tt>dists</tt>
+directories. They contain everything needed to apt-get from.
+Tell apt to include it by adding the following to your <tt>sources.list</tt>:
+<pre class="shell">
+deb file:///$YOURBASEDIR etch main
+</pre>
+or make it available via http or ftp.</li>
+</ul>
+<h2><a name="dirbasics">Repository basics</a></h2>
+An <tt>apt-get</tt>able repository of Debian packages consists of two parts:
+the index files describing what is available and where it is and the actual
+Debian binary (<tt class="suffix">.deb</tt>),
+installer binary (<tt class="suffix">.udeb</tt>),
+and source (<tt class="suffix">.dsc</tt> together with
+<tt class="suffix">.tar.gz</tt> or
+<tt class="suffix">.orig.tar.gz</tt> and
+<tt class="suffix">.diff.gz</tt>) packages.
+<br>
+While you do not know how these look like to use reprepro, it's always a good
+idea to know what you are creating.
+<h3>Index files</h3>
+All index files are in subdirectories of a directory called
+<tt class="dirname">dists</tt>. Apt is very decided what names those should
+have, including the name of <tt class="dirname">dists</tt>.
+Including all optional and extensional files, the hierarchy looks like this:
+
+<dl class="dir">
+<dt class="dir">dists</dt><dd>
+	<dl class="dir">
+	<dt class="dir">CODENAME</dt><dd>
+Each distribution has it's own subdirectory here, named by it's codename.
+		<dl class="dir">
+		<dt class="file">Release</dt><dd>
+This file describes what distribution this is and the checksums of
+all index files included.
+		</dd>
+		<dt class="file">Release.gpg</dt><dd>
+This is the optional detached gpg signature of the Release file.
+Take a look at the <a name="#signing">section about signing</a> for how to
+active this.
+		</dd>
+		<dt class="file">Contents-ARCHITECTURE.gz</dt><dd>
+This optional file lists all files and which packages they belong to.
+It's downloaded and used by tools like
+<a href="http://packages.debian.org/apt-file">apt-file</a>
+to allow users to determine which package to install to get a specific file.
+<br>
+To activate generating of these files by reprepro, you need a <a href="#contents">Contents</a>
+header in your distribution declaration.
+		</dd>
+		<dt class="dir">COMPONENT1</dt><dd>
+Each component has it's own subdirectory here. They can be named whatever users
+can be bothered to write into their <tt class="filename">sources.list</tt>, but
+things like <tt>main</tt>, <tt>non-free</tt> and <tt>contrib</tt> are common.
+But funny names like <tt>bad</tt> or <tt>universe</tt> are just as possible.
+			<dl class="dir">
+			<dt class="dir">source</dt><dd>
+If this distribution supports sources, this directory lists which source
+packages are available in this component.
+				<dl class="dir">
+				<dt class="file">Release</dt><dd>
+This file contains a copy of those information about the distribution
+applicable to this directory.
+				</dd>
+				<dt class="file">Sources</dt>
+				<dt class="file">Sources.gz</dt>
+				<dt class="file">Sources.bz2</dt><dd>
+These files contain the actual description of the source Packages. By default
+only the <tt class="suffix">.gz</tt> file created, to create all three add the
+following to the declarations of the distributions:
+<pre class="config">
+DscIndices Sources Release . .gz .bz2
+</pre>
+That header can also be used to name those files differently, but then apt
+will no longer find them...
+				</dd>
+				<dt class="dir">Sources.diff</dt><dd>
+This optional directory contains diffs, so that only parts of the index
+file must be downloaded if it changed. While reprepro cannot generate these
+so-called <tt>pdiff</tt>s itself, it ships both with a program called rredtool
+and with an example python script to generate those.
+				</dd>
+				</dl>
+			</dd>
+			</dl>
+			<dl class="dir">
+			<dt class="dir">binary-ARCHITECTURE</dt><dd>
+Each architecture has its own directory in each component.
+				<dl class="dir">
+				<dt class="file">Release</dt><dd>
+This file contains a copy of those information about the distribution
+applicable to this directory.
+				</dd>
+				<dt class="file">Packages</dt>
+				<dt class="file">Packages.gz</dt>
+				<dt class="file">Packages.bz2</dt><dd>
+These files contain the actual description of the binary Packages. By default
+only the uncompressed and <tt class="suffix">.gz</tt> files are created.
+To create all three, add the following to the declarations of the distributions:
+<pre class="config">
+DebIndices Packages Release . .gz .bz2
+</pre>
+That header can also be used to name those files differently, but then apt
+will no longer find them...
+				</dd>
+				<dt class="dir">Packages.diff</dt><dd>
+This optional directory contains diffs, so that only parts of the index
+file must be downloaded if it changed. While reprepro cannot generate these
+so-called <tt>pdiff</tt>s itself, it ships both with a program called rredtool
+and with an example python script to generate those.
+				</dd>
+				</dl>
+			</dd>
+			<dt class="dir">debian-installer</dt><dd>
+This directory contains information about the <tt class="suffix">.udeb</tt>
+modules for the <a href="http://www.debian.org/devel/debian-installer/">Debian-Installer</a>.
+Those are actually just a very stripped down form of normal <tt class="suffix">.deb</tt>
+packages and this the hierarchy looks very similar:
+
+				<dl class="dir">
+				<dt class="dir">binary-ARCHITECTURE</dt><dd>
+					<dl class="dir">
+					<dt class="file">Packages</dt><dd></dd>
+					<dt class="file">Packages.gz</dt><dd></dd>
+					</dl>
+				</dd>
+				</dl>
+			</dd>
+			</dl>
+		</dd>
+		<dt class="dir">COMPONENT2</dt><dd>
+There is one dir for every component. All look just the same.
+		</dd>
+		</dl>
+	</dd>
+	<dt class="symlink">SUITE -> CODENAME</dt><dd>
+To allow accessing distribution by function instead of by name, there are often
+symbolic links from suite to codenames. That way users can write
+<pre class="config">
+deb http://some.domain.tld/debian SUITE COMPONENT1 COMPONENT2
+</pre>
+instead of
+<pre class="config">
+deb http://some.domain.tld/debian CODENAME COMPONENT1 COMPONENT2
+</pre>
+in their <tt class="filename">/etc/apt/sources.list</tt> and totally get
+surprised by getting something new after a release.
+	</dd>
+	</dl>
+</dd></dl>
+<h3>Package pool</h3>
+While the index files have a required filename, the actual files
+are given just as relative path to the base directory you specify
+in your sources list. That means apt can get them no matter what
+scheme is used to place them. The classical way Debian used till
+woody was to just put them in subdirectories of the
+<tt class="dir">binary-ARCHITECTURE</tt> directories, with the exception
+of the architecture-independent packages, which were put into a
+artificial <tt class="dir">binary-all</tt> directory. This was replaced
+for the official repository with package pools, which reprepro also uses.
+(Actually reprepro stores everything in pool a bit longer than the official
+repositories, that's why it recalculates all filenames without exception).
+<br>
+In a package pool, all package files of all distributions in that repository
+are stored in a common directory hierarchy starting with <tt class="dir">pool/</tt>,
+only separated by the component they belong to and the source package name.
+As everything this has disadvantages and advantages:
+<ul><li>disadvantages
+	<ul><li>different files in different distributions must have different filenames
+	</li><li>it's impossible to determine which distribution a file belongs to by path and filename (think mirroring)
+	</li><li>packages can no longer be grouped together in common subdirectories by having similar functions
+	</li></ul>
+</li><li>advantages
+	<ul><li>the extremely confusing situation of having differently build packages with the same version if different distributions gets impossible by design.
+	</li><li>the source (well, if it exists) is in the same directory as the binaries generated from it
+	</li><li>same files in different distributions need disk-space and bandwidth only once
+	</li><li>each package can be found only knowing component and sourcename
+	</li></ul>
+</li></ul>
+Now let's look at the actual structure of a pool (there is currently no difference
+between the pool structure of official Debian repositories and those generated by
+reprepro):
+
+<dl class="dir">
+<dt class="dir">pool</dt><dd>
+	The directory all this resides in is normally called <tt class="dir">pool</tt>.
+	That's nowhere hard coded in apt but that only looks at the relative
+	directory names in the index files. But there is also no reason to name
+	it differently.
+	<dl class="dir">
+	<dt class="dir">COMPONENT1</dt><dd>
+Each component has it's own subdirectory here.
+They can be named whatever users
+can be bothered to write into their <tt class="filename">sources.list</tt>, but
+things like <tt>main</tt>, <tt>non-free</tt> and <tt>contrib</tt> are common.
+But funny names like <tt>bad</tt> or <tt>universe</tt> are just as possible.
+		<dl class="dir">
+		<dt class="dir">a</dt><dd>
+As there are really many different source packages,
+the directory would be too full when all put here.
+So they are separated in different directories.
+Source packages starting with <tt class="constant">lib</tt> are put into a
+directory named after the first four letters of the source name.
+Everything else is put in a directory having the first letter as name.
+			<dl class="dir">
+			<dt class="dir">asource</dt><dd>
+Then the source package name follows.
+So this directory <tt class="dir">pool/COMPONENT1/a/asource/</tt> would contain
+all files of different versions of the hypothetical package <tt class="constant">asource</tt>.
+				<dl class="dir">
+				<dt class="dir">asource</dt><dd>
+				<dt class="file">a-source_version.dsc</dt>
+				<dt>a-source_version.tar.gz</dt><dd>
+The actual source package consists of its description file (<tt class="suffix">.dsc</tt>)
+and the files references by that.
+				</dd>
+				<dt class="file">binary_version_ARCH1deb</dt>
+				<dt class="file">binary_version_ARCH2.deb</dt>
+				<dt class="file">binary2_version_all.deb</dt><dd>
+				<dt class="file">di-module_version_ARCH1.udeb</dt><dd>
+Binary packages are stored here to.
+So to know where a binary package is stored you need to know what its source package
+name is.
+				</dd>
+				</dl>
+			</dd>
+			</dl>
+		</dd>
+		<dt class="dir">liba</dt><dd>
+As described before packages starting with <tt class="constant">lib</tt> are not stored
+in <tt class="dir">l</tt> but get a bit more context.
+		</dd>
+		</dl>
+	</dd>
+	<dt class="dir">COMPONENT2</dt><dd>
+There is one dir for every component. All look just the same.
+	</dd>
+	</dl>
+</dd></dl>
+As said before, you don't need to know this hierarchy in normal operation.
+reprepro will put everything to where it belong, keep account what is there
+and needed by what distribution or snapshot, and delete files no longer needed.
+(Unless told otherwise or when you are using the low-level commands).
+<h2><a name="config">Config files</a></h2>
+Configuring a reprepro repository is done by writing some config files
+into a directory.
+This directory is currently the <tt class="dir">conf</tt> subdirectory of the
+base directory of the repository,
+unless you specify <tt class="option">--confdir</tt> or set the
+environment variable <tt class="env">REPREPRO_CONFIG_DIR</tt>.
+
+<dl class="dir">
+<dt class="dir">options</dt><dd>
+If this file exists, reprepro will consider each line an additional
+command line option.
+Arguments must be in the same line after an equal sign.
+
+Options specified on the command line take precedence.
+</dd>
+<dt class="dir">distributions</dt><dd>
+This is the main configuration file and the only that is needed in all
+cases.
+It lists the distributions this repository contains and their properties.
+<br>
+See <a href="#firststeps">First steps</a> for a short example or the manpage
+for a list of all possible fields.
+</dd>
+<dt class="dir">updates</dt><dd>
+Rules about where to download packages from other repositories.
+See the section <a href="#mirroring">Mirroring / Updating</a>
+for more examples or the man page for a full reference.
+</dd>
+<dt class="dir">pulls</dt><dd>
+Rules about how to move packages in bulk between distributions
+where to download packages from other repositories.
+See the section <a href="#propagation">Propagation of packages</a>
+for an example or the man page for full reference.
+</dd>
+<dt class="dir">incoming</dt><dd>
+Rules for incoming queues as processed by <tt class="command">processincoming</tt>.
+See <a href="#processincoming-incoming-config">Processing an incoming queue</a> for more information.
+</dd>
+</dl>
+<h2><a name="export">Generation of index files</a></h2>
+<h3>Deciding when to generate</h3>
+As reprepro stores all state in its database,
+you can decide when you want them to be written to the <tt class="dir">dists/</tt>
+directory.
+You can always tell reprepro to generate those files with the <tt>export</tt> command:
+<pre class="command">
+reprepro -b $YOURBASEDIR export $CODENAMES
+</pre>
+This can be especially useful, if you just edited <tt class="file">conf/distributions</tt>
+and want to test what it generates.
+<p>
+While that command regenerates all files, in normal operation reprepro will only
+regenerate files where something just changed or that are missing.
+With <tt class="option">--export</tt> option you can control when this fill happen:
+<dl><dt>never</dt><dd>Don't touch any index files.
+This can be useful for doing multiple operations in a row and not wanting to regenerate
+the indices all the time.
+Note that unless you do an explicit export or change the same parts later without that
+option, the generated index files may be permanently out of date.
+</dd><dt>silent-never</dt><dd>Like never, but be more silent about it.
+</dd><dt>changed</dt><dd>This is the default behaviour since 3.0.1.
+Only export distributions where something changed
+(and no error occurred that makes an inconsistent state likely).
+And in those distributions only (re-)generate files which content should have been changed
+by the current action or which are missing.
+</dd><dt>lookedat</dt><dd>New name for <tt>normal</tt> since 3.0.1.
+</dd><dt>normal</dt><dd>This was the default behaviour until 3.0.0 (changed in 3.0.1).
+In this mode all distributions are processed that were looked at without error
+(where error means only errors happening while the package was open so have a chance
+to cause strange contents).
+This ensures that even after a operation that had nothing to do the looked at
+distribution has all the files exported needed to access it. (But still only files
+missing or that content would change with this action are regenerated).
+</dd><dt>force</dt><dd>Also try to write the current state if some error occurred.
+In all other modes reprepro will not write the index files if there was a problem.
+While this keeps the repository usable for users, it means that you will need an
+explicit export to write possible other changes done before that in the same run.
+(reprepro will tell you that at the end of the run with error, but you should not
+miss it).
+</dd></dl>
+<h3>Distribution specific fields</h3>
+There are a lot of <tt class="file">conf/distributions</tt> headers to control
+what index files to generate for some distribution, how to name
+them, how to postprocess them and so on. The most important are:
+<h4>Fields for the Release files</h4>
+The following headers are copied verbatim to the Release file, if they exist:
+<tt class="header">Origin</tt>,
+<tt class="header">Label</tt>,
+<tt class="header">Codename</tt>,
+<tt class="header">Suite</tt>,
+<tt class="header">Architectures</tt> (excluding a possible value "<tt>source</tt>"),
+<tt class="header">Components</tt>,
+<tt class="header">Description</tt>, and
+<tt class="header">NotAutomatic</tt>,
+<tt class="header">ButAutomaticUpgrades</tt>.
+<h4><a name="compression">Choosing compression and file names</a></h4>
+Depending on the type of the index files, different files are generated.
+No specifying anything is equivalent to:
+<pre class="config">
+ DscIndices Sources Release .gz
+ DebIndices Packages Release . .gz
+ UDebIndices Packages . .gz
+</pre>
+This means to generate <tt>Release</tt>, <tt>Sources.gz</tt> for sources,
+<tt>Release</tt>, <tt>Packages</tt> and <tt>Packages.gz</tt> for binaries
+and <tt>Packages</tt> and <tt>Packages.gz</tt> for installer modules.
+<br>
+The format of these headers is the name of index file to generate, followed
+by the optional name for a per-directory release description
+(when no name is specified, no file is generated).
+Then a list of compressions:
+A single dot (<tt>.</tt>) means generating an uncompressed index,
+<tt>.gz</tt> means generating a gzipped output,
+while <tt>.bz2</tt> requests and bzip2ed file.
+(<tt>.bz2</tt> is not available when disabled at compile time).
+After the compressions a script can be given that is called to generate/update
+additional forms, see <a href="#exporthook">&quot;Additional index files&quot;</a>.
+<h4><a name="signing">Signing</a></h4>
+If there is a <tt class="config">SignWith</tt> header, reprepro will try
+to generate a <tt class="file">Release.gpg</tt> file using libgpgme.
+If the value of the header is <tt>yes</tt> it will use the first key
+it finds, otherwise it will give the option to libgpgme to determine the
+key. (Which means fingerprints and keyids work fine, and whatever libgpgme
+supports, which might include most that gpg supports to select a key).
+<br>
+The best way to deal with keys needing passphrases is to use
+<a href="http://packages.debian.org/gnupg-agent">gpg-agent</a>.
+The only way to specify which keyring to use is to set the
+<tt class="env">GNUPGHOME</tt> environment variable, which will effect all
+distributions.
+<h4><a name="contents">Contents files</a></h4>
+Reprepro can generate files called
+<tt class="file">dists/CODENAME/Contents-ARCHITECTURE.gz</tt>
+listing all files in all binary packages available for the selected
+architecture in that distribution and which package they belong to.
+<br>
+This file can either be used by humans directly or via downloaded
+and searched with tools like
+<a href="http://packages.debian.org/apt-file">apt-file</a>.
+<br>
+To activate generating of these files by reprepro, you need a <tt class="config">Contents</tt> header in that distribution's declaration in <tt class="file">conf/distributions</tt>,
+like:
+<pre class="config">
+Contents:
+</pre>
+Versions before 3.0.0 need a ratio number there, like:
+<pre class="config">
+Contents: 1
+</pre>
+The number is the inverse ratio of not yet looked at and cached files to process in
+every run. The larger the more packages are missing. 1 means to list everything.
+<br>
+The arguments of the Contents field and other fields control
+which Architectures to generate Contents files for and which
+Components to include in those. For example
+<pre class="config">
+Contents: udebs nodebs . .gz .bz2
+ContentsArchitectures: ia64
+ContentsComponents:
+ContentsUComponents: main
+</pre>
+means to not skip any packages, generate Contents for <tt class="suffix">.udeb</tt>
+files, not generating Contents for <tt class="suffix">.deb</tt>s. Also it is only
+generated for the <tt>ia64</tt> architecture and only packages in component
+<tt>main</tt> are included.
+<h4><a name="exporthook">Additional index files (like .diff)</a></h4>
+Index files reprepro cannot generate itself, can be generated by telling
+it to call a script.
+<h5>using rredtool to generate pdiff files</h5>
+Starting with version 4.1.0, the <tt>rredtool</tt> coming with reprepro
+can be used as hook to create and update <tt>Packages.diff/Index</tt> files.
+<br>
+Unlike dak (which created the official Debian repositories) or the pdiff.py
+script (see below) derived from dak, an user will only need to download
+one of those patches, as new changes are merged into the old files.
+<br>
+To use it, make sure you have
+<a href="http://packages.debian.org/diff">diff</a> and
+<a href="http://packages.debian.org/gzip">gzip</a>
+installed.
+Then add something like the following to the headers of the distributions
+that should use this in <tt class="file">conf/distributions</tt>:
+<pre class="config">
+ DscIndices: Sources Release . .gz /usr/bin/rredtool
+ DebIndices: Packages Release . .gz /usr/bin/rredtool
+</pre>
+<h5>the pdiff example hook script (generates pdiff files)</h5>
+This example generates <tt class="file">Packages.diff</tt> and/or
+<tt class="file">Sources.diff</tt> directories containing a set of
+ed-style patches, so that people do not redownload the whole index
+for just some small changes.
+<br>
+To use it, copy <tt class="file">pdiff.example</tt> from the examples directory
+into your <tt class="dir">conf</tt> directory.
+(or any other directory, then you will need to give an absolute path later).
+Unpack, if needed. Rename it to pdiff.py and make it executable.
+Make sure you have
+<a href="http://packages.debian.org/python3-apt">python3-apt</a>,
+<a href="http://packages.debian.org/diff">diff</a> and
+<a href="http://packages.debian.org/gzip">gzip</a>
+installed.
+Then add something like the following to the headers of the distributions
+that should use this in <tt class="file">conf/distributions</tt>:
+<pre class="config">
+ DscIndices: Sources Release . .gz pdiff.py
+ DebIndices: Packages Release . .gz pdiff.py
+</pre>
+More information can be found in the file itself. You should read it.
+<h5>the bzip2 example hook script</h5>
+This is an very simple example.
+Simple and mostly useless,
+as reprepro has built in <tt>.bz2</tt> generation support,
+unless you compiled it your own with <tt>--without-libbz2</tt> or
+with no <tt>libbz2-dev</tt> installed.
+<br>
+To use it, copy <tt class="file">bzip.example</tt> from the examples directory
+into your <tt class="dir">conf</tt> directory.
+(or any other directory, then you will need to give an absolute path later).
+Unpack, if needed. Rename it to bzip2.sh and make it executable.
+Then add something like the following to the headers of the distributions
+that should use this in <tt class="file">conf/distributions</tt>:
+<pre class="config">
+ DscIndices: Sources Release . .gz bzip2.sh
+ DebIndices: Packages Release . .gz bzip2.sh
+ UDebIndices: Packages . .gz bzip2.sh
+</pre>
+The script will compress the index file using the
+<a href="http://packages.debian.org/bzip2">bzip2</a> program and tell
+reprepro which files to include in the Release file of the distribution.
+<h5>internals</h5>
+TO BE CONTINUED
+<h4>...</h4>
+TO BE CONTINUED
+<h2><a name="localpackages">Local packages</a></h2>
+There are two ways to get packages not yet in any repository into yours.
+<dl><dt>includedsc, includedeb, include</dt><dd>
+These are for including packages at the command line.
+Many options are available to control what actually happens.
+You can easily force components, section and priority and/or choose to
+include only some files or only in specific architectures.
+(Can be quite useful for architecture all packages depending on some
+packages you will some time before building for some of your architectures).
+Files can be moved instead of copied and most sanity checks overwritten.
+They are also optimized towards being fast and simply try things instead of
+checking a long time if they would succeed.
+</dd><dt>processincoming</dt><dd>
+This command checks for changes files in an incoming directory.
+Being optimized for automatic processing (i.e. trying to checking
+everything before actually doing anything), it can be slower
+(as every file is copied at least once to sure the owner is correct,
+with multiple partitions another copy can follow).
+Component, section and priority can only be changed via the distribution's
+override files. Every inclusion needs a <tt class="suffix">.changes</tt> file.
+<br>
+This method is also relatively new (only available since 2.0.0), thus
+optimisation for automatic procession will happen even more.
+</dd></dl>
+<h3><a name="include">Including via command line</a></h3>
+There are three commands to directly include packages into your repository:
+<tt class="command">includedeb</tt>, <tt class="command">includedsc</tt>
+and <tt class="command">includechanges</tt>.
+Each needs to codename of the distribution you want to put your package into
+as first argument and a file of the appropriate type
+(<tt class="suffix">.deb</tt>, <tt class="suffix">.dsc</tt> or
+ <tt class="suffix">.changes</tt>, respectively) as second argument.
+<br>
+If no component is specified via <tt class="option">--component</tt>
+(or short <tt class="option">-C</tt>), it will be guessed looking at its
+section and the components of that distribution.
+<br>
+If there are no <tt class="option">--section</tt>
+(or short <tt class="option">-S</tt>) option, and it is not specified
+by the (binary or source, depending on the type) override file of the
+distribution, the value from the <tt class="suffix">.changes</tt>-file
+is used (if the command is <tt class="command">includechanges</tt>)
+or it is extracted out of the file (if it is a
+<tt class="suffix">.deb</tt>-file, future versions might also try to
+extract it from a <tt class="suffix">.dsc</tt>'s diff or tarball).
+<br>
+Same with the priority and the <tt class="option">--priority</tt>
+(or short <tt class="option">-P</tt>) option.
+<br>
+With the <tt class="option">--architecture</tt> (or short <tt class="option">-A</tt>)
+option, the scope of the command is limited to that architecture.
+<tt class="command">includdeb</tt> will add a Architecture <tt>all</tt>
+packages only to that architecture (and complain about Debian packages for
+other architectures).
+<tt class="command">include</tt> will do the same and ignore packages for
+other architectures (source packages will only be included if the value
+for <tt class="option">--architecture</tt> is <tt>source</tt>).
+<br>
+To limit the scope to a specify type of package, use the
+<tt class="option">--packagetype</tt> or short <tt class="option">-T</tt>
+option. Possible values are <tt>deb</tt>, <tt>udeb</tt> and <tt>dsc</tt>.
+<br>
+When using the <tt class="option">--delete</tt> option, files will
+be moved or deleted after copying them.
+Repeating the <tt class="option">--delete</tt> option will also delete
+unused files.
+<br>
+TO BE CONTINUED.
+<h3><a name="incoming">Processing an incoming queue</a></h3>
+Using the <tt class="command">processincoming</tt> command reprepro
+can automatically process incoming queues.
+While this is still improveable (reprepro still misses ways to send
+mails and especially an easy way to send rejection mails to the
+uploader directly), it makes it easy to have an directory where you
+place your packages and reprepro will automatically include them.
+<br>
+To get this working you need three things:
+<ul>
+<li><a href="#processincoming-incoming-config">
+a file <tt class="file">conf/incoming</tt> describing your incoming directories,
+</a></li>
+<li><a href="#processincoming-dist-config">
+a <tt class="file">conf/distribution</tt> file describing your distributions
+(as always with reprepro)
+and
+</a></li>
+<li><a href="#processincoming-calling">
+a way to get reprepro called to process it.
+</a></li>
+</ul>
+<a name="processincoming-incoming-config">
+<h4>The file <tt class="file">conf/incoming</tt></h4></a>
+describes the different incoming queues.
+As usual the different chunks are separated by empty lines.
+Each chunk can have the following fields:
+<dl><dt>Name</dt><dd>This
+is the name of the incoming queue, that <tt class="command">processincoming</tt>
+wants as argument.</dd>
+<dt>IncomingDir</dt><dd>The actual directory to look for
+<tt class="suffix">.changes</tt> files.</dd>
+<dt>TempDir</dt><dd>To ensure integrity of the processed files and their
+permissions,
+every file is first copied from the incoming directory to this directory.
+Only the user reprepro runs as needs write permissions here.
+It speeds things up if this directory is in the same partition as the pool.
+<dt>Allow</dt><dd>
+This field lists the distributions this incoming queue might inject packages
+into.
+Each item can be a pair of a name of a distribution to accept and a distribution
+to put it into.
+Each upload has each item in its <tt class="field">Distribution:</tt> field
+compared first to last to each of this items and is put in the first distribution
+accepting it. For example
+<pre class="line">
+Allow: stable>etch stable>etch-proposed-updates mystuff unstable>sid
+</pre>
+will put a <tt class="suffix">.changes</tt> file with
+<tt class="field">Distribution: stable</tt> into etch.
+If that is not possible (e.g. because etch has a
+<tt class="field">UploadersList</tt> option not allowing this) it will
+be put into etch-proposed-updates.
+And a <tt class="suffix">.changes</tt> file with
+<tt class="field">Distribution: unstable</tt> will be put into sid, while
+with <tt class="field">Distribution: mystuff</tt> will end up in mystuff.
+<br>
+If there is a <tt class="field">Default</tt> field, the <tt class="field">Allow</tt>
+field is optional.</dd>
+<dt>Default</dt><dd>
+Every upload not caught by an item of the <tt class="field">Allow</tt>
+field is put into the distribution specified by this.
+<br>
+If there is a <tt class="field">Allow</tt> field, the <tt class="field">Default</tt>
+field is optional.</dd>
+<dt>Multiple</dt><dd>
+This field only makes a difference if a <tt class="suffix">.changes</tt> file
+has multiple distributions listed in its <tt class="field">Distribution:</tt>
+field.
+Without this field each of those distributions is tried according to the
+above rules until the package is added to one (or none accepts it).
+With this field it is tried for each distribution, so a package can be upload
+to multiple distributions at the same time.
+</dd>
+<dt>Permit</dt><dd>
+A list of options to allow things otherwise causing errors.
+(see the manpage for possible values).
+<br>This field os optional.</dd>
+<dt>Cleanup</dt><dd>
+Determines when and what files to delete from the incoming queue.
+By default only successfully processed <tt class="suffix">.changes</tt> files
+and the files references by those are deleted.
+For a list of possible options take a look into the man page.
+<br>This field os optional.</dd>
+</dl>
+<a name="processincoming-dist-config">
+<h4><tt class="file">conf/distribution</tt> for <tt class="command">processincoming</tt></h4></a>
+There are no special requirements on the <tt class="file">conf/distribution</tt>
+file by processincoming. So even a simple
+<pre class="file">
+Codename: mystuff
+Architectures: i386 source
+Components: main non-free contrib bad
+</pre>
+will work.
+<br>
+The <tt class="field">Uploaders</tt> field can list a file limiting
+uploads to this distribution to specific keys and
+<tt class="field">AlsoAcceptFor</tt> is used to resolve unknown names
+in <tt class="file">conf/incoming</tt>'s <tt class="field">Allow</tt>
+and <tt class="field">Default</tt> fields.
+<a name="processincoming-calling">
+<h4>Getting <tt class="command">processincoming</tt> called.</h4></a>
+While you can just call <tt class="command">reprepro processincoming</tt> manually,
+having an incoming queue needing manual intervention takes all the fun out of
+having an incoming queue, so usually so automatic way is chosen:
+<ul>
+<li>Dupload and dput have ways to call an hook after an package was uploaded.
+This can be an ssh to the host calling reprepro.
+The disavantage is having to configure this in every
+<tt class="file">.dupload.conf</tt> on every host you want to upload and give
+everyone access to ssh and permissions on the archive who should upload.
+The advantage is you can configure reprepro to have interactive scripts or
+ask for passphrases.
+</li>
+<li>Install a cron-job calling reprepro every 5 minutes. Cron is usually
+available everywhere and getting the output sent by mail to you or a mailing
+list is easy.
+The annoying part is having to wait almost 5 minutes for the processing.
+</li>
+<li>Use something like <a href="http://packages.debian.org/inoticoming"><tt class="external">inoticoming</tt></a>.
+Linux has a syscall called inotify, allowing a program to be run whenever
+something happens to a file.
+One program making use of this is inoticoming. It watches a directory using
+this facility and whenever a <tt class="suffix">.changes</tt> file is completed
+it can call reprepro for you.
+(As this happens directly, make sure you always upload the <tt class="suffix">.changes</tt>
+file last, dupload and dput always ensure this).
+This can be combined with Debian's cron-extension to have a program started at
+boot time with the <tt>@reboot</tt> directive.
+For example with a crontab like:
+<pre class="file">
+MAILTO=myaddress@somewhere.tld
+
+@reboot inoticoming --logfile /my/basedir/logs/i.log /my/basedir/incoming/ --stderr-to-log --stdout-to-log --suffix '.changes' --chdir /my/basedir reprepro -b /my/basedir --waitforlock 100 processincoming local {} \;
+</pre>
+</li>
+</ul>
+<h2><a name="mirroring">Mirroring / Updating</a></h2>
+Reprepro can fetch packages from other repositories.
+For this it uses apt's methods from <tt class="dir">/usr/lib/apt/methods/</tt>
+so everything (http, ftp, ...) that works with apt should also work with reprepro.
+Note that this works on the level of packages, even though you can tell reprepro
+to create a distribution having always the same packages as some remote repository,
+the repository as a whole may not look exactly the same but only have the same set
+of packages in the same versions.
+<br>
+You can also only mirror a specific subset of packages, merge multiple repositories
+into one distribution, or even have distributions mixing remote and local packages.
+<br>
+Each distribution to receive packages from other repositories needs an
+<tt class="field">Update:</tt> field listing the update rules applied to it.
+Those update rules are listed in <tt class="file">conf/updates</tt>.
+There is also the magic <tt>-</tt> update rule, which tells reprepro to delete
+all packages not re-added by later rules.
+<br>
+To make reprepro to update all distributions call <tt>reprepro update</tt>
+without further arguments, or give the distributions to update as additional
+arguments.
+<br>
+Let's start with some examples:
+<h3><a name="update-examples">Updating examples</a></h3>
+Let's assume you have the following <tt class="file">conf/distributions</tt>
+<pre class="file">
+Codename: etch
+Architectures: i386 source
+Components: main contrib
+Update: local - debian security
+
+Codename: mystuff
+Architectures: abacus source
+Components: main bad
+Update: debiantomystuff
+</pre>
+and the following <tt class="file">conf/updates</tt>
+<pre class="file">
+Name: local
+Method: http://ftp.myorg.tld/debian
+
+Name: debian
+Method: http://ftp.de.debian.org/debian
+VerifyRelease: A70DAF536070D3A1
+Config: Acquire::Http::Proxy=http://proxy.yours.org:8080
+
+Name: security
+Suite: */updates
+Method: http://security.eu.debian.org/
+Fallback: http://security.debian.org/
+VerifyRelease: A70DAF536070D3A1
+Config: Acquire::Http::Proxy=http://proxy.yours.org:8080
+
+Name: debiantomystuff
+Suite: sid
+Method: http://ftp.de.debian.org/debian
+Architectures: i386&gt;abacus source
+Components: main non-free&gt;bad contrib&gt;bad
+FilterFormula: Architecture (== all)| !Architecture
+FilterList: deinstall list
+</pre>
+and a file <tt class="file">conf/list</tt> with some
+output as <tt>dpkg --get-selections</tt> is printing.
+<br>
+If you then run
+<tt class="command">reprepro update etch</tt> or
+<tt class="command">reprepro checkupdate etch</tt>,
+reprepro looks at etch's <tt class="field">Update:</tt> line
+and finds four rules. The first is the <tt>local</tt> rule,
+which only has a method, so that means it will download the
+<tt class="file">Release</tt> file from
+<tt>http://ftp.myorg.tld/debian/dists/etch/Release</tt> and
+(unless it already has downloaded them before or that
+repository does not have all of them) downloads the
+<tt>binary-i386/Packages.gz</tt>
+and <tt>source/Sources.gz</tt> files for main and contrib.
+The same is done for the <tt>debian</tt> and <tt>security</tt>
+rules.
+As they have a <tt class="field">VerifyRelease</tt> field,
+Release.gpg is also downloaded and checked to be signed with the
+given key
+(which you should have imported to you <tt class="external">gpg</tt>
+keyring before).
+As security has a <tt class="field">Suite:</tt> field, not the codename,
+but the content of this field (with an possible<tt>*</tt> replaced by the codename),
+is used as distribution to get.
+<br>
+Then it will parse for each part of the distribution, parse the files it
+get from left to right.
+For each package it starts with the version currently in the distribution,
+if there is a newer on in <tt>local</tt> it will mark this.
+Then there is the delete rule <tt>-</tt>, which will mark it to be deleted
+(but remembers what was there, so if later the version in the distribution
+or the version in <tt>local</tt> are newest, it will get them from here avoiding
+slow downloads from far away). Then it will look into <tt>debian</tt> and then
+in <tt>security</tt>, if they have a newer version (or the same version, clearing
+the deletion mark).
+<br>
+If you issued <tt class="command">checkupdate</tt> reprepro will print what it would
+do now, otherwise it tries to download all the needed files and when it got all,
+change the packages in the distribution to the new ones, export the index files
+for this distribution and finally delete old files no longer needed.
+<br>
+TO BE CONTINUED.
+<h2><a name="propagation">Propagation of packages</a></h2>
+You can copy packages between distributions using the
+<tt class="command">pull</tt> and <tt class="command">copy</tt> commands.
+<br>
+With the <tt class="command">copy</tt> command you can copy packages
+by name from one distribution to the other within the same repository.
+<br>
+With the <tt class="command">pull</tt> command you can pull all packages
+(or a subset defined by some list, or exceptions by some list, or by some
+formula, or ...) from one distribution to another within the same formula.
+<br>
+Note that both assume the filenames of the corresponding packages in the
+pool will not differ, so you cannot move packages from one component to another.
+<br>
+Let's just look at a little example, more information can be found in the man page.
+<br>
+Assume you upload all new packages to a distribution and you want another
+so you can keep using an old version until you know the newer works, too.
+One way would be to use something like the following
+<tt class="file">conf/distributions</tt>:
+<pre class="file">
+Codename: development
+Suite: unstable
+Components: main extra
+Architectures: i386 source
+
+Codename: bla
+Suite: testing
+Components: main extra
+Architectures: i386 source
+Pull: from_development
+</pre>
+and <tt class="file">conf/pulls</tt>:
+<pre class="file">
+Name: from_development
+From: development
+</pre>
+i.e. you have two distributions, bla and development.
+Now you can just upload stuff to development (or it's alias unstable).
+And when you want a single package to go to testing, you can use the copy
+command:
+<pre class="shell">
+reprepro copy bla development name1 name2 name3
+</pre>
+If you do not want to copy all packages of a given name, but only some
+of them, you can use <tt>-A</tt>, <tt>-T</tt> and <tt>-C</tt>:
+<pre class="shell">
+reprepro -T deb -A i386 copy bla development name1
+</pre>
+will copy <tt class="suffix">.deb</tt> packages called name1 from the i386
+parts of the distribution.
+<br>
+TO BE CONTINUED
+<h2><a name="snapshosts">Snapshots</a></h2>
+There is a gensnapshot command.<br>
+TO BE DOCUMENTED
+<h2><a name="tracking">Source package tracking</a></h2>
+TO BE DOCUMENTED
+<h2><a name="hooks">Extending reprepro / Hooks and more</a></h2>
+When reprepro misses some functionality,
+it often can be added by some kind of hook.
+<br>
+Currently you can execute your own scripts at the following occasions:
+<ul>
+<li><a href="#addhook">after adding or removing packages</a></li>
+<li><a href="#byhandhook">to process byhand files</a></li>
+<li><a href="#exporthook">when creating index files (Packages.gz, Sources.gz)</a></li>
+<li><a href="#signhook">when signing releases</a></li>
+<li><a href="#outhook">after changing the visible files of the repository managed</a></li>
+<li><a href="#endhook">when reprepro finished</a></li>
+</ul>
+<h3><a name="addhook">Scripts to be run when adding or removing packages</a></h3>
+Whenever a package is added or removed,
+you can tell reprepro to log that to some file and/or call a script using the
+<tt>Log:</tt> directive in <tt class="file">conf/distributions</tt>.
+<br>
+This script can send out mails and do other logging stuff,
+but despite the name, it is not restricted to logging.
+<br>
+<h4>Automatically extracting changelog and copyright information</h4>
+reprepro ships with an example script to extract <tt class="file">debian/changelog</tt>
+and <tt class="file">debian/copyright</tt>
+files from source packages into a hierarchy loosely resembling the way changelogs
+are made available at
+<a href="http://packages.debian.org/changelogs/">http://packages.debian.org/changelogs/</a>.
+<br>
+All you have to do is to copy (or unpack if compressed) the file
+<tt class="file">changelogs.example</tt> from the examples directory
+in the reprepro source or
+<a href="file:///usr/share/doc/reprepro/examples/">/usr/share/doc/reprepro/examples/</a>
+of your installed reprepro package into your <tt class="directory">conf/</tt> directory
+(or somewhere else, then you will need an absolute path later), perhaps
+change some directories specified in it
+and add something like the following lines
+to all distributions in <tt class="file">conf/distributions</tt> that should use
+this feature:
+<pre class="config">
+Log:
+ --type=dsc changelogs.example
+</pre>
+If you still want to log to some file, just keep the filename there:
+<pre class="config">
+Log: mylogfilename
+ --type=dsc changelogs.example
+</pre>
+Then cause those files to be generated for all existing files via
+<pre class="command">
+reprepro rerunnotifiers
+</pre>
+and all future source packages added or removed will get this list automatically
+updated.
+<h4>Writing your own Log: scripts</h4>
+You can list an arbitrary amount of scripts, to be called at specified times
+(which can overlap or even be the same):
+<pre class="config">
+Log: logfilename
+ --type=dsc script-to-run-on-source-package-changes
+ script-to-run-on-package-changes
+ another-script-to-run-on-package-changes
+ --type=dsc --component=main script-to-run-on-main-source-packages
+ --architecture=i386 --type=udeb script-to-run-on-i386-udebs
+ --changes script-to-run-on-include-or-processincoming
+</pre>
+There are two kind of scripts:
+The first one is called when a package was added or removed.
+Using the <tt class="option">--archtecture=</tt>,
+<tt class="option">--component=</tt> and
+<tt class="option">--type=</tt> options you can limit it to specific parts
+of the distribution.
+The second kind is marked with <tt class="option">--changes</tt> and is
+called when a <tt class="suffix">.changes</tt>-file was added with
+<tt class="command">include</tt> or <tt class="command">processincoming</tt>.
+Both are called asynchronous in the background <emph>after</emph> everything was done,
+but before no longer referenced files are deleted (so the files of the
+replaced or deleted package are still around).
+<h5>Calling conventions for package addition/removal scripts</h5>
+This type of script is called with a variable number of arguments.
+The first argument is the action. This is either
+<tt>add</tt>, <tt>remove</tt> or <tt>replace</tt>.
+The next four arguments are the codename of the affected distribution
+and the packagetype, component and architecture in that distribution
+affected.
+The sixth argument is the package's name.
+After that is the version of the added package (<tt>add</tt> and <tt>replace</tt>)
+and the version of the removed package (<tt>remove</tt> and <tt>replace</tt>).
+Finally the filekeys of the new (<tt>add</tt> and <tt>replace</tt>) and/or
+removed (<tt>remove</tt> and <tt>replace</tt>) package are listed
+starting with the marker &quot;<tt>--</tt>&quot; followed by each filekey
+(the name of the file in the <tt class="dir">pool/</tt>
+relative to <tt class="env">REPREPRO_OUT_DIR</tt>)
+as its own argument.
+<br>
+The environment variable <tt class="env">REPREPRO_CAUSING_COMMAND</tt>
+contains the command of the action causing this change.
+The environment variable
+<tt class="env">REPREPRO_CAUSING_FILE</tt> contains the name of the file
+given at the command line causing this package to be changed,
+if there is one.
+(i.e. with <tt class="command">includedeb</tt>,
+<tt class="command">includedsc</tt> and <tt class="command">include</tt>).
+The environment variables
+<tt class="env">REPREPRO_CAUSING_RULE</tt> and
+<tt class="env">REPREPRO_FROM</tt> are
+the name of the update or pull rule pulling in a package
+and the name of the distribution a package is coming from.
+What this name is depends on the command and for most commands
+it is simply not set at all.
+And of course all the <tt class="env">REPREPRO_*_DIR</tt> variables are set.
+<h5>Calling conventions for <tt class="suffix">.changes</tt> scripts</h5>
+This type of script is called with 5 or 6 arguments.
+The first is always &quot;<tt>accepted</tt>&quot;, to make it easier to
+check it is configured the right way.
+The second argument is the codename of the distribution the
+<tt class="suffix">.changes</tt>-file was added to.
+The third argument is the source name, the forth the version.
+The fifth name is the <tt class="suffix">.changes</tt> itself
+(in case of <tt class="command">processingcoming</tt> the secure copy in the
+temporary dir).
+There is a sixth argument if the <tt class="suffix">.changes</tt>-file was
+added to the <tt class="dir">pool/</tt>:
+The filekey of the added .changes file
+(i.e. the filename relative to <tt class="env">REPREPRO_OUT_DIR</tt>).
+<br>
+The environment variable <tt class="env">REPREPRO_CAUSING_COMMAND</tt>
+contains the command of the action causing this change.
+The environment variable
+<tt class="env">REPREPRO_CAUSING_FILE</tt> contains the name of the file
+given at the command line, if there is one
+(e.g. with <tt class="command">include</tt>).
+And of course all the <tt class="env">REPREPRO_*_DIR</tt> variables are set.
+<h3><a name="byhandhook">Scripts to be run to process byhand files</a></h3>
+<tt class="suffix">.changes</tt> files can (beside the usual packages files
+to be included in the repository) contain additional files to be processed
+specially.
+Those are marked by the special section <tt class="constant">byhand</tt> (in Debian)
+or <tt class="constant">raw-</tt>something (in Ubuntu).
+Besides storing them just in the pool besides the packages using the
+<tt class="constant">includebyhand</tt> value in the <tt class="field">Tracking</tt>
+settings you can also let reprepro process a hook to process them when encountering
+them in the <tt class="action">processincomming</tt> action
+(Typical usages are uploading documentation files this way that are unpacked next
+to the repository, or installer images or stuff like that).
+
+To use them add to the distribution's defining stanca in <tt class="filename">conf/distributions</tt> a field like:
+<pre class="config">
+ByhandHooks:
+ byhand * manifesto.txt handle-byhand.sh
+</pre>
+This will call the hook script <tt class="constant">handle-byhand.sh</tt> for every byhand file with section <tt class="constant">byhand</tt>, any priority and filename <tt class="constant">manifesto.txt</tt>. (The first three fields allow glob characters for matching).
+
+The script will then be called with 5 arguments:
+the codename of the distribution,
+the section,
+the priority,
+the filename as found in the changes file and
+the filename of where the script can find the actual file.
+
+<h3>Scripts to be run when creating index files (Packages.gz, Sources.gz)</h3>
+this hook is described in the section <a href="#exporthook">&quot;Additional index files&quot;</a>.
+
+<h3><a name="signhook">Scripts to be run when signing releases</a></h3>
+Instead of creating <tt class="filename">InRelease</tt> and
+<tt class="filename">Release.gpg</tt> files using libgpgme,
+the <tt class="option">SignWith</tt> option can also contain
+a exclamation mark followed by a space and the name of a hook script to call.
+
+The script gets three arguments:
+The filename to sign,
+the filename of the InRelease file to create and
+the filename of the Release.gpg to create
+(a Release.gpg does not need to be created. reprepro will assume you do not care about that legacy file if it is not created).
+
+Reprepro will wait for the script to continue and only do the renaming
+and deleting of old files after that, so the script might wait for example
+for someone to copy files from the system, signing and copying them it,
+for example.
+
+<h3><a name="outhook">Scripts to be run after changing the visible files of the repository managed</a></h3>
+When using the <tt class="option">--outhook</tt> command line option (or the corresponding
+<tt class="constant">outhook</tt> in the <tt class="filename">options</tt> file),
+reprepro will create a <tt class="suffix">.outlog</tt> file in the log directory describing
+any changes done to the out dir and calls the hook script given as argument with this
+file as argument.
+
+The <tt class="suffix">.outlog</tt> file consists of lines each starting with a keyword
+and then some arguments separated by tab characters.
+
+The possible keywords are:
+<ul>
+<li><tt class="constant">POOLNEW</tt>:
+One argument is the filekey of a file newly added to the pool.
+<li><tt class="constant">POOLDELETE</tt>:
+One argument is the filekey of a file removed from the pool.
+<li><tt class="constant">START-DISTRIBUTION</tt>, <tt class="constant">END-DISTRIBUTION</tt>:
+two or three arguments: the codename, the directory,
+and the suite (if set).
+<li><tt class="constant">START-SNAPSHOT</tt>, <tt class="constant">END-SNAPSHOT</tt>:
+three arguments: the codename, the directory, and the name of the snapshot generated.
+<li><tt class="constant">DISTFILE</tt>:
+three arguments: the directory of the distribution (relative to out dir), the name relative to that directory, and the filename generated by reprepro.
+<li><tt class="constant">DISTSYMLINK</tt>:
+three arguments: the directory of the distribution (relative to out dir), the name relative to that directory, and the symlink target (relative to that directory).
+<li><tt class="constant">DISTDELETE</tt>:
+two arguments: the directory of the distribution (relative to out dir), the name relative to that directory of a file no longer there.
+<li><tt class="constant">DISTKEEP</tt> (not yet generated):
+two arguments: the directory of the distribution (relative to out dir), the name relative to that directory.
+</ul>
+
+All <tt class="constant">POOLNEW</tt> come before any distribution changes referencing them
+and all <tt class="constant">POOLDELETE</tt> will be afterwards.
+Each line belonging to a distribution is guaranteed to be between the corresponding
+<tt class="constant">START-DISTRIBUTION</tt> and
+<tt class="constant">END-DISTRIBUTION</tt> or between a
+<tt class="constant">START-SNAPSHOT</tt> and
+<tt class="constant">END-SNAPSHOT</tt> or between a
+with the same directory
+(i.e. there is some redundancy so you can choose to parse the information where it is more convenient for you).
+
+The lines starting with <tt class="constant">DIST</tt> describe new or modified files in the distribution description exported by reprepro. No hint is given if that file was previously non-existent, a proper file or a symlink (i.e. if you copy stuff, do not make any assumptions about that).
+Future versions of reprepro might create <tt class="constant">DISTKEEP</tt> lines to denote files that have not changed (i.e. just ignore those lines to be future-proof).
+
+The directories for the distribution entries are what apt expects them (i.e. always starting with <tt class="constant">dists/</tt>, while the third argument to <tt class="constant">DISTFILE</tt> is the name reprepro generated (i.e. starts with the distdir value, which can be configured to not end with <tt class="constant">dists/</tt>).
+
+<h3><a name="endhook">when reprepro finished</a></h3>
+With the <tt class="option">--endhook</tt> command line option (or the corresponding
+<tt class="constant">endhook</tt> in the <tt class="filename">options</tt> file) you
+can specify a hook to be executed after reprepro finished but before reprepro returns
+the to calling process.
+The hook gets all the command line arguments after the options (i.e. starting with
+the name of the action) and the exit code reprepro would have produces.
+For an example see the man page.
+<h2><a name="maintenance">Maintenance</a></h2>
+This section lists some commands you can use to check and improve the health
+of you repository.
+<br>
+Normally nothing of this should be needed, but taking a look from time to time
+cannot harm.
+<pre class="command">
+reprepro -b $YOURBASEDIR dumpunreferenced
+</pre>
+This lists all files reprepro knows about that are not marked as needed by anything.
+Unless you called reprepro with the <tt class="option">--keepunreferenced</tt>
+option, those should never occur. Though if reprepro is confused or interrupted it
+may sometimes prefer keeping files around instead of deleting them.
+<pre class="command">
+reprepro -b $YOURBASEDIR deleteunreferenced
+</pre>
+This is like the command before, only that such files are directly forgotten and
+deleted.
+<pre class="command">
+reprepro -b $YOURBASEDIR check
+</pre>
+Look if all needed files are in fact marked needed and known.
+<pre class="command">
+reprepro -b $YOURBASEDIR checkpool
+</pre>
+Make sure all known files are still there and still have the same checksum.
+<pre class="command">
+reprepro -b $YOURBASEDIR checkpool fast
+</pre>
+As the command above, but do not compute checksums.
+<pre class="command">
+reprepro -b $YOURBASEDIR tidytracks
+</pre>
+If you use source package tracking, check for files kept because of this
+that should no longer by the current rules.
+<br>
+If you fear your tracking data could have became outdated,
+you can also try the retrack command:
+<pre class="command">
+reprepro -b $YOURBASEDIR retrack
+</pre>
+That refreshes the tracking information about packages used and then
+runs a tidytracks. (Beware: don't do this with reprepro versions before 3.0.0).
+<h2><a name="internals">Internals</a></h2>
+reprepro stores the data it collects in Berkeley DB file (<tt class="suffix">.db</tt>)
+in a directory called <tt class="dir">db/</tt> or whatever you specified via command
+line. With a few exceptions, those files are NO CACHES, but the actual data.
+While some of those data can be regained when you lose those files, they are better
+not deleted.
+<h3>packages.db</h3>
+This file contains the actual package information.
+<br>
+It contains a database for every (codename,component,architecture,packagetype) quadruple
+available.
+<br>
+Each is indexed by package name and essentially contains the information written do
+the Packages and Sources files.
+<br>
+Note that if you change your <tt class="file">conf/distributions</tt> to no longer
+list some codenames, architectures or components,
+that will not remove the associated databases in this file.
+That needs an explicit call to <tt class="command">clearvanished</tt>.
+<h3>references.db</h3>
+This file contains a single database that lists for every file why this file
+is still needed.
+This is either an identifier for a package database, an tracked source package,
+or a snapshot.
+<br>
+Some low level commands to access this are (take a look at the manpage for how to use them):
+<dl class="commands">
+<dt class="command">rereference</dt><dd>recreate references (i.e. forget old and create newly)</dd>
+<dt class="command">dumpreferences</dt><dd>print a list of all references</dd>
+<dt class="command">_removereferences</dt><dd>remove everything referenced by a given identifier</dd>
+<dt class="command">_addreference</dt><dd>manually add a reference</dd>
+<dt class="command">_addreferences</dt><dd>manually add multiple references</dd>
+</dl>
+<h3>files.db / checksums.db</h3>
+These files contains what reprepro knows about your <tt class="dir">pool/</tt> directory,
+i.e. what files it things are there with what sizes and checksums.
+The file <tt class="filename">files.db</tt> is used by reprepro before version 3.3
+and kept for backwards compatibility.
+If your repository was only used with newer versions you can safely delete it.
+Otherwise you should run <tt class="command">collectnewchecksums</tt> before deleting
+it.
+The file <tt class="filename">checksums.db</tt> is the new file used since
+version 3.3.
+It can store more checksums types (<tt class="filename">files.db</tt> only contained
+md5sums, <tt class="filename">checksums.db</tt> can store arbitrary checksums and
+reprepro can even cope with it containing checksum types it does not yet know of)
+but for compatibility with pre-3.3 versions is not the canonical source of information
+as long as a <tt class="filename">files.db</tt> file exists).
+<br>
+If you manually put files in the pool or remove them, you should tell reprepro about that.
+(it sometimes looks for files there without being told, but it never forgets files
+except when it would have deleted them anyway).
+Some low level commands (take a look at the man page for how to use them):
+<dl class="commands">
+<dt class="command">collectnewchecksums</dt><dd>Make sure every file is listed in <tt class="filename">checksums.db</tt> and with all checksum types your reprepro supports.</dd>
+<dt class="command">checkpool fast</dt><dd>Make sure all files are still there.</dd>
+<dt class="command">checkpool</dt><dd>Make sure all files are still there and correct.</dd>
+<dt class="command">dumpunreferenced</dt><dd>Show all known files without reference.</dd>
+<dt class="command">deleteunreferenced</dt><dd>Delete all known files without reference.</dd>
+<dt class="command">_listmd5sums</dt><dd>Dump this database (old style)</dd>
+<dt class="command">_listchecksums</dt><dd>Dump this database (new style)</dd>
+<dt class="command">_detect</dt><dd>Add files to the database</dd>
+<dt class="command">_forget</dt><dd>Forget that some file is there</dd>
+<dt class="command">_addmd5sums</dt><dd>Create the database from dumped data</dd>
+<dt class="command">_addchecksums</dt><dd>dito</dd>
+<h3>release.cache.db</h3>
+In this file reprepro remembers what it already wrote to the <tt class="dir">dists</tt>
+directory,
+so that it can write their checksums (including the checksums of the uncompressed variant,
+even if that was never written to disk)
+in a newly to create <tt class="file">Release</tt>
+file without having to trust those files or having to unpack them.
+<h3>contents.cache.db</h3>
+This file contains all the lists of files of binary package files where reprepro
+already needed them. (which can only happen if you requested Contents files to be
+generated).
+<h3>tracking.db</h3>
+This file contains the information of the <a href="#tracking">source package tracking</a>.
+<h2><a name="recovery">Disaster recovery</a></h2>
+TO BE DOCUMENTED (see the
+<a href="http://git.debian.org/?p=mirrorer/reprepro.git;a=blob_plain;f=docs/recovery;hb=HEAD">recovery</a>
+file until then)
+<h2><a name="paranoia">Paranoia</a></h2>
+As all software, reprepro might have bugs.
+And it uses libraries not written by myself,
+which I'm thus even more sure that they will have bugs.
+Some of those bugs might be security relevant.
+This section contains some tips, to reduce the impact of those.
+<ul>
+<li>Never run reprepro as root.<br>
+All reprepro needs to work are permissions to files,
+there is no excuse for running it as root.
+</li>
+<li>Don't publish your db/ directory.<br>
+The contents of the db directory are not needed by everyone else.
+Having them available to everyone may make it easier for them to
+exploit some hypothetical problem in libdb and makes it easier to
+know in advance how exactly reprepro will act in a given circumstances,
+thus easier to exploit some hypothetical problem.
+</li>
+<li>Don't accept untrusted data without need.<br>
+If an attacker cannot do anything, they cannot do anything harmful, either.
+So if there is no need, don't offer an anonymous incoming queue.
+<tt class="program">dput</tt> supports uploading via scp, so just having
+an only group-writable incoming directory, or even better multiple incoming
+directories can be a better alternative.
+</li>
+</ul>
+External stuff being used and attack vectors opened by it:
+<dl>
+<dt>libgpgme/gpg</dt><dd>
+Almost anything is run through <tt>libgpgme</tt> and thus <tt>gpg</tt>.
+It will be used to check the <tt class="filename">Release.gpg</tt> file,
+or to read <tt class="suffix">.dsc</tt> and <tt class="suffix">.changes</tt>
+files (even when there is no key to look for specified,
+as that is the best way to get the data from the signed block).
+Avoiding this by just accepting stuff without looking for signatures on
+untrusted data is not really an option, so I know nothing to prevent this
+type of problems.
+</dd>
+<dt>libarchive</dt><dd>
+The <tt class="suffix">.tar</tt> files within <tt class="suffix">.deb</tt>
+files are normally (unless that library was
+not available while compiling) read using libarchive.
+This happens when a <tt class="suffix">.deb</tt> file is to be added
+(though only after deciding if it should be added, so if it does not have
+the correct checksum or the .changes did not have the signatures you specified,
+it is not) or when the file list is to be extracted
+(when creating <tt class="filename">Contents</tt> files).
+Note that they are not processed when only mirroring them (of course unless
+<tt class="filename">Contents</tt> files are generated), as then only the
+information from the Packages file is copied.
+</dd>
+<dt>dpkg-deb/tar</dt><dd>
+If reprepro was compiled without libarchive,
+<tt class="program">dpkg-deb</tt> is used instead, which most likely will
+call <tt class="program">tar</tt>. Otherwise just the same like the last
+item.
+</dd>
+<dt>zlib</dt><dd>
+When mirroring packages, the downloaded
+<tt class="filename">Packages.gz</tt> and <tt class="filename">Sources.gz</tt> files
+are read using zlib. Also the generated <tt class="suffix">.gz</tt> files
+are generated using it. There is no option but hoping there is no security
+relevant problem in that library.
+</dd>
+<dt>libbz2</dt><dd>
+Only used to generate <tt class="suffix">.bz2</tt> files.
+If you fear simple blockwise writing using that library has a security problem
+that can be exploited by data enough harmless looking to be written to the
+generated index files, you can always decide to no tell reprepro to generate
+<tt class="suffix">.bz2</tt> files.
+</dd>
+</dl>
+<h2><a name="counterindications">What reprepro cannot do</a></h2>
+There are some things reprepro does not do:
+<dl><dt>Verbatim mirroring</dt><dd>
+Reprepro aims to put all files into a coherent <tt>pool/</tt> hierarchy.
+Thus it cannot guarantee that files will have the same relatives path as in the
+original repository (especially if those have no pool).
+It also creates the index files from its own indices.
+While this leads to a tidy repository and possible savings of disk-space, the
+signatures of the repositories you mirror cannot be used to authenticate the mirror,
+but you will have to sign (or tell reprepro to sign for you) the result.
+While this is perfect when you only mirror some parts or specific packages or
+also have local packages that need local signing anyway, reprepro is no suitable tool
+for creating a full mirror that can be authenticated without adding the key of this
+repository.
+</dd>
+<dt>Placing your files on your own</dt><dd>
+Reprepro does all the calculation of filenames to save files as,
+bookkeeping what files are there and what are needed and so on.
+This cannot be switched off or disabled.
+You can place files where reprepro will expect them and reprepro will use
+them if their md5sum matches.
+But reprepro is not suited if you want those files outside of a pool or in
+places reprepro does not consider their canonical ones.
+</dd>
+<dt>Having different files with the same name</dt><dd>
+take a look in the <a href="http://git.debian.org/?p=mirrorer/reprepro.git;a=blob_plain;f=docs/FAQ;hb=HEAD">FAQ</a> (currently question 1.2) why and how to avoid the problem.
+
+</dd>
+</dl>
+</body>
+</html>
author	Daniel Baumann <daniel.baumann@progress-linux.org>	2024-04-15 19:12:14 +0000
committer	Daniel Baumann <daniel.baumann@progress-linux.org>	2024-04-15 19:12:14 +0000
commit	4b8a0f3f3dcf60dac2ce308ea08d413a535af29f (patch)
tree	0f09c0ad2a4d0f535d89040a63dc3a866a6606e6 /docs/manual.html
parent	Initial commit. (diff)
download	reprepro-4b8a0f3f3dcf60dac2ce308ea08d413a535af29f.tar.xz reprepro-4b8a0f3f3dcf60dac2ce308ea08d413a535af29f.zip