1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
|
# How to build and install rsync
When building rsync, you'll want to install various libraries in order to get
all the features enabled. The configure script will alert you when the
newest libraries are missing and tell you the appropriate `--disable-LIB`
option to use if you want to just skip that feature. What follows are various
support libraries that you may want to install to build rsync with the maximum
features (the impatient can skip down to the package summary):
## The basic setup
You need to have a C compiler installed and optionally a C++ compiler in order
to try to build some hardware-accelerated checksum routines. Rsync also needs
a modern awk, which might be provided via gawk or nawk on some OSes.
## Autoconf & manpages
If you're installing from the git repo (instead of a release tar file) you'll
also need the GNU autotools (autoconf & automake) and your choice of 2 python3
markdown libraries: cmarkgfm or commonmark (needed to generate the manpages).
If your OS doesn't provide a python3-cmarkgfm or python3-commonmark package,
you can run the following to install the commonmark python library for your
build user (after installing python3's pip package):
> python3 -mpip install --user commonmark
You can test if you've got it fixed by running (from the rsync checkout):
> ./md-convert --test rsync-ssl.1.md
Alternately, you can avoid generating the manpages by fetching the very latest
versions (that match the latest git source) from the [generated-files][6] dir.
One way to do that is to run:
> ./prepare-source fetchgen
[6]: https://download.samba.org/pub/rsync/generated-files/
## ACL support
To support copying ACL file information, make sure you have an acl
development library installed. It also helps to have the helper programs
installed to manipulate ACLs and to run the rsync testsuite.
## Xattr support
To support copying xattr file information, make sure you have an attr
development library installed. It also helps to have the helper programs
installed to manipulate xattrs and to run the rsync testsuite.
## xxhash
The [xxHash library][1] provides extremely fast checksum functions that can
make the "rsync algorithm" run much more quickly, especially when matching
blocks in large files. Installing this development library adds xxhash
checksums as the default checksum algorithm. You'll need at least v0.8.0
if you want rsync to include the full range of its checksum algorithms.
[1]: https://cyan4973.github.io/xxHash/
## zstd
The [zstd library][2] compression algorithm that uses less CPU than
the default zlib algorithm at the same compression level. Note that you
need at least version 1.4, so you might need to skip the zstd compression if
you can only install a 1.3 release. Installing this development library
adds zstd compression as the default compression algorithm.
[2]: http://facebook.github.io/zstd/
## lz4
The [lz4 library][3] compression algorithm that uses very little CPU, though
it also has the smallest compression ratio of other algorithms. Installing
this development library adds lz4 compression as an available compression
algorithm.
[3]: https://lz4.github.io/lz4/
## openssl crypto
The [openssl crypto library][4] provides some hardware accelerated checksum
algorithms for MD4 and MD5. Installing this development library makes rsync
use the (potentially) faster checksum routines when computing MD4 & MD5
checksums.
[4]: https://www.openssl.org/docs/man1.0.2/man3/crypto.html
## Package summary
To help you get the libraries installed, here are some package install commands
for various OSes. The commands are split up to correspond with the above
items, but feel free to combine the package names into a single install, if you
like.
- For Debian and Ubuntu (Debian Buster users may want to briefly(?) enable
buster-backports to update zstd from 1.3 to 1.4):
> sudo apt install -y gcc g++ gawk autoconf automake python3-cmarkgfm
> sudo apt install -y acl libacl1-dev
> sudo apt install -y attr libattr1-dev
> sudo apt install -y libxxhash-dev
> sudo apt install -y libzstd-dev
> sudo apt install -y liblz4-dev
> sudo apt install -y libssl-dev
- For CentOS (use EPEL for python3-pip):
> sudo yum -y install epel-release
> sudo yum -y install gcc g++ gawk autoconf automake python3-pip
> sudo yum -y install acl libacl-devel
> sudo yum -y install attr libattr-devel
> sudo yum -y install xxhash-devel
> sudo yum -y install libzstd-devel
> sudo yum -y install lz4-devel
> sudo yum -y install openssl-devel
> python3 -mpip install --user commonmark
- For Fedora 33:
> sudo dnf -y install acl libacl-devel
> sudo dnf -y install attr libattr-devel
> sudo dnf -y install xxhash-devel
> sudo dnf -y install libzstd-devel
> sudo dnf -y install lz4-devel
> sudo dnf -y install openssl-devel
- For FreeBSD (this assumes that the python3 version is 3.7):
> sudo pkg install -y autotools python3 py37-CommonMark
> sudo pkg install -y xxhash
> sudo pkg install -y zstd
> sudo pkg install -y liblz4
- For macOS:
> brew install automake
> brew install xxhash
> brew install zstd
> brew install lz4
> brew install openssl
- For Cygwin (with all cygwin programs stopped, run the appropriate setup program from a cmd shell):
> setup-x86_64 --quiet-mode -P make,gawk,autoconf,automake,gcc-core,python38,python38-pip
> setup-x86_64 --quiet-mode -P attr,libattr-devel
> setup-x86_64 --quiet-mode -P libzstd-devel
> setup-x86_64 --quiet-mode -P liblz4-devel
> setup-x86_64 --quiet-mode -P libssl-devel
Sometimes cygwin has commonmark packaged and sometimes it doesn't. Now that
its python38 has stabilized, you could install python38-commonmark. Or just
avoid the issue by running this from a bash shell as your build user:
> python3 -mpip install --user commonmark
## Build and install
After installing the various libraries, you need to configure, build, and
install the source:
> ./configure
> make
> sudo make install
The default install path is /usr/local/bin, but you can set the installation
directory and other parameters using options to ./configure. To see them, use:
> ./configure --help
Configure tries to figure out if the local system uses group "nobody" or
"nogroup" by looking in the /etc/group file. (This is only used for the
default group of an rsync daemon, which attempts to run with "nobody"
user and group permissions.) You can change the default user and group
for the daemon by editing the NOBODY_USER and NOBODY_GROUP defines in
config.h, or just override them in your /etc/rsyncd.conf file.
As of 2.4.7, rsync uses Eric Troan's popt option-parsing library. A
cut-down copy of a recent release is included in the rsync distribution,
and will be used if there is no popt library on your build host, or if
the `--with-included-popt` option is passed to ./configure.
If you configure using `--enable-maintainer-mode`, then rsync will try
to pop up an xterm on DISPLAY=:0 if it crashes. You might find this
useful, but it should be turned off for production builds.
If you want to automatically use a separate "build" directory based on
the current git branch name, start with a pristine git checkout and run
"mkdir auto-build-save" before you run the first ./configure command.
That will cause a fresh build dir to spring into existence along with a
special Makefile symlink that allows you to run "make" and "./configure"
from the source dir (the "build" dir gets auto switched based on branch).
This is helpful when using the branch-from-patch and patch-update scripts
to maintain the official rsync patches. If you ever need to build from
a "detached head" git position then you'll need to manually chdir into
the build dir to run make. I also like to create 2 more symlinks in the
source dir: `ln -s build/rsync . ; ln -s build/testtmp .`
## Make compatibility
Note that Makefile.in has a rule that uses a wildcard in a prerequisite. If
your make has a problem with this rule, you will see an error like this:
Don't know how to make ./*.c
You can change the "proto.h-tstamp" target in Makefile.in to list all the \*.c
filenames explicitly in order to avoid this issue.
## RPM notes
Under packaging you will find .spec files for several distributions.
The .spec file in packaging/lsb can be used for Linux systems that
adhere to the Linux Standards Base (e.g., RedHat and others).
## HP-UX notes
The HP-UX 10.10 "bundled" C compiler seems not to be able to cope with
ANSI C. You may see this error message in config.log if ./configure
fails:
(Bundled) cc: "configure", line 2162: error 1705: Function prototypes are an ANSI feature.
Install gcc or HP's "ANSI/C Compiler".
## Mac OS X notes
Some versions of Mac OS X (Darwin) seem to have an IPv6 stack, but do
not completely implement the "New Sockets" API.
[This site][5] says that Apple started to support IPv6 in 10.2 (Jaguar). If
your build fails, try again after running configure with `--disable-ipv6`.
[5]: http://www.ipv6.org/impl/mac.html
## IBM AIX notes
IBM AIX has a largefile problem with mkstemp. See IBM PR-51921.
The workaround is to append the following to config.h:
> #ifdef _LARGE_FILES
> #undef HAVE_SECURE_MKSTEMP
> #endif
|
