1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
|
.. _faq:
iperf3 FAQ
==========
What is the history of iperf3, and what is the difference between iperf2 and iperf3?
iperf2 was orphaned in the late 2000s at version 2.0.5, despite some
known bugs and issues. After spending some time trying to fix
iperf2's problems, ESnet decided by 2010 that a new, simpler tool
was needed, and began development of iperf3. The goal was make the
tool as simple as possible, so others could contribute to the code
base. For this reason, it was decided to make the tool single
threaded, and not worry about backwards compatibility with
iperf2. Many of the feature requests for iperf3 came from the
perfSONAR project (http://www.perfsonar.net).
Then in 2014, Bob (Robert) McMahon from Broadcom restarted
development of iperf2 (See
https://sourceforge.net/projects/iperf2/). He fixed many of the
problems with iperf2, and added a number of new features similar to
iperf3. iperf2.0.8, released in 2015, made iperf2 a useful tool. iperf2's
current development is focused is on using UDP for latency testing, as well
as broad platform support.
As of this writing (2017), both iperf2 and iperf3 are being actively
(although independently) developed. We recommend being familiar with
both tools, and use whichever tool’s features best match your needs.
A feature comparison of iperf2, iperf3, and nuttcp is available at:
https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/throughput-tool-comparision/
iperf3 parallel stream performance is much less than iperf2. Why?
iperf3 is single threaded, and iperf2 is multi-threaded. We
recommend using iperf2 for parallel streams.
If you want to use multiple iperf3 streams use the method described `here <https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/>`_.
I’m trying to use iperf3 on Windows, but having trouble. What should I do?
iperf3 is not officially supported on Windows, but iperf2 is. We
recommend you use iperf2.
Some people are using Cygwin to run iperf3 in Windows, but not all
options will work. Some community-provided binaries of iperf3 for
Windows exist.
How can I build a statically-linked executable of iperf3?
There are a number of reasons for building an iperf3 executable with
no dependencies on any shared libraries. Unfortunately this isn't
quite a straight-forward process.
The steps below have nominally been tested on CentOS 7.4, but
can probably be adapted for use with other Linux distributions:
#. If necessary, install the static C libraries; for CentOS this is
the ``glibc-static`` package.
#. If OpenSSL is installed, be sure that its static libraries are
also installed, from the ``openssl-static`` package.
#. Be sure that ``lksctp-*`` packages are not installed, because
as of this writing, there do not appear to be any static
libraries available for SCTP.
#. Configure iperf3 thusly: ``configure "LDFLAGS=--static"
--disable-shared`` These options are necessary to disable the
generation of shared libraries and link the executable
statically. For iperf-3.8 or later, configuring as ``configure
--enable-static-bin`` is another, shorter way to accomplish
this. If SCTP is installed on the system it might also be
necessary to pass the ``--without-sctp`` flag at configure
time.
#. Compile as normal.
It appears that for FreeBSD (tested on FreeBSD 11.1-RELEASE), only
the last two steps are needed to produce a static executable.
How can I build on a system that doesn't support profiled executables?
This problem has been noted by users attempting to build iperf3 for
Android systems, as well as some recent versions of macOS.
There are several workarounds. In order from least
effort to most effort:
#. Beginning with iperf-3.8, profiled executables are actually not
built by default, so this question becomes somewhat moot. Pass
the ``--enable-profiling`` flag to ``configure`` to build
profiled executables.
#. In iperf-3.6 and iperf-3.7, the ``--disable-profiling`` flag can be
passed to ``configure`` to disable the building of profiled
object files and the profiled executable.
#. At the time the linking of the iperf3 profiled executable fails,
the "normal" iperf3 executable is probably already created. So if
you are willing to accept the error exit from the make process
(and a little bit of wasted work on the build host), you might
not need to do anything.
#. After the configure step, there will be a definition in
``src/Makefile`` that looks like this::
noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) \
iperf3_profile$(EXEEXT)
If you edit it to look like this, it will disable the build of the profiled iperf3::
noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT)
#. Similar to item 2 above, but more permanent...if you edit
``src/Makefile.am`` and change the line reading like this::
noinst_PROGRAMS = t_timer t_units t_uuid iperf3_profile
To look like this::
noinst_PROGRAMS = t_timer t_units t_uuid
And then run ``./bootstrap.sh``, that will regenerate the project
Makefiles to make the exclusion of the profiled iperf3 executable
permanent (within that source tree).
I'm seeing quite a bit of unexpected UDP loss. Why?
First, confirm you are using iperf 3.1.5 or higher. There was an
issue with the default UDP send size that was fixed in
3.1.5. Second, try adding the flag ``-w2M`` to increase the socket
buffer sizes. That seems to make a big difference on some hosts.
iperf3 UDP does not seem to work at bandwidths less than 100Kbps. Why?
You'll need to reduce the default packet length to get UDP rates of less that 100Kbps. Try ``-l100``.
TCP throughput drops to (almost) zero during a test, what's going on?
A drop in throughput to almost zero, except maybe for the first
reported interval(s), may be related to problems in NIC TCP Offload,
which is used to offload TCP functionality to the NIC (see
https://en.wikipedia.org/wiki/TCP_offload_engine). The goal of TCP
Offload is to save main CPU performance, mainly in the areas of
segmentation and reassembly of large packets and checksum
computation.
When TCP packets are sent with the "Don't Fragment" flag set, which
is the recommended setting, segmentation is done by the TCP stack
based on the reported next hop MSS in the ICMP Fragmentation Needed
message. With TCP Offload, active segmentation is done by the NIC on
the sending side, which is known as TCP Segmentation offload (TSO)
or in Windows as Large Send Offload (LSO). It seems that there are
TSO/LSO implementations which for some reason ignore the reported
MSS and therefore don’t perform segmentation. In these cases, when
large packets are sent, e.g. the default iperf3 128KB (131,072
bytes), iperf3 will show that data was sent in the first interval,
but since the packets don’t get to the server, no ack is received
and therefore no data is sent in the following intervals. It may
happen that after certain timeout the main CPU will re-send the
packet by re-segmenting it, and in these cases data will get to the
server after a while. However, it seems that segmentation is not
automatically continued with the next packet, so the data transfer
rate be very low.
The recommended solution in such a case is to disable TSO/LSO, at
least on the relevant port. See for example:
https://atomicit.ca/kb/articles/slow-network-speed-windows-10/. If
that doesn’t help then "Don't Fragment" TCP flag may be
disabled. See for example:
https://support.microsoft.com/en-us/help/900926/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than. However,
note that disabling the “Don’t Fragment” flag may cause other
issues.
To test whether TSO/LSO may be the problem, do the following:
* If different machine configurations are used for the client and
server, try the iperf3 reverse mode (``-R``). If TSO/LSO is only
enabled on the client machine, this test should succeed.
* Reduce the sending length to a small value that should not require
segmentation, using the iperf3 ``-l`` option, e.g. ``-l 512``. It
may also help to reduce the MTU by using the iperf3 ``-M`` option,
e.g. ``-M 1460``.
* Using tools like Wireshark, identify the required MSS in the ICMP
Fragmentation Needed messages (if reported). Run tests with the
``-l`` value set to 2 times the MSS and then 4 times, 6 times,
etc. With TSO/LSO issue in each test the throughput should be
reduced more. It may help to increase the testing time beyond the
default 10 seconds to better see the behavior (iperf3 ``-t``
option).
What congestion control algorithms are supported?
On Linux, run this command to see the available congestion control
algorithms (note that some algorithms are packaged as kernel
modules, which must be loaded before they can be used)::
/sbin/sysctl net.ipv4.tcp_available_congestion_control
On FreeBSD, the equivalent command is::
/sbin/sysctl net.inet.tcp.cc.available
I’m using the ``--logfile`` option. How do I see file output in real time?
Use the ``--forceflush`` flag.
I'm using the --fq-rate flag, but it does not seem to be working. Why?
You need to add 'net.core.default_qdisc = fq' to /etc/sysctl.conf for that option to work.
I'm having trouble getting iperf3 to work on Windows, Android, etc. Where can I get help?
iperf3 only supports Linux, FreeBSD, and OSX. For other platforms we recommend using iperf2.
I managed to get a Windows executable built, but why do I get a BSOD on Windows 7?
There seems to be a bug in Windows 7 where running iperf3 from a
network filesystem can cause a system crash (in other words Blue
Screen of Death, or BSOD). This is a Windows bug addressed in kb2839149:
https://support.microsoft.com/en-us/help/2839149/stop-error-0x00000027-in-the-rdbss-sys-process-in-windows-7-or-windows
A hotfix is available under kb2732673:
https://support.microsoft.com/en-us/help/2732673/-delayed-write-failed-error-message-when--pst-files-are-stored-on-a-ne
Why can’t I run a UDP client with no server?
This is potentially dangerous, and an attacker could use this for a
denial of service attack. We don't want iperf3 to be an attack tool.
I'm trying to use iperf3 to test a 40G/100G link...What do I need to know?
See the following pages on fasterdata.es.net:
- https://fasterdata.es.net/host-tuning/100g-tuning/
- https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/
My receiver didn't get all the bytes that got sent but there was no loss. Huh?
iperf3 uses a control connection between the client and server to
manage the start and end of each test. Sometimes the commands on
the control connection can be received and acted upon before all of
the test data has been processed. Thus the test ends with data
still in flight. This effect can be significant for short (a few
seconds) tests, but is probably negligible for longer tests.
A file sent using the ``-F`` option got corrupted...what happened?
The ``-F`` option to iperf3 is not a file transfer utility. It's a
way of testing the end-to-end performance of a file transfer,
including filesystem and disk overheads. So while the test will
mimic an actual file transfer, the data stored to disk may not be
the same as what was sent. In particular, the file size will be
rounded up to the next larger multiple of the transfer block size,
and for UDP tests, iperf's metadata (containing timestamps and
sequence numbers) will overwrite the start of every UDP packet
payload.
I have a question regarding iperf3...what's the best way to get help?
Searching on the Internet is a good first step.
http://stackoverflow.com/ has a number of iperf3-related questions
and answers, but a simple query into your favorite search engine can
also yield some results.
There is a mailing list nominally used for iperf3 development,
iperf-dev@googlegroups.com.
We discourage the use of the iperf3 issue tracker on GitHub for
support questions. Actual bug reports, enhancement requests, or
pull requests are encouraged, however.
|
