summaryrefslogtreecommitdiffstats
path: root/src/c-ares/test/README.md
blob: 988abb674cdbca4ad2d9752e64ab36f14e0f8fc5 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
c-ares Unit Test Suite
======================


This directory holds unit tests for the c-ares library.  To build the tests:

 - Build the main c-ares library first, in the directory above this.  To
   enable tests of internal functions, configure the library build to expose
   hidden symbols with `./configure --disable-symbol-hiding`.
 - Generate a `configure` file by running `autoreconf -iv` (which requires
   a local installation of
   [autotools](https://www.gnu.org/software/automake/manual/html_node/Autotools-Introduction.html)).
 - `./configure`
 - `make`
 - Run the tests with `./arestest`, or `./arestest -v` for extra debug info.

Points to note:

 - The tests are written in C++11, and so need a C++ compiler that supports
   this.  To avoid adding this as a requirement for the library, the
   configuration and build of the tests is independent from the library.
 - The tests include some live queries, which will fail when run on a machine
   without internet connectivity.  To skip live tests, run with
   `./arestest --gtest_filter=-*.Live*`.
 - The tests include queries of a mock DNS server.  This server listens on port
   5300 by default, but the port can be changed with the `-p 5300` option to
   `arestest`.


Test Types
----------

The test suite includes various different types of test.

 - There are live tests (`ares-test-live.cc`), which assume that the
   current machine has a valid DNS setup and connection to the
   internet; these tests issue queries for real domains but don't
   particularly check what gets returned.  The tests will fail on
   an offline machine.
 - There are some mock tests (`ares-test-mock.cc`) that set up a fake DNS
   server and inject its port into the c-ares library configuration.
   These tests allow specific response messages to be crafted and
   injected, and so are likely to be used for many more tests in
   future.
    - To make this generation/injection easier, the `dns-proto.h`
      file includes C++ helper classes for building DNS packets.
 - Other library entrypoints that don't require network activity
   (e.g. `ares_parse_*_reply`) are tested directly.
 - A couple of the tests use a helper method of the test fixture to
   inject memory allocation failures, using a recent change to the
   c-ares library that allows override of `malloc`/`free`.
 - There are some tests of the internal entrypoints of the library
   (`ares-test-internal.c`), but these are only enabled if the library
   was configured with `--disable-symbol-hiding` and/or
   `--enable-expose-statics`.
 - There is also an entrypoint to allow Clang's
   [libfuzzer](http://llvm.org/docs/LibFuzzer.html) to drive
   the packet parsing code in `ares_parse_*_reply`, together with a
   standalone wrapper for it (`./aresfuzz`) to allow use of command
   line fuzzers (such as [afl-fuzz](http://lcamtuf.coredump.cx/afl/))
   for further [fuzz testing](#fuzzing).


Code Coverage Information
-------------------------

To generate code coverage information:

 - Configure both the library and the tests with `./configure
   --enable-code-coverage` before building. This requires the relevant code
   coverage tools ([gcov](https://gcc.gnu.org/onlinedocs/gcc/Gcov.html),
   [lcov](http://ltp.sourceforge.net/coverage/lcov.php)) to be installed locally.
 - Run the tests with `test/arestest`.
 - Generate code coverage output with `make code-coverage-capture` in the
   library directory (i.e. not in `test/`).


Fuzzing
-------

### libFuzzer

To fuzz the packet parsing code with libFuzzer, follow the main
[libFuzzer build instructions](http://llvm.org/docs/LibFuzzer.html#building):

 - Configure the c-ares library and test suite with a recent Clang and a sanitizer, for example:

   ```console
   % export CFLAGS="-fsanitize=address -fsanitize-coverage=edge"
   % export CC=clang
   % ./configure --disable-shared && make
   ```
 - Download and build the libFuzzer code:

   ```console
   % cd test
   % svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer
   % clang++ -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer
   % ar ruv libFuzzer.a Fuzzer*.o
   ```
 - Link each of the fuzzer entrypoints in with `ares-fuzz.cc`:

   ```
   % $CC $CFLAGS -I.. -c ares-test-fuzz.c
   % $CC $CFLAGS -I.. -c ares-test-fuzz-name.c
   % clang++ $CFLAGS ares-test-fuzz.o ../.libs/libcares.a libFuzzer.a -o ares-libfuzzer
   % clang++ $CFLAGS ares-test-fuzz-name.o ../.libs/libcares.a libFuzzer.a -o ares-libfuzzer-name
   ```
 - Run the fuzzer using the starting corpus with:

   ```console
   % ./ares-libfuzzer fuzzinput/  # OR
   % ./ares-libfuzzer-name fuzznames/
   ```

### AFL

To fuzz using AFL, follow the
[AFL quick start guide](http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt):

 - Download and build AFL.
 - Configure the c-ares library and test tool to use AFL's compiler wrappers:

   ```console
   % export CC=$AFLDIR/afl-gcc
   % ./configure --disable-shared && make
   % cd test && ./configure && make aresfuzz aresfuzzname
   ```

 - Run the AFL fuzzer against the starting corpus:

   ```console
   % mkdir fuzzoutput
   % $AFLDIR/afl-fuzz -i fuzzinput -o fuzzoutput -- ./aresfuzz  # OR
   % $AFLDIR/afl-fuzz -i fuzznames -o fuzzoutput -- ./aresfuzzname
   ```

### AFL Persistent Mode

If a recent version of Clang is available, AFL can use its built-in compiler
instrumentation; this configuration also allows the use of a (much) faster
persistent mode, where multiple fuzz inputs are run for each process invocation.

 - Download and build a recent AFL, and run `make` in the `llvm_mode`
   subdirectory to ensure that `afl-clang-fast` gets built.
 - Configure the c-ares library and test tool to use AFL's clang wrappers that
   use compiler instrumentation:

   ```console
   % export CC=$AFLDIR/afl-clang-fast
   % ./configure --disable-shared && make
   % cd test && ./configure && make aresfuzz
   ```

 - Run the AFL fuzzer (in persistent mode) against the starting corpus:

   ```console
   % mkdir fuzzoutput
   % $AFLDIR/afl-fuzz -i fuzzinput -o fuzzoutput -- ./aresfuzz
   ```