1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
|
Coverage collection explained
=============================
Hi mate. In short, you don't wanna know this. Believe me, you don't. Please, close this file and forget about it.
Surely? You still here?
Please, stop it until it's too late.
You were warned.
Preamble
--------
RSPAMD is written mainly in two languages: C and Lua. Coverage for each of them is being collected using different
tools and approaches and is sent into [coveralls.io](https://coveralls.io).
Each approach is not quite compatible to other tools. This document describes how we crutch them to work together.
C coverage
----------
In general, pretty boring. When you run `cmake` with "-DENABLE_COVERAGE=ON" flag, it adds "--coverage" flag to both
CFLAGS and LDFLAGS. So that each run of generated binary will create `*.gcda` file containing coverage data.
However, there are some moment to highlight:
- RSPAMD is run under "nobody" user. Hence, directories and files should be writable for this user.
- To make it possible, we explicitly run `umask 0000` in "build" and "functional" stages in .circleci/config.yml
- After run, we persist coverage data in "coverage.${CIRCLE\_JOB}.dump" during this build flow, see `capture_coverage_data`,
to use it on the final stage.
- we use `cpp-coveralls` because it is able to save data for coveralls without actually sending it. We send on our own
along with Lua-coverage.
Lua coverage
------------
Lua coverage is collected for unit-tests and functional test suite.
First part contains nothing interesting, just see `test/lua/tests.lua`.
"Functional" part is completely unobvious.
1. Coverage collecting is initiated and dumped in `test/functional/lua/test_coverage.lua` (there are a lot of comments inside).
This file should be included on the very early stage of test run. Usually it's included via config.
2. Coverage is dumped into ${TMPDIR}/%{worker_name}.luacov.stats.out
3. All worker coverage reports are merged into `lua_coverage_report.json` (see `collect_lua_coverage()`)
4. finally, `lua_coverage_report.json` is persisted in build flow (see `functional` stage)
Altogether
----------
Finally, we get all the reports:
- `coverage.functional.dump`
- `coverage.rspamd-test.dump`
- `lua_coverage_report.json`
- `unit_test_lua.json`
and merge them and send the resulting report using `test/functional/util/merge_coveralls.py`. Also, this scripts maps installed
paths into corresponding repository paths and removes unneeded files (i.e. test sources).
|
