1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
|
# Suricata
[](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:suricata)
[](https://codecov.io/gh/OISF/suricata)
## Introduction
[Suricata](https://suricata.io) is a network IDS, IPS and NSM engine
developed by the [OISF](https://oisf.net) and the Suricata community.
## Resources
- [Home Page](https://suricata.io)
- [Bug Tracker](https://redmine.openinfosecfoundation.org/projects/suricata)
- [User Guide](https://docs.suricata.io)
- [Installation Guide](https://docs.suricata.io/en/latest/install.html)
- [User Support Forum](https://forum.suricata.io)
## Contributing
We're happily taking patches and other contributions. Please see our
[Contribution
Process](https://docs.suricata.io/en/latest/devguide/codebase/contributing/contribution-process.html)
for how to get started.
Suricata is a complex piece of software dealing with mostly untrusted
input. Mishandling this input will have serious consequences:
* in IPS mode a crash may knock a network offline
* in passive mode a compromise of the IDS may lead to loss of critical
and confidential data
* missed detection may lead to undetected compromise of the network
In other words, we think the stakes are pretty high, especially since
in many common cases the IDS/IPS will be directly reachable by an
attacker.
For this reason, we have developed a QA process that is quite
extensive. A consequence is that contributing to Suricata can be a
somewhat lengthy process.
On a high level, the steps are:
1. GitHub-CI based checks. This runs automatically when a pull request
is made.
2. Review by devs from the team and community
3. QA runs from private QA setups. These are private due to the nature
of the test traffic.
### Overview of Suricata's QA steps
OISF team members are able to submit builds to our private QA
setup. It will run a series of build tests and a regression suite to
confirm no existing features break.
The final QA runs takes a few hours minimally, and generally runs
overnight. It currently runs:
- extensive build tests on different OS', compilers, optimization
levels, configure features
- static code analysis using cppcheck, scan-build
- runtime code analysis using valgrind, AddressSanitizer,
LeakSanitizer
- regression tests for past bugs
- output validation of logging
- unix socket testing
- pcap based fuzz testing using ASAN and LSAN
- traffic replay based IDS and IPS tests
Next to these tests, based on the type of code change further tests
can be run manually:
- traffic replay testing (multi-gigabit)
- large pcap collection processing (multi-terabytes)
- fuzz testing (might take multiple days or even weeks)
- pcap based performance testing
- live performance testing
- various other manual tests based on evaluation of the proposed
changes
It's important to realize that almost all of the tests above are used
as acceptance tests. If something fails, it's up to you to address
this in your code.
One step of the QA is currently run post-merge. We submit builds to
the Coverity Scan program. Due to limitations of this (free) service,
we can submit once a day max. Of course it can happen that after the
merge the community will find issues. For both cases we request you to
help address the issues as they may come up.
## FAQ
__Q: Will you accept my PR?__
A: That depends on a number of things, including the code
quality. With new features it also depends on whether the team and/or
the community think the feature is useful, how much it affects other
code and features, the risk of performance regressions, etc.
__Q: When will my PR be merged?__
A: It depends, if it's a major feature or considered a high risk
change, it will probably go into the next major version.
__Q: Why was my PR closed?__
A: As documented in the [Suricata GitHub
workflow](https://docs.suricata.io/en/latest/devguide/codebase/contributing/github-pr-workflow.html),
we expect a new pull request for every change.
Normally, the team (or community) will give feedback on a pull request
after which it is expected to be replaced by an improved PR. So look
at the comments. If you disagree with the comments we can still
discuss them in the closed PR.
If the PR was closed without comments it's likely due to QA
failure. If the GitHub-CI checks failed, the PR should be fixed right
away. No need for a discussion about it, unless you believe the QA
failure is incorrect.
__Q: The compiler/code analyser/tool is wrong, what now?__
A: To assist in the automation of the QA, we're not accepting warnings
or errors to stay. In some cases this could mean that we add a
suppression if the tool supports that (e.g. valgrind, DrMemory). Some
warnings can be disabled. In some exceptional cases the only
'solution' is to refactor the code to work around a static code
checker limitation false positive. While frustrating, we prefer this
over leaving warnings in the output. Warnings tend to get ignored and
then increase risk of hiding other warnings.
__Q: I think your QA test is wrong__
A: If you really think it is, we can discuss how to improve it. But
don't come to this conclusion too quickly, more often it's the code
that turns out to be wrong.
__Q: Do you require signing of a contributor license agreement?__
A: Yes, we do this to keep the ownership of Suricata in one hand: the
Open Information Security Foundation. See
http://suricata.io/about/open-source/ and
http://suricata.io/about/contribution-agreement/
|
