1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
|
.\"
.\" Automated Testing Framework (atf)
.\"
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd August 28, 2010
.Dt ATF 7
.Os
.Sh NAME
.Nm ATF
.Nd introduction to the Automated Testing Framework
.Sh DESCRIPTION
.Em IMPORTANT: If you are here because you want to know how to run the tests in
.Pa __TESTSDIR__ ,
.Em you most likely want to read the
.Xr tests 7
.Em manual page instead.
.Pp
The Automated Testing Framework
.Pf ( Nm )
is a collection of libraries and utilities designed to ease unattended
application testing in the hands of developers and end users of a specific
piece of software.
.Pp
As regards developers,
.Nm
provides the necessary means to easily create
test suites composed of multiple test programs, which in turn are a
collection of test cases.
It also attempts to simplify the debugging of problems when these test
cases detect an error by providing as much information as possible
about the failure.
.Pp
As regards users, it simplifies the process of running the test suites and,
in special, encourages end users to run them often: they do not need to
have source trees around nor any other development tools installed to be
able to certify that a given piece of software works on their machine as
advertised.
.Pp
If your operating systems distributes
.Nm ,
it is possible that it provides an introductory
.Xr tests 7
manual page.
You are encouraged to read it now.
.Ss License
.Nm
is distributed under the terms of the TNF License, a 2-clause BSD license.
For more details please see:
.Bd -literal -offset indent
.Pa __DOCDIR__/COPYING
.Ed
.Ss Components
.Nm
is a highly modular piece of software.
It provides a couple of libraries to ease the implementation of test
programs: one for the C and C++ languages and another one for shell
scripts.
It also includes multiple small utilities that follow the principle of
doing a single thing but doing it right.
This section outlines which these components are.
.Pp
Public utilities:
.Bl -tag -width atfXtestXprogramXXXXX
.It Xr atf-check 1
Executes a command and checks that its exit code, its standard output
and its standard error output match pre-specified expected values.
.It Xr atf-config 1
Queries static configuration information.
.It Xr atf-report 1
Converts the output of
.Nm atf-run
to user-friendly and/or machine-parseable reports.
.It Xr atf-run 1
Automates the execution of a series of test programs and collects their
results in a unified report.
.It Xr atf-sh 1
Shell interpreter for shell-based test programs.
.El
.Pp
Programming interfaces:
.Bl -tag -width atfXtestXprogramXXXXX
.It Xr atf-c-api 3
C programming interface for test programs.
.It Xr atf-c++-api 3
C++ programming interface for test programs.
.It Xr atf-sh-api 3
POSIX shell programming interface for test programs.
.El
.Pp
Other:
.Bl -tag -width atfXtestXprogramXXXXX
.It Xr atf-formats 5
Description of the machine-parseable data formats used by the tools.
.It Xr atf-test-case 4
Generic description of test cases, independent of the language they are
implemented in.
.It Xr atf-test-program 1
Common interface provided by the test programs written using the
.Nm
libraries.
.El
.Ss Recommended reading order
For end users wishing to run tests:
.Bl -enum -compact
.It
.Xr tests 7
(only if provided by your operating system).
.It
.Xr atf-test-program 1
.It
.Xr atf-run 1
.It
.Xr atf-report 1
.It
.Xr atf-config 1
.El
.Pp
For developers wanting to write their own tests:
.Bl -enum -compact
.It
Everything recommended to users.
.It
.Xr atf-test-case 4
.It
.Xr atf-c-api 3
.It
.Xr atf-c++-api 3
.It
.Xr atf-sh-api 3
.It
.Xr atf-sh 1
.It
.Xr atf-check 1
.El
.Pp
For those interested in
.Nm
internals:
.Bl -enum -compact
.It
Everything recommended to users.
.It
Everything recommended to developers.
.It
.Xr atf-formats 5
.El
.Sh SEE ALSO
.Xr tests 7
.Sh HISTORY
.Nm
started as a Google Summer of Code 2007 project mentored by The NetBSD
Foundation.
Its original goal was to provide a testing framework for The NetBSD
Operating System, but it grew as an independent project because the
framework itself did not need to be tied to a specific operating system.
.Pp
For more details on this subject, please see:
.Bd -literal -offset indent
.Pa __DOCDIR__/NEWS
.Pa __DOCDIR__/ROADMAP
.Ed
.Sh AUTHORS
For more details on the people that made
.Nm
possible, please see:
.Bd -literal -offset indent
.Pa __DOCDIR__/AUTHORS
.Ed
|
