1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
|
.\"
.\" Automated Testing Framework (atf)
.\"
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd December 16, 2011
.Dt ATF-REPORT 1
.Os
.Sh NAME
.Nm atf-report
.Nd transforms the output of atf-run to different formats
.Sh SYNOPSIS
.Nm
.Op Fl o Ar fmt1:path1 Op .. Fl o Ar fmtN:pathN
.Nm
.Fl h
.Sh DESCRIPTION
.Nm
reads the output of
.Nm atf-run
and transforms it to different formats.
Some of these are user-friendly and others are machine-parseable, which
opens a wide range of possibilities to analyze the results of a test
suite's execution.
See
.Sx Output formats
below for more details on which these formats are.
.Pp
In the first synopsis form,
.Nm
reads the output of
.Nm atf-run
through its standard input and, if no
.Fl o
options are given, prints a user-friendly report on its standard
output using the
.Sq ticker
format.
If
.Fl o
options are provided (more than one are allowed), they specify the complete
list of reports to generate.
They are all generated simultaneously, and for obvious reasons, two reports
cannot be written to the same file.
Note that the default output is suppressed when
.Fl o
is provided.
.Pp
In the second synopsis form,
.Nm
will print information about all supported options and their purpose.
.Pp
The following options are available:
.Bl -tag -width XoXfmtXpathXX
.It Fl h
Shows a short summary of all available options and their purpose.
.It Fl o Ar fmt:path
Adds a new output format.
.Ar fmt
is one of the formats described later on in
.Sx Output formats .
.Ar path
specifies where the report will be written to.
Depending on the chosen format, this may refer to a single file or to
a directory.
For those formats that write to a single file, specifying a
.Sq -
as the path will redirect the report to the standard output.
.El
.Ss Output formats
The following output formats are allowed:
.Bl -tag -width tickerXX
.It csv
A machine-parseable Comma-Separated Values (CSV) file.
This file contains the results for all test cases and test programs.
Test cases are logged using the following syntax:
.Bd -literal -offset indent
tc, duration, test-program, test-case, result[, reason]
.Ed
.Pp
The
.Sq result
field for test cases is always one of
.Sq passed ,
.Sq skipped
or
.Sq failed .
The last two are always followed by a reason.
.Pp
Test programs are logged with the following syntax:
.Bd -literal -offset indent
tp, duration, test-program, result[, reason]
.Ed
.Pp
In this case, the
.Sq result
can be one of:
.Sq passed ,
which denotes test programs that ran without any failure;
.Sq failed ,
which refers to test programs in which one or more test cases failed;
or
.Sq bogus ,
which mentions those test programs that failed to execute by some reason.
The reason field is only available in the last case.
.Pp
The time required to execute each test case and test program is
also provided.
You should not rely on the order of the entries in the resulting output.
.It ticker
A user-friendly report that shows the progress of the test suite's
execution as it operates.
This type of report should always be redirected to a virtual terminal,
not a file, as it may use control sequences that will make the output
unreadable in regular files.
.It xml
A report contained in a single XML file.
Ideal for later processing with
.Xr xsltproc 1
to generate nice HTML reports.
.El
.Sh EXAMPLES
The most simple way of running a test suite is to pipe the output of
.Nm atf-run
through
.Nm
without any additional flags.
This will use the default output format, which is suitable to most users:
.Bd -literal -offset indent
atf-run | atf-report
.Ed
.Pp
In some situations, it may be interesting to get a machine-parseable file
aside from the standard report.
This can be done as follows:
.Bd -literal -offset indent
atf-run | atf-report -o csv:testsuite.csv -o ticker:-
.Ed
.Pp
Or if the standard report is not desired, thus achieving completely silent
operation:
atf-run | atf-report -o csv:testsuite.csv
.Sh SEE ALSO
.Xr atf-run 1 ,
.Xr atf 7
|
