1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
|
# Running Tests from the Local System
The tests are designed to be run from your local computer.
## System Setup
Running the tests requires `python`, `pip` and `virtualenv`, as well as updating
the system `hosts` file.
WPT requires Python 3.6 or higher.
The required setup is different depending on your operating system.
### Linux Setup
If not already present, use the system package manager to install `python`,
`pip` and `virtualenv`.
On Debian or Ubuntu:
```bash
sudo apt-get install python python-pip virtualenv
```
It is important to have a package that provides a `python` binary. On Fedora,
for example, that means installing the `python-unversioned-command` package. On
Ubuntu Focal and later, the package is called `python-is-python3`.
### macOS Setup
The system-provided Python can be used, while `pip` and `virtualenv` can be
installed for the user only:
```bash
python -m ensurepip --user
export PATH="$PATH:$HOME/Library/Python/2.7/bin"
pip install --user virtualenv
```
To make the `PATH` change persistent, add it to your `~/.bash_profile` file or
wherever you currently set your PATH.
See also [additional setup required to run Safari](safari.md).
### Windows Setup
Download and install [Python 3](https://www.python.org/downloads). The
installer includes `pip` by default.
Add `C:\Python39` and `C:\Python39\Scripts` to your `%Path%`
[environment variable](http://www.computerhope.com/issues/ch000549.htm).
Finally, install `virtualenv`:
```bash
pip install virtualenv
```
The standard Windows shell requires that all `wpt` commands are prefixed
by the Python binary i.e. assuming `python` is on your path the server is
started using:
```bash
python wpt serve
```
#### Windows Subsystem for Linux
Optionally on Windows you can use the [Windows Subsystem for
Linux](https://docs.microsoft.com/en-us/windows/wsl/about) (WSL). If doing so,
installation and usage are similar to the Linux instructions. Be aware that WSL
may attempt to override `/etc/hosts` each time it is launched, which would then
require you to re-run [`hosts` File Setup](#hosts-file-setup). This behavior
[can be configured](https://docs.microsoft.com/en-us/windows/wsl/wsl-config#network).
### `hosts` File Setup
To get the tests running, you need to set up the test domains in your
[`hosts` file](http://en.wikipedia.org/wiki/Hosts_%28file%29%23Location_in_the_file_system).
On Linux, macOS or other UNIX-like system:
```bash
./wpt make-hosts-file | sudo tee -a /etc/hosts
```
And on Windows (this must be run in a PowerShell session with Administrator privileges):
```
python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
```
If you are behind a proxy, you also need to make sure the domains above are
excluded from your proxy lookups.
## Via the browser
The test environment can then be started using
./wpt serve
This will start HTTP servers on two ports and a websockets server on
one port. By default the web servers start on ports 8000 and 8443 and the other
ports are randomly-chosen free ports. Tests must be loaded from the
*first* HTTP server in the output. To change the ports,
create a `config.json` file in the wpt root directory, and add
port definitions of your choice e.g.:
```
{
"ports": {
"http": [1234, "auto"],
"https":[5678]
}
}
```
After your `hosts` file is configured, the servers will be locally accessible at:
http://web-platform.test:8000/<br>
https://web-platform.test:8443/ *
To use the web-based runner point your browser to:
http://web-platform.test:8000/tools/runner/index.html<br>
https://web-platform.test:8443/tools/runner/index.html *
This server has all the capabilities of the publicly-deployed version--see
[Running the Tests from the Web](from-web.md).
\**See [Trusting Root CA](../tools/certs/README.md)*
## Via the command line
Many tests can be automatically executed in a new browser instance using
./wpt run [browsername] [tests]
This will automatically load the tests in the chosen browser and extract the
test results. For example to run the `dom/historical.html` tests in a local
copy of Chrome:
./wpt run chrome dom/historical.html
Or to run in a specified copy of Firefox:
./wpt run --binary ~/local/firefox/firefox firefox dom/historical.html
For details on the supported products and a large number of other options for
customising the test run:
./wpt run --help
[A complete listing of the command-line arguments is available
here](command-line-arguments.html#run).
```eval_rst
.. toctree::
:hidden:
command-line-arguments
```
### Browser-specific instructions
```eval_rst
.. toctree::
chrome
chrome_android
android_webview
safari
webkitgtk_minibrowser
```
### Running in parallel
To speed up the testing process, use the `--processes` option to run multiple
browser instances in parallel. For example, to run the tests in dom/ with six
Firefox instances in parallel:
./wpt run --processes=6 firefox dom/
But note that behaviour in this mode is necessarily less deterministic than with
a single process (the default), so there may be more noise in the test results.
### Output formats
By default, `./wpt run` outputs test results and a summary in a human readable
format. For debugging, `--log-mach` can give more verbose output. For example:
./wpt run --log-mach=- --log-mach-level=info firefox dom/
A machine readable JSON report can be produced using `--log-wptreport`. This
together with `--log-wptscreenshot` is what is used to produce results for
[wpt.fyi](https://wpt.fyi). For example:
./wpt run --log-wptreport=report.json --log-wptscreenshot=screenshots.txt firefox css/css-grid/
(See [wpt.fyi documentation](https://github.com/web-platform-tests/wpt.fyi/blob/main/api/README.md#results-creation)
for how results are uploaded.)
### Expectation data
For use in continuous integration systems, and other scenarios where regression
tracking is required, the command-line interface supports storing and loading
the expected result of each test in a test run. See [Expectations
Data](../../tools/wptrunner/docs/expectation) for more information on creating
and maintaining these files.
## Testing polyfills
Polyfill scripts can be tested using the `--inject-script` argument to either
`wpt run` or `wpt serve`. See [Testing Polyfills](testing-polyfills) for
details.
|
