diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 09:22:09 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 09:22:09 +0000 |
commit | 43a97878ce14b72f0981164f87f2e35e14151312 (patch) | |
tree | 620249daf56c0258faa40cbdcf9cfba06de2a846 /testing/web-platform/tests/docs/running-tests/from-local-system.md | |
parent | Initial commit. (diff) | |
download | firefox-upstream.tar.xz firefox-upstream.zip |
Adding upstream version 110.0.1.upstream/110.0.1upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'testing/web-platform/tests/docs/running-tests/from-local-system.md')
-rw-r--r-- | testing/web-platform/tests/docs/running-tests/from-local-system.md | 215 |
1 files changed, 215 insertions, 0 deletions
diff --git a/testing/web-platform/tests/docs/running-tests/from-local-system.md b/testing/web-platform/tests/docs/running-tests/from-local-system.md new file mode 100644 index 0000000000..77f4799fb2 --- /dev/null +++ b/testing/web-platform/tests/docs/running-tests/from-local-system.md @@ -0,0 +1,215 @@ +# Running Tests from the Local System + +The tests are designed to be run from your local computer. + +## System Setup + +Running the tests requires `python`, `pip` and `virtualenv`, as well as updating +the system `hosts` file. + +WPT requires Python 3.6 or higher. + +The required setup is different depending on your operating system. + +### Linux Setup + +If not already present, use the system package manager to install `python`, +`pip` and `virtualenv`. + +On Debian or Ubuntu: + +```bash +sudo apt-get install python python-pip virtualenv +``` + +It is important to have a package that provides a `python` binary. On Fedora, +for example, that means installing the `python-unversioned-command` package. On +Ubuntu Focal and later, the package is called `python-is-python3`. + +### macOS Setup + +The system-provided Python can be used, while `pip` and `virtualenv` can be +installed for the user only: + +```bash +python -m ensurepip --user +export PATH="$PATH:$HOME/Library/Python/2.7/bin" +pip install --user virtualenv +``` + +To make the `PATH` change persistent, add it to your `~/.bash_profile` file or +wherever you currently set your PATH. + +See also [additional setup required to run Safari](safari.md). + +### Windows Setup + +Download and install [Python 3](https://www.python.org/downloads). The +installer includes `pip` by default. + +Add `C:\Python39` and `C:\Python39\Scripts` to your `%Path%` +[environment variable](http://www.computerhope.com/issues/ch000549.htm). + +Finally, install `virtualenv`: + +```bash +pip install virtualenv +``` + +The standard Windows shell requires that all `wpt` commands are prefixed +by the Python binary i.e. assuming `python` is on your path the server is +started using: + +```bash +python wpt serve +``` + +#### Windows Subsystem for Linux + +Optionally on Windows you can use the [Windows Subsystem for +Linux](https://docs.microsoft.com/en-us/windows/wsl/about) (WSL). If doing so, +installation and usage are similar to the Linux instructions. Be aware that WSL +may attempt to override `/etc/hosts` each time it is launched, which would then +require you to re-run [`hosts` File Setup](#hosts-file-setup). This behavior +[can be configured](https://docs.microsoft.com/en-us/windows/wsl/wsl-config#network). + +### `hosts` File Setup + +To get the tests running, you need to set up the test domains in your +[`hosts` file](http://en.wikipedia.org/wiki/Hosts_%28file%29%23Location_in_the_file_system). + +On Linux, macOS or other UNIX-like system: + +```bash +./wpt make-hosts-file | sudo tee -a /etc/hosts +``` + +And on Windows (this must be run in a PowerShell session with Administrator privileges): + +``` +python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append +``` + +If you are behind a proxy, you also need to make sure the domains above are +excluded from your proxy lookups. + +## Via the browser + +The test environment can then be started using + + ./wpt serve + +This will start HTTP servers on two ports and a websockets server on +one port. By default the web servers start on ports 8000 and 8443 and the other +ports are randomly-chosen free ports. Tests must be loaded from the +*first* HTTP server in the output. To change the ports, +create a `config.json` file in the wpt root directory, and add +port definitions of your choice e.g.: + +``` +{ + "ports": { + "http": [1234, "auto"], + "https":[5678] + } +} +``` + +After your `hosts` file is configured, the servers will be locally accessible at: + +http://web-platform.test:8000/<br> +https://web-platform.test:8443/ * + +To use the web-based runner point your browser to: + +http://web-platform.test:8000/tools/runner/index.html<br> +https://web-platform.test:8443/tools/runner/index.html * + +This server has all the capabilities of the publicly-deployed version--see +[Running the Tests from the Web](from-web.md). + +\**See [Trusting Root CA](../tools/certs/README.md)* + +## Via the command line + +Many tests can be automatically executed in a new browser instance using + + ./wpt run [browsername] [tests] + +This will automatically load the tests in the chosen browser and extract the +test results. For example to run the `dom/historical.html` tests in a local +copy of Chrome: + + ./wpt run chrome dom/historical.html + +Or to run in a specified copy of Firefox: + + ./wpt run --binary ~/local/firefox/firefox firefox dom/historical.html + +For details on the supported products and a large number of other options for +customising the test run: + + ./wpt run --help + +[A complete listing of the command-line arguments is available +here](command-line-arguments.html#run). + +```eval_rst +.. toctree:: + :hidden: + + command-line-arguments +``` + +### Browser-specific instructions + +```eval_rst +.. toctree:: + + chrome + chrome_android + android_webview + safari + webkitgtk_minibrowser +``` + +### Running in parallel + +To speed up the testing process, use the `--processes` option to run multiple +browser instances in parallel. For example, to run the tests in dom/ with six +Firefox instances in parallel: + + ./wpt run --processes=6 firefox dom/ + +But note that behaviour in this mode is necessarily less deterministic than with +a single process (the default), so there may be more noise in the test results. + +### Output formats + +By default, `./wpt run` outputs test results and a summary in a human readable +format. For debugging, `--log-mach` can give more verbose output. For example: + + ./wpt run --log-mach=- --log-mach-level=info firefox dom/ + +A machine readable JSON report can be produced using `--log-wptreport`. This +together with `--log-wptscreenshot` is what is used to produce results for +[wpt.fyi](https://wpt.fyi). For example: + + ./wpt run --log-wptreport=report.json --log-wptscreenshot=screenshots.txt firefox css/css-grid/ + +(See [wpt.fyi documentation](https://github.com/web-platform-tests/wpt.fyi/blob/main/api/README.md#results-creation) +for how results are uploaded.) + +### Expectation data + +For use in continuous integration systems, and other scenarios where regression +tracking is required, the command-line interface supports storing and loading +the expected result of each test in a test run. See [Expectations +Data](../../tools/wptrunner/docs/expectation) for more information on creating +and maintaining these files. + +## Testing polyfills + +Polyfill scripts can be tested using the `--inject-script` argument to either +`wpt run` or `wpt serve`. See [Testing Polyfills](testing-polyfills) for +details. |