summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/docs/running-tests
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 19:33:14 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 19:33:14 +0000
commit36d22d82aa202bb199967e9512281e9a53db42c9 (patch)
tree105e8c98ddea1c1e4784a60a5a6410fa416be2de /testing/web-platform/tests/docs/running-tests
parentInitial commit. (diff)
downloadfirefox-esr-36d22d82aa202bb199967e9512281e9a53db42c9.tar.xz
firefox-esr-36d22d82aa202bb199967e9512281e9a53db42c9.zip
Adding upstream version 115.7.0esr.upstream/115.7.0esr
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'testing/web-platform/tests/docs/running-tests')
-rw-r--r--testing/web-platform/tests/docs/running-tests/android_webview.md51
-rw-r--r--testing/web-platform/tests/docs/running-tests/chrome-chromium-installation-detection.md96
-rw-r--r--testing/web-platform/tests/docs/running-tests/chrome.md33
-rw-r--r--testing/web-platform/tests/docs/running-tests/chrome_android.md22
-rw-r--r--testing/web-platform/tests/docs/running-tests/command-line-arguments.md14
-rw-r--r--testing/web-platform/tests/docs/running-tests/custom-runner.md21
-rw-r--r--testing/web-platform/tests/docs/running-tests/from-ci.md32
-rw-r--r--testing/web-platform/tests/docs/running-tests/from-local-system.md215
-rw-r--r--testing/web-platform/tests/docs/running-tests/from-web.md27
-rw-r--r--testing/web-platform/tests/docs/running-tests/index.md24
-rw-r--r--testing/web-platform/tests/docs/running-tests/safari.md47
-rw-r--r--testing/web-platform/tests/docs/running-tests/testing-polyfills.md67
-rw-r--r--testing/web-platform/tests/docs/running-tests/webkitgtk_minibrowser.md20
13 files changed, 669 insertions, 0 deletions
diff --git a/testing/web-platform/tests/docs/running-tests/android_webview.md b/testing/web-platform/tests/docs/running-tests/android_webview.md
new file mode 100644
index 0000000000..4a86814fdf
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/android_webview.md
@@ -0,0 +1,51 @@
+# Android WebView
+
+To run WPT on WebView on an Android device, some additional set-up is required.
+
+Currently, Android WebView support is experimental.
+
+## Prerequisites
+
+Please check [Chrome for Android](chrome_android.md) for the common instructions for Android support first.
+
+Ensure you have a userdebug or eng Android build installed on the device.
+
+Install an up-to-date version of system webview shell:
+1. Go to [chromium-browser-snapshots](https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Android/)
+2. Find the subdirectory with the highest number and click it, this number can be found
+ in the "Commit Position" column of row "LAST_CHANGE" (at bottom of page).
+3. Download `chrome-android.zip` file and unzip it.
+4. Install `SystemWebViewShell.apk`.
+5. On emulator, system webview shell may already be installed by default. Then you may need to remove the existing apk:
+ * Choose a userdebug build.
+ * Run an emulator with
+ [writable system partition from command line](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/android_emulator.md/)
+
+If you have an issue with ChromeDriver version mismatch, try one of the following:
+ * Try removing `_venv/bin/chromedriver` such that wpt runner can install a matching version
+ automatically. Failing that, please check your environment path and make
+ sure that no other ChromeDriver is used.
+ * Download the [ChromeDriver binary](https://chromedriver.chromium.org/) matching your WebView's major version and specify it on the command line
+ ```
+ ./wpt run --webdriver-binary <binary path> ...
+ ```
+
+Configure host remap rules in the [webview commandline file](https://cs.chromium.org/chromium/src/android_webview/docs/commandline-flags.md?l=57):
+```
+adb shell "echo '_ --host-resolver-rules=\"MAP nonexistent.*.test ~NOTFOUND, MAP *.test 127.0.0.1\"' > /data/local/tmp/webview-command-line"
+```
+
+Ensure that `adb` can be found on your system's PATH.
+
+## Running Tests
+
+Example command line:
+
+```bash
+./wpt run --test-type=testharness android_webview <TESTS>
+```
+
+* Note that there is no support for channel or automatic installation. The test
+ will be run against the current WebView version installed on the device.
+
+* Reftests are not supported at the moment.
diff --git a/testing/web-platform/tests/docs/running-tests/chrome-chromium-installation-detection.md b/testing/web-platform/tests/docs/running-tests/chrome-chromium-installation-detection.md
new file mode 100644
index 0000000000..63fcd8688d
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/chrome-chromium-installation-detection.md
@@ -0,0 +1,96 @@
+# Detection and Installation of Browser and WebDriver Binaries and for Chrome and Chromium
+
+This is a detailed description of the process in which WPT detects and installs the browser
+components for Chrome and Chromium. This process can seem convoluted and difficult to
+understand at first glance, but the reason for this process is to best ensure these components
+are compatible with each other and are the intended items that the user is trying to test.
+
+## Chrome
+
+### Detection
+**Browser**: Because WPT does not offer installation of Chrome browser binaries, it will
+not attempt to detect a Chrome browser binary in the virtual environment directory.
+Instead, commonly-used installation locations on various operating systems are checked to
+detect a valid Chrome binary. This detection process is only used if the user has not passed
+a binary path as an argument using the `--binary` flag.
+
+**WebDriver**: ChromeDriver detection for Chrome will only occur if a valid browser binary
+has been found. Once the browser binary version is detected, the virtual environment
+directory will be checked to see if a matching ChromeDriver version is already installed.
+If the browser and ChromeDriver versions do not match, the ChromeDriver binary will be
+removed from the directory and the user will be prompted to begin the webdriver installation
+process. A ChromeDriver version is considered matching the browser version if ChromeDriver shares
+the same major version, or next major version when testing Chrome Dev. For example, Chrome 98.x.x.x
+is considered to match ChromeDriver version 98.x.x.x, or also ChromeDriver 99.x.x.x if testing
+Chrome Dev.
+
+Note: Both Chrome and Chromium’s versions of ChromeDriver are stored in separate
+directories in the virtual environment directory i.e
+`_venv3/bin/{chrome|chromium}/{chromedriver}`. This safeguards from accidentally
+using Chromium’s ChromeDriver for a Chrome run and vice versa. Additionally, there
+is no need to reinstall ChromeDriver versions if switching between testing Chrome and Chromium.
+
+### Installation
+**Browser**: Browser binary installation is not provided through WPT and will throw a
+`NotImplementedError` if attempted via `./wpt install`. The user will need to
+have a browser binary on their system that can be detected or provide a path explicitly
+using the `--binary` flag.
+
+**WebDriver**: A version of ChromeDriver will only be installed once a Chrome browser binary
+has been given or detected. A `FileNotFoundError` will be raised if the user tries to download
+ChromeDriver via `./wpt install` and a browser binary is not located. After browser binary
+detection, a version of ChromeDriver that matches the browser binary will be installed.
+The download source for this ChromeDriver is
+[described here](https://chromedriver.chromium.org/downloads/version-selection).
+If a matching ChromeDriver version cannot be found using this process, it is assumed that
+the Chrome browser binary is a dev version which does not have a ChromeDriver version available
+through official releases. In this case, the Chromium revision associated with this version is
+detected from [OmahaProxy](https://omahaproxy.appspot.com/) and used to download
+Chromium's version of ChromeDriver for use from Chromium snapshots, as this is currently
+the closest version we can match for Chrome Dev. Finally, if the revision number detected is
+not available in Chromium snapshots, or if the version does not match any revision number,
+the latest revision of Chromium's ChromeDriver is installed from Chromium snapshots.
+
+## Chromium
+
+### Detection
+**Browser**: Chromium browser binary detection is only done in the virtual
+environment directory `_venv3/browsers/{channel}/`, not on the user’s system
+outside of this directory. This detection process is only used if the user has
+not passed a binary path as an argument using the `--binary` flag.
+
+**WebDriver**: ChromeDriver detection for Chromium will only occur if a valid browser binary has
+been found. Once the browser binary version is detected, the virtual environment directory will
+be checked to see if a matching ChromeDriver version is already installed. If the versions do not
+match, the ChromeDriver binary will be removed from the directory and the user will be prompted to
+begin the webdriver installation process. For Chromium, the ChromeDriver and browser versions must be
+the same to be considered matching. For example, Chromium 99.0.4844.74 will only match ChromeDriver
+99.0.4844.74.
+
+### Installation
+**Browser**: Chromium’s browser binary will be installed from
+[Chromium snapshots storage](https://storage.googleapis.com/chromium-browser-snapshots/index.html).
+The last revision associated with the user’s operating system will be downloaded
+(this revision is obtained by the LAST_CHANGE designation from the snapshots bucket).
+Chromium does not have varying channels, so the installation uses the default `nightly`
+designation. The install path is `_venv3/browsers/nightly/{chromium_binary}`.
+
+Note: If this download process is successful, the Chromium snapshot URL that the browser
+binary was downloaded from will be kept during the current invocation. If a Chromium ChromeDriver
+is also downloaded later to match this browser binary, the same URL is used for that download to
+ensure both components are downloaded from the same source.
+
+**WebDriver**: A version of ChromeDriver will only be installed once a Chromium browser binary
+has been given or detected. A FileNotFoundError will be raised if the user tries to download
+ChromeDriver via the install command and a browser binary is not located. A version of
+ChromeDriver that matches the version of the browser binary will be installed. The download
+source for this ChromeDriver will be the Chromium snapshots. If a Chromium browser
+binary and webdriver are installed in the same invocation of `./wpt run`
+(for example, by passing both `--install-browser` and `--install-webdriver` flags), then the
+browser binary and ChromeDriver will be pulled from the same Chromium snapshots URL (see Note
+from browser installation). Although unusual, if a Chromium browser binary is detected and
+it is not the tip-of-tree revision and the browser binary was not downloaded and installed
+during this invocation of `./wpt run` and the currently installed ChromeDriver version does
+not match the browser version, then an attempt will be made to detect the revision number from
+the browser binary version using the [OmahaProxy](https://omahaproxy.appspot.com/)
+and download the matching ChromeDriver using this revision number from Chromium snapshots.
diff --git a/testing/web-platform/tests/docs/running-tests/chrome.md b/testing/web-platform/tests/docs/running-tests/chrome.md
new file mode 100644
index 0000000000..45293af65e
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/chrome.md
@@ -0,0 +1,33 @@
+# Chrome and Chromium
+
+When running Chrome, there are some useful command line arguments.
+
+You can inform `wpt` of the release channel of Chrome using `--channel`.
+`wpt` is able to find the correct binary in the following cases:
+* On Linux for stable, beta and dev channels if
+ `google-chrome-{stable,beta,unstable}` are in `PATH`;
+* On Mac for stable and canary channels if the official DMGs are installed.
+
+In other cases, you will need to specify the path to the Chrome binary with
+`--binary`. For example:
+
+```bash
+./wpt run --channel dev --binary /path/to/non-default/google-chrome chrome
+```
+
+Note: when the channel is "dev", `wpt` will *automatically* enable all
+[experimental web platform features][1]
+(chrome://flags/#enable-experimental-web-platform-features) by passing
+`--enable-experimental-web-platform-features` to Chrome.
+
+If you want to enable a specific [runtime enabled feature][1], use
+`--binary-arg` to specify the flag(s) that you want to pass to Chrome:
+
+```bash
+./wpt run --binary-arg=--enable-blink-features=AsyncClipboard chrome clipboard-apis/
+```
+
+[A detailed explanation is available](running-tests/chrome-chromium-installation-detection.html)
+for more information on how wpt detects and installs the components for Chrome and Chromium.
+
+[1]: https://chromium.googlesource.com/chromium/src/+/main/third_party/blink/renderer/platform/RuntimeEnabledFeatures.md
diff --git a/testing/web-platform/tests/docs/running-tests/chrome_android.md b/testing/web-platform/tests/docs/running-tests/chrome_android.md
new file mode 100644
index 0000000000..a216a8a68b
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/chrome_android.md
@@ -0,0 +1,22 @@
+# Chrome for Android
+
+To run WPT on Chrome on an Android device, some additional set up is required.
+
+As with usual Android development, you need to have `adb` and be able to
+connect to the device. Run `adb devices` to verify.
+
+Currently, Android support is a prototype with some known issues:
+
+* If you have previously run `./wpt run` against Chrome, you might need to
+ remove `_venv/bin/chromedriver` so that we can install the correct
+ ChromeDriver corresponding to your Chrome for Android version.
+* We do not support reftests at the moment.
+* You will need to manually kill Chrome (all channels) before running tests.
+
+Note: rooting the device or installing a root CA is no longer required.
+
+Example (assuming you have Chrome Canary installed on your phone):
+
+```bash
+./wpt run --test-type=testharness --channel=canary chrome_android TESTS
+```
diff --git a/testing/web-platform/tests/docs/running-tests/command-line-arguments.md b/testing/web-platform/tests/docs/running-tests/command-line-arguments.md
new file mode 100644
index 0000000000..598c9da2a1
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/command-line-arguments.md
@@ -0,0 +1,14 @@
+# Command-Line Arguments
+
+The `wpt` command-line application offers a number of features for interacting
+with WPT. The functionality is organized into "sub-commands", and each accepts
+a different set of command-line arguments.
+
+This page documents all of the available sub-commands and associated arguments.
+
+```eval_rst
+.. argparse::
+ :module: tools.wpt.wpt
+ :func: create_complete_parser
+ :prog: wpt
+```
diff --git a/testing/web-platform/tests/docs/running-tests/custom-runner.md b/testing/web-platform/tests/docs/running-tests/custom-runner.md
new file mode 100644
index 0000000000..029a8771c1
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/custom-runner.md
@@ -0,0 +1,21 @@
+# Writing Your Own Runner
+
+Most test runners have two stages: finding all tests, followed by
+executing them (or a subset thereof).
+
+To find all tests in the repository, it is **strongly** recommended to
+use the included `wpt manifest` tool: the required behaviors are more
+complex than what are documented (especially when it comes to
+precedence of the various possibilities and some undocumented legacy
+ways to define test types), and hence its behavior should be
+considered the canonical definition of how to enumerate tests and find
+their type in the repository.
+
+For test execution, please read the documentation for the various test
+types very carefully and then check your understanding on the [mailing
+list][public-test-infra] or [matrix][matrix]. It's possible edge-case
+behavior isn't properly documented!
+
+[public-test-infra]: https://lists.w3.org/Archives/Public/public-test-infra/
+[matrix]: https://app.element.io/#/room/#wpt:matrix.org
+[web irc]: http://irc.w3.org
diff --git a/testing/web-platform/tests/docs/running-tests/from-ci.md b/testing/web-platform/tests/docs/running-tests/from-ci.md
new file mode 100644
index 0000000000..9ea142bb4b
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/from-ci.md
@@ -0,0 +1,32 @@
+# Running Tests on CI
+
+Contributors with write access to the repository can trigger full runs in the
+same CI systems used to produce results for [wpt.fyi](https://wpt.fyi). The runs
+are triggered by pushing to branch names on the form `triggers/$browser_$channel`
+and the results will be automatically submitted to wpt.fyi.
+
+This is useful when making infrastructure changes that could affect very many
+tests, in order to avoid regressions.
+
+Note: Full runs use a lot of CI resources, so please take care to not trigger
+them more than necessary.
+
+Instructions:
+
+ * Base your changes on a commit for which there are already results in wpt.fyi.
+
+ * Determine which branch name to push to by looking for `refs/heads/triggers/`
+ in `.azure-pipelines.yml` and `.taskcluster.yml`. For example, to trigger a
+ full run of Safari Technology Preview, the branch name is
+ `triggers/safari_preview`.
+
+ * Force push to the branch, for example:
+ `git push --force-with-lease origin HEAD:triggers/safari_preview`.
+ The `--force-with-lease` argument is to detect if someone else has just
+ pushed. When this happens wait for the checkout step of their triggered run
+ to finish before you force push again.
+
+You can see if the run started from the commit status on GitHub's commits listing
+([example](https://github.com/web-platform-tests/wpt/commits/triggers/safari_preview))
+and if successful the results will show up on wpt.fyi within 10 minutes
+([example](https://wpt.fyi/runs?product=safari)).
diff --git a/testing/web-platform/tests/docs/running-tests/from-local-system.md b/testing/web-platform/tests/docs/running-tests/from-local-system.md
new file mode 100644
index 0000000000..3865038ef6
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/from-local-system.md
@@ -0,0 +1,215 @@
+# Running Tests from the Local System
+
+The tests are designed to be run from your local computer.
+
+## System Setup
+
+Running the tests requires `python`, `pip` and `virtualenv`, as well as updating
+the system `hosts` file.
+
+WPT requires Python 3.7 or higher.
+
+The required setup is different depending on your operating system.
+
+### Linux Setup
+
+If not already present, use the system package manager to install `python`,
+`pip` and `virtualenv`.
+
+On Debian or Ubuntu:
+
+```bash
+sudo apt-get install python python-pip virtualenv
+```
+
+It is important to have a package that provides a `python` binary. On Fedora,
+for example, that means installing the `python-unversioned-command` package. On
+Ubuntu Focal and later, the package is called `python-is-python3`.
+
+### macOS Setup
+
+The system-provided Python can be used, while `pip` and `virtualenv` can be
+installed for the user only:
+
+```bash
+python -m ensurepip --user
+export PATH="$PATH:$( python3 -m site --user-base )/bin"
+pip install --user virtualenv
+```
+
+To make the `PATH` change persistent, add it to your `~/.bash_profile` file or
+wherever you currently set your PATH.
+
+See also [additional setup required to run Safari](safari.md).
+
+### Windows Setup
+
+Download and install [Python 3](https://www.python.org/downloads). The
+installer includes `pip` by default.
+
+Add `C:\Python39` and `C:\Python39\Scripts` to your `%Path%`
+[environment variable](http://www.computerhope.com/issues/ch000549.htm).
+
+Finally, install `virtualenv`:
+
+```bash
+pip install virtualenv
+```
+
+The standard Windows shell requires that all `wpt` commands are prefixed
+by the Python binary i.e. assuming `python` is on your path the server is
+started using:
+
+```bash
+python wpt serve
+```
+
+#### Windows Subsystem for Linux
+
+Optionally on Windows you can use the [Windows Subsystem for
+Linux](https://docs.microsoft.com/en-us/windows/wsl/about) (WSL). If doing so,
+installation and usage are similar to the Linux instructions. Be aware that WSL
+may attempt to override `/etc/hosts` each time it is launched, which would then
+require you to re-run [`hosts` File Setup](#hosts-file-setup). This behavior
+[can be configured](https://docs.microsoft.com/en-us/windows/wsl/wsl-config#network).
+
+### `hosts` File Setup
+
+To get the tests running, you need to set up the test domains in your
+[`hosts` file](http://en.wikipedia.org/wiki/Hosts_%28file%29%23Location_in_the_file_system).
+
+On Linux, macOS or other UNIX-like system:
+
+```bash
+./wpt make-hosts-file | sudo tee -a /etc/hosts
+```
+
+And on Windows (this must be run in a PowerShell session with Administrator privileges):
+
+```
+python wpt make-hosts-file | Out-File $env:SystemRoot\System32\drivers\etc\hosts -Encoding ascii -Append
+```
+
+If you are behind a proxy, you also need to make sure the domains above are
+excluded from your proxy lookups.
+
+## Via the browser
+
+The test environment can then be started using
+
+ ./wpt serve
+
+This will start HTTP servers on two ports and a websockets server on
+one port. By default the web servers start on ports 8000 and 8443 and the other
+ports are randomly-chosen free ports. Tests must be loaded from the
+*first* HTTP server in the output. To change the ports,
+create a `config.json` file in the wpt root directory, and add
+port definitions of your choice e.g.:
+
+```
+{
+ "ports": {
+ "http": [1234, "auto"],
+ "https":[5678]
+ }
+}
+```
+
+After your `hosts` file is configured, the servers will be locally accessible at:
+
+http://web-platform.test:8000/<br>
+https://web-platform.test:8443/ *
+
+To use the web-based runner point your browser to:
+
+http://web-platform.test:8000/tools/runner/index.html<br>
+https://web-platform.test:8443/tools/runner/index.html *
+
+This server has all the capabilities of the publicly-deployed version--see
+[Running the Tests from the Web](from-web.md).
+
+\**See [Trusting Root CA](../tools/certs/README.md)*
+
+## Via the command line
+
+Many tests can be automatically executed in a new browser instance using
+
+ ./wpt run [browsername] [tests]
+
+This will automatically load the tests in the chosen browser and extract the
+test results. For example to run the `dom/historical.html` tests in a local
+copy of Chrome:
+
+ ./wpt run chrome dom/historical.html
+
+Or to run in a specified copy of Firefox:
+
+ ./wpt run --binary ~/local/firefox/firefox firefox dom/historical.html
+
+For details on the supported products and a large number of other options for
+customising the test run:
+
+ ./wpt run --help
+
+[A complete listing of the command-line arguments is available
+here](command-line-arguments.html#run).
+
+```eval_rst
+.. toctree::
+ :hidden:
+
+ command-line-arguments
+```
+
+### Browser-specific instructions
+
+```eval_rst
+.. toctree::
+
+ chrome
+ chrome_android
+ android_webview
+ safari
+ webkitgtk_minibrowser
+```
+
+### Running in parallel
+
+To speed up the testing process, use the `--processes` option to run multiple
+browser instances in parallel. For example, to run the tests in dom/ with six
+Firefox instances in parallel:
+
+ ./wpt run --processes=6 firefox dom/
+
+But note that behaviour in this mode is necessarily less deterministic than with
+a single process (the default), so there may be more noise in the test results.
+
+### Output formats
+
+By default, `./wpt run` outputs test results and a summary in a human readable
+format. For debugging, `--log-mach` can give more verbose output. For example:
+
+ ./wpt run --log-mach=- --log-mach-level=info firefox dom/
+
+A machine readable JSON report can be produced using `--log-wptreport`. This
+together with `--log-wptscreenshot` is what is used to produce results for
+[wpt.fyi](https://wpt.fyi). For example:
+
+ ./wpt run --log-wptreport=report.json --log-wptscreenshot=screenshots.txt firefox css/css-grid/
+
+(See [wpt.fyi documentation](https://github.com/web-platform-tests/wpt.fyi/blob/main/api/README.md#results-creation)
+for how results are uploaded.)
+
+### Expectation data
+
+For use in continuous integration systems, and other scenarios where regression
+tracking is required, the command-line interface supports storing and loading
+the expected result of each test in a test run. See [Expectations
+Data](../../tools/wptrunner/docs/expectation) for more information on creating
+and maintaining these files.
+
+## Testing polyfills
+
+Polyfill scripts can be tested using the `--inject-script` argument to either
+`wpt run` or `wpt serve`. See [Testing Polyfills](testing-polyfills) for
+details.
diff --git a/testing/web-platform/tests/docs/running-tests/from-web.md b/testing/web-platform/tests/docs/running-tests/from-web.md
new file mode 100644
index 0000000000..157598da36
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/from-web.md
@@ -0,0 +1,27 @@
+# Running Tests from the Web
+
+Tests that have been merged on GitHub are mirrored at
+[wpt.live](https://wpt.live) and [w3c-test.org](https://w3c-test.org).
+[On properly-configured systems](from-local-system), local files may also be
+served from the URL [http://web-platform.test](http://web-platform.test).
+
+Not all tests can be executed in-browser, as some tests rely on automation
+(e.g. via [testdriver.js](../writing-tests/testdriver)) that is not available
+when running a browser in a normal user session.
+
+## Web test runner
+
+For running multiple tests inside a browser, there is a test runner
+located at `/tools/runner/index.html`.
+
+This allows all the tests, or those matching a specific prefix
+(e.g. all tests under `/dom/`) to be run. For testharness.js tests,
+the results will be automatically collected, while the runner
+provides a simple UI for manually comparing reftest rendering and
+running manual tests.
+
+Note, however, it does not currently handle more complex reftests with
+more than one reference involved.
+
+Because it runs entirely in-browser, this runner cannot deal with
+edge-cases like tests that cause the browser to crash or hang.
diff --git a/testing/web-platform/tests/docs/running-tests/index.md b/testing/web-platform/tests/docs/running-tests/index.md
new file mode 100644
index 0000000000..17b361dde8
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/index.md
@@ -0,0 +1,24 @@
+# Running Tests
+
+```eval_rst
+.. toctree::
+
+ from-web
+ from-local-system
+ from-ci
+ custom-runner
+ ../tools/certs/README.md
+```
+
+The simplest way to run the tests is via the public website. More detail on
+that approach is available in [Running tests from the Web](from-web).
+
+Contributors who are interested in modifying and creating tests should refer to
+[Running Tests from the Local System](from-local-system).
+
+Contributors with write access to the repository can also trigger full runs
+in our CI setups, see [Running Tests on CI](from-ci).
+
+Advanced use cases may call for a customized method of executing the tests.
+Guidelines for writing a custom "runner" are available at [Writing Your Own
+Runner](custom-runner).
diff --git a/testing/web-platform/tests/docs/running-tests/safari.md b/testing/web-platform/tests/docs/running-tests/safari.md
new file mode 100644
index 0000000000..eed5225471
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/safari.md
@@ -0,0 +1,47 @@
+# Safari
+
+To run Safari on macOS, some manual setup is required. Some steps are different
+for Safari and Safari Technology Preview.
+
+ * Allow Safari to be controlled by SafariDriver:
+ * `safaridriver --enable` or
+ * `"/Applications/Safari Technology Preview.app/Contents/MacOS/safaridriver" --enable`
+
+ * Allow pop-up windows:
+ * `defaults write com.apple.Safari WebKitJavaScriptCanOpenWindowsAutomatically 1` or
+ * `defaults write com.apple.SafariTechnologyPreview WebKitJavaScriptCanOpenWindowsAutomatically 1`
+
+ * Turn on additional experimental features Safari Technology Preview:
+ * `defaults write com.apple.SafariTechnologyPreview ExperimentalServerTimingEnabled 1`
+
+ * Trust the certificate:
+ * `security add-trusted-cert -k "$(security default-keychain | cut -d\" -f2)" tools/certs/cacert.pem`
+
+ * Set `no_proxy='*'` in your environment. This is a
+ workaround for a known
+ [macOS High Sierra issue](https://github.com/web-platform-tests/wpt/issues/9007).
+
+Now, run the tests using the `safari` product:
+```
+./wpt run safari [test_list]
+```
+
+This will use the `safaridriver` found on the path, which will be stable Safari.
+To run Safari Technology Preview instead, use the `--channel=preview` argument:
+```
+./wpt run --channel=preview safari [test_list]
+```
+
+## Debugging
+
+To debug problems with `safaridriver`, add the `--webdriver-arg=--diagnose`
+argument:
+```
+./wpt run --channel=preview --webdriver-arg=--diagnose safari [test_list]
+```
+
+The logs will be in `~/Library/Logs/com.apple.WebDriver/`.
+See `man safaridriver` for more information.
+
+To enable safaridriver diagnostics in Azure Pipelines, set
+`safaridriver_diagnose` to `true` in `.azure-pipelines.yml`.
diff --git a/testing/web-platform/tests/docs/running-tests/testing-polyfills.md b/testing/web-platform/tests/docs/running-tests/testing-polyfills.md
new file mode 100644
index 0000000000..468bfa2e03
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/testing-polyfills.md
@@ -0,0 +1,67 @@
+# Testing polyfills
+
+## Preparing the polyfill
+
+The polyfill script-injection feature currently only supports scripts which
+are immediately invoked. The script must be prepared as a single file whose
+contents will be inlined into a script tag served as part of every test page.
+
+If your polyfill is only available as an asynchronous module with dependent
+scripts, you can use a tool such as
+[microbundle](https://github.com/developit/microbundle) to repackage it as a
+single synchronous script file, e.g.:
+
+```bash
+microbundle -f iife -i polyfill/src/main.js -o polyfill.js
+```
+
+## Running the tests
+
+Follow the steps for [Running Tests from the Local System](from-local-system) to
+set up your test environment. When running tests via the browser or via the
+command line, add the `--inject-script=polyfill.js` to either command, e.g.
+
+Via the browser:
+
+```bash
+./wpt serve --inject-script=polyfill.js
+```
+
+Then visit http://web-platform.test:8000/ or https://web-platform.test:8443/ to
+run the tests in your browser.
+
+Via the command line:
+
+```bash
+./wpt run --inject-script=polyfill.js [browsername] [tests]
+```
+
+## Limitations
+
+Polyfill scripts are injected to an inline script tag which removes itself from
+the DOM after executing. This is done by modifying the server response for
+documents with a `text/html` MIME type to insert the following before the first tag in
+the served response:
+
+```html
+<script>
+// <-- The polyfill file is inlined here
+// Remove the injected script tag from the DOM.
+document.currentScript.remove();
+```
+
+This approach has a couple limitations:
+* This requires that the polyfill is self-contained and executes
+synchronously in a single inline script. See [Preparing the
+polyfill](#preparing-the-polyfill) for suggestions on transforming polyfills to
+run in that way.
+* Does not inject into python handlers which write directly to the output
+ stream.
+* Does not inject into the worker context of `.any.js` tests.
+
+### Observability
+
+The script tag is removed from the DOM before any other script has run, and runs
+from an inline script. As such, it should not affect mutation observers on the
+same page or resource timing APIs, as it is not a separate resource. The polyfill
+may be observable by a mutation observer added by a parent frame before load.
diff --git a/testing/web-platform/tests/docs/running-tests/webkitgtk_minibrowser.md b/testing/web-platform/tests/docs/running-tests/webkitgtk_minibrowser.md
new file mode 100644
index 0000000000..7aac81e5fc
--- /dev/null
+++ b/testing/web-platform/tests/docs/running-tests/webkitgtk_minibrowser.md
@@ -0,0 +1,20 @@
+# WebKitGTK MiniBrowser
+
+
+To be able to run tests with the WebKitGTK MiniBrowser you need the
+following packages installed:
+
+* Fedora: `webkit2gtk3-devel`
+* Debian or Ubuntu: `webkit2gtk-driver`
+
+
+The WebKitGTK MiniBrowser is not installed on the default binary path.
+The `wpt` script will try to automatically locate it, but if you need
+to run it manually you can find it on any of this paths:
+
+* Fedora: `/usr/libexec/webkit2gtk-4.0/MiniBrowser`
+* Debian or Ubuntu: `/usr/lib/x86_64-linux-gnu/webkit2gtk-4.0/MiniBrowser`
+ * Note: if the machine architecture is not `x86_64`, then it will be located
+ inside:
+ `/usr/lib/${TRIPLET}/webkit2gtk-4.0/MiniBrowser`
+ where `TRIPLET=$(gcc -dumpmachine)`