diff options
Diffstat (limited to 'testing/web-platform/tests/docs/writing-tests/testharness.md')
-rw-r--r-- | testing/web-platform/tests/docs/writing-tests/testharness.md | 285 |
1 files changed, 285 insertions, 0 deletions
diff --git a/testing/web-platform/tests/docs/writing-tests/testharness.md b/testing/web-platform/tests/docs/writing-tests/testharness.md new file mode 100644 index 0000000000..fd4450f440 --- /dev/null +++ b/testing/web-platform/tests/docs/writing-tests/testharness.md @@ -0,0 +1,285 @@ +# JavaScript Tests (testharness.js) + +JavaScript tests are the correct type of test to write in any +situation where you are not specifically interested in the rendering +of a page, and where human interaction isn't required; these tests are +written in JavaScript using a framework called `testharness.js`. + +A high-level overview is provided below and more information can be found here: + + * [testharness.js Documentation](testharness-api.md) — An introduction + to the library and a detailed API reference. [The tutorial on writing a + testharness.js test](testharness-tutorial) provides a concise guide to writing + a test — a good place to start for newcomers to the project. + + * [testdriver.js Automation](testdriver.md) — Automating end user actions, such as moving or + clicking a mouse. See also the + [testdriver.js extension tutorial](testdriver-extension-tutorial.md) for adding new commands. + + * [idlharness.js](idlharness.md) — A library for testing + IDL interfaces using `testharness.js`. + + * [Message Channels](channels.md) - A way to communicate between + different globals, including window globals not in the same + browsing context group. + + * [Server features](server-features.md) - Advanced testing features + that are commonly used with JavaScript tests. + +See also the [general guidelines](general-guidelines.md) for all test types. + +## Window tests + +### Without HTML boilerplate (`.window.js`) + +Create a JavaScript file whose filename ends in `.window.js` to have the necessary HTML boilerplate +generated for you at `.window.html`. I.e., for `test.window.js` the server will ensure +`test.window.html` is available. + +In this JavaScript file you can place one or more tests, as follows: +```js +test(() => { + // Place assertions and logic here + assert_equals(document.characterSet, "UTF-8"); +}, "Ensure HTML boilerplate uses UTF-8"); // This is the title of the test +``` + +If you only need to test a [single thing](testharness-api.html#single-page-tests), you could also use: +```js +// META: title=Ensure HTML boilerplate uses UTF-8 +setup({ single_test: true }); +assert_equals(document.characterSet, "UTF-8"); +done(); +``` + +See [asynchronous (`async_test()`)](testharness-api.html#asynchronous-tests) and +[promise tests (`promise_test()`)](testharness-api.html#promise-tests) for more involved setups. + +### With HTML boilerplate + +You need to be a bit more explicit and include the `testharness.js` framework directly as well as an +additional file used by implementations: + +```html +<!doctype html> +<meta charset=utf-8> +<script src="/resources/testharness.js"></script> +<script src="/resources/testharnessreport.js"></script> +<body> + <script> + test(() => { + assert_equals(document.characterSet, "UTF-8"); + }, "Ensure UTF-8 declaration is observed"); + </script> +``` + +Here too you could avoid the wrapper `test()` function: + +```html +<!doctype html> +<meta charset=utf-8> +<title>Ensure UTF-8 declaration is observed</title> +<script src="/resources/testharness.js"></script> +<script src="/resources/testharnessreport.js"></script> +<body> + <script> + setup({ single_test: true }); + assert_equals(document.characterSet, "UTF-8"); + done(); + </script> +``` + +In this case the test title is taken from the `title` element. + +## Dedicated worker tests (`.worker.js`) + +Create a JavaScript file that imports `testharness.js` and whose filename ends in `.worker.js` to +have the necessary HTML boilerplate generated for you at `.worker.html`. + +For example, one could write a test for the `FileReaderSync` API by +creating a `FileAPI/FileReaderSync.worker.js` as follows: + +```js +importScripts("/resources/testharness.js"); +test(function () { + const blob = new Blob(["Hello"]); + const fr = new FileReaderSync(); + assert_equals(fr.readAsText(blob), "Hello"); +}, "FileReaderSync#readAsText."); +done(); +``` + +This test could then be run from `FileAPI/FileReaderSync.worker.html`. + +(Removing the need for `importScripts()` and `done()` is tracked in +[issue #11529](https://github.com/web-platform-tests/wpt/issues/11529).) + +## Tests for other or multiple globals (`.any.js`) + +Tests for features that exist in multiple global scopes can be written in a way +that they are automatically run in several scopes. In this case, the test is a +JavaScript file with extension `.any.js`, which can use all the usual APIs. + +By default, the test runs in a window scope and a dedicated worker scope. + +For example, one could write a test for the `Blob` constructor by +creating a `FileAPI/Blob-constructor.any.js` as follows: + +```js +test(function () { + const blob = new Blob(); + assert_equals(blob.size, 0); + assert_equals(blob.type, ""); + assert_false(blob.isClosed); +}, "The Blob constructor."); +``` + +This test could then be run from `FileAPI/Blob-constructor.any.worker.html` as well +as `FileAPI/Blob-constructor.any.html`. + +It is possible to customize the set of scopes with a metadata comment, such as + +``` +// META: global=sharedworker +// ==> would run in the shared worker scope +// META: global=window,serviceworker +// ==> would only run in the window and service worker scope +// META: global=dedicatedworker +// ==> would run in the default dedicated worker scope +// META: global=dedicatedworker-module +// ==> would run in the dedicated worker scope as a module +// META: global=worker +// ==> would run in the dedicated, shared, and service worker scopes +``` + +For a test file <code><var>x</var>.any.js</code>, the available scope keywords +are: + +* `window` (default): to be run at <code><var>x</var>.any.html</code> +* `dedicatedworker` (default): to be run at <code><var>x</var>.any.worker.html</code> +* `dedicatedworker-module` to be run at <code><var>x</var>.any.worker-module.html</code> +* `serviceworker`: to be run at <code><var>x</var>.any.serviceworker.html</code> (`.https` is + implied) +* `serviceworker-module`: to be run at <code><var>x</var>.any.serviceworker-module.html</code> + (`.https` is implied) +* `sharedworker`: to be run at <code><var>x</var>.any.sharedworker.html</code> +* `sharedworker-module`: to be run at <code><var>x</var>.any.sharedworker-module.html</code> +* `jsshell`: to be run in a JavaScript shell, without access to the DOM + (currently only supported in SpiderMonkey, and skipped in wptrunner) +* `worker`: shorthand for the dedicated, shared, and service worker scopes +* `shadowrealm`: runs the test code in a + [ShadowRealm](https://github.com/tc39/proposal-shadowrealm) context hosted in + an ordinary Window context; to be run at <code><var>x</var>.any.shadowrealm.html</code> + +To check if your test is run from a window or worker you can use the following two methods that will +be made available by the framework: + + self.GLOBAL.isWindow() + self.GLOBAL.isWorker() + +Although [the global `done()` function must be explicitly invoked for most +dedicated worker tests and shared worker +tests](testharness-api.html#determining-when-all-tests-are-complete), it is +automatically invoked for tests defined using the "multi-global" pattern. + +## Other features of `.window.js`, `.worker.js` and `.any.js` + +### Specifying a test title + +Use `// META: title=This is the title of the test` at the beginning of the resource. + +### Including other JavaScript files + +Use `// META: script=link/to/resource.js` at the beginning of the resource. For example, + +``` +// META: script=/common/utils.js +// META: script=resources/utils.js +``` + +can be used to include both the global and a local `utils.js` in a test. + +In window environments, the script will be included using a classic `<script>` tag. In classic +worker environments, the script will be imported using `importScripts()`. In module worker +environments, the script will be imported using a static `import`. + +### Specifying a timeout of long + +Use `// META: timeout=long` at the beginning of the resource. + +### Specifying test [variants](#variants) + +Use `// META: variant=url-suffix` at the beginning of the resource. For example, + +``` +// META: variant= +// META: variant=?wss +``` + +## Variants + +A test file can have multiple variants by including `meta` elements, +for example: + +```html +<meta name="variant" content=""> +<meta name="variant" content="?wss"> +``` + +Test runners will execute the test for each variant specified, appending the corresponding content +attribute value to the URL of the test as they do so. + +`/common/subset-tests.js` and `/common/subset-tests-by-key.js` are two utility scripts that work +well together with variants, allowing a test to be split up into subtests in cases when there are +otherwise too many tests to complete inside the timeout. For example: + +```html +<!doctype html> +<title>Testing variants</title> +<meta name="variant" content="?1-1000"> +<meta name="variant" content="?1001-2000"> +<meta name="variant" content="?2001-last"> +<script src="/resources/testharness.js"></script> +<script src="/resources/testharnessreport.js"></script> +<script src="/common/subset-tests.js"></script> +<script> + const tests = [ + { fn: t => { ... }, name: "..." }, + ... lots of tests ... + ]; + for (const test of tests) { + subsetTest(async_test, test.fn, test.name); + } +</script> +``` + +With `subsetTestByKey`, the key is given as the first argument, and the +query string can include or exclude a key (which will be matched as a regular +expression). + +```html +<!doctype html> +<title>Testing variants by key</title> +<meta name="variant" content="?include=Foo"> +<meta name="variant" content="?include=Bar"> +<meta name="variant" content="?exclude=(Foo|Bar)"> +<script src="/resources/testharness.js"></script> +<script src="/resources/testharnessreport.js"></script> +<script src="/common/subset-tests-by-key.js"></script> +<script> + subsetTestByKey("Foo", async_test, () => { ... }, "Testing foo"); + ... +</script> +``` + +## Table of Contents + +```eval_rst +.. toctree:: + :maxdepth: 1 + + testharness-api + testdriver + testdriver-extension-tutorial + idlharness +``` |