diff options
Diffstat (limited to 'testing/web-platform/tests/resources/test/README.md')
-rw-r--r-- | testing/web-platform/tests/resources/test/README.md | 83 |
1 files changed, 83 insertions, 0 deletions
diff --git a/testing/web-platform/tests/resources/test/README.md b/testing/web-platform/tests/resources/test/README.md new file mode 100644 index 0000000000..edc03ef214 --- /dev/null +++ b/testing/web-platform/tests/resources/test/README.md @@ -0,0 +1,83 @@ +# `testharness.js` test suite + +The test suite for the `testharness.js` testing framework. + +## Executing Tests + +Install the following dependencies: + +- [Python 2.7.9+](https://www.python.org/) +- [the tox Python package](https://tox.readthedocs.io/en/latest/) +- [the Mozilla Firefox web browser](https://mozilla.org/firefox) +- [the GeckoDriver server](https://github.com/mozilla/geckodriver) + +Make sure `geckodriver` can be found in your `PATH`. + +Currently, the tests should be run with the latest *Firefox Nightly*. In order to +specify the path to Firefox Nightly, use the following command-line option: + + tox -- --binary=/path/to/FirefoxNightly + +### Automated Script + +Alternatively, you may run `tools/ci/ci_resources_unittest.sh`, which only depends on +Python 2. The script will install other dependencies automatically and start `tox` with +the correct arguments. + +## Authoring Tests + +Test cases are expressed as `.html` files located within the `tests/unit/` and +`tests/functional/` sub-directories. Each test should include the +`testharness.js` library with the following markup: + + <script src="/resources/testharness.js"></script> + <script src="/resources/testharnessreport.js"></script> + +This should be followed by one or more `<script>` tags that interface with the +`testharness.js` API in some way. For example: + + <script> + test(function() { + 1 = 1; + }, 'This test is expected to fail.'); + </script> + +### Unit tests + +The "unit test" type allows for concisely testing the expected behavior of +assertion methods. These tests may define any number of sub-tests; the +acceptance criteria is simply that all tests executed pass. + +### Functional tests + +Thoroughly testing the behavior of the harness itself requires ensuring a +number of considerations which cannot be verified with the "unit testing" +strategy. These include: + +- Ensuring that some tests are not run +- Ensuring conditions that cause test failures +- Ensuring conditions that cause harness errors + +Functional tests allow for these details to be verified. Every functional test +must include a summary of the expected results as a JSON string within a +`<script>` tag with an `id` of `"expected"`, e.g.: + + <script type="text/json" id="expected"> + { + "summarized_status": { + "message": null, + "stack": null, + "status_string": "OK" + }, + "summarized_tests": [ + { + "message": "ReferenceError: invalid assignment left-hand side", + "name": "Sample HTML5 API Tests", + "properties": {}, + "stack": "(implementation-defined)", + "status_string": "FAIL" + } + ], + "type": "complete" + } + </script> |