summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/resources/test/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'testing/web-platform/tests/resources/test/README.md')
-rw-r--r--testing/web-platform/tests/resources/test/README.md83
1 files changed, 83 insertions, 0 deletions
diff --git a/testing/web-platform/tests/resources/test/README.md b/testing/web-platform/tests/resources/test/README.md
new file mode 100644
index 0000000000..edc03ef214
--- /dev/null
+++ b/testing/web-platform/tests/resources/test/README.md
@@ -0,0 +1,83 @@
+# `testharness.js` test suite
+
+The test suite for the `testharness.js` testing framework.
+
+## Executing Tests
+
+Install the following dependencies:
+
+- [Python 2.7.9+](https://www.python.org/)
+- [the tox Python package](https://tox.readthedocs.io/en/latest/)
+- [the Mozilla Firefox web browser](https://mozilla.org/firefox)
+- [the GeckoDriver server](https://github.com/mozilla/geckodriver)
+
+Make sure `geckodriver` can be found in your `PATH`.
+
+Currently, the tests should be run with the latest *Firefox Nightly*. In order to
+specify the path to Firefox Nightly, use the following command-line option:
+
+ tox -- --binary=/path/to/FirefoxNightly
+
+### Automated Script
+
+Alternatively, you may run `tools/ci/ci_resources_unittest.sh`, which only depends on
+Python 2. The script will install other dependencies automatically and start `tox` with
+the correct arguments.
+
+## Authoring Tests
+
+Test cases are expressed as `.html` files located within the `tests/unit/` and
+`tests/functional/` sub-directories. Each test should include the
+`testharness.js` library with the following markup:
+
+ <script src="/resources/testharness.js"></script>
+ <script src="/resources/testharnessreport.js"></script>
+
+This should be followed by one or more `<script>` tags that interface with the
+`testharness.js` API in some way. For example:
+
+ <script>
+ test(function() {
+ 1 = 1;
+ }, 'This test is expected to fail.');
+ </script>
+
+### Unit tests
+
+The "unit test" type allows for concisely testing the expected behavior of
+assertion methods. These tests may define any number of sub-tests; the
+acceptance criteria is simply that all tests executed pass.
+
+### Functional tests
+
+Thoroughly testing the behavior of the harness itself requires ensuring a
+number of considerations which cannot be verified with the "unit testing"
+strategy. These include:
+
+- Ensuring that some tests are not run
+- Ensuring conditions that cause test failures
+- Ensuring conditions that cause harness errors
+
+Functional tests allow for these details to be verified. Every functional test
+must include a summary of the expected results as a JSON string within a
+`<script>` tag with an `id` of `"expected"`, e.g.:
+
+ <script type="text/json" id="expected">
+ {
+ "summarized_status": {
+ "message": null,
+ "stack": null,
+ "status_string": "OK"
+ },
+ "summarized_tests": [
+ {
+ "message": "ReferenceError: invalid assignment left-hand side",
+ "name": "Sample HTML5 API Tests",
+ "properties": {},
+ "stack": "(implementation-defined)",
+ "status_string": "FAIL"
+ }
+ ],
+ "type": "complete"
+ }
+ </script>