From 36d22d82aa202bb199967e9512281e9a53db42c9 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 7 Apr 2024 21:33:14 +0200 Subject: Adding upstream version 115.7.0esr. Signed-off-by: Daniel Baumann --- dom/webgpu/tests/cts/checkout/docs/intro/plans.md | 82 +++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 dom/webgpu/tests/cts/checkout/docs/intro/plans.md (limited to 'dom/webgpu/tests/cts/checkout/docs/intro/plans.md') diff --git a/dom/webgpu/tests/cts/checkout/docs/intro/plans.md b/dom/webgpu/tests/cts/checkout/docs/intro/plans.md new file mode 100644 index 0000000000..f8d7af3a78 --- /dev/null +++ b/dom/webgpu/tests/cts/checkout/docs/intro/plans.md @@ -0,0 +1,82 @@ +# Adding or Editing Test Plans + +## 1. Write a test plan + +For new tests, if some notes exist already, incorporate them into your plan. + +A detailed test plan should be written and reviewed before substantial test code is written. +This allows reviewers a chance to identify additional tests and cases, opportunities for +generalizations that would improve the strength of tests, similar existing tests or test plans, +and potentially useful [helpers](../helper_index.txt). + +**A test plan must serve two functions:** + +- Describes the test, succinctly, but in enough detail that a reader can read *only* the test + plans and evaluate coverage completeness of a file/directory. +- Describes the test precisely enough that, when code is added, the reviewer can ensure that the + test really covers what the test plan says. + +There should be one test plan for each test. It should describe what it tests, how, and describe +important cases that need to be covered. Here's an example: + +```ts +g.test('x,some_detail') + .desc( + ` +Tests [some detail] about x. Tests calling x in various 'mode's { mode1, mode2 }, +with various values of 'arg', and checks correctness of the result. +Tries to trigger [some conditional path]. + +- Valid values (control case) // <- (to make sure the test function works well) +- Unaligned values (should fail) // <- (only validation tests need to intentionally hit invalid cases) +- Extreme values` + ) + .params(u => + u // + .combine('mode', ['mode1', 'mode2']) + .beginSubcases() + .combine('arg', [ + // Valid // <- Comment params as you see fit. + 4, + 8, + 100, + // Invalid + 2, + 6, + 1e30, + ]) + ) + .unimplemented(); +``` + +"Cases" each appear as individual items in the `/standalone/` runner. +"Subcases" run inside each case, like a for-loop wrapping the `.fn(`test function`)`. +Documentation on the parameter builder can be found in the [helper index](../helper_index.txt). + +It's often impossible to predict the exact case/subcase structure before implementing tests, so they +can be added during implementation, instead of planning. + +For any notes which are not specific to a single test, or for preliminary notes for tests that +haven't been planned in full detail, put them in the test file's `description` variable at +the top. Or, if they aren't associated with a test file, put them in a `README.txt` file. + +**Any notes about missing test coverage must be marked with the word `TODO` inside a +description or README.** This makes them appear on the `/standalone/` page. + +## 2. Open a pull request + +Open a PR, and work with the reviewer(s) to revise the test plan. + +Usually (probably), plans will be landed in separate PRs before test implementations. + +## Conventions used in test plans + +- `Iff`: If and only if +- `x=`: "cartesian-cross equals", like `+=` for cartesian product. + Used for combinatorial test coverage. + - Sometimes this will result in too many test cases; simplify/reduce as needed + during planning *or* implementation. +- `{x,y,z}`: list of cases to test + - e.g. `x= texture format {r8unorm, r8snorm}` +- *Control case*: a case included to make sure that the rest of the cases aren't + missing their target by testing some other error case. -- cgit v1.2.3