Adding upstream version 110.0.1.upstream/110.0.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 230 insertions, 0 deletions

diff --git a/testing/web-platform/tests/docs/writing-tests/general-guidelines.md b/testing/web-platform/tests/docs/writing-tests/general-guidelines.md
new file mode 100644
index 0000000000..1689c064a3
--- /dev/null
+++ b/testing/web-platform/tests/docs/writing-tests/general-guidelines.md
@@ -0,0 +1,230 @@
+# General Test Guidelines
+
+### File Paths and Names
+
+When choosing where in the directory structure to put any new tests,
+try to follow the structure of existing tests for that specification;
+if there are no existing tests, it is generally recommended to create
+subdirectories for each section.
+
+Due to path length limitations on Windows, test paths must be less
+that 150 characters relative to the test root directory (this gives
+vendors just over 100 characters for their own paths when running in
+automation).
+
+File names should generally be somewhat descriptive of what is being
+tested; very generic names like `001.html` are discouraged. A common
+format is `test-topic-001.html`, where `test-topic` is a short
+identifier that describes the test. It should avoid conjunctions,
+articles, and prepositions as it should be as concise as possible. The
+integer that follows is normally just increased incrementally, and
+padded to three digits. (If you'd end up with more than 999 tests,
+your `test-topic` is probably too broad!)
+
+The test filename is significant in enabling specific optional features, such as HTTPS
+or server-side substitution. See the documentation on [file names flags][file-name-flags]
+for more details.
+
+In the css directory, the file names should be unique within the whole
+css/ directory, regardless of where they are in the directory structure.
+
+### HTTPS
+
+By default, tests are served over plain HTTP. If a test requires HTTPS
+it must be given a filename containing `.https.` e.g.,
+`test-secure.https.html`, or be the generated service worker test of a
+`.https`-less `.any` test. For more details see the documentation on
+[file names][file-name-flags].
+
+### HTTP2
+
+If a test must be served from an HTTP/2 server, it must be given a
+filename containing `.h2`.
+
+#### Support Files
+
+Various support files are available in in the directories named `/common/`,
+`/media/`, and `/css/support/`. Reusing existing resources is encouraged where
+possible, as is adding generally-useful files to these common areas rather than
+to specific test suites.
+
+
+#### Tools
+
+Sometimes you may want to add a script to the repository that's meant
+to be used from the command line, not from a browser (e.g., a script
+for generating test files). If you want to ensure (e.g., for security
+reasons) that such scripts will only be usable from the command line
+but won't be handled by the HTTP server then place them in a `tools`
+subdirectory at the appropriate level—the server will then return a
+404 if they are requested.
+
+For example, if you wanted to add a script for use with tests in the
+`notifications` directory, create the `notifications/tools`
+subdirectory and put your script there.
+
+
+### File Formats
+
+Tests are generally formatted as HTML (including XHTML) or XML (including SVG).
+Some test types support other formats:
+
+- [testharness.js tests](testharness) may be expressed as JavaScript files
+  ([the WPT server automatically generates the HTML documents for these][server
+  features])
+- [WebDriver specification tests](wdspec) are expressed as Python files
+
+The best way to determine how to format a new test is to look at how
+similar tests have been formatted. You can also ask for advice in [the
+project's matrix channel][matrix].
+
+
+### Character Encoding
+
+Except when specifically testing encoding, files must be encoded in
+UTF-8. In file formats where UTF-8 is not the default encoding, they
+must contain metadata to mark them as such (e.g., `<meta
+charset=utf-8>` in HTML files) or be pure ASCII.
+
+
+### Server Side Support
+
+The custom web server
+supports [a variety of features][server features] useful for testing
+browsers, including (but not limited to!) support for writing out
+appropriate domains and custom (per-file and per-directory) HTTP
+headers.
+
+
+### Be Short
+
+Tests should be as short as possible. For reftests in particular
+scrollbars at 800&#xD7;600px window size must be avoided unless scrolling
+behavior is specifically being tested. For all tests extraneous
+elements on the page should be avoided so it is clear what is part of
+the test (for a typical testharness test, the only content on the page
+will be rendered by the harness itself).
+
+
+### Be Conservative
+
+Tests should generally avoid depending on edge case behavior of
+features that they don't explicitly intend on testing. For example,
+except where testing parsing, tests should contain
+no [parse errors](https://validator.nu).
+
+This is not, however, to discourage testing of edge cases or
+interactions between multiple features; such tests are an essential
+part of ensuring interoperability of the web platform. When possible, use the
+canonical support libraries provided by features; for more information, see the documentation on [testing interactions between features][interacting-features].
+
+Tests should pass when the feature under test exposes the expected behavior,
+and they should fail when the feature under test is not implemented or is
+implemented incorrectly. Tests should not rely on unrelated features if doing
+so causes failures in the latest stable release of [Apple
+Safari][apple-safari], [Google Chrome][google-chrome], or [Mozilla
+Firefox][mozilla-firefox]. They should, therefore, not rely on any features
+aside from the one under test unless they are supported in all three browsers.
+
+Existing tests can be used as a guide to identify acceptable features. For
+language features that are not used in existing tests, community-maintained
+projects such as [the ECMAScript compatibility tables][es-compat] and
+[caniuse.com][caniuse] provide an overview of basic feature support across the
+browsers listed above.
+
+For JavaScript code that is re-used across many tests (e.g. `testharness.js`
+and the files located in the directory named `common`), only use language
+features that have been supported by each of the major browser engines above
+for over a year. This practice avoids introducing test failures for consumers
+maintaining older JavaScript runtimes.
+
+Patches to make tests run on older versions or other browsers will be accepted
+provided they are relatively simple and do not add undue complexity to the
+test.
+
+
+### Be Cross-Platform
+
+Tests should be as cross-platform as reasonably possible, working
+across different devices, screen resolutions, paper sizes, etc. The
+assumptions that can be relied on are documented [here][assumptions];
+tests that rely on anything else should be manual tests that document
+their assumptions.
+
+Fonts cannot be relied on to be either installed or to have specific
+metrics. As such, in most cases when a known font is needed, [Ahem][ahem]
+should be used and loaded as a web font. In other cases, `@font-face`
+should be used.
+
+
+### Be Self-Contained
+
+Tests must not depend on external network resources. When these tests
+are run on CI systems, they are typically configured with access to
+external resources disabled, so tests that try to access them will
+fail. Where tests want to use multiple hosts, this is possible through
+a known set of subdomains and the [text substitution features of
+wptserve](server-features).
+
+
+### Be Self-Describing
+
+Tests should make it obvious when they pass and when they fail. It
+shouldn't be necessary to consult the specification to figure out
+whether a test has passed of failed.
+
+
+### Style Rules
+
+A number of style rules should be applied to the test file. These are
+not uniformly enforced throughout the existing tests, but will be for
+new tests. Any of these rules may be broken if the test demands it:
+
+ * No trailing whitespace
+ * Use spaces rather than tabs for indentation
+ * Use UNIX-style line endings (i.e. no CR characters at EOL)
+
+We have a lint tool for catching these and other common mistakes. You
+can run it manually by starting the `wpt` executable from the root of
+your local web-platform-tests working directory, and invoking the
+`lint` subcommand, like this:
+
+```
+./wpt lint
+```
+
+The lint tool is also run automatically for every submitted pull request,
+and reviewers will not merge branches with tests that have lint errors, so
+you must fix any errors the lint tool reports. For details on doing that,
+see the [lint-tool documentation][lint-tool].
+
+But in the unusual case of error reports for things essential to a certain
+test or that for other exceptional reasons shouldn't prevent a merge of a
+test, update and commit the `lint.ignore` file in the web-platform-tests
+root directory to suppress the error reports. For details on doing that,
+see the [lint-tool documentation][lint-tool].
+
+
+## CSS-Specific Requirements
+
+In order to be included in an official specification test suite, tests
+for CSS have some additional requirements for:
+
+* [Metadata][css-metadata], and
+* [User style sheets][css-user-styles].
+
+
+[server features]: server-features
+[assumptions]: assumptions
+[ahem]: ahem
+[matrix]: https://app.element.io/#/room/#wpt:matrix.org
+[lint-tool]: lint-tool
+[css-metadata]: css-metadata
+[css-user-styles]: css-user-styles
+[file-name-flags]: file-names
+[interacting-features]: interacting-features
+[mozilla-firefox]: https://mozilla.org/firefox
+[google-chrome]: https://google.com/chrome/browser/desktop/
+[apple-safari]: https://apple.com/safari
+[es-compat]: https://kangax.github.io/compat-table/
+[caniuse]: https://caniuse.com/