diff options
Diffstat (limited to 'testing/web-platform/tests/common/security-features')
45 files changed, 4283 insertions, 0 deletions
diff --git a/testing/web-platform/tests/common/security-features/README.md b/testing/web-platform/tests/common/security-features/README.md new file mode 100644 index 0000000000..f957541f75 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/README.md @@ -0,0 +1,460 @@ +This directory contains the common infrastructure for the following tests (also referred below as projects). + +- referrer-policy/ +- mixed-content/ +- upgrade-insecure-requests/ + +Subdirectories: + +- `resources`: + Serves JavaScript test helpers. +- `subresource`: + Serves subresources, with support for redirects, stash, etc. + The subresource paths are managed by `subresourceMap` and + fetched in `requestVia*()` functions in `resources/common.js`. +- `scope`: + Serves nested contexts, such as iframe documents or workers. + Used from `invokeFrom*()` functions in `resources/common.js`. +- `tools`: + Scripts that generate test HTML files. Not used while running tests. +- `/referrer-policy/generic/subresource-test`: + Sanity checking tests for subresource invocation + (This is still placed outside common/) + +# Test generator + +The test generator ([common/security-features/tools/generate.py](tools/generate.py)) generates test HTML files from templates and a seed (`spec.src.json`) that defines all the test scenarios. + +The project (i.e. a WPT subdirectory, for example `referrer-policy/`) that uses the generator should define per-project data and invoke the common generator logic in `common/security-features/tools`. + +This is the overview of the project structure: + +``` +common/security-features/ +└── tools/ - the common test generator logic + ├── spec.src.json + └── template/ - the test files templates +project-directory/ (e.g. referrer-policy/) +├── spec.src.json +├── generic/ +│ ├── test-case.sub.js - Per-project test helper +│ ├── sanity-checker.js (Used by debug target only) +│ └── spec_json.js (Used by debug target only) +└── gen/ - generated tests +``` + +## Generating the tests + +Note: When the repository already contains generated tests, [remove all generated tests](#removing-all-generated-tests) first. + +```bash +# Install json5 module if needed. +pip install --user json5 + +# Generate the test files under gen/ (HTMLs and .headers files). +path/to/common/security-features/tools/generate.py --spec path/to/project-directory/ + +# Add all generated tests to the repo. +git add path/to/project-directory/gen/ && git commit -m "Add generated tests" +``` + +This will parse the spec JSON5 files and determine which tests to generate (or skip) while using templates. + +- The default spec JSON5: `common/security-features/tools/spec.src.json`. + - Describes common configurations, such as subresource types, source context types, etc. +- The per-project spec JSON5: `project-directory/spec.src.json`. + - Describes project-specific configurations, particularly those related to test generation patterns (`specification`), policy deliveries (e.g. `delivery_type`, `delivery_value`) and `expectation`. + +For how these two spec JSON5 files are merged, see [Sub projects](#sub-projects) section. + +Note: `spec.src.json` is transitioning to JSON5 [#21710](https://github.com/web-platform-tests/wpt/issues/21710). + +During the generation, the spec is validated by ```common/security-features/tools/spec_validator.py```. This is specially important when you're making changes to `spec.src.json`. Make sure it's a valid JSON (no comments or trailing commas). The validator reports specific errors (missing keys etc.), if any. + +### Removing all generated tests + +Simply remove all files under `project-directory/gen/`. + +```bash +rm -r path/to/project-directory/gen/ +``` + +### Options for generating tests + +Note: this section is currently obsolete. Only the release template is working. + +The generator script has two targets: ```release``` and ```debug```. + +* Using **release** for the target will produce tests using a template for optimizing size and performance. The release template is intended for the official web-platform-tests and possibly other test suites. No sanity checking is done in release mode. Use this option whenever you're checking into web-platform-tests. + +* When generating for ```debug```, the produced tests will contain more verbosity and sanity checks. Use this target to identify problems with the test suites when making changes locally. Make sure you don't check in tests generated with the debug target. + +Note that **release** is the default target when invoking ```generate.py```. + + +## Sub projects + +Projects can be nested, for example to reuse a single `spec.src.json` across similar but slightly different sets of generated tests. +The directory structure would look like: + +``` +project-directory/ (e.g. referrer-policy/) +├── spec.src.json - Parent project's spec JSON +├── generic/ +│ └── test-case.sub.js - Parent project's test helper +├── gen/ - parent project's generated tests +└── sub-project-directory/ (e.g. 4K) + ├── spec.src.json - Child project's spec JSON + ├── generic/ + │ └── test-case.sub.js - Child project's test helper + └── gen/ - child project's generated tests +``` + +`generate.py --spec project-directory/sub-project-directory` generates test files under `project-directory/sub-project-directory/gen`, based on `project-directory/spec.src.json` and `project-directory/sub-project-directory/spec.src.json`. + +- The child project's `spec.src.json` is merged into parent project's `spec.src.json`. + - Two spec JSON objects are merged recursively. + - If a same key exists in both objects, the child's value overwrites the parent's value. + - If both (child's and parent's) values are arrays, then the child's value is concatenated to the parent's value. + - For debugging, `generate.py` dumps the merged spec JSON object as `generic/debug-output.spec.src.json`. +- The child project's generated tests include both of the parent and child project's `test-case.sub.js`: + ```html + <script src="project-directory/test-case.sub.js"></script> + <script src="project-directory/sub-project-directory/test-case.sub.js"></script> + <script> + TestCase(...); + </script> + ``` + + +## Updating the tests + +The main test logic lives in ```project-directory/generic/test-case.sub.js``` with helper functions defined in ```/common/security-features/resources/common.js``` so you should probably start there. + +For updating the test suites you will most likely do **a subset** of the following: + +* Add a new subresource type: + + * Add a new sub-resource python script to `/common/security-features/subresource/`. + * Add a sanity check test for a sub-resource to `referrer-policy/generic/subresource-test/`. + * Add a new entry to `subresourceMap` in `/common/security-features/resources/common.js`. + * Add a new entry to `valid_subresource_names` in `/common/security-features/tools/spec_validator.py`. + * Add a new entry to `subresource_schema` in `spec.src.json`. + * Update `source_context_schema` to specify in which source context the subresource can be used. + +* Add a new subresource redirection type + + * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18939](https://github.com/web-platform-tests/wpt/pull/18939) + +* Add a new subresource origin type + + * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18940](https://github.com/web-platform-tests/wpt/pull/18940) + +* Add a new source context (e.g. "module sharedworker global scope") + + * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18904](https://github.com/web-platform-tests/wpt/pull/18904) + +* Add a new source context list (e.g. "subresource request from a dedicated worker in a `<iframe srcdoc>`") + + * TODO: to be documented. + +* Implement new or update existing assertions in ```project-directory/generic/test-case.sub.js```. + +* Exclude or add some tests by updating ```spec.src.json``` test expansions. + +* Implement a new delivery method. + + * TODO: to be documented. Currently the support for delivery methods are implemented in many places across `common/security-features/`. + +* Regenerate the tests and MANIFEST.json + +## How the generator works + +This section describes how `spec.src.json` is turned into scenario data in test HTML files which are then processed by JavaScript test helpers and server-side scripts, and describes the objects/types used in the process. + +### The spec JSON + +`spec.src.json` is the input for the generator that defines what to generate. For examples of spec JSON files, see [referrer-policy/spec.src.json](../../referrer-policy/spec.src.json) or [mixed-content/spec.src.json](../../mixed-content/spec.src.json). + +#### Main sections + +* **`specification`** + + Top level requirements with description fields and a ```test_expansion``` rule. + This is closely mimicking the [Referrer Policy specification](http://w3c.github.io/webappsec/specs/referrer-policy/) structure. + +* **`excluded_tests`** + + List of ```test_expansion``` patterns expanding into selections which get skipped when generating the tests (aka. blocklisting/suppressing) + +* **`test_expansion_schema`** + + Provides valid values for each field. + Each test expansion can only contain fields and values defined by this schema (or `"*"` values that indicate all the valid values defined this schema). + +* **`subresource_schema`** + + Provides metadata of subresources, e.g. supported delivery types for each subresource. + +* **`source_context_schema`** + + Provides metadata of each single source context, e.g. supported delivery types and subresources that can be sent from the context. + +* **`source_context_list_schema`** + + Provides possible nested combinations of source contexts. See [SourceContexts Resolution](#sourcecontexts-resolution) section below for details. + +### Test Expansion Pattern Object + +Test expansion patterns (`test_expansion`s in `specification` section) define the combinations of test configurations (*selections*) to be generated. +Each field in a test expansion can be in one of the following formats: + +* Single match: ```"value"``` + +* Match any of: ```["value1", "value2", ...]``` + +* Match all: ```"*"``` + +The following fields have special meaning: + +- **`name`**: just ignored. (Previously this was used as a part of filenames but now this is merely a label for human and is never used by generator. This field might be removed in the future (https://github.com/web-platform-tests/wpt/issues/21708)) +- **`expansion`**: if there is more than one pattern expanding into a same selection, the pattern appearing later in the spec JSON will overwrite a previously generated selection. To make clear this is intentional, set the value of the ```expansion``` field to ```default``` for an expansion appearing earlier and ```override``` for the one appearing later. + +For example a test expansion pattern (taken from [referrer-policy/spec.src.json](../../referrer-policy/spec.src.json), sorted/formatted for explanation): + +```json +{ + "name": "insecure-protocol", + "expansion": "default", + + "delivery_type": "*", + "delivery_value": "no-referrer-when-downgrade", + "source_context_list": "*", + + "expectation": "stripped-referrer", + "origin": ["same-http", "cross-http"], + "redirection": "*", + "source_scheme": "http", + "subresource": "*" +} +``` + +means: "All combinations with all possible `delivery_type`, `delivery_value`=`no-referrer-when-downgrade`, all possible `source_context_list`, `expectation`=`stripped-referrer`, `origin`=`same-http` or `cross-http`, all possible `redirection`, `source_scheme`=`http`, and all possible `subresource`. + +### Selection Object + +A selection is an object that defines a single test, with keys/values from `test_expansion_schema`. + +A single test expansion pattern gets expanded into a list of selections as follows: + +* Expand each field's pattern (single, any of, or all) to list of allowed values (defined by the ```test_expansion_schema```) + +* Permute - Recursively enumerate all selections across all fields + +The following field has special meaning: + +- **`delivery_key`**: This doesn't exist in test expansion patterns, and instead is taken from `delivery_key` field of the spec JSON and added into selections. (TODO(https://github.com/web-platform-tests/wpt/issues/21708): probably this should be added to test expansion patterns to remove this special handling) + +For example, the test expansion in the example above generates selections like the following selection (which eventually generates [this test file](../../referrer-policy/gen/worker-classic.http-rp/no-referrer-when-downgrade/fetch/same-http.no-redirect.http.html )): + +```json +{ + "delivery_type": "http-rp", + "delivery_key": "referrerPolicy", + "delivery_value": "no-referrer-when-downgrade", + "source_context_list": "worker-classic", + + "expectation": "stripped-referrer", + "origin": "same-http", + "redirection": "no-redirect", + "source_scheme": "http", + "subresource": "fetch" +} +``` + +### Excluding Test Expansion Patterns + +The ```excluded_tests``` section have objects with the same format as [Test Expansion Patterns](#test-expansion-patterns) that define selections to be excluded. + +Taking the spec JSON, the generator follows this algorithm: + +* Expand all ```excluded_tests``` to create a denylist of selections + +* For each `specification` entries: Expand the ```test_expansion``` pattern into selections and check each against the denylist, if not marked as suppresed, generate the test resources for the selection + +### SourceContext Resolution + +The `source_context_list_schema` section of `spec.src.json` defines templates of policy deliveries and source contexts. +The `source_context_list` value in a selection specifies the key of the template to be used in `source_context_list_schema`, and the following fields in the selection are filled into the template (these three values define the **target policy delivery** to be tested): + +- `delivery_type` +- `delivery_key` +- `delivery_value` + +#### Source Context List Schema + +Each entry of **`source_context_list_schema`**, defines a single template of how/what policies to be delivered in what source contexts (See also [PolicyDelivery](types.md#policydelivery) and [SourceContext](types.md#sourcecontext)). + +- The key: the name of the template which matches with the `source_context_list` value in a selection. +- `sourceContextList`: an array of `SourceContext` objects that represents a (possibly nested) context. + - `sourceContextType` of the first entry of `sourceContextList` should be always `"top"`, which represents the top-level generated test HTML. This entry is omitted in the scenario JSON object passed to JavaScript runtime, but the policy deliveries specified here are handled by the generator, e.g. written as `<meta>` elements in the generated test HTML. +- `subresourcePolicyDeliveries`: an array of `PolicyDelivery` objects that represents policies specified at subresource requests (e.g. `referrerPolicy` attribute of `<img>` elements). + +#### PolicyDelivery placeholders + +Instead to ordinal `PolicyDelivery` objects, the following placeholder strings can be used in `sourceContextList` or `subresourcePolicyDeliveries`. + +- `"policy"`: + - Replaced with the target policy delivery. +- `"policyIfNonNull"`: + - Replaced with the target policy delivery, only if `delivery_value` is not `null`. + If `delivery_value` is `null`, then the test is not generated. +- `"anotherPolicy"`: + - Replaced with a `PolicyDelivery` object that has a different value from + the target policy delivery. + - Can be used to specify e.g. a policy that should be overridden by + the target policy delivery. + +#### `source_context_schema` and `subresource_schema` + +These represent supported delivery types and subresources +for each source context or subresource type. These are used + +- To filter out test files for unsupported combinations of delivery types, + source contexts and subresources during SourceContext resolution. +- To determine what delivery types can be used for `anotherPolicy` + placeholder. + +#### Example + +For example, the following entry in `source_context_list_schema`: + +```json +"worker-classic": { + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "worker-classic", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] +} +``` + +Defines a template to be instantiated with `delivery_key`, `delivery_type` and `delivery_value` values defined outside `source_context_list_schema`, which reads: + +- A classic `WorkerGlobalScope` is created under the top-level Document, and has a policy defined by `delivery_key`, `delivery_type` and `delivery_value`. +- The top-level Document has a policy different from the policy given to the classic worker (to confirm that the policy of the classic worker, not of the top-level Document, is used). +- The subresource request is sent from the classic `WorkerGlobalScope`, with no additional policies specified at the subresource request. + +And when filled with the following values from a selection: + +- `delivery_type`: `"http-rp"` +- `delivery_key`: `"referrerPolicy"` +- `delivery_value`: `"no-referrer-when-downgrade"` + +This becomes: + +```json +"worker-classic": { + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + { + "deliveryType": "meta", + "key": "referrerPolicy", + "value": "no-referrer" + } + ] + }, + { + "sourceContextType": "worker-classic", + "policyDeliveries": [ + { + "deliveryType": "http-rp", + "key": "referrerPolicy", + "value": "no-referrer-when-downgrade" + } + ] + } + ], + "subresourcePolicyDeliveries": [] +} +``` + +which means + +- The top-level Document has `<meta name="referrer" content="no-referrer">`. +- The classic worker is created with + `Referrer-Policy: no-referrer-when-downgrade` HTTP response headers. + +### Scenario Object + +The **scenario** object is the JSON object written to the generated HTML files, and passed to JavaScript test runtime (as an argument of `TestCase`). +A scenario object is an selection object, minus the keys used in [SourceContext Resolution](#sourceContext-resolution): + +- `source_context_list` +- `delivery_type` +- `delivery_key` +- `delivery_value` + +plus the keys instantiated by [SourceContext Resolution](#sourceContext-resolution): + +- `source_context_list`, except for the first `"top"` entry. +- `subresource_policy_deliveries` + +For example: + +```json +{ + "source_context_list": [ + { + "sourceContextType": "worker-classic", + "policyDeliveries": [ + { + "deliveryType": "http-rp", + "key": "referrerPolicy", + "value": "no-referrer-when-downgrade" + } + ] + } + ], + "subresource_policy_deliveries": [], + + "expectation": "stripped-referrer", + "origin": "same-http", + "redirection": "no-redirect", + "source_scheme": "http", + "subresource": "fetch" +} +``` + +### TopLevelPolicyDelivery Object + +The ***TopLevelPolicyDelivery** object is the first `"top"` entry of `SourceContextList` instantiated by [SourceContext Resolution](#sourceContext-resolution), which represents the policy delivery of the top-level HTML Document. + +The generator generates `<meta>` elements and `.headers` files of the top-level HTML files from the TopLevelPolicyDelivery object. + +This is handled separately by the generator from other parts of selection objects and scenario objects, because the `<meta>` and `.headers` are hard-coded directly to the files in the WPT repository, while policies of subcontexts are generated via server-side `common/security-features/scope` scripts. + +TODO(https://github.com/web-platform-tests/wpt/issues/21710): Currently the name `TopLevelPolicyDelivery` doesn't appear in the code. + +## How the test runtime works + +All the information needed at runtime is contained in an scenario object. See the code/comments of the following files. + +- `project-directory/generic/test-case.js` defines `TestCase`, the entry point that receives a scenario object. `resources/common.sub.js` does the most of common JavaScript work. + - Subresource URLs (which point to `subresource/` scripts) are calculated from `origin` and `redirection` values. + - Initiating fetch requests based on `subresource` and `subresource_policy_deliveries`. +- `scope/` server-side scripts serve non-toplevel contexts, while the top-level Document is generated by the generator. + TODO(https://github.com/web-platform-tests/wpt/issues/21709): Merge the logics of `scope/` and the generator. +- `subresource/` server-side scripts serve subresource responses. diff --git a/testing/web-platform/tests/common/security-features/__init__.py b/testing/web-platform/tests/common/security-features/__init__.py new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/__init__.py diff --git a/testing/web-platform/tests/common/security-features/resources/common.sub.js b/testing/web-platform/tests/common/security-features/resources/common.sub.js new file mode 100644 index 0000000000..96ca280597 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/resources/common.sub.js @@ -0,0 +1,1311 @@ +/** + * @fileoverview Utilities for mixed-content in web-platform-tests. + * @author burnik@google.com (Kristijan Burnik) + * Disclaimer: Some methods of other authors are annotated in the corresponding + * method's JSDoc. + */ + +// =============================================================== +// Types +// =============================================================== +// Objects of the following types are used to represent what kind of +// subresource requests should be sent with what kind of policies, +// from what kind of possibly nested source contexts. +// The objects are represented as JSON objects (not JavaScript/Python classes +// in a strict sense) to be passed between JavaScript/Python code. +// +// See also common/security-features/Types.md for high-level description. + +/** + @typedef PolicyDelivery + @type {object} + Referrer policy etc. can be applied/delivered in several ways. + A PolicyDelivery object specifies what policy is delivered and how. + + @property {string} deliveryType + Specifies how the policy is delivered. + The valid deliveryType are: + + "attr" + [A] DOM attributes e.g. referrerPolicy. + + "rel-noref" + [A] <link rel="noreferrer"> (referrer-policy only). + + "http-rp" + [B] HTTP response headers. + + "meta" + [B] <meta> elements. + + @property {string} key + @property {string} value + Specifies what policy to be delivered. The valid keys are: + + "referrerPolicy" + Referrer Policy + https://w3c.github.io/webappsec-referrer-policy/ + Valid values are those listed in + https://w3c.github.io/webappsec-referrer-policy/#referrer-policy + (except that "" is represented as null/None) + + A PolicyDelivery can be specified in several ways: + + - (for [A]) Associated with an individual subresource request and + specified in `Subresource.policies`, + e.g. referrerPolicy attributes of DOM elements. + This is handled in invokeRequest(). + + - (for [B]) Associated with an nested environmental settings object and + specified in `SourceContext.policies`, + e.g. HTTP referrer-policy response headers of HTML/worker scripts. + This is handled in server-side under /common/security-features/scope/. + + - (for [B]) Associated with the top-level HTML document. + This is handled by the generators.d +*/ + +/** + @typedef Subresource + @type {object} + A Subresource represents how a subresource request is sent. + + @property{SubresourceType} subresourceType + How the subresource request is sent, + e.g. "img-tag" for sending a request via <img src>. + See the keys of `subresourceMap` for valid values. + + @property{string} url + subresource's URL. + Typically this is constructed by getRequestURLs() below. + + @property{PolicyDelivery} policyDeliveries + Policies delivered specific to the subresource request. +*/ + +/** + @typedef SourceContext + @type {object} + + @property {string} sourceContextType + Kind of the source context to be used. + Valid values are the keys of `sourceContextMap` below. + + @property {Array<PolicyDelivery>} policyDeliveries + A list of PolicyDelivery applied to the source context. +*/ + +// =============================================================== +// General utility functions +// =============================================================== + +function timeoutPromise(t, ms) { + return new Promise(resolve => { t.step_timeout(resolve, ms); }); +} + +/** + * Normalizes the target port for use in a URL. For default ports, this is the + * empty string (omitted port), otherwise it's a colon followed by the port + * number. Ports 80, 443 and an empty string are regarded as default ports. + * @param {number} targetPort The port to use + * @return {string} The port portion for using as part of a URL. + */ +function getNormalizedPort(targetPort) { + return ([80, 443, ""].indexOf(targetPort) >= 0) ? "" : ":" + targetPort; +} + +/** + * Creates a GUID. + * See: https://en.wikipedia.org/wiki/Globally_unique_identifier + * Original author: broofa (http://www.broofa.com/) + * Sourced from: http://stackoverflow.com/a/2117523/4949715 + * @return {string} A pseudo-random GUID. + */ +function guid() { + return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) { + var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8); + return v.toString(16); + }); +} + +/** + * Initiates a new XHR via GET. + * @param {string} url The endpoint URL for the XHR. + * @param {string} responseType Optional - how should the response be parsed. + * Default is "json". + * See: https://xhr.spec.whatwg.org/#dom-xmlhttprequest-responsetype + * @return {Promise} A promise wrapping the success and error events. + */ +function xhrRequest(url, responseType) { + return new Promise(function(resolve, reject) { + var xhr = new XMLHttpRequest(); + xhr.open('GET', url, true); + xhr.responseType = responseType || "json"; + + xhr.addEventListener("error", function() { + reject(Error("Network Error")); + }); + + xhr.addEventListener("load", function() { + if (xhr.status != 200) + reject(Error(xhr.statusText)); + else + resolve(xhr.response); + }); + + xhr.send(); + }); +} + +/** + * Sets attributes on a given DOM element. + * @param {DOMElement} element The element on which to set the attributes. + * @param {object} An object with keys (serving as attribute names) and values. + */ +function setAttributes(el, attrs) { + attrs = attrs || {} + for (var attr in attrs) { + if (attr !== 'src') + el.setAttribute(attr, attrs[attr]); + } + // Workaround for Chromium: set <img>'s src attribute after all other + // attributes to ensure the policy is applied. + for (var attr in attrs) { + if (attr === 'src') + el.setAttribute(attr, attrs[attr]); + } +} + +/** + * Binds to success and error events of an object wrapping them into a promise + * available through {@code element.eventPromise}. The success event + * resolves and error event rejects. + * This method adds event listeners, and then removes all the added listeners + * when one of listened event is fired. + * @param {object} element An object supporting events on which to bind the + * promise. + * @param {string} resolveEventName [="load"] The event name to bind resolve to. + * @param {string} rejectEventName [="error"] The event name to bind reject to. + */ +function bindEvents(element, resolveEventName, rejectEventName) { + element.eventPromise = + bindEvents2(element, resolveEventName, element, rejectEventName); +} + +// Returns a promise wrapping success and error events of objects. +// This is a variant of bindEvents that can accept separate objects for each +// events and two events to reject, and doesn't set `eventPromise`. +// +// When `resolveObject`'s `resolveEventName` event (default: "load") is +// fired, the promise is resolved with the event. +// +// When `rejectObject`'s `rejectEventName` event (default: "error") or +// `rejectObject2`'s `rejectEventName2` event (default: "error") is +// fired, the promise is rejected. +// +// `rejectObject2` is optional. +function bindEvents2(resolveObject, resolveEventName, rejectObject, rejectEventName, rejectObject2, rejectEventName2) { + return new Promise(function(resolve, reject) { + const actualResolveEventName = resolveEventName || "load"; + const actualRejectEventName = rejectEventName || "error"; + const actualRejectEventName2 = rejectEventName2 || "error"; + + const resolveHandler = function(event) { + cleanup(); + resolve(event); + }; + + const rejectHandler = function(event) { + // Chromium starts propagating errors from worker.onerror to + // window.onerror. This handles the uncaught exceptions in tests. + event.preventDefault(); + cleanup(); + reject(event); + }; + + const cleanup = function() { + resolveObject.removeEventListener(actualResolveEventName, resolveHandler); + rejectObject.removeEventListener(actualRejectEventName, rejectHandler); + if (rejectObject2) { + rejectObject2.removeEventListener(actualRejectEventName2, rejectHandler); + } + }; + + resolveObject.addEventListener(actualResolveEventName, resolveHandler); + rejectObject.addEventListener(actualRejectEventName, rejectHandler); + if (rejectObject2) { + rejectObject2.addEventListener(actualRejectEventName2, rejectHandler); + } + }); +} + +/** + * Creates a new DOM element. + * @param {string} tagName The type of the DOM element. + * @param {object} attrs A JSON with attributes to apply to the element. + * @param {DOMElement} parent Optional - an existing DOM element to append to + * If not provided, the returned element will remain orphaned. + * @param {boolean} doBindEvents Optional - Whether to bind to load and error + * events and provide the promise wrapping the events via the element's + * {@code eventPromise} property. Default value evaluates to false. + * @return {DOMElement} The newly created DOM element. + */ +function createElement(tagName, attrs, parentNode, doBindEvents) { + var element = document.createElement(tagName); + + if (doBindEvents) { + bindEvents(element); + if (element.tagName == "IFRAME" && !('srcdoc' in attrs || 'src' in attrs)) { + // If we're loading a frame, ensure we spin the event loop after load to + // paper over the different event timing in Gecko vs Blink/WebKit + // see https://github.com/whatwg/html/issues/4965 + element.eventPromise = element.eventPromise.then(() => { + return new Promise(resolve => setTimeout(resolve, 0)) + }); + } + } + // We set the attributes after binding to events to catch any + // event-triggering attribute changes. E.g. form submission. + // + // But be careful with images: unlike other elements they will start the load + // as soon as the attr is set, even if not in the document yet, and sometimes + // complete it synchronously, so the append doesn't have the effect we want. + // So for images, we want to set the attrs after appending, whereas for other + // elements we want to do it before appending. + var isImg = (tagName == "img"); + if (!isImg) + setAttributes(element, attrs); + + if (parentNode) + parentNode.appendChild(element); + + if (isImg) + setAttributes(element, attrs); + + return element; +} + +function createRequestViaElement(tagName, attrs, parentNode) { + return createElement(tagName, attrs, parentNode, true).eventPromise; +} + +function wrapResult(server_data) { + if (typeof(server_data) === "string") { + throw server_data; + } + return { + referrer: server_data.headers.referer, + headers: server_data.headers + } +} + +// =============================================================== +// Subresources +// =============================================================== + +/** + @typedef RequestResult + @type {object} + Represents the result of sending an request. + All properties are optional. See the comments for + requestVia*() and invokeRequest() below to see which properties are set. + + @property {Array<Object<string, string>>} headers + HTTP request headers sent to server. + @property {string} referrer - Referrer. + @property {string} location - The URL of the subresource. + @property {string} sourceContextUrl + the URL of the global object where the actual request is sent. +*/ + +/** + requestVia*(url, additionalAttributes) functions send a subresource + request from the current environment settings object. + + @param {string} url + The URL of the subresource. + @param {Object<string, string>} additionalAttributes + Additional attributes set to DOM elements + (element-initiated requests only). + + @returns {Promise} that are resolved with a RequestResult object + on successful requests. + + - Category 1: + `headers`: set. + `referrer`: set via `document.referrer`. + `location`: set via `document.location`. + See `template/document.html.template`. + - Category 2: + `headers`: set. + `referrer`: set to `headers.referer` by `wrapResult()`. + `location`: not set. + - Category 3: + All the keys listed above are NOT set. + `sourceContextUrl` is not set here. + + -------------------------------- -------- -------------------------- + Function name Category Used in + -------- ------- --------- + referrer mixed- upgrade- + policy content insecure- + policy content request + -------------------------------- -------- -------- ------- --------- + requestViaAnchor 1 Y Y - + requestViaArea 1 Y Y - + requestViaAudio 3 - Y - + requestViaDedicatedWorker 2 Y Y Y + requestViaFetch 2 Y Y - + requestViaForm 2 - Y - + requestViaIframe 1 Y Y - + requestViaImage 2 Y Y - + requestViaLinkPrefetch 3 - Y - + requestViaLinkStylesheet 3 - Y - + requestViaObject 3 - Y - + requestViaPicture 3 - Y - + requestViaScript 2 Y Y - + requestViaSendBeacon 3 - Y - + requestViaSharedWorker 2 Y Y Y + requestViaVideo 3 - Y - + requestViaWebSocket 3 - Y - + requestViaWorklet 3 - Y Y + requestViaXhr 2 Y Y - + -------------------------------- -------- -------- ------- --------- +*/ + +/** + * Creates a new iframe, binds load and error events, sets the src attribute and + * appends it to {@code document.body} . + * @param {string} url The src for the iframe. + * @return {Promise} The promise for success/error events. + */ +function requestViaIframe(url, additionalAttributes) { + const iframe = createElement( + "iframe", + Object.assign({"src": url}, additionalAttributes), + document.body, + false); + return bindEvents2(window, "message", iframe, "error", window, "error") + .then(event => { + if (event.source !== iframe.contentWindow) + return Promise.reject(new Error('Unexpected event.source')); + return event.data; + }); +} + +/** + * Creates a new image, binds load and error events, sets the src attribute and + * appends it to {@code document.body} . + * @param {string} url The src for the image. + * @return {Promise} The promise for success/error events. + */ +function requestViaImage(url, additionalAttributes) { + const img = createElement( + "img", + // crossOrigin attribute is added to read the pixel data of the response. + Object.assign({"src": url, "crossOrigin": "Anonymous"}, additionalAttributes), + document.body, true); + return img.eventPromise.then(() => wrapResult(decodeImageData(img))); +} + +// Helper for requestViaImage(). +function decodeImageData(img) { + var canvas = document.createElement("canvas"); + var context = canvas.getContext('2d'); + context.drawImage(img, 0, 0); + var imgData = context.getImageData(0, 0, img.clientWidth, img.clientHeight); + const rgba = imgData.data; + + let decodedBytes = new Uint8ClampedArray(rgba.length); + let decodedLength = 0; + + for (var i = 0; i + 12 <= rgba.length; i += 12) { + // A single byte is encoded in three pixels. 8 pixel octets (among + // 9 octets = 3 pixels * 3 channels) are used to encode 8 bits, + // the most significant bit first, where `0` and `255` in pixel values + // represent `0` and `1` in bits, respectively. + // This encoding is used to avoid errors due to different color spaces. + const bits = []; + for (let j = 0; j < 3; ++j) { + bits.push(rgba[i + j * 4 + 0]); + bits.push(rgba[i + j * 4 + 1]); + bits.push(rgba[i + j * 4 + 2]); + // rgba[i + j * 4 + 3]: Skip alpha channel. + } + // The last one element is not used. + bits.pop(); + + // Decode a single byte. + let byte = 0; + for (let j = 0; j < 8; ++j) { + byte <<= 1; + if (bits[j] >= 128) + byte |= 1; + } + + // Zero is the string terminator. + if (byte == 0) + break; + + decodedBytes[decodedLength++] = byte; + } + + // Remove trailing nulls from data. + decodedBytes = decodedBytes.subarray(0, decodedLength); + var string_data = (new TextDecoder("ascii")).decode(decodedBytes); + + return JSON.parse(string_data); +} + +/** + * Initiates a new XHR GET request to provided URL. + * @param {string} url The endpoint URL for the XHR. + * @return {Promise} The promise for success/error events. + */ +function requestViaXhr(url) { + return xhrRequest(url).then(result => wrapResult(result)); +} + +/** + * Initiates a new GET request to provided URL via the Fetch API. + * @param {string} url The endpoint URL for the Fetch. + * @return {Promise} The promise for success/error events. + */ +function requestViaFetch(url) { + return fetch(url) + .then(res => res.json()) + .then(j => wrapResult(j)); +} + +function dedicatedWorkerUrlThatFetches(url) { + return `data:text/javascript, + fetch('${url}') + .then(r => r.json()) + .then(j => postMessage(j)) + .catch((e) => postMessage(e.message));`; +} + +function workerUrlThatImports(url, additionalAttributes) { + let csp = ""; + if (additionalAttributes && additionalAttributes.contentSecurityPolicy) { + csp=`&contentSecurityPolicy=${additionalAttributes.contentSecurityPolicy}`; + } + return `/common/security-features/subresource/static-import.py` + + `?import_url=${encodeURIComponent(url)}${csp}`; +} + +function workerDataUrlThatImports(url) { + return `data:text/javascript,import '${url}';`; +} + +/** + * Creates a new Worker, binds message and error events wrapping them into. + * {@code worker.eventPromise} and posts an empty string message to start + * the worker. + * @param {string} url The endpoint URL for the worker script. + * @param {object} options The options for Worker constructor. + * @return {Promise} The promise for success/error events. + */ +function requestViaDedicatedWorker(url, options) { + var worker; + try { + worker = new Worker(url, options); + } catch (e) { + return Promise.reject(e); + } + worker.postMessage(''); + return bindEvents2(worker, "message", worker, "error") + .then(event => wrapResult(event.data)); +} + +function requestViaSharedWorker(url, options) { + var worker; + try { + worker = new SharedWorker(url, options); + } catch(e) { + return Promise.reject(e); + } + const promise = bindEvents2(worker.port, "message", worker, "error") + .then(event => wrapResult(event.data)); + worker.port.start(); + return promise; +} + +// Returns a reference to a worklet object corresponding to a given type. +function get_worklet(type) { + if (type == 'animation') + return CSS.animationWorklet; + if (type == 'layout') + return CSS.layoutWorklet; + if (type == 'paint') + return CSS.paintWorklet; + if (type == 'audio') + return new OfflineAudioContext(2,44100*40,44100).audioWorklet; + + throw new Error('unknown worklet type is passed.'); +} + +function requestViaWorklet(type, url) { + try { + return get_worklet(type).addModule(url); + } catch (e) { + return Promise.reject(e); + } +} + +/** + * Creates a navigable element with the name `navigableElementName` + * (<a>, <area>, or <form>) under `parentNode`, and + * performs a navigation by `trigger()` (e.g. clicking <a>). + * To avoid navigating away from the current execution context, + * a target attribute is set to point to a new helper iframe. + * @param {string} navigableElementName + * @param {object} additionalAttributes The attributes of the navigable element. + * @param {DOMElement} parentNode + * @param {function(DOMElement} trigger A callback called after the navigable + * element is inserted and should trigger navigation using the element. + * @return {Promise} The promise for success/error events. + */ +function requestViaNavigable(navigableElementName, additionalAttributes, + parentNode, trigger) { + const name = guid(); + + const iframe = + createElement("iframe", {"name": name, "id": name}, parentNode, false); + + const navigable = createElement( + navigableElementName, + Object.assign({"target": name}, additionalAttributes), + parentNode, false); + + const promise = + bindEvents2(window, "message", iframe, "error", window, "error") + .then(event => { + if (event.source !== iframe.contentWindow) + return Promise.reject(new Error('Unexpected event.source')); + return event.data; + }); + trigger(navigable); + return promise; +} + +/** + * Creates a new anchor element, appends it to {@code document.body} and + * performs the navigation. + * @param {string} url The URL to navigate to. + * @return {Promise} The promise for success/error events. + */ +function requestViaAnchor(url, additionalAttributes) { + return requestViaNavigable( + "a", + Object.assign({"href": url, "innerHTML": "Link to resource"}, + additionalAttributes), + document.body, a => a.click()); +} + +/** + * Creates a new area element, appends it to {@code document.body} and performs + * the navigation. + * @param {string} url The URL to navigate to. + * @return {Promise} The promise for success/error events. + */ +function requestViaArea(url, additionalAttributes) { + // TODO(kristijanburnik): Append to map and add image. + return requestViaNavigable( + "area", + Object.assign({"href": url}, additionalAttributes), + document.body, area => area.click()); +} + +/** + * Creates a new script element, sets the src to url, and appends it to + * {@code document.body}. + * @param {string} url The src URL. + * @return {Promise} The promise for success/error events. + */ +function requestViaScript(url, additionalAttributes) { + const script = createElement( + "script", + Object.assign({"src": url}, additionalAttributes), + document.body, + false); + + return bindEvents2(window, "message", script, "error", window, "error") + .then(event => wrapResult(event.data)); +} + +/** + * Creates a new script element that performs a dynamic import to `url`, and + * appends the script element to {@code document.body}. + * @param {string} url The src URL. + * @return {Promise} The promise for success/error events. + */ +function requestViaDynamicImport(url, additionalAttributes) { + const scriptUrl = `data:text/javascript,import("${url}");`; + const script = createElement( + "script", + Object.assign({"src": scriptUrl}, additionalAttributes), + document.body, + false); + + return bindEvents2(window, "message", script, "error", window, "error") + .then(event => wrapResult(event.data)); +} + +/** + * Creates a new form element, sets attributes, appends it to + * {@code document.body} and submits the form. + * @param {string} url The URL to submit to. + * @return {Promise} The promise for success/error events. + */ +function requestViaForm(url, additionalAttributes) { + return requestViaNavigable( + "form", + Object.assign({"action": url, "method": "POST"}, additionalAttributes), + document.body, form => form.submit()); +} + +/** + * Creates a new link element for a stylesheet, binds load and error events, + * sets the href to url and appends it to {@code document.head}. + * @param {string} url The URL for a stylesheet. + * @return {Promise} The promise for success/error events. + */ +function requestViaLinkStylesheet(url) { + return createRequestViaElement("link", + {"rel": "stylesheet", "href": url}, + document.head); +} + +/** + * Creates a new link element for a prefetch, binds load and error events, sets + * the href to url and appends it to {@code document.head}. + * @param {string} url The URL of a resource to prefetch. + * @return {Promise} The promise for success/error events. + */ +function requestViaLinkPrefetch(url) { + var link = document.createElement('link'); + if (link.relList && link.relList.supports && link.relList.supports("prefetch")) { + return createRequestViaElement("link", + {"rel": "prefetch", "href": url}, + document.head); + } else { + return Promise.reject("This browser does not support 'prefetch'."); + } +} + +/** + * Initiates a new beacon request. + * @param {string} url The URL of a resource to prefetch. + * @return {Promise} The promise for success/error events. + */ +async function requestViaSendBeacon(url) { + function wait(ms) { + return new Promise(resolve => step_timeout(resolve, ms)); + } + if (!navigator.sendBeacon(url)) { + // If mixed-content check fails, it should return false. + throw new Error('sendBeacon() fails.'); + } + // We don't have a means to see the result of sendBeacon() request + // for sure. Let's wait for a while and let the generic test function + // ask the server for the result. + await wait(500); + return 'allowed'; +} + +/** + * Creates a new media element with a child source element, binds loadeddata and + * error events, sets attributes and appends to document.body. + * @param {string} type The type of the media element (audio/video/picture). + * @param {object} media_attrs The attributes for the media element. + * @param {object} source_attrs The attributes for the child source element. + * @return {DOMElement} The newly created media element. + */ +function createMediaElement(type, media_attrs, source_attrs) { + var mediaElement = createElement(type, {}); + + var sourceElement = createElement("source", {}); + + mediaElement.eventPromise = new Promise(function(resolve, reject) { + mediaElement.addEventListener("loadeddata", function (e) { + resolve(e); + }); + + // Safari doesn't fire an `error` event when blocking mixed content. + mediaElement.addEventListener("stalled", function(e) { + reject(e); + }); + + sourceElement.addEventListener("error", function(e) { + reject(e); + }); + }); + + setAttributes(mediaElement, media_attrs); + setAttributes(sourceElement, source_attrs); + + mediaElement.appendChild(sourceElement); + document.body.appendChild(mediaElement); + + return mediaElement; +} + +/** + * Creates a new video element, binds loadeddata and error events, sets + * attributes and source URL and appends to {@code document.body}. + * @param {string} url The URL of the video. + * @return {Promise} The promise for success/error events. + */ +function requestViaVideo(url) { + return createMediaElement("video", + {}, + {"src": url}).eventPromise; +} + +/** + * Creates a new audio element, binds loadeddata and error events, sets + * attributes and source URL and appends to {@code document.body}. + * @param {string} url The URL of the audio. + * @return {Promise} The promise for success/error events. + */ +function requestViaAudio(url) { + return createMediaElement("audio", + {}, + {"type": "audio/wav", "src": url}).eventPromise; +} + +/** + * Creates a new picture element, binds loadeddata and error events, sets + * attributes and source URL and appends to {@code document.body}. Also + * creates new image element appending it to the picture + * @param {string} url The URL of the image for the source and image elements. + * @return {Promise} The promise for success/error events. + */ +function requestViaPicture(url) { + var picture = createMediaElement("picture", {}, {"srcset": url, + "type": "image/png"}); + return createRequestViaElement("img", {"src": url}, picture); +} + +/** + * Creates a new object element, binds load and error events, sets the data to + * url, and appends it to {@code document.body}. + * @param {string} url The data URL. + * @return {Promise} The promise for success/error events. + */ +function requestViaObject(url) { + return createRequestViaElement("object", {"data": url, "type": "text/html"}, document.body); +} + +/** + * Creates a new WebSocket pointing to {@code url} and sends a message string + * "echo". The {@code message} and {@code error} events are triggering the + * returned promise resolve/reject events. + * @param {string} url The URL for WebSocket to connect to. + * @return {Promise} The promise for success/error events. + */ +function requestViaWebSocket(url) { + return new Promise(function(resolve, reject) { + var websocket = new WebSocket(url); + + websocket.addEventListener("message", function(e) { + resolve(e.data); + }); + + websocket.addEventListener("open", function(e) { + websocket.send("echo"); + }); + + websocket.addEventListener("error", function(e) { + reject(e) + }); + }) + .then(data => { + return JSON.parse(data); + }); +} + +/** + @typedef SubresourceType + @type {string} + + Represents how a subresource is sent. + The keys of `subresourceMap` below are the valid values. +*/ + +// Subresource paths and invokers. +const subresourceMap = { + "a-tag": { + path: "/common/security-features/subresource/document.py", + invoker: requestViaAnchor, + }, + "area-tag": { + path: "/common/security-features/subresource/document.py", + invoker: requestViaArea, + }, + "audio-tag": { + path: "/common/security-features/subresource/audio.py", + invoker: requestViaAudio, + }, + "beacon": { + path: "/common/security-features/subresource/empty.py", + invoker: requestViaSendBeacon, + }, + "fetch": { + path: "/common/security-features/subresource/xhr.py", + invoker: requestViaFetch, + }, + "form-tag": { + path: "/common/security-features/subresource/document.py", + invoker: requestViaForm, + }, + "iframe-tag": { + path: "/common/security-features/subresource/document.py", + invoker: requestViaIframe, + }, + "img-tag": { + path: "/common/security-features/subresource/image.py", + invoker: requestViaImage, + }, + "link-css-tag": { + path: "/common/security-features/subresource/empty.py", + invoker: requestViaLinkStylesheet, + }, + "link-prefetch-tag": { + path: "/common/security-features/subresource/empty.py", + invoker: requestViaLinkPrefetch, + }, + "object-tag": { + path: "/common/security-features/subresource/empty.py", + invoker: requestViaObject, + }, + "picture-tag": { + path: "/common/security-features/subresource/image.py", + invoker: requestViaPicture, + }, + "script-tag": { + path: "/common/security-features/subresource/script.py", + invoker: requestViaScript, + }, + "script-tag-dynamic-import": { + path: "/common/security-features/subresource/script.py", + invoker: requestViaDynamicImport, + }, + "video-tag": { + path: "/common/security-features/subresource/video.py", + invoker: requestViaVideo, + }, + "xhr": { + path: "/common/security-features/subresource/xhr.py", + invoker: requestViaXhr, + }, + + "worker-classic": { + path: "/common/security-features/subresource/worker.py", + invoker: url => requestViaDedicatedWorker(url), + }, + "worker-module": { + path: "/common/security-features/subresource/worker.py", + invoker: url => requestViaDedicatedWorker(url, {type: "module"}), + }, + "worker-import": { + path: "/common/security-features/subresource/worker.py", + invoker: (url, additionalAttributes) => + requestViaDedicatedWorker(workerUrlThatImports(url, additionalAttributes), {type: "module"}), + }, + "worker-import-data": { + path: "/common/security-features/subresource/worker.py", + invoker: url => + requestViaDedicatedWorker(workerDataUrlThatImports(url), {type: "module"}), + }, + "sharedworker-classic": { + path: "/common/security-features/subresource/shared-worker.py", + invoker: url => requestViaSharedWorker(url), + }, + "sharedworker-module": { + path: "/common/security-features/subresource/shared-worker.py", + invoker: url => requestViaSharedWorker(url, {type: "module"}), + }, + "sharedworker-import": { + path: "/common/security-features/subresource/shared-worker.py", + invoker: (url, additionalAttributes) => + requestViaSharedWorker(workerUrlThatImports(url, additionalAttributes), {type: "module"}), + }, + "sharedworker-import-data": { + path: "/common/security-features/subresource/shared-worker.py", + invoker: url => + requestViaSharedWorker(workerDataUrlThatImports(url), {type: "module"}), + }, + + "websocket": { + path: "/stash_responder", + invoker: requestViaWebSocket, + }, +}; +for (const workletType of ['animation', 'audio', 'layout', 'paint']) { + subresourceMap[`worklet-${workletType}`] = { + path: "/common/security-features/subresource/worker.py", + invoker: url => requestViaWorklet(workletType, url) + }; + subresourceMap[`worklet-${workletType}-import-data`] = { + path: "/common/security-features/subresource/worker.py", + invoker: url => + requestViaWorklet(workletType, workerDataUrlThatImports(url)) + }; +} + +/** + @typedef RedirectionType + @type {string} + + Represents what redirects should occur to the subresource request + after initial request. + See preprocess_redirection() in + /common/security-features/subresource/subresource.py for valid values. +*/ + +/** + Construct subresource (and related) origin. + + @param {string} originType + @returns {object} the origin of the subresource. +*/ +function getSubresourceOrigin(originType) { + const httpProtocol = "http"; + const httpsProtocol = "https"; + const wsProtocol = "ws"; + const wssProtocol = "wss"; + + const sameOriginHost = "{{host}}"; + const crossOriginHost = "{{domains[www1]}}"; + + // These values can evaluate to either empty strings or a ":port" string. + const httpPort = getNormalizedPort(parseInt("{{ports[http][0]}}", 10)); + const httpsRawPort = parseInt("{{ports[https][0]}}", 10); + const httpsPort = getNormalizedPort(httpsRawPort); + const wsPort = getNormalizedPort(parseInt("{{ports[ws][0]}}", 10)); + const wssRawPort = parseInt("{{ports[wss][0]}}", 10); + const wssPort = getNormalizedPort(wssRawPort); + + /** + @typedef OriginType + @type {string} + + Represents the origin of the subresource request URL. + The keys of `originMap` below are the valid values. + + Note that there can be redirects from the specified origin + (see RedirectionType), and thus the origin of the subresource + response URL might be different from what is specified by OriginType. + */ + const originMap = { + "same-https": httpsProtocol + "://" + sameOriginHost + httpsPort, + "same-http": httpProtocol + "://" + sameOriginHost + httpPort, + "cross-https": httpsProtocol + "://" + crossOriginHost + httpsPort, + "cross-http": httpProtocol + "://" + crossOriginHost + httpPort, + "same-wss": wssProtocol + "://" + sameOriginHost + wssPort, + "same-ws": wsProtocol + "://" + sameOriginHost + wsPort, + "cross-wss": wssProtocol + "://" + crossOriginHost + wssPort, + "cross-ws": wsProtocol + "://" + crossOriginHost + wsPort, + + // The following origin types are used for upgrade-insecure-requests tests: + // These rely on some unintuitive cleverness due to WPT's test setup: + // 'Upgrade-Insecure-Requests' does not upgrade the port number, + // so we use URLs in the form `http://[domain]:[https-port]`, + // which will be upgraded to `https://[domain]:[https-port]`. + // If the upgrade fails, the load will fail, as we don't serve HTTP over + // the secure port. + "same-http-downgrade": + httpProtocol + "://" + sameOriginHost + ":" + httpsRawPort, + "cross-http-downgrade": + httpProtocol + "://" + crossOriginHost + ":" + httpsRawPort, + "same-ws-downgrade": + wsProtocol + "://" + sameOriginHost + ":" + wssRawPort, + "cross-ws-downgrade": + wsProtocol + "://" + crossOriginHost + ":" + wssRawPort, + }; + + return originMap[originType]; +} + +/** + Construct subresource (and related) URLs. + + @param {SubresourceType} subresourceType + @param {OriginType} originType + @param {RedirectionType} redirectionType + @returns {object} with following properties: + {string} testUrl + The subresource request URL. + {string} announceUrl + {string} assertUrl + The URLs to be used for detecting whether `testUrl` is actually sent + to the server. + 1. Fetch `announceUrl` first, + 2. then possibly fetch `testUrl`, and + 3. finally fetch `assertUrl`. + The fetch result of `assertUrl` should indicate whether + `testUrl` is actually sent to the server or not. +*/ +function getRequestURLs(subresourceType, originType, redirectionType) { + const key = guid(); + const value = guid(); + + // We use the same stash path for both HTTP/S and WS/S stash requests. + const stashPath = encodeURIComponent("/mixed-content"); + + const stashEndpoint = "/common/security-features/subresource/xhr.py?key=" + + key + "&path=" + stashPath; + return { + testUrl: + getSubresourceOrigin(originType) + + subresourceMap[subresourceType].path + + "?redirection=" + encodeURIComponent(redirectionType) + + "&action=purge&key=" + key + + "&path=" + stashPath, + announceUrl: stashEndpoint + "&action=put&value=" + value, + assertUrl: stashEndpoint + "&action=take", + }; +} + +// =============================================================== +// Source Context +// =============================================================== +// Requests can be sent from several source contexts, +// such as the main documents, iframes, workers, or so, +// possibly nested, and possibly with <meta>/http headers added. +// invokeRequest() and invokeFrom*() functions handles +// SourceContext-related setup in client-side. + +/** + invokeRequest() invokes a subresource request + (specified as `subresource`) + from a (possibly nested) environment settings object + (specified as `sourceContextList`). + + For nested contexts, invokeRequest() calls an invokeFrom*() function + that creates a nested environment settings object using + /common/security-features/scope/, which calls invokeRequest() + again inside the nested environment settings object. + This cycle continues until all specified + nested environment settings object are created, and + finally invokeRequest() calls a requestVia*() function to start the + subresource request from the inner-most environment settings object. + + @param {Subresource} subresource + @param {Array<SourceContext>} sourceContextList + + @returns {Promise} A promise that is resolved with an RequestResult object. + `sourceContextUrl` is always set. For whether other properties are set, + see the comments for requestVia*() above. +*/ +function invokeRequest(subresource, sourceContextList) { + if (sourceContextList.length === 0) { + // No further nested global objects. Send the subresource request here. + + const additionalAttributes = {}; + /** @type {PolicyDelivery} policyDelivery */ + for (const policyDelivery of (subresource.policyDeliveries || [])) { + // Depending on the delivery method, extend the subresource element with + // these attributes. + if (policyDelivery.deliveryType === "attr") { + additionalAttributes[policyDelivery.key] = policyDelivery.value; + } else if (policyDelivery.deliveryType === "rel-noref") { + additionalAttributes["rel"] = "noreferrer"; + } else if (policyDelivery.deliveryType === "http-rp") { + additionalAttributes[policyDelivery.key] = policyDelivery.value; + } else if (policyDelivery.deliveryType === "meta") { + additionalAttributes[policyDelivery.key] = policyDelivery.value; + } + } + + return subresourceMap[subresource.subresourceType].invoker( + subresource.url, + additionalAttributes) + .then(result => Object.assign( + {sourceContextUrl: location.toString()}, + result)); + } + + // Defines invokers for each valid SourceContext.sourceContextType. + const sourceContextMap = { + "srcdoc": { // <iframe srcdoc></iframe> + invoker: invokeFromIframe, + }, + "iframe": { // <iframe src="same-origin-URL"></iframe> + invoker: invokeFromIframe, + }, + "iframe-blank": { // <iframe></iframe> + invoker: invokeFromIframe, + }, + "worker-classic": { + // Classic dedicated worker loaded from same-origin. + invoker: invokeFromWorker.bind(undefined, "worker", false, {}), + }, + "worker-classic-data": { + // Classic dedicated worker loaded from data: URL. + invoker: invokeFromWorker.bind(undefined, "worker", true, {}), + }, + "worker-module": { + // Module dedicated worker loaded from same-origin. + invoker: invokeFromWorker.bind(undefined, "worker", false, {type: 'module'}), + }, + "worker-module-data": { + // Module dedicated worker loaded from data: URL. + invoker: invokeFromWorker.bind(undefined, "worker", true, {type: 'module'}), + }, + "sharedworker-classic": { + // Classic shared worker loaded from same-origin. + invoker: invokeFromWorker.bind(undefined, "sharedworker", false, {}), + }, + "sharedworker-classic-data": { + // Classic shared worker loaded from data: URL. + invoker: invokeFromWorker.bind(undefined, "sharedworker", true, {}), + }, + "sharedworker-module": { + // Module shared worker loaded from same-origin. + invoker: invokeFromWorker.bind(undefined, "sharedworker", false, {type: 'module'}), + }, + "sharedworker-module-data": { + // Module shared worker loaded from data: URL. + invoker: invokeFromWorker.bind(undefined, "sharedworker", true, {type: 'module'}), + }, + }; + + return sourceContextMap[sourceContextList[0].sourceContextType].invoker( + subresource, sourceContextList); +} + +// Quick hack to expose invokeRequest when common.sub.js is loaded either +// as a classic or module script. +self.invokeRequest = invokeRequest; + +/** + invokeFrom*() functions are helper functions with the same parameters + and return values as invokeRequest(), that are tied to specific types + of top-most environment settings objects. + For example, invokeFromIframe() is the helper function for the cases where + sourceContextList[0] is an iframe. +*/ + +/** + @param {string} workerType + "worker" (for dedicated worker) or "sharedworker". + @param {boolean} isDataUrl + true if the worker script is loaded from data: URL. + Otherwise, the script is loaded from same-origin. + @param {object} workerOptions + The `options` argument for Worker constructor. + + Other parameters and return values are the same as those of invokeRequest(). +*/ +function invokeFromWorker(workerType, isDataUrl, workerOptions, + subresource, sourceContextList) { + const currentSourceContext = sourceContextList[0]; + let workerUrl = + "/common/security-features/scope/worker.py?policyDeliveries=" + + encodeURIComponent(JSON.stringify( + currentSourceContext.policyDeliveries || [])); + if (workerOptions.type === 'module') { + workerUrl += "&type=module"; + } + + let promise; + if (isDataUrl) { + promise = fetch(workerUrl) + .then(r => r.text()) + .then(source => { + return 'data:text/javascript;base64,' + btoa(source); + }); + } else { + promise = Promise.resolve(workerUrl); + } + + return promise + .then(url => { + if (workerType === "worker") { + const worker = new Worker(url, workerOptions); + worker.postMessage({subresource: subresource, + sourceContextList: sourceContextList.slice(1)}); + return bindEvents2(worker, "message", worker, "error", window, "error"); + } else if (workerType === "sharedworker") { + const worker = new SharedWorker(url, workerOptions); + worker.port.start(); + worker.port.postMessage({subresource: subresource, + sourceContextList: sourceContextList.slice(1)}); + return bindEvents2(worker.port, "message", worker, "error", window, "error"); + } else { + throw new Error('Invalid worker type: ' + workerType); + } + }) + .then(event => { + if (event.data.error) + return Promise.reject(event.data.error); + return event.data; + }); +} + +function invokeFromIframe(subresource, sourceContextList) { + const currentSourceContext = sourceContextList[0]; + const frameUrl = + "/common/security-features/scope/document.py?policyDeliveries=" + + encodeURIComponent(JSON.stringify( + currentSourceContext.policyDeliveries || [])); + + let iframe; + let promise; + if (currentSourceContext.sourceContextType === 'srcdoc') { + promise = fetch(frameUrl) + .then(r => r.text()) + .then(srcdoc => { + iframe = createElement( + "iframe", {srcdoc: srcdoc}, document.body, true); + return iframe.eventPromise; + }); + } else if (currentSourceContext.sourceContextType === 'iframe') { + iframe = createElement("iframe", {src: frameUrl}, document.body, true); + promise = iframe.eventPromise; + } else if (currentSourceContext.sourceContextType === 'iframe-blank') { + let frameContent; + promise = fetch(frameUrl) + .then(r => r.text()) + .then(t => { + frameContent = t; + iframe = createElement("iframe", {}, document.body, true); + return iframe.eventPromise; + }) + .then(() => { + // Reinitialize `iframe.eventPromise` with a new promise + // that catches the load event for the document.write() below. + bindEvents(iframe); + + iframe.contentDocument.write(frameContent); + iframe.contentDocument.close(); + return iframe.eventPromise; + }); + } + + return promise + .then(() => { + const promise = bindEvents2( + window, "message", iframe, "error", window, "error"); + iframe.contentWindow.postMessage( + {subresource: subresource, + sourceContextList: sourceContextList.slice(1)}, + "*"); + return promise; + }) + .then(event => { + if (event.data.error) + return Promise.reject(event.data.error); + return event.data; + }); +} + +// SanityChecker does nothing in release mode. See sanity-checker.js for debug +// mode. +function SanityChecker() {} +SanityChecker.prototype.checkScenario = function() {}; +SanityChecker.prototype.setFailTimeout = function(test, timeout) {}; +SanityChecker.prototype.checkSubresourceResult = function() {}; diff --git a/testing/web-platform/tests/common/security-features/resources/common.sub.js.headers b/testing/web-platform/tests/common/security-features/resources/common.sub.js.headers new file mode 100644 index 0000000000..cb762eff80 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/resources/common.sub.js.headers @@ -0,0 +1 @@ +Access-Control-Allow-Origin: * diff --git a/testing/web-platform/tests/common/security-features/scope/__init__.py b/testing/web-platform/tests/common/security-features/scope/__init__.py new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/__init__.py diff --git a/testing/web-platform/tests/common/security-features/scope/document.py b/testing/web-platform/tests/common/security-features/scope/document.py new file mode 100644 index 0000000000..9a9f045e64 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/document.py @@ -0,0 +1,36 @@ +import os, sys, json + +from wptserve.utils import isomorphic_decode, isomorphic_encode + +import importlib +util = importlib.import_module("common.security-features.scope.util") + +def main(request, response): + policyDeliveries = json.loads(request.GET.first(b"policyDeliveries", b"[]")) + maybe_additional_headers = {} + meta = u'' + error = u'' + for delivery in policyDeliveries: + if delivery[u'deliveryType'] == u'meta': + if delivery[u'key'] == u'referrerPolicy': + meta += u'<meta name="referrer" content="%s">' % delivery[u'value'] + else: + error = u'invalid delivery key' + elif delivery[u'deliveryType'] == u'http-rp': + if delivery[u'key'] == u'referrerPolicy': + maybe_additional_headers[b'Referrer-Policy'] = isomorphic_encode(delivery[u'value']) + else: + error = u'invalid delivery key' + else: + error = u'invalid deliveryType' + + handler = lambda: util.get_template(u"document.html.template") % ({ + u"meta": meta, + u"error": error + }) + util.respond( + request, + response, + payload_generator=handler, + content_type=b"text/html", + maybe_additional_headers=maybe_additional_headers) diff --git a/testing/web-platform/tests/common/security-features/scope/template/document.html.template b/testing/web-platform/tests/common/security-features/scope/template/document.html.template new file mode 100644 index 0000000000..37e29f8e97 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/template/document.html.template @@ -0,0 +1,30 @@ +<!DOCTYPE html> +<html> + <head> + %(meta)s + <script src="/common/security-features/resources/common.sub.js"></script> + <script> + // Receive a message from the parent and start the test. + function onMessageFromParent(event) { + // Because this window might receive messages from child iframe during + // tests, we first remove the listener here before staring the test. + window.removeEventListener('message', onMessageFromParent); + + const configurationError = "%(error)s"; + if (configurationError.length > 0) { + parent.postMessage({error: configurationError}, "*"); + return; + } + + invokeRequest(event.data.subresource, + event.data.sourceContextList) + .then(result => parent.postMessage(result, "*")) + .catch(e => { + const message = (e.error && e.error.stack) || e.message || "Error"; + parent.postMessage({error: message}, "*"); + }); + } + window.addEventListener('message', onMessageFromParent); + </script> + </head> +</html> diff --git a/testing/web-platform/tests/common/security-features/scope/template/worker.js.template b/testing/web-platform/tests/common/security-features/scope/template/worker.js.template new file mode 100644 index 0000000000..7a2a6e05c4 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/template/worker.js.template @@ -0,0 +1,29 @@ +%(import)s + +if ('DedicatedWorkerGlobalScope' in self && + self instanceof DedicatedWorkerGlobalScope) { + self.onmessage = event => onMessageFromParent(event, self); +} else if ('SharedWorkerGlobalScope' in self && + self instanceof SharedWorkerGlobalScope) { + onconnect = event => { + const port = event.ports[0]; + port.onmessage = event => onMessageFromParent(event, port); + }; +} + +// Receive a message from the parent and start the test. +function onMessageFromParent(event, port) { + const configurationError = "%(error)s"; + if (configurationError.length > 0) { + port.postMessage({error: configurationError}); + return; + } + + invokeRequest(event.data.subresource, + event.data.sourceContextList) + .then(result => port.postMessage(result)) + .catch(e => { + const message = (e.error && e.error.stack) || e.message || "Error"; + port.postMessage({error: message}); + }); +} diff --git a/testing/web-platform/tests/common/security-features/scope/util.py b/testing/web-platform/tests/common/security-features/scope/util.py new file mode 100644 index 0000000000..da5aacf35e --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/util.py @@ -0,0 +1,43 @@ +import os + +from wptserve.utils import isomorphic_decode + +def get_template(template_basename): + script_directory = os.path.dirname(os.path.abspath(isomorphic_decode(__file__))) + template_directory = os.path.abspath( + os.path.join(script_directory, u"template")) + template_filename = os.path.join(template_directory, template_basename) + + with open(template_filename, "r") as f: + return f.read() + + +def __noop(request, response): + return u"" + + +def respond(request, + response, + status_code=200, + content_type=b"text/html", + payload_generator=__noop, + cache_control=b"no-cache; must-revalidate", + access_control_allow_origin=b"*", + maybe_additional_headers=None): + response.add_required_headers = False + response.writer.write_status(status_code) + + if access_control_allow_origin != None: + response.writer.write_header(b"access-control-allow-origin", + access_control_allow_origin) + response.writer.write_header(b"content-type", content_type) + response.writer.write_header(b"cache-control", cache_control) + + additional_headers = maybe_additional_headers or {} + for header, value in additional_headers.items(): + response.writer.write_header(header, value) + + response.writer.end_headers() + + payload = payload_generator() + response.writer.write(payload) diff --git a/testing/web-platform/tests/common/security-features/scope/worker.py b/testing/web-platform/tests/common/security-features/scope/worker.py new file mode 100644 index 0000000000..6b321e7de1 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/scope/worker.py @@ -0,0 +1,44 @@ +import os, sys, json + +from wptserve.utils import isomorphic_decode, isomorphic_encode +import importlib +util = importlib.import_module("common.security-features.scope.util") + +def main(request, response): + policyDeliveries = json.loads(request.GET.first(b'policyDeliveries', b'[]')) + worker_type = request.GET.first(b'type', b'classic') + commonjs_url = u'%s://%s:%s/common/security-features/resources/common.sub.js' % ( + request.url_parts.scheme, request.url_parts.hostname, + request.url_parts.port) + if worker_type == b'classic': + import_line = u'importScripts("%s");' % commonjs_url + else: + import_line = u'import "%s";' % commonjs_url + + maybe_additional_headers = {} + error = u'' + for delivery in policyDeliveries: + if delivery[u'deliveryType'] == u'meta': + error = u'<meta> cannot be used in WorkerGlobalScope' + elif delivery[u'deliveryType'] == u'http-rp': + if delivery[u'key'] == u'referrerPolicy': + maybe_additional_headers[b'Referrer-Policy'] = isomorphic_encode(delivery[u'value']) + elif delivery[u'key'] == u'mixedContent' and delivery[u'value'] == u'opt-in': + maybe_additional_headers[b'Content-Security-Policy'] = b'block-all-mixed-content' + elif delivery[u'key'] == u'upgradeInsecureRequests' and delivery[u'value'] == u'upgrade': + maybe_additional_headers[b'Content-Security-Policy'] = b'upgrade-insecure-requests' + else: + error = u'invalid delivery key for http-rp: %s' % delivery[u'key'] + else: + error = u'invalid deliveryType: %s' % delivery[u'deliveryType'] + + handler = lambda: util.get_template(u'worker.js.template') % ({ + u'import': import_line, + u'error': error + }) + util.respond( + request, + response, + payload_generator=handler, + content_type=b'text/javascript', + maybe_additional_headers=maybe_additional_headers) diff --git a/testing/web-platform/tests/common/security-features/subresource/__init__.py b/testing/web-platform/tests/common/security-features/subresource/__init__.py new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/__init__.py diff --git a/testing/web-platform/tests/common/security-features/subresource/audio.py b/testing/web-platform/tests/common/security-features/subresource/audio.py new file mode 100644 index 0000000000..f16a0f7fbb --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/audio.py @@ -0,0 +1,18 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(request, server_data): + file = os.path.join(request.doc_root, u"webaudio", u"resources", + u"sin_440Hz_-6dBFS_1s.wav") + return open(file, "rb").read() + + +def main(request, response): + handler = lambda data: generate_payload(request, data) + subresource.respond(request, + response, + payload_generator = handler, + access_control_allow_origin = b"*", + content_type = b"audio/wav") diff --git a/testing/web-platform/tests/common/security-features/subresource/document.py b/testing/web-platform/tests/common/security-features/subresource/document.py new file mode 100644 index 0000000000..52b684a4d9 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/document.py @@ -0,0 +1,12 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + return subresource.get_template(u"document.html.template") % server_data + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload) diff --git a/testing/web-platform/tests/common/security-features/subresource/empty.py b/testing/web-platform/tests/common/security-features/subresource/empty.py new file mode 100644 index 0000000000..312e12cbed --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/empty.py @@ -0,0 +1,14 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + return u'' + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload, + access_control_allow_origin = b"*", + content_type = b"text/plain") diff --git a/testing/web-platform/tests/common/security-features/subresource/font.py b/testing/web-platform/tests/common/security-features/subresource/font.py new file mode 100644 index 0000000000..7900079cdf --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/font.py @@ -0,0 +1,76 @@ +import os, sys +from base64 import decodebytes + +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + + +def generate_payload(request, server_data): + data = (u'{"headers": %(headers)s}') % server_data + if b"id" in request.GET: + request.server.stash.put(request.GET[b"id"], data) + # Simple base64 encoded .tff font + return decodebytes(b"AAEAAAANAIAAAwBQRkZUTU6u6MkAAAXcAAAAHE9TLzJWYW" + b"QKAAABWAAAAFZjbWFwAA8D7wAAAcAAAAFCY3Z0IAAhAnkA" + b"AAMEAAAABGdhc3D//wADAAAF1AAAAAhnbHlmCC6aTwAAAx" + b"QAAACMaGVhZO8ooBcAAADcAAAANmhoZWEIkAV9AAABFAAA" + b"ACRobXR4EZQAhQAAAbAAAAAQbG9jYQBwAFQAAAMIAAAACm" + b"1heHAASQA9AAABOAAAACBuYW1lehAVOgAAA6AAAAIHcG9z" + b"dP+uADUAAAWoAAAAKgABAAAAAQAAMhPyuV8PPPUACwPoAA" + b"AAAMU4Lm0AAAAAxTgubQAh/5wFeAK8AAAACAACAAAAAAAA" + b"AAEAAAK8/5wAWgXcAAAAAAV4AAEAAAAAAAAAAAAAAAAAAA" + b"AEAAEAAAAEAAwAAwAAAAAAAgAAAAEAAQAAAEAALgAAAAAA" + b"AQXcAfQABQAAAooCvAAAAIwCigK8AAAB4AAxAQIAAAIABg" + b"kAAAAAAAAAAAABAAAAAAAAAAAAAAAAUGZFZABAAEEAQQMg" + b"/zgAWgK8AGQAAAABAAAAAAAABdwAIQAAAAAF3AAABdwAZA" + b"AAAAMAAAADAAAAHAABAAAAAAA8AAMAAQAAABwABAAgAAAA" + b"BAAEAAEAAABB//8AAABB////wgABAAAAAAAAAQYAAAEAAA" + b"AAAAAAAQIAAAACAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + b"AAAAAAAAAAAAAAAAAAAhAnkAAAAqACoAKgBGAAAAAgAhAA" + b"ABKgKaAAMABwAusQEALzyyBwQA7TKxBgXcPLIDAgDtMgCx" + b"AwAvPLIFBADtMrIHBgH8PLIBAgDtMjMRIREnMxEjIQEJ6M" + b"fHApr9ZiECWAAAAwBk/5wFeAK8AAMABwALAAABNSEVATUh" + b"FQE1IRUB9AH0/UQDhPu0BRQB9MjI/tTIyP7UyMgAAAAAAA" + b"4ArgABAAAAAAAAACYATgABAAAAAAABAAUAgQABAAAAAAAC" + b"AAYAlQABAAAAAAADACEA4AABAAAAAAAEAAUBDgABAAAAAA" + b"AFABABNgABAAAAAAAGAAUBUwADAAEECQAAAEwAAAADAAEE" + b"CQABAAoAdQADAAEECQACAAwAhwADAAEECQADAEIAnAADAA" + b"EECQAEAAoBAgADAAEECQAFACABFAADAAEECQAGAAoBRwBD" + b"AG8AcAB5AHIAaQBnAGgAdAAgACgAYwApACAAMgAwADAAOA" + b"AgAE0AbwB6AGkAbABsAGEAIABDAG8AcgBwAG8AcgBhAHQA" + b"aQBvAG4AAENvcHlyaWdodCAoYykgMjAwOCBNb3ppbGxhIE" + b"NvcnBvcmF0aW9uAABNAGEAcgBrAEEAAE1hcmtBAABNAGUA" + b"ZABpAHUAbQAATWVkaXVtAABGAG8AbgB0AEYAbwByAGcAZQ" + b"AgADIALgAwACAAOgAgAE0AYQByAGsAQQAgADoAIAA1AC0A" + b"MQAxAC0AMgAwADAAOAAARm9udEZvcmdlIDIuMCA6IE1hcm" + b"tBIDogNS0xMS0yMDA4AABNAGEAcgBrAEEAAE1hcmtBAABW" + b"AGUAcgBzAGkAbwBuACAAMAAwADEALgAwADAAMAAgAABWZX" + b"JzaW9uIDAwMS4wMDAgAABNAGEAcgBrAEEAAE1hcmtBAAAA" + b"AgAAAAAAAP+DADIAAAABAAAAAAAAAAAAAAAAAAAAAAAEAA" + b"AAAQACACQAAAAAAAH//wACAAAAAQAAAADEPovuAAAAAMU4" + b"Lm0AAAAAxTgubQ==") + +def generate_report_headers_payload(request, server_data): + stashed_data = request.server.stash.take(request.GET[b"id"]) + return stashed_data + +def main(request, response): + handler = lambda data: generate_payload(request, data) + content_type = b'application/x-font-truetype' + + if b"report-headers" in request.GET: + handler = lambda data: generate_report_headers_payload(request, data) + content_type = b'application/json' + + subresource.respond(request, + response, + payload_generator = handler, + content_type = content_type, + access_control_allow_origin = b"*") diff --git a/testing/web-platform/tests/common/security-features/subresource/image.py b/testing/web-platform/tests/common/security-features/subresource/image.py new file mode 100644 index 0000000000..5c9a0c063c --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/image.py @@ -0,0 +1,116 @@ +import os, sys, array, math + +from io import BytesIO + +from wptserve.utils import isomorphic_decode + +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +class Image: + """This class partially implements the interface of the PIL.Image.Image. + One day in the future WPT might support the PIL module or another imaging + library, so this hacky BMP implementation will no longer be required. + """ + def __init__(self, width, height): + self.width = width + self.height = height + self.img = bytearray([0 for i in range(3 * width * height)]) + + @staticmethod + def new(mode, size, color=0): + return Image(size[0], size[1]) + + def _int_to_bytes(self, number): + packed_bytes = [0, 0, 0, 0] + for i in range(4): + packed_bytes[i] = number & 0xFF + number >>= 8 + + return packed_bytes + + def putdata(self, color_data): + for y in range(self.height): + for x in range(self.width): + i = x + y * self.width + if i > len(color_data) - 1: + return + + self.img[i * 3: i * 3 + 3] = color_data[i][::-1] + + def save(self, f, type): + assert type == "BMP" + # 54 bytes of preambule + image color data. + filesize = 54 + 3 * self.width * self.height + # 14 bytes of header. + bmpfileheader = bytearray([ord('B'), ord('M')] + self._int_to_bytes(filesize) + + [0, 0, 0, 0, 54, 0, 0, 0]) + # 40 bytes of info. + bmpinfoheader = bytearray([40, 0, 0, 0] + + self._int_to_bytes(self.width) + + self._int_to_bytes(self.height) + + [1, 0, 24] + (25 * [0])) + + padlength = (4 - (self.width * 3) % 4) % 4 + bmppad = bytearray([0, 0, 0]) + padding = bmppad[0 : padlength] + + f.write(bmpfileheader) + f.write(bmpinfoheader) + + for i in range(self.height): + offset = self.width * (self.height - i - 1) * 3 + f.write(self.img[offset : offset + 3 * self.width]) + f.write(padding) + +def encode_string_as_bmp_image(string_data): + data_bytes = array.array("B", string_data.encode("utf-8")) + + num_bytes = len(data_bytes) + + # Encode data bytes to color data (RGB), one bit per channel. + # This is to avoid errors due to different color spaces used in decoding. + color_data = [] + for byte in data_bytes: + p = [int(x) * 255 for x in '{0:08b}'.format(byte)] + color_data.append((p[0], p[1], p[2])) + color_data.append((p[3], p[4], p[5])) + color_data.append((p[6], p[7], 0)) + + # Render image. + num_pixels = len(color_data) + sqrt = int(math.ceil(math.sqrt(num_pixels))) + img = Image.new("RGB", (sqrt, sqrt), "black") + img.putdata(color_data) + + # Flush image to string. + f = BytesIO() + img.save(f, "BMP") + f.seek(0) + + return f.read() + +def generate_payload(request, server_data): + data = (u'{"headers": %(headers)s}') % server_data + if b"id" in request.GET: + request.server.stash.put(request.GET[b"id"], data) + data = encode_string_as_bmp_image(data) + return data + +def generate_report_headers_payload(request, server_data): + stashed_data = request.server.stash.take(request.GET[b"id"]) + return stashed_data + +def main(request, response): + handler = lambda data: generate_payload(request, data) + content_type = b'image/bmp' + + if b"report-headers" in request.GET: + handler = lambda data: generate_report_headers_payload(request, data) + content_type = b'application/json' + + subresource.respond(request, + response, + payload_generator = handler, + content_type = content_type, + access_control_allow_origin = b"*") diff --git a/testing/web-platform/tests/common/security-features/subresource/referrer.py b/testing/web-platform/tests/common/security-features/subresource/referrer.py new file mode 100644 index 0000000000..e36631479e --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/referrer.py @@ -0,0 +1,4 @@ +def main(request, response): + referrer = request.headers.get(b"referer", b"") + response_headers = [(b"Content-Type", b"text/javascript")] + return (200, response_headers, b"window.referrer = '" + referrer + b"'") diff --git a/testing/web-platform/tests/common/security-features/subresource/script.py b/testing/web-platform/tests/common/security-features/subresource/script.py new file mode 100644 index 0000000000..9701816b9f --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/script.py @@ -0,0 +1,14 @@ +import os, sys +from wptserve.utils import isomorphic_decode + +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + return subresource.get_template(u"script.js.template") % server_data + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload, + content_type = b"application/javascript") diff --git a/testing/web-platform/tests/common/security-features/subresource/shared-worker.py b/testing/web-platform/tests/common/security-features/subresource/shared-worker.py new file mode 100644 index 0000000000..bdfb61bbb3 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/shared-worker.py @@ -0,0 +1,13 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + return subresource.get_template(u"shared-worker.js.template") % server_data + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload, + content_type = b"application/javascript") diff --git a/testing/web-platform/tests/common/security-features/subresource/static-import.py b/testing/web-platform/tests/common/security-features/subresource/static-import.py new file mode 100644 index 0000000000..3c3a6f6871 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/static-import.py @@ -0,0 +1,61 @@ +import os, sys, json +from urllib.parse import unquote + +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def get_csp_value(value): + ''' + Returns actual CSP header values (e.g. "worker-src 'self'") for the + given string used in PolicyDelivery's value (e.g. "worker-src-self"). + ''' + + # script-src + # Test-related scripts like testharness.js and inline scripts containing + # test bodies. + # 'unsafe-inline' is added as a workaround here. This is probably not so + # bad, as it shouldn't intefere non-inline-script requests that we want to + # test. + if value == 'script-src-wildcard': + return "script-src * 'unsafe-inline'" + if value == 'script-src-self': + return "script-src 'self' 'unsafe-inline'" + # Workaround for "script-src 'none'" would be more complicated, because + # - "script-src 'none' 'unsafe-inline'" is handled somehow differently from + # "script-src 'none'", i.e. + # https://w3c.github.io/webappsec-csp/#match-url-to-source-list Step 3 + # handles the latter but not the former. + # - We need nonce- or path-based additional values to allow same-origin + # test scripts like testharness.js. + # Therefore, we disable 'script-src-none' tests for now in + # `/content-security-policy/spec.src.json`. + if value == 'script-src-none': + return "script-src 'none'" + + # worker-src + if value == 'worker-src-wildcard': + return 'worker-src *' + if value == 'worker-src-self': + return "worker-src 'self'" + if value == 'worker-src-none': + return "worker-src 'none'" + raise Exception('Invalid delivery_value: %s' % value) + +def generate_payload(request): + import_url = unquote(isomorphic_decode(request.GET[b'import_url'])) + return subresource.get_template(u"static-import.js.template") % { + u"import_url": import_url + } + +def main(request, response): + def payload_generator(_): return generate_payload(request) + maybe_additional_headers = {} + if b'contentSecurityPolicy' in request.GET: + csp = unquote(isomorphic_decode(request.GET[b'contentSecurityPolicy'])) + maybe_additional_headers[b'Content-Security-Policy'] = get_csp_value(csp) + subresource.respond(request, + response, + payload_generator = payload_generator, + content_type = b"application/javascript", + maybe_additional_headers = maybe_additional_headers) diff --git a/testing/web-platform/tests/common/security-features/subresource/stylesheet.py b/testing/web-platform/tests/common/security-features/subresource/stylesheet.py new file mode 100644 index 0000000000..05db249250 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/stylesheet.py @@ -0,0 +1,61 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(request, server_data): + data = (u'{"headers": %(headers)s}') % server_data + type = b'image' + if b"type" in request.GET: + type = request.GET[b"type"] + + if b"id" in request.GET: + request.server.stash.put(request.GET[b"id"], data) + + if type == b'image': + return subresource.get_template(u"image.css.template") % {u"id": isomorphic_decode(request.GET[b"id"])} + + elif type == b'font': + return subresource.get_template(u"font.css.template") % {u"id": isomorphic_decode(request.GET[b"id"])} + + elif type == b'svg': + return subresource.get_template(u"svg.css.template") % { + u"id": isomorphic_decode(request.GET[b"id"]), + u"property": isomorphic_decode(request.GET[b"property"])} + + # A `'stylesheet-only'`-type stylesheet has no nested resources; this is + # useful in tests that cover referrers for stylesheet fetches (e.g. fetches + # triggered by `@import` statements). + elif type == b'stylesheet-only': + return u'' + +def generate_import_rule(request, server_data): + return u"@import url('%(url)s');" % { + u"url": subresource.create_url(request, swap_origin=True, + query_parameter_to_remove=u"import-rule") + } + +def generate_report_headers_payload(request, server_data): + stashed_data = request.server.stash.take(request.GET[b"id"]) + return stashed_data + +def main(request, response): + payload_generator = lambda data: generate_payload(request, data) + content_type = b"text/css" + referrer_policy = b"unsafe-url" + if b"import-rule" in request.GET: + payload_generator = lambda data: generate_import_rule(request, data) + + if b"report-headers" in request.GET: + payload_generator = lambda data: generate_report_headers_payload(request, data) + content_type = b'application/json' + + if b"referrer-policy" in request.GET: + referrer_policy = request.GET[b"referrer-policy"] + + subresource.respond( + request, + response, + payload_generator = payload_generator, + content_type = content_type, + maybe_additional_headers = { b"Referrer-Policy": referrer_policy }) diff --git a/testing/web-platform/tests/common/security-features/subresource/subresource.py b/testing/web-platform/tests/common/security-features/subresource/subresource.py new file mode 100644 index 0000000000..b3c055a93a --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/subresource.py @@ -0,0 +1,199 @@ +import os, json +from urllib.parse import parse_qsl, SplitResult, urlencode, urlsplit, urlunsplit + +from wptserve.utils import isomorphic_decode, isomorphic_encode + +def get_template(template_basename): + script_directory = os.path.dirname(os.path.abspath(isomorphic_decode(__file__))) + template_directory = os.path.abspath(os.path.join(script_directory, + u"template")) + template_filename = os.path.join(template_directory, template_basename) + + with open(template_filename, "r") as f: + return f.read() + + +def redirect(url, response): + response.add_required_headers = False + response.writer.write_status(301) + response.writer.write_header(b"access-control-allow-origin", b"*") + response.writer.write_header(b"location", isomorphic_encode(url)) + response.writer.end_headers() + response.writer.write(u"") + + +# TODO(kristijanburnik): subdomain_prefix is a hardcoded value aligned with +# referrer-policy-test-case.js. The prefix should be configured in one place. +def __get_swapped_origin_netloc(netloc, subdomain_prefix = u"www1."): + if netloc.startswith(subdomain_prefix): + return netloc[len(subdomain_prefix):] + else: + return subdomain_prefix + netloc + + +# Creates a URL (typically a redirect target URL) that is the same as the +# current request URL `request.url`, except for: +# - When `swap_scheme` or `swap_origin` is True, its scheme/origin is changed +# to the other one. (http <-> https, ws <-> wss, etc.) +# - For `downgrade`, we redirect to a URL that would be successfully loaded +# if and only if upgrade-insecure-request is applied. +# - `query_parameter_to_remove` parameter is removed from query part. +# Its default is "redirection" to avoid redirect loops. +def create_url(request, + swap_scheme=False, + swap_origin=False, + downgrade=False, + query_parameter_to_remove=u"redirection"): + parsed = urlsplit(request.url) + destination_netloc = parsed.netloc + + scheme = parsed.scheme + if swap_scheme: + scheme = u"http" if parsed.scheme == u"https" else u"https" + hostname = parsed.netloc.split(u':')[0] + port = request.server.config[u"ports"][scheme][0] + destination_netloc = u":".join([hostname, str(port)]) + + if downgrade: + # These rely on some unintuitive cleverness due to WPT's test setup: + # 'Upgrade-Insecure-Requests' does not upgrade the port number, + # so we use URLs in the form `http://[domain]:[https-port]`, + # which will be upgraded to `https://[domain]:[https-port]`. + # If the upgrade fails, the load will fail, as we don't serve HTTP over + # the secure port. + if parsed.scheme == u"https": + scheme = u"http" + elif parsed.scheme == u"wss": + scheme = u"ws" + else: + raise ValueError(u"Downgrade redirection: Invalid scheme '%s'" % + parsed.scheme) + hostname = parsed.netloc.split(u':')[0] + port = request.server.config[u"ports"][parsed.scheme][0] + destination_netloc = u":".join([hostname, str(port)]) + + if swap_origin: + destination_netloc = __get_swapped_origin_netloc(destination_netloc) + + parsed_query = parse_qsl(parsed.query, keep_blank_values=True) + parsed_query = [x for x in parsed_query if x[0] != query_parameter_to_remove] + + destination_url = urlunsplit(SplitResult( + scheme = scheme, + netloc = destination_netloc, + path = parsed.path, + query = urlencode(parsed_query), + fragment = None)) + + return destination_url + + +def preprocess_redirection(request, response): + if b"redirection" not in request.GET: + return False + + redirection = request.GET[b"redirection"] + + if redirection == b"no-redirect": + return False + elif redirection == b"keep-scheme": + redirect_url = create_url(request, swap_scheme=False) + elif redirection == b"swap-scheme": + redirect_url = create_url(request, swap_scheme=True) + elif redirection == b"downgrade": + redirect_url = create_url(request, downgrade=True) + elif redirection == b"keep-origin": + redirect_url = create_url(request, swap_origin=False) + elif redirection == b"swap-origin": + redirect_url = create_url(request, swap_origin=True) + else: + raise ValueError(u"Invalid redirection type '%s'" % isomorphic_decode(redirection)) + + redirect(redirect_url, response) + return True + + +def preprocess_stash_action(request, response): + if b"action" not in request.GET: + return False + + action = request.GET[b"action"] + + key = request.GET[b"key"] + stash = request.server.stash + path = request.GET[b"path"] if b"path" in request.GET \ + else isomorphic_encode(request.url.split(u'?')[0]) + + if action == b"put": + value = isomorphic_decode(request.GET[b"value"]) + stash.take(key=key, path=path) + stash.put(key=key, value=value, path=path) + response_data = json.dumps({u"status": u"success", u"result": isomorphic_decode(key)}) + elif action == b"purge": + value = stash.take(key=key, path=path) + return False + elif action == b"take": + value = stash.take(key=key, path=path) + if value is None: + status = u"allowed" + else: + status = u"blocked" + response_data = json.dumps({u"status": status, u"result": value}) + else: + return False + + response.add_required_headers = False + response.writer.write_status(200) + response.writer.write_header(b"content-type", b"text/javascript") + response.writer.write_header(b"cache-control", b"no-cache; must-revalidate") + response.writer.end_headers() + response.writer.write(response_data) + return True + + +def __noop(request, response): + return u"" + + +def respond(request, + response, + status_code = 200, + content_type = b"text/html", + payload_generator = __noop, + cache_control = b"no-cache; must-revalidate", + access_control_allow_origin = b"*", + maybe_additional_headers = None): + if preprocess_redirection(request, response): + return + + if preprocess_stash_action(request, response): + return + + response.add_required_headers = False + response.writer.write_status(status_code) + + if access_control_allow_origin != None: + response.writer.write_header(b"access-control-allow-origin", + access_control_allow_origin) + response.writer.write_header(b"content-type", content_type) + response.writer.write_header(b"cache-control", cache_control) + + additional_headers = maybe_additional_headers or {} + for header, value in additional_headers.items(): + response.writer.write_header(header, value) + + response.writer.end_headers() + + new_headers = {} + new_val = [] + for key, val in request.headers.items(): + if len(val) == 1: + new_val = isomorphic_decode(val[0]) + else: + new_val = [isomorphic_decode(x) for x in val] + new_headers[isomorphic_decode(key)] = new_val + + server_data = {u"headers": json.dumps(new_headers, indent = 4)} + + payload = payload_generator(server_data) + response.writer.write(payload) diff --git a/testing/web-platform/tests/common/security-features/subresource/svg.py b/testing/web-platform/tests/common/security-features/subresource/svg.py new file mode 100644 index 0000000000..9c569e3bf5 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/svg.py @@ -0,0 +1,37 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(request, server_data): + data = (u'{"headers": %(headers)s}') % server_data + if b"id" in request.GET: + with request.server.stash.lock: + request.server.stash.take(request.GET[b"id"]) + request.server.stash.put(request.GET[b"id"], data) + return u"<svg xmlns='http://www.w3.org/2000/svg'></svg>" + +def generate_payload_embedded(request, server_data): + return subresource.get_template(u"svg.embedded.template") % { + u"id": isomorphic_decode(request.GET[b"id"]), + u"property": isomorphic_decode(request.GET[b"property"])} + +def generate_report_headers_payload(request, server_data): + stashed_data = request.server.stash.take(request.GET[b"id"]) + return stashed_data + +def main(request, response): + handler = lambda data: generate_payload(request, data) + content_type = b'image/svg+xml' + + if b"embedded-svg" in request.GET: + handler = lambda data: generate_payload_embedded(request, data) + + if b"report-headers" in request.GET: + handler = lambda data: generate_report_headers_payload(request, data) + content_type = b'application/json' + + subresource.respond(request, + response, + payload_generator = handler, + content_type = content_type) diff --git a/testing/web-platform/tests/common/security-features/subresource/template/document.html.template b/testing/web-platform/tests/common/security-features/subresource/template/document.html.template new file mode 100644 index 0000000000..141711c148 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/document.html.template @@ -0,0 +1,16 @@ +<!DOCTYPE html> +<html> + <head> + <title>This page reports back it's request details to the parent frame</title> + </head> + <body> + <script> + var result = { + location: document.location.toString(), + referrer: document.referrer.length > 0 ? document.referrer : undefined, + headers: %(headers)s + }; + parent.postMessage(result, "*"); + </script> + </body> +</html> diff --git a/testing/web-platform/tests/common/security-features/subresource/template/font.css.template b/testing/web-platform/tests/common/security-features/subresource/template/font.css.template new file mode 100644 index 0000000000..9d1e9c421c --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/font.css.template @@ -0,0 +1,9 @@ +@font-face { + font-family: 'wpt'; + font-style: normal; + font-weight: normal; + src: url(/common/security-features/subresource/font.py?id=%(id)s) format('truetype'); +} +body { + font-family: 'wpt'; +} diff --git a/testing/web-platform/tests/common/security-features/subresource/template/image.css.template b/testing/web-platform/tests/common/security-features/subresource/template/image.css.template new file mode 100644 index 0000000000..dfe41f1bf1 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/image.css.template @@ -0,0 +1,3 @@ +div.styled::before { + content:url(/common/security-features/subresource/image.py?id=%(id)s) +} diff --git a/testing/web-platform/tests/common/security-features/subresource/template/script.js.template b/testing/web-platform/tests/common/security-features/subresource/template/script.js.template new file mode 100644 index 0000000000..e2edf21819 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/script.js.template @@ -0,0 +1,3 @@ +postMessage({ + "headers": %(headers)s +}, "*"); diff --git a/testing/web-platform/tests/common/security-features/subresource/template/shared-worker.js.template b/testing/web-platform/tests/common/security-features/subresource/template/shared-worker.js.template new file mode 100644 index 0000000000..c3f109e4a9 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/shared-worker.js.template @@ -0,0 +1,5 @@ +onconnect = function(e) { + e.ports[0].postMessage({ + "headers": %(headers)s + }); +}; diff --git a/testing/web-platform/tests/common/security-features/subresource/template/static-import.js.template b/testing/web-platform/tests/common/security-features/subresource/template/static-import.js.template new file mode 100644 index 0000000000..095459b547 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/static-import.js.template @@ -0,0 +1 @@ +import '%(import_url)s'; diff --git a/testing/web-platform/tests/common/security-features/subresource/template/svg.css.template b/testing/web-platform/tests/common/security-features/subresource/template/svg.css.template new file mode 100644 index 0000000000..c2e509cc3b --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/svg.css.template @@ -0,0 +1,3 @@ +path { + %(property)s: url(/common/security-features/subresource/svg.py?id=%(id)s#invalidFragment); +} diff --git a/testing/web-platform/tests/common/security-features/subresource/template/svg.embedded.template b/testing/web-platform/tests/common/security-features/subresource/template/svg.embedded.template new file mode 100644 index 0000000000..5986c4800a --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/svg.embedded.template @@ -0,0 +1,5 @@ +<?xml version='1.0' standalone='no'?> +<?xml-stylesheet href='stylesheet.py?id=%(id)s&type=svg&property=%(property)s' type='text/css'?> +<svg xmlns='http://www.w3.org/2000/svg'> + <path d='M 50,5 95,100 5,100 z' /> +</svg> diff --git a/testing/web-platform/tests/common/security-features/subresource/template/worker.js.template b/testing/web-platform/tests/common/security-features/subresource/template/worker.js.template new file mode 100644 index 0000000000..817dd8c87a --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/template/worker.js.template @@ -0,0 +1,3 @@ +postMessage({ + "headers": %(headers)s +}); diff --git a/testing/web-platform/tests/common/security-features/subresource/video.py b/testing/web-platform/tests/common/security-features/subresource/video.py new file mode 100644 index 0000000000..9db8e9fbb5 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/video.py @@ -0,0 +1,17 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(request, server_data): + file = os.path.join(request.doc_root, u"media", u"movie_5.webm") + return open(file, "rb").read() + + +def main(request, response): + handler = lambda data: generate_payload(request, data) + subresource.respond(request, + response, + payload_generator = handler, + access_control_allow_origin = b"*", + content_type = b"video/webm") diff --git a/testing/web-platform/tests/common/security-features/subresource/worker.py b/testing/web-platform/tests/common/security-features/subresource/worker.py new file mode 100644 index 0000000000..f655633b5d --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/worker.py @@ -0,0 +1,13 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + return subresource.get_template(u"worker.js.template") % server_data + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload, + content_type = b"application/javascript") diff --git a/testing/web-platform/tests/common/security-features/subresource/xhr.py b/testing/web-platform/tests/common/security-features/subresource/xhr.py new file mode 100644 index 0000000000..75921e9156 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/subresource/xhr.py @@ -0,0 +1,16 @@ +import os, sys +from wptserve.utils import isomorphic_decode +import importlib +subresource = importlib.import_module("common.security-features.subresource.subresource") + +def generate_payload(server_data): + data = (u'{"headers": %(headers)s}') % server_data + return data + +def main(request, response): + subresource.respond(request, + response, + payload_generator = generate_payload, + access_control_allow_origin = b"*", + content_type = b"application/json", + cache_control = b"no-store") diff --git a/testing/web-platform/tests/common/security-features/tools/format_spec_src_json.py b/testing/web-platform/tests/common/security-features/tools/format_spec_src_json.py new file mode 100644 index 0000000000..d1bf5817ad --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/format_spec_src_json.py @@ -0,0 +1,24 @@ +import collections +import json +import os + + +def main(): + '''Formats spec.src.json.''' + script_directory = os.path.dirname(os.path.abspath(__file__)) + for dir in [ + 'mixed-content', 'referrer-policy', 'referrer-policy/4K-1', + 'referrer-policy/4K', 'referrer-policy/4K+1', + 'upgrade-insecure-requests' + ]: + filename = os.path.join(script_directory, '..', '..', '..', dir, + 'spec.src.json') + spec = json.load( + open(filename, 'r'), object_pairs_hook=collections.OrderedDict) + with open(filename, 'w') as f: + f.write(json.dumps(spec, indent=2, separators=(',', ': '))) + f.write('\n') + + +if __name__ == '__main__': + main() diff --git a/testing/web-platform/tests/common/security-features/tools/generate.py b/testing/web-platform/tests/common/security-features/tools/generate.py new file mode 100755 index 0000000000..d1cd33147f --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/generate.py @@ -0,0 +1,462 @@ +#!/usr/bin/env python3 + +import argparse +import collections +import copy +import json +import os +import sys + +import spec_validator +import util + + +def expand_pattern(expansion_pattern, test_expansion_schema): + expansion = {} + for artifact_key in expansion_pattern: + artifact_value = expansion_pattern[artifact_key] + if artifact_value == '*': + expansion[artifact_key] = test_expansion_schema[artifact_key] + elif isinstance(artifact_value, list): + expansion[artifact_key] = artifact_value + elif isinstance(artifact_value, dict): + # Flattened expansion. + expansion[artifact_key] = [] + values_dict = expand_pattern(artifact_value, + test_expansion_schema[artifact_key]) + for sub_key in values_dict.keys(): + expansion[artifact_key] += values_dict[sub_key] + else: + expansion[artifact_key] = [artifact_value] + + return expansion + + +def permute_expansion(expansion, + artifact_order, + selection={}, + artifact_index=0): + assert isinstance(artifact_order, list), "artifact_order should be a list" + + if artifact_index >= len(artifact_order): + yield selection + return + + artifact_key = artifact_order[artifact_index] + + for artifact_value in expansion[artifact_key]: + selection[artifact_key] = artifact_value + for next_selection in permute_expansion(expansion, artifact_order, + selection, artifact_index + 1): + yield next_selection + + +# Dumps the test config `selection` into a serialized JSON string. +def dump_test_parameters(selection): + return json.dumps( + selection, + indent=2, + separators=(',', ': '), + sort_keys=True, + cls=util.CustomEncoder) + + +def get_test_filename(spec_directory, spec_json, selection): + '''Returns the filname for the main test HTML file''' + + selection_for_filename = copy.deepcopy(selection) + # Use 'unset' rather than 'None' in test filenames. + if selection_for_filename['delivery_value'] is None: + selection_for_filename['delivery_value'] = 'unset' + + return os.path.join( + spec_directory, + spec_json['test_file_path_pattern'] % selection_for_filename) + + +def get_csp_value(value): + ''' + Returns actual CSP header values (e.g. "worker-src 'self'") for the + given string used in PolicyDelivery's value (e.g. "worker-src-self"). + ''' + + # script-src + # Test-related scripts like testharness.js and inline scripts containing + # test bodies. + # 'unsafe-inline' is added as a workaround here. This is probably not so + # bad, as it shouldn't intefere non-inline-script requests that we want to + # test. + if value == 'script-src-wildcard': + return "script-src * 'unsafe-inline'" + if value == 'script-src-self': + return "script-src 'self' 'unsafe-inline'" + # Workaround for "script-src 'none'" would be more complicated, because + # - "script-src 'none' 'unsafe-inline'" is handled somehow differently from + # "script-src 'none'", i.e. + # https://w3c.github.io/webappsec-csp/#match-url-to-source-list Step 3 + # handles the latter but not the former. + # - We need nonce- or path-based additional values to allow same-origin + # test scripts like testharness.js. + # Therefore, we disable 'script-src-none' tests for now in + # `/content-security-policy/spec.src.json`. + if value == 'script-src-none': + return "script-src 'none'" + + # worker-src + if value == 'worker-src-wildcard': + return 'worker-src *' + if value == 'worker-src-self': + return "worker-src 'self'" + if value == 'worker-src-none': + return "worker-src 'none'" + raise Exception('Invalid delivery_value: %s' % value) + +def handle_deliveries(policy_deliveries): + ''' + Generate <meta> elements and HTTP headers for the given list of + PolicyDelivery. + TODO(hiroshige): Merge duplicated code here, scope/document.py, etc. + ''' + + meta = '' + headers = {} + + for delivery in policy_deliveries: + if delivery.value is None: + continue + if delivery.key == 'referrerPolicy': + if delivery.delivery_type == 'meta': + meta += \ + '<meta name="referrer" content="%s">' % delivery.value + elif delivery.delivery_type == 'http-rp': + headers['Referrer-Policy'] = delivery.value + # TODO(kristijanburnik): Limit to WPT origins. + headers['Access-Control-Allow-Origin'] = '*' + else: + raise Exception( + 'Invalid delivery_type: %s' % delivery.delivery_type) + elif delivery.key == 'mixedContent': + assert (delivery.value == 'opt-in') + if delivery.delivery_type == 'meta': + meta += '<meta http-equiv="Content-Security-Policy" ' + \ + 'content="block-all-mixed-content">' + elif delivery.delivery_type == 'http-rp': + headers['Content-Security-Policy'] = 'block-all-mixed-content' + else: + raise Exception( + 'Invalid delivery_type: %s' % delivery.delivery_type) + elif delivery.key == 'contentSecurityPolicy': + csp_value = get_csp_value(delivery.value) + if delivery.delivery_type == 'meta': + meta += '<meta http-equiv="Content-Security-Policy" ' + \ + 'content="' + csp_value + '">' + elif delivery.delivery_type == 'http-rp': + headers['Content-Security-Policy'] = csp_value + else: + raise Exception( + 'Invalid delivery_type: %s' % delivery.delivery_type) + elif delivery.key == 'upgradeInsecureRequests': + # https://w3c.github.io/webappsec-upgrade-insecure-requests/#delivery + assert (delivery.value == 'upgrade') + if delivery.delivery_type == 'meta': + meta += '<meta http-equiv="Content-Security-Policy" ' + \ + 'content="upgrade-insecure-requests">' + elif delivery.delivery_type == 'http-rp': + headers[ + 'Content-Security-Policy'] = 'upgrade-insecure-requests' + else: + raise Exception( + 'Invalid delivery_type: %s' % delivery.delivery_type) + else: + raise Exception('Invalid delivery_key: %s' % delivery.key) + return {"meta": meta, "headers": headers} + + +def generate_selection(spec_json, selection): + ''' + Returns a scenario object (with a top-level source_context_list entry, + which will be removed in generate_test_file() later). + ''' + + target_policy_delivery = util.PolicyDelivery(selection['delivery_type'], + selection['delivery_key'], + selection['delivery_value']) + del selection['delivery_type'] + del selection['delivery_key'] + del selection['delivery_value'] + + # Parse source context list and policy deliveries of source contexts. + # `util.ShouldSkip()` exceptions are raised if e.g. unsuppported + # combinations of source contexts and policy deliveries are used. + source_context_list_scheme = spec_json['source_context_list_schema'][ + selection['source_context_list']] + selection['source_context_list'] = [ + util.SourceContext.from_json(source_context, target_policy_delivery, + spec_json['source_context_schema']) + for source_context in source_context_list_scheme['sourceContextList'] + ] + + # Check if the subresource is supported by the innermost source context. + innermost_source_context = selection['source_context_list'][-1] + supported_subresource = spec_json['source_context_schema'][ + 'supported_subresource'][innermost_source_context.source_context_type] + if supported_subresource != '*': + if selection['subresource'] not in supported_subresource: + raise util.ShouldSkip() + + # Parse subresource policy deliveries. + selection[ + 'subresource_policy_deliveries'] = util.PolicyDelivery.list_from_json( + source_context_list_scheme['subresourcePolicyDeliveries'], + target_policy_delivery, spec_json['subresource_schema'] + ['supported_delivery_type'][selection['subresource']]) + + # Generate per-scenario test description. + selection['test_description'] = spec_json[ + 'test_description_template'] % selection + + return selection + + +def generate_test_file(spec_directory, test_helper_filenames, + test_html_template_basename, test_filename, scenarios): + ''' + Generates a test HTML file (and possibly its associated .headers file) + from `scenarios`. + ''' + + # Scenarios for the same file should have the same `source_context_list`, + # including the top-level one. + # Note: currently, non-top-level source contexts aren't necessarily required + # to be the same, but we set this requirement as it will be useful e.g. when + # we e.g. reuse a worker among multiple scenarios. + for scenario in scenarios: + assert (scenario['source_context_list'] == scenarios[0] + ['source_context_list']) + + # We process the top source context below, and do not include it in + # the JSON objects (i.e. `scenarios`) in generated HTML files. + top_source_context = scenarios[0]['source_context_list'].pop(0) + assert (top_source_context.source_context_type == 'top') + for scenario in scenarios[1:]: + assert (scenario['source_context_list'].pop(0) == top_source_context) + + parameters = {} + + # Sort scenarios, to avoid unnecessary diffs due to different orders in + # `scenarios`. + serialized_scenarios = sorted( + [dump_test_parameters(scenario) for scenario in scenarios]) + + parameters['scenarios'] = ",\n".join(serialized_scenarios).replace( + "\n", "\n" + " " * 10) + + test_directory = os.path.dirname(test_filename) + + parameters['helper_js'] = "" + for test_helper_filename in test_helper_filenames: + parameters['helper_js'] += ' <script src="%s"></script>\n' % ( + os.path.relpath(test_helper_filename, test_directory)) + parameters['sanity_checker_js'] = os.path.relpath( + os.path.join(spec_directory, 'generic', 'sanity-checker.js'), + test_directory) + parameters['spec_json_js'] = os.path.relpath( + os.path.join(spec_directory, 'generic', 'spec_json.js'), + test_directory) + + test_headers_filename = test_filename + ".headers" + + test_html_template = util.get_template(test_html_template_basename) + disclaimer_template = util.get_template('disclaimer.template') + + html_template_filename = os.path.join(util.template_directory, + test_html_template_basename) + generated_disclaimer = disclaimer_template \ + % {'generating_script_filename': os.path.relpath(sys.argv[0], + util.test_root_directory), + 'spec_directory': os.path.relpath(spec_directory, + util.test_root_directory)} + + # Adjust the template for the test invoking JS. Indent it to look nice. + parameters['generated_disclaimer'] = generated_disclaimer.rstrip() + + # Directory for the test files. + try: + os.makedirs(test_directory) + except: + pass + + delivery = handle_deliveries(top_source_context.policy_deliveries) + + if len(delivery['headers']) > 0: + with open(test_headers_filename, "w") as f: + for header in delivery['headers']: + f.write('%s: %s\n' % (header, delivery['headers'][header])) + + parameters['meta_delivery_method'] = delivery['meta'] + # Obey the lint and pretty format. + if len(parameters['meta_delivery_method']) > 0: + parameters['meta_delivery_method'] = "\n " + \ + parameters['meta_delivery_method'] + + # Write out the generated HTML file. + util.write_file(test_filename, test_html_template % parameters) + + +def generate_test_source_files(spec_directory, test_helper_filenames, + spec_json, target): + test_expansion_schema = spec_json['test_expansion_schema'] + specification = spec_json['specification'] + + if target == "debug": + spec_json_js_template = util.get_template('spec_json.js.template') + util.write_file( + os.path.join(spec_directory, "generic", "spec_json.js"), + spec_json_js_template % {'spec_json': json.dumps(spec_json)}) + util.write_file( + os.path.join(spec_directory, "generic", + "debug-output.spec.src.json"), + json.dumps(spec_json, indent=2, separators=(',', ': '))) + + # Choose a debug/release template depending on the target. + html_template = "test.%s.html.template" % target + + artifact_order = list(test_expansion_schema.keys()) + artifact_order.remove('expansion') + + excluded_selection_pattern = '' + for key in artifact_order: + excluded_selection_pattern += '%(' + key + ')s/' + + # Create list of excluded tests. + exclusion_dict = set() + for excluded_pattern in spec_json['excluded_tests']: + excluded_expansion = \ + expand_pattern(excluded_pattern, test_expansion_schema) + for excluded_selection in permute_expansion(excluded_expansion, + artifact_order): + excluded_selection['delivery_key'] = spec_json['delivery_key'] + exclusion_dict.add(excluded_selection_pattern % excluded_selection) + + # `scenarios[filename]` represents the list of scenario objects to be + # generated into `filename`. + scenarios = {} + + for spec in specification: + # Used to make entries with expansion="override" override preceding + # entries with the same |selection_path|. + output_dict = {} + + for expansion_pattern in spec['test_expansion']: + expansion = expand_pattern(expansion_pattern, + test_expansion_schema) + for selection in permute_expansion(expansion, artifact_order): + selection['delivery_key'] = spec_json['delivery_key'] + selection_path = spec_json['selection_pattern'] % selection + if selection_path in output_dict: + if expansion_pattern['expansion'] != 'override': + print("Error: expansion is default in:") + print(dump_test_parameters(selection)) + print("but overrides:") + print(dump_test_parameters( + output_dict[selection_path])) + sys.exit(1) + output_dict[selection_path] = copy.deepcopy(selection) + + for selection_path in output_dict: + selection = output_dict[selection_path] + if (excluded_selection_pattern % selection) in exclusion_dict: + print('Excluding selection:', selection_path) + continue + try: + test_filename = get_test_filename(spec_directory, spec_json, + selection) + scenario = generate_selection(spec_json, selection) + scenarios[test_filename] = scenarios.get(test_filename, + []) + [scenario] + except util.ShouldSkip: + continue + + for filename in scenarios: + generate_test_file(spec_directory, test_helper_filenames, + html_template, filename, scenarios[filename]) + + +def merge_json(base, child): + for key in child: + if key not in base: + base[key] = child[key] + continue + # `base[key]` and `child[key]` both exists. + if isinstance(base[key], list) and isinstance(child[key], list): + base[key].extend(child[key]) + elif isinstance(base[key], dict) and isinstance(child[key], dict): + merge_json(base[key], child[key]) + else: + base[key] = child[key] + + +def main(): + parser = argparse.ArgumentParser( + description='Test suite generator utility') + parser.add_argument( + '-t', + '--target', + type=str, + choices=("release", "debug"), + default="release", + help='Sets the appropriate template for generating tests') + parser.add_argument( + '-s', + '--spec', + type=str, + default=os.getcwd(), + help='Specify a file used for describing and generating the tests') + # TODO(kristijanburnik): Add option for the spec_json file. + args = parser.parse_args() + + spec_directory = os.path.abspath(args.spec) + + # Read `spec.src.json` files, starting from `spec_directory`, and + # continuing to parent directories as long as `spec.src.json` exists. + spec_filenames = [] + test_helper_filenames = [] + spec_src_directory = spec_directory + while len(spec_src_directory) >= len(util.test_root_directory): + spec_filename = os.path.join(spec_src_directory, "spec.src.json") + if not os.path.exists(spec_filename): + break + spec_filenames.append(spec_filename) + test_filename = os.path.join(spec_src_directory, 'generic', + 'test-case.sub.js') + assert (os.path.exists(test_filename)) + test_helper_filenames.append(test_filename) + spec_src_directory = os.path.abspath( + os.path.join(spec_src_directory, "..")) + + spec_filenames = list(reversed(spec_filenames)) + test_helper_filenames = list(reversed(test_helper_filenames)) + + if len(spec_filenames) == 0: + print('Error: No spec.src.json is found at %s.' % spec_directory) + return + + # Load the default spec JSON file, ... + default_spec_filename = os.path.join(util.script_directory, + 'spec.src.json') + spec_json = collections.OrderedDict() + if os.path.exists(default_spec_filename): + spec_json = util.load_spec_json(default_spec_filename) + + # ... and then make spec JSON files in subdirectories override the default. + for spec_filename in spec_filenames: + child_spec_json = util.load_spec_json(spec_filename) + merge_json(spec_json, child_spec_json) + + spec_validator.assert_valid_spec_json(spec_json) + generate_test_source_files(spec_directory, test_helper_filenames, + spec_json, args.target) + + +if __name__ == '__main__': + main() diff --git a/testing/web-platform/tests/common/security-features/tools/spec.src.json b/testing/web-platform/tests/common/security-features/tools/spec.src.json new file mode 100644 index 0000000000..4a84493f47 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/spec.src.json @@ -0,0 +1,533 @@ +{ + "selection_pattern": "%(source_context_list)s.%(delivery_type)s/%(delivery_value)s/%(subresource)s/%(origin)s.%(redirection)s.%(source_scheme)s", + "test_file_path_pattern": "gen/%(source_context_list)s.%(delivery_type)s/%(delivery_value)s/%(subresource)s.%(source_scheme)s.html", + "excluded_tests": [ + { + // Workers are same-origin only + "expansion": "*", + "source_scheme": "*", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": "*", + "subresource": [ + "worker-classic", + "worker-module", + "sharedworker-classic", + "sharedworker-module" + ], + "origin": [ + "cross-https", + "cross-http", + "cross-http-downgrade", + "cross-wss", + "cross-ws", + "cross-ws-downgrade" + ], + "expectation": "*" + }, + { + // Workers are same-origin only (redirects) + "expansion": "*", + "source_scheme": "*", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": [ + "swap-origin", + "swap-scheme" + ], + "subresource": [ + "worker-classic", + "worker-module", + "sharedworker-classic", + "sharedworker-module" + ], + "origin": "*", + "expectation": "*" + }, + { + // Websockets are ws/wss-only + "expansion": "*", + "source_scheme": "*", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": "*", + "subresource": "websocket", + "origin": [ + "same-https", + "same-http", + "same-http-downgrade", + "cross-https", + "cross-http", + "cross-http-downgrade" + ], + "expectation": "*" + }, + { + // Redirects are intentionally forbidden in browsers: + // https://fetch.spec.whatwg.org/#concept-websocket-establish + // Websockets are no-redirect only + "expansion": "*", + "source_scheme": "*", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": [ + "keep-origin", + "swap-origin", + "keep-scheme", + "swap-scheme", + "downgrade" + ], + "subresource": "websocket", + "origin": "*", + "expectation": "*" + }, + { + // ws/wss are websocket-only + "expansion": "*", + "source_scheme": "*", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": "*", + "subresource": [ + "a-tag", + "area-tag", + "audio-tag", + "beacon", + "fetch", + "iframe-tag", + "img-tag", + "link-css-tag", + "link-prefetch-tag", + "object-tag", + "picture-tag", + "script-tag", + "script-tag-dynamic-import", + "sharedworker-classic", + "sharedworker-import", + "sharedworker-import-data", + "sharedworker-module", + "video-tag", + "worker-classic", + "worker-import", + "worker-import-data", + "worker-module", + "worklet-animation", + "worklet-animation-import-data", + "worklet-audio", + "worklet-audio-import-data", + "worklet-layout", + "worklet-layout-import-data", + "worklet-paint", + "worklet-paint-import-data", + "xhr" + ], + "origin": [ + "same-wss", + "same-ws", + "same-ws-downgrade", + "cross-wss", + "cross-ws", + "cross-ws-downgrade" + ], + "expectation": "*" + }, + { + // Worklets are HTTPS contexts only + "expansion": "*", + "source_scheme": "http", + "source_context_list": "*", + "delivery_type": "*", + "delivery_value": "*", + "redirection": "*", + "subresource": [ + "worklet-animation", + "worklet-animation-import-data", + "worklet-audio", + "worklet-audio-import-data", + "worklet-layout", + "worklet-layout-import-data", + "worklet-paint", + "worklet-paint-import-data" + ], + "origin": "*", + "expectation": "*" + } + ], + "source_context_schema": { + "supported_subresource": { + "top": "*", + "iframe": "*", + "iframe-blank": "*", + "srcdoc": "*", + "worker-classic": [ + "xhr", + "fetch", + "websocket", + "worker-classic", + "worker-module" + ], + "worker-module": [ + "xhr", + "fetch", + "websocket", + "worker-classic", + "worker-module" + ], + "worker-classic-data": [ + "xhr", + "fetch", + "websocket" + ], + "worker-module-data": [ + "xhr", + "fetch", + "websocket" + ], + "sharedworker-classic": [ + "xhr", + "fetch", + "websocket" + ], + "sharedworker-module": [ + "xhr", + "fetch", + "websocket" + ], + "sharedworker-classic-data": [ + "xhr", + "fetch", + "websocket" + ], + "sharedworker-module-data": [ + "xhr", + "fetch", + "websocket" + ] + } + }, + "source_context_list_schema": { + // Warning: Currently, some nested patterns of contexts have different + // inheritance rules for different kinds of policies. + // The generated tests will be used to test/investigate the policy + // inheritance rules, and eventually the policy inheritance rules will + // be unified (https://github.com/w3ctag/design-principles/issues/111). + "top": { + "description": "Policy set by the top-level Document", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "req": { + "description": "Subresource request's policy should override Document's policy", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + } + ], + "subresourcePolicyDeliveries": [ + "nonNullPolicy" + ] + }, + "srcdoc-inherit": { + "description": "srcdoc iframe without its own policy should inherit parent Document's policy", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "srcdoc" + } + ], + "subresourcePolicyDeliveries": [] + }, + "srcdoc": { + "description": "srcdoc iframe's policy should override parent Document's policy", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "srcdoc", + "policyDeliveries": [ + "nonNullPolicy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "iframe": { + "description": "external iframe's policy should override parent Document's policy", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "iframe", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "iframe-blank-inherit": { + "description": "blank iframe should inherit parent Document's policy", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "iframe-blank" + } + ], + "subresourcePolicyDeliveries": [] + }, + "worker-classic": { + // This is applicable to referrer-policy tests. + // Use "worker-classic-inherit" for CSP (mixed-content, etc.). + "description": "dedicated workers shouldn't inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "worker-classic", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "worker-classic-data": { + "description": "data: dedicated workers should inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "worker-classic-data", + "policyDeliveries": [] + } + ], + "subresourcePolicyDeliveries": [] + }, + "worker-module": { + // This is applicable to referrer-policy tests. + "description": "dedicated workers shouldn't inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "worker-module", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "worker-module-data": { + "description": "data: dedicated workers should inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "worker-module-data", + "policyDeliveries": [] + } + ], + "subresourcePolicyDeliveries": [] + }, + "sharedworker-classic": { + "description": "shared workers shouldn't inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "sharedworker-classic", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "sharedworker-classic-data": { + "description": "data: shared workers should inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "sharedworker-classic-data", + "policyDeliveries": [] + } + ], + "subresourcePolicyDeliveries": [] + }, + "sharedworker-module": { + "description": "shared workers shouldn't inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "anotherPolicy" + ] + }, + { + "sourceContextType": "sharedworker-module", + "policyDeliveries": [ + "policy" + ] + } + ], + "subresourcePolicyDeliveries": [] + }, + "sharedworker-module-data": { + "description": "data: shared workers should inherit its parent's policy.", + "sourceContextList": [ + { + "sourceContextType": "top", + "policyDeliveries": [ + "policy" + ] + }, + { + "sourceContextType": "sharedworker-module-data", + "policyDeliveries": [] + } + ], + "subresourcePolicyDeliveries": [] + } + }, + "test_expansion_schema": { + "expansion": [ + "default", + "override" + ], + "source_scheme": [ + "http", + "https" + ], + "source_context_list": [ + "top", + "req", + "srcdoc-inherit", + "srcdoc", + "iframe", + "iframe-blank-inherit", + "worker-classic", + "worker-classic-data", + "worker-module", + "worker-module-data", + "sharedworker-classic", + "sharedworker-classic-data", + "sharedworker-module", + "sharedworker-module-data" + ], + "redirection": [ + "no-redirect", + "keep-origin", + "swap-origin", + "keep-scheme", + "swap-scheme", + "downgrade" + ], + "origin": [ + "same-https", + "same-http", + "same-http-downgrade", + "cross-https", + "cross-http", + "cross-http-downgrade", + "same-wss", + "same-ws", + "same-ws-downgrade", + "cross-wss", + "cross-ws", + "cross-ws-downgrade" + ], + "subresource": [ + "a-tag", + "area-tag", + "audio-tag", + "beacon", + "fetch", + "iframe-tag", + "img-tag", + "link-css-tag", + "link-prefetch-tag", + "object-tag", + "picture-tag", + "script-tag", + "script-tag-dynamic-import", + "sharedworker-classic", + "sharedworker-import", + "sharedworker-import-data", + "sharedworker-module", + "video-tag", + "websocket", + "worker-classic", + "worker-import", + "worker-import-data", + "worker-module", + "worklet-animation", + "worklet-animation-import-data", + "worklet-audio", + "worklet-audio-import-data", + "worklet-layout", + "worklet-layout-import-data", + "worklet-paint", + "worklet-paint-import-data", + "xhr" + ] + } +} diff --git a/testing/web-platform/tests/common/security-features/tools/spec_validator.py b/testing/web-platform/tests/common/security-features/tools/spec_validator.py new file mode 100755 index 0000000000..e4c9e145f8 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/spec_validator.py @@ -0,0 +1,251 @@ +#!/usr/bin/env python3 + +import json, sys + + +def assert_non_empty_string(obj, field): + assert field in obj, 'Missing field "%s"' % field + assert isinstance(obj[field], str), \ + 'Field "%s" must be a string' % field + assert len(obj[field]) > 0, 'Field "%s" must not be empty' % field + + +def assert_non_empty_list(obj, field): + assert isinstance(obj[field], list), \ + '%s must be a list' % field + assert len(obj[field]) > 0, \ + '%s list must not be empty' % field + + +def assert_non_empty_dict(obj, field): + assert isinstance(obj[field], dict), \ + '%s must be a dict' % field + assert len(obj[field]) > 0, \ + '%s dict must not be empty' % field + + +def assert_contains(obj, field): + assert field in obj, 'Must contain field "%s"' % field + + +def assert_value_from(obj, field, items): + assert obj[field] in items, \ + 'Field "%s" must be from: %s' % (field, str(items)) + + +def assert_atom_or_list_items_from(obj, field, items): + if isinstance(obj[field], str) or isinstance( + obj[field], int) or obj[field] is None: + assert_value_from(obj, field, items) + return + + assert isinstance(obj[field], list), '%s must be a list' % field + for allowed_value in obj[field]: + assert allowed_value != '*', "Wildcard is not supported for lists!" + assert allowed_value in items, \ + 'Field "%s" must be from: %s' % (field, str(items)) + + +def assert_contains_only_fields(obj, expected_fields): + for expected_field in expected_fields: + assert_contains(obj, expected_field) + + for actual_field in obj: + assert actual_field in expected_fields, \ + 'Unexpected field "%s".' % actual_field + + +def leaf_values(schema): + if isinstance(schema, list): + return schema + ret = [] + for _, sub_schema in schema.iteritems(): + ret += leaf_values(sub_schema) + return ret + + +def assert_value_unique_in(value, used_values): + assert value not in used_values, 'Duplicate value "%s"!' % str(value) + used_values[value] = True + + +def assert_valid_artifact(exp_pattern, artifact_key, schema): + if isinstance(schema, list): + assert_atom_or_list_items_from(exp_pattern, artifact_key, + ["*"] + schema) + return + + for sub_artifact_key, sub_schema in schema.iteritems(): + assert_valid_artifact(exp_pattern[artifact_key], sub_artifact_key, + sub_schema) + + +def validate(spec_json, details): + """ Validates the json specification for generating tests. """ + + details['object'] = spec_json + assert_contains_only_fields(spec_json, [ + "selection_pattern", "test_file_path_pattern", + "test_description_template", "test_page_title_template", + "specification", "delivery_key", "subresource_schema", + "source_context_schema", "source_context_list_schema", + "test_expansion_schema", "excluded_tests" + ]) + assert_non_empty_list(spec_json, "specification") + assert_non_empty_dict(spec_json, "test_expansion_schema") + assert_non_empty_list(spec_json, "excluded_tests") + + specification = spec_json['specification'] + test_expansion_schema = spec_json['test_expansion_schema'] + excluded_tests = spec_json['excluded_tests'] + + valid_test_expansion_fields = test_expansion_schema.keys() + + # Should be consistent with `sourceContextMap` in + # `/common/security-features/resources/common.sub.js`. + valid_source_context_names = [ + "top", "iframe", "iframe-blank", "srcdoc", "worker-classic", + "worker-module", "worker-classic-data", "worker-module-data", + "sharedworker-classic", "sharedworker-module", + "sharedworker-classic-data", "sharedworker-module-data" + ] + + valid_subresource_names = [ + "a-tag", "area-tag", "audio-tag", "form-tag", "iframe-tag", "img-tag", + "link-css-tag", "link-prefetch-tag", "object-tag", "picture-tag", + "script-tag", "script-tag-dynamic-import", "video-tag" + ] + ["beacon", "fetch", "xhr", "websocket"] + [ + "worker-classic", "worker-module", "worker-import", + "worker-import-data", "sharedworker-classic", "sharedworker-module", + "sharedworker-import", "sharedworker-import-data", + "serviceworker-classic", "serviceworker-module", + "serviceworker-import", "serviceworker-import-data" + ] + [ + "worklet-animation", "worklet-audio", "worklet-layout", + "worklet-paint", "worklet-animation-import", "worklet-audio-import", + "worklet-layout-import", "worklet-paint-import", + "worklet-animation-import-data", "worklet-audio-import-data", + "worklet-layout-import-data", "worklet-paint-import-data" + ] + + # Validate each single spec. + for spec in specification: + details['object'] = spec + + # Validate required fields for a single spec. + assert_contains_only_fields(spec, [ + 'title', 'description', 'specification_url', 'test_expansion' + ]) + assert_non_empty_string(spec, 'title') + assert_non_empty_string(spec, 'description') + assert_non_empty_string(spec, 'specification_url') + assert_non_empty_list(spec, 'test_expansion') + + for spec_exp in spec['test_expansion']: + details['object'] = spec_exp + assert_contains_only_fields(spec_exp, valid_test_expansion_fields) + + for artifact in test_expansion_schema: + details['test_expansion_field'] = artifact + assert_valid_artifact(spec_exp, artifact, + test_expansion_schema[artifact]) + del details['test_expansion_field'] + + # Validate source_context_schema. + details['object'] = spec_json['source_context_schema'] + assert_contains_only_fields( + spec_json['source_context_schema'], + ['supported_delivery_type', 'supported_subresource']) + assert_contains_only_fields( + spec_json['source_context_schema']['supported_delivery_type'], + valid_source_context_names) + for source_context in spec_json['source_context_schema'][ + 'supported_delivery_type']: + assert_valid_artifact( + spec_json['source_context_schema']['supported_delivery_type'], + source_context, test_expansion_schema['delivery_type']) + assert_contains_only_fields( + spec_json['source_context_schema']['supported_subresource'], + valid_source_context_names) + for source_context in spec_json['source_context_schema'][ + 'supported_subresource']: + assert_valid_artifact( + spec_json['source_context_schema']['supported_subresource'], + source_context, leaf_values(test_expansion_schema['subresource'])) + + # Validate subresource_schema. + details['object'] = spec_json['subresource_schema'] + assert_contains_only_fields(spec_json['subresource_schema'], + ['supported_delivery_type']) + assert_contains_only_fields( + spec_json['subresource_schema']['supported_delivery_type'], + leaf_values(test_expansion_schema['subresource'])) + for subresource in spec_json['subresource_schema'][ + 'supported_delivery_type']: + assert_valid_artifact( + spec_json['subresource_schema']['supported_delivery_type'], + subresource, test_expansion_schema['delivery_type']) + + # Validate the test_expansion schema members. + details['object'] = test_expansion_schema + assert_contains_only_fields(test_expansion_schema, [ + 'expansion', 'source_scheme', 'source_context_list', 'delivery_type', + 'delivery_value', 'redirection', 'subresource', 'origin', 'expectation' + ]) + assert_atom_or_list_items_from(test_expansion_schema, 'expansion', + ['default', 'override']) + assert_atom_or_list_items_from(test_expansion_schema, 'source_scheme', + ['http', 'https']) + assert_atom_or_list_items_from( + test_expansion_schema, 'source_context_list', + spec_json['source_context_list_schema'].keys()) + + # Should be consistent with `preprocess_redirection` in + # `/common/security-features/subresource/subresource.py`. + assert_atom_or_list_items_from(test_expansion_schema, 'redirection', [ + 'no-redirect', 'keep-origin', 'swap-origin', 'keep-scheme', + 'swap-scheme', 'downgrade' + ]) + for subresource in leaf_values(test_expansion_schema['subresource']): + assert subresource in valid_subresource_names, "Invalid subresource %s" % subresource + # Should be consistent with getSubresourceOrigin() in + # `/common/security-features/resources/common.sub.js`. + assert_atom_or_list_items_from(test_expansion_schema, 'origin', [ + 'same-http', 'same-https', 'same-ws', 'same-wss', 'cross-http', + 'cross-https', 'cross-ws', 'cross-wss', 'same-http-downgrade', + 'cross-http-downgrade', 'same-ws-downgrade', 'cross-ws-downgrade' + ]) + + # Validate excluded tests. + details['object'] = excluded_tests + for excluded_test_expansion in excluded_tests: + assert_contains_only_fields(excluded_test_expansion, + valid_test_expansion_fields) + details['object'] = excluded_test_expansion + for artifact in test_expansion_schema: + details['test_expansion_field'] = artifact + assert_valid_artifact(excluded_test_expansion, artifact, + test_expansion_schema[artifact]) + del details['test_expansion_field'] + + del details['object'] + + +def assert_valid_spec_json(spec_json): + error_details = {} + try: + validate(spec_json, error_details) + except AssertionError as err: + print('ERROR:', err) + print(json.dumps(error_details, indent=4)) + sys.exit(1) + + +def main(): + spec_json = load_spec_json() + assert_valid_spec_json(spec_json) + print("Spec JSON is valid.") + + +if __name__ == '__main__': + main() diff --git a/testing/web-platform/tests/common/security-features/tools/template/disclaimer.template b/testing/web-platform/tests/common/security-features/tools/template/disclaimer.template new file mode 100644 index 0000000000..ba9458cb31 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/template/disclaimer.template @@ -0,0 +1 @@ +<!-- DO NOT EDIT! Generated by `%(generating_script_filename)s --spec %(spec_directory)s/` --> diff --git a/testing/web-platform/tests/common/security-features/tools/template/spec_json.js.template b/testing/web-platform/tests/common/security-features/tools/template/spec_json.js.template new file mode 100644 index 0000000000..e4cbd03425 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/template/spec_json.js.template @@ -0,0 +1 @@ +var SPEC_JSON = %(spec_json)s; diff --git a/testing/web-platform/tests/common/security-features/tools/template/test.debug.html.template b/testing/web-platform/tests/common/security-features/tools/template/test.debug.html.template new file mode 100644 index 0000000000..b6be088f61 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/template/test.debug.html.template @@ -0,0 +1,26 @@ +<!DOCTYPE html> +%(generated_disclaimer)s +<html> + <head> + <meta charset="utf-8"> + <meta name="timeout" content="long">%(meta_delivery_method)s + <script src="/resources/testharness.js"></script> + <script src="/resources/testharnessreport.js"></script> + <script src="/common/security-features/resources/common.sub.js"></script> + <!-- The original specification JSON for validating the scenario. --> + <script src="%(spec_json_js)s"></script> + <!-- Internal checking of the tests --> + <script src="%(sanity_checker_js)s"></script> +%(helper_js)s </head> + <body> + <script> + TestCase( + [ + %(scenarios)s + ], + new SanityChecker() + ).start(); + </script> + <div id="log"></div> + </body> +</html> diff --git a/testing/web-platform/tests/common/security-features/tools/template/test.release.html.template b/testing/web-platform/tests/common/security-features/tools/template/test.release.html.template new file mode 100644 index 0000000000..bac2d5b5a4 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/template/test.release.html.template @@ -0,0 +1,22 @@ +<!DOCTYPE html> +%(generated_disclaimer)s +<html> + <head> + <meta charset="utf-8"> + <meta name="timeout" content="long">%(meta_delivery_method)s + <script src="/resources/testharness.js"></script> + <script src="/resources/testharnessreport.js"></script> + <script src="/common/security-features/resources/common.sub.js"></script> +%(helper_js)s </head> + <body> + <script> + TestCase( + [ + %(scenarios)s + ], + new SanityChecker() + ).start(); + </script> + <div id="log"></div> + </body> +</html> diff --git a/testing/web-platform/tests/common/security-features/tools/util.py b/testing/web-platform/tests/common/security-features/tools/util.py new file mode 100644 index 0000000000..5da06f9d51 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/tools/util.py @@ -0,0 +1,228 @@ +import os, sys, json, json5, re +import collections + +script_directory = os.path.dirname(os.path.abspath(__file__)) +template_directory = os.path.abspath( + os.path.join(script_directory, 'template')) +test_root_directory = os.path.abspath( + os.path.join(script_directory, '..', '..', '..')) + + +def get_template(basename): + with open(os.path.join(template_directory, basename), "r") as f: + return f.read() + + +def write_file(filename, contents): + with open(filename, "w") as f: + f.write(contents) + + +def read_nth_line(fp, line_number): + fp.seek(0) + for i, line in enumerate(fp): + if (i + 1) == line_number: + return line + + +def load_spec_json(path_to_spec): + re_error_location = re.compile('line ([0-9]+) column ([0-9]+)') + with open(path_to_spec, "r") as f: + try: + return json5.load(f, object_pairs_hook=collections.OrderedDict) + except ValueError as ex: + print(ex.message) + match = re_error_location.search(ex.message) + if match: + line_number, column = int(match.group(1)), int(match.group(2)) + print(read_nth_line(f, line_number).rstrip()) + print(" " * (column - 1) + "^") + sys.exit(1) + + +class ShouldSkip(Exception): + ''' + Raised when the given combination of subresource type, source context type, + delivery type etc. are not supported and we should skip that configuration. + ShouldSkip is expected in normal generator execution (and thus subsequent + generation continues), as we first enumerate a broad range of configurations + first, and later raise ShouldSkip to filter out unsupported combinations. + + ShouldSkip is distinguished from other general errors that cause immediate + termination of the generator and require fix. + ''' + def __init__(self): + pass + + +class PolicyDelivery(object): + ''' + See `@typedef PolicyDelivery` comments in + `common/security-features/resources/common.sub.js`. + ''' + + def __init__(self, delivery_type, key, value): + self.delivery_type = delivery_type + self.key = key + self.value = value + + def __eq__(self, other): + return type(self) is type(other) and self.__dict__ == other.__dict__ + + @classmethod + def list_from_json(cls, list, target_policy_delivery, + supported_delivery_types): + # type: (dict, PolicyDelivery, typing.List[str]) -> typing.List[PolicyDelivery] + ''' + Parses a JSON object `list` that represents a list of `PolicyDelivery` + and returns a list of `PolicyDelivery`, plus supporting placeholders + (see `from_json()` comments below or + `common/security-features/README.md`). + + Can raise `ShouldSkip`. + ''' + if list is None: + return [] + + out = [] + for obj in list: + policy_delivery = PolicyDelivery.from_json( + obj, target_policy_delivery, supported_delivery_types) + # Drop entries with null values. + if policy_delivery.value is None: + continue + out.append(policy_delivery) + return out + + @classmethod + def from_json(cls, obj, target_policy_delivery, supported_delivery_types): + # type: (dict, PolicyDelivery, typing.List[str]) -> PolicyDelivery + ''' + Parses a JSON object `obj` and returns a `PolicyDelivery` object. + In addition to dicts (in the same format as to_json() outputs), + this method accepts the following placeholders: + "policy": + `target_policy_delivery` + "policyIfNonNull": + `target_policy_delivery` if its value is not None. + "anotherPolicy": + A PolicyDelivery that has the same key as + `target_policy_delivery` but a different value. + The delivery type is selected from `supported_delivery_types`. + + Can raise `ShouldSkip`. + ''' + + if obj == "policy": + policy_delivery = target_policy_delivery + elif obj == "nonNullPolicy": + if target_policy_delivery.value is None: + raise ShouldSkip() + policy_delivery = target_policy_delivery + elif obj == "anotherPolicy": + if len(supported_delivery_types) == 0: + raise ShouldSkip() + policy_delivery = target_policy_delivery.get_another_policy( + supported_delivery_types[0]) + elif isinstance(obj, dict): + policy_delivery = PolicyDelivery(obj['deliveryType'], obj['key'], + obj['value']) + else: + raise Exception('policy delivery is invalid: ' + obj) + + # Omit unsupported combinations of source contexts and delivery type. + if policy_delivery.delivery_type not in supported_delivery_types: + raise ShouldSkip() + + return policy_delivery + + def to_json(self): + # type: () -> dict + return { + "deliveryType": self.delivery_type, + "key": self.key, + "value": self.value + } + + def get_another_policy(self, delivery_type): + # type: (str) -> PolicyDelivery + if self.key == 'referrerPolicy': + # Return 'unsafe-url' (i.e. more unsafe policy than `self.value`) + # as long as possible, to make sure the tests to fail if the + # returned policy is used unexpectedly instead of `self.value`. + # Using safer policy wouldn't be distinguishable from acceptable + # arbitrary policy enforcement by user agents, as specified at + # Step 7 of + # https://w3c.github.io/webappsec-referrer-policy/#determine-requests-referrer: + # "The user agent MAY alter referrerURL or referrerOrigin at this + # point to enforce arbitrary policy considerations in the + # interests of minimizing data leakage." + # See also the comments at `referrerUrlResolver` in + # `wpt/referrer-policy/generic/test-case.sub.js`. + if self.value != 'unsafe-url': + return PolicyDelivery(delivery_type, self.key, 'unsafe-url') + else: + return PolicyDelivery(delivery_type, self.key, 'no-referrer') + elif self.key == 'mixedContent': + if self.value == 'opt-in': + return PolicyDelivery(delivery_type, self.key, None) + else: + return PolicyDelivery(delivery_type, self.key, 'opt-in') + elif self.key == 'contentSecurityPolicy': + if self.value is not None: + return PolicyDelivery(delivery_type, self.key, None) + else: + return PolicyDelivery(delivery_type, self.key, 'worker-src-none') + elif self.key == 'upgradeInsecureRequests': + if self.value == 'upgrade': + return PolicyDelivery(delivery_type, self.key, None) + else: + return PolicyDelivery(delivery_type, self.key, 'upgrade') + else: + raise Exception('delivery key is invalid: ' + self.key) + + +class SourceContext(object): + def __init__(self, source_context_type, policy_deliveries): + # type: (unicode, typing.List[PolicyDelivery]) -> None + self.source_context_type = source_context_type + self.policy_deliveries = policy_deliveries + + def __eq__(self, other): + return type(self) is type(other) and self.__dict__ == other.__dict__ + + @classmethod + def from_json(cls, obj, target_policy_delivery, source_context_schema): + ''' + Parses a JSON object `obj` and returns a `SourceContext` object. + + `target_policy_delivery` and `source_context_schema` are used for + policy delivery placeholders and filtering out unsupported + delivery types. + + Can raise `ShouldSkip`. + ''' + source_context_type = obj.get('sourceContextType') + policy_deliveries = PolicyDelivery.list_from_json( + obj.get('policyDeliveries'), target_policy_delivery, + source_context_schema['supported_delivery_type'] + [source_context_type]) + return SourceContext(source_context_type, policy_deliveries) + + def to_json(self): + return { + "sourceContextType": self.source_context_type, + "policyDeliveries": [x.to_json() for x in self.policy_deliveries] + } + + +class CustomEncoder(json.JSONEncoder): + ''' + Used to dump dicts containing `SourceContext`/`PolicyDelivery` into JSON. + ''' + def default(self, obj): + if isinstance(obj, SourceContext): + return obj.to_json() + if isinstance(obj, PolicyDelivery): + return obj.to_json() + return json.JSONEncoder.default(self, obj) diff --git a/testing/web-platform/tests/common/security-features/types.md b/testing/web-platform/tests/common/security-features/types.md new file mode 100644 index 0000000000..17079916c1 --- /dev/null +++ b/testing/web-platform/tests/common/security-features/types.md @@ -0,0 +1,62 @@ +# Types around the generator and generated tests + +This document describes types and concepts used across JavaScript and Python parts of this test framework. +Please refer to the JSDoc in `common.sub.js` or docstrings in Python scripts (if any). + +## Scenario + +### Properties + +- All keys of `test_expansion_schema` in `spec.src.json`, except for `expansion`, `delivery_type`, `delivery_value`, and `source_context_list`. Their values are **string**s specified in `test_expansion_schema`. +- `source_context_list` +- `subresource_policy_deliveries` + +### Types + +- Generator (`spec.src.json`): JSON object +- Generator (Python): `dict` +- Runtime (JS): JSON object +- Runtime (Python): N/A + +## `PolicyDelivery` + +### Types + +- Generator (`spec.src.json`): JSON object +- Generator (Python): `util.PolicyDelivery` +- Runtime (JS): JSON object (`@typedef PolicyDelivery` in `common.sub.js`) +- Runtime (Python): N/A + +## `SourceContext` + +Subresource requests can be possibly sent from various kinds of fetch client's environment settings objects. For example: + +- top-level windows, +- `<iframe>`s, or +- `WorkerGlobalScope`s. + +A **`SourceContext`** object specifies one environment settings object, and an Array of `SourceContext` specifies a possibly nested context, from the outer-most to inner-most environment settings objects. + +Note: The top-level document is processed and trimmed by the generator, and is not included in the `sourceContextList` field of `Scenario` in the generated output. + +For example, `[{sourceContextType: "srcdoc"}, {sourceContextType: "worker-classic"}]` means that a subresource request is to be sent from a classic dedicated worker created from `<iframe srcdoc>` inside the top-level HTML Document. + +Note: A `SourceContext` (or an array of `SourceContext`) is set based on the fetch client's settings object that is used for the subresource fetch, NOT on the module map settings object nor on the inner-most settings object that appears in the test. +For example, the `sourceContextList` field of `Scenario` is `[]` (indicating the top-level Window): + +- When testing top-level worker script fetch, e.g. `new Worker('worker.js')`. There is `WorkerGlobalScope` created from `worker.js`, but it isn't the fetch client's settings object used for fetching `worker.js` itself. +- When testing worker script imported from the root worker script, e.g. `new Worker('top.js', {type: 'module'})` where `top.js` has `import 'worker.js'`. Again, the fetch client's settings object used for `worker.js` is the top-level Window, not `WorkerGlobalScope` created by `top.js`. + +### Properties + +- `sourceContextType`: A string specifying the kind of the source context to be used. + Valid values are the keys of `sourceContextMap` in `common.sub.js`, or `"top"` indicating the top-level Document (`"top"` is valid/used only in the generator). + +- `policyDeliveries`: A list of `PolicyDelivery` applied to the source context. + +### Types + +- Generator (`spec.src.json`): JSON object +- Generator (Python): `util.SourceContext` +- Runtime (JS): JSON object (`@typedef SourceContext` in `common.sub.js`) +- Runtime (Python): N/A |