summaryrefslogtreecommitdiffstats
path: root/remote/doc/cdp/PuppeteerVendor.md
diff options
context:
space:
mode:
Diffstat (limited to 'remote/doc/cdp/PuppeteerVendor.md')
-rw-r--r--remote/doc/cdp/PuppeteerVendor.md103
1 files changed, 103 insertions, 0 deletions
diff --git a/remote/doc/cdp/PuppeteerVendor.md b/remote/doc/cdp/PuppeteerVendor.md
new file mode 100644
index 0000000000..248f5b7c18
--- /dev/null
+++ b/remote/doc/cdp/PuppeteerVendor.md
@@ -0,0 +1,103 @@
+# Vendoring Puppeteer
+
+As mentioned in the chapter on [Testing], we run the full [Puppeteer
+test suite] on try. These tests are vendored in central under
+_remote/test/puppeteer/_ and we have a script to pull in upstream changes.
+
+We periodically perform a manual two-way sync. Below is an outline of the
+process interspersed with some tips.
+
+## Check for not-yet upstreamed changes
+
+Before vendoring a new Puppeteer release make sure that there are no Puppeteer
+specific changes in mozilla-central repository that haven't been upstreamed yet
+since the last vendor happened. Run one of the following commands and check the
+listed bugs or related upstream code to verify:
+
+```shell
+% hg log remote/test/puppeteer
+% git log remote/test/puppeteer
+```
+
+If an upstream pull request is needed please see their [contributing.md].
+Typically, the changes we push to Puppeteer include unskipping newly passing
+unit tests for Firefox along with minor fixes to the tests or
+to Firefox-specific browser-fetching and launch code.
+
+Be sure to [run tests against both Chromium and Firefox] in the Puppeteer
+repo. You can specify your local Firefox build when you do so:
+
+```shell
+% BINARY=<path-to-objdir-binary> npm run funit
+```
+
+## Prepare the Puppeteer Repository
+
+Clone the Puppeteer git repository and checkout the release tag you want
+to vendor into mozilla-central.
+
+```shell
+% git checkout tags/puppeteer-%version%
+```
+
+You might want to [install the project] at this point and make sure unit tests pass.
+Check the project's `package.json` for relevant testing commands.
+
+## Update the Puppeteer code in mozilla-central
+
+You can run the following mach command to copy over the Puppeteer branch you
+just prepared. The mach command has flags to specify a local or remote
+repository as well as a commit:
+
+```shell
+% ./mach remote vendor-puppeteer --commitish puppeteer-%version%
+```
+
+By default, this command also installs the newly-pulled Puppeteer package in
+order to generate a new `package-lock.json` file for the purpose of pinning
+Puppeteer dependencies for our CI. There is a `--no-install` option if you want
+to skip this step; for example, if you want to run installation separately at
+a later point.
+
+### Validate that the new code works
+
+Use `./mach puppeteer-test` (see [Testing]) to run Puppeteer tests against both
+Chromium and Firefox in headless mode. Again, only running a subset of tests
+against Firefox is fine -- at this point you just want to check that the
+typescript compiles and the browser binaries are launched successfully.
+
+If something at this stage fails, you might want to check changes in
+`remote/test/puppeteer/package.json` and update `remote/mach_commands.py`
+with new npm scripts.
+
+### Verify the expectation meta data
+
+Next, you want to make sure that the expectation meta data is correct. Check
+changes in [TestExpectations.json]. If there are
+newly skipped tests for Firefox, you might need to update these expectations.
+To do this, run the Puppeteer test job on try (see [Testing]). If these tests
+are specific for Chrome or time out, we want to keep them skipped, if they fail
+we want to have `FAIL` status for all platforms in the expectation meta data.
+You can see, if the meta data needs to be updated, at the end of the log file.
+
+Examine the job logs and make sure the run didn't get interrupted early by a
+crash or a hang, especially if you see a lot of `TEST-UNEXPECTED-MISSING` in
+the Treeherder Failure Summary. You might have to fix some new bug in the unit
+tests. This is the fun part.
+
+Some tests can also unexpectedly pass. Make sure it's correct, and if needed
+update the expectation data by following the instructions at the end of the
+log file.
+
+### Submit the code changes
+
+Once you are happy with the metadata and are ready to submit the sync patch
+up for review, run the Puppeteer test job on try again with `--rebuild 10`
+to check for stability.
+
+[Testing]: ../Testing.md
+[Puppeteer test suite]: https://github.com/GoogleChrome/puppeteer/tree/master/test
+[install the project]: https://github.com/puppeteer/puppeteer/blob/main/docs/contributing.md#getting-started
+[run tests against both Chromium and Firefox]: https://github.com/puppeteer/puppeteer/blob/main/test/README.md#running-tests
+[TestExpectations.json]: https://searchfox.org/mozilla-central/source/remote/test/puppeteer/test/TestExpectations.json
+[contributing.md]: https://github.com/puppeteer/puppeteer/blob/main/docs/contributing.md