Adding upstream version 1:115.7.0.upstream/1%115.7.0upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 735 insertions, 0 deletions

diff --git a/testing/talos/perfdocs/index.rst b/testing/talos/perfdocs/index.rst
new file mode 100644
index 0000000000..472088cbf7
--- /dev/null
+++ b/testing/talos/perfdocs/index.rst
@@ -0,0 +1,735 @@
+=====
+Talos
+=====
+
+Talos is a cross-platform Python performance testing framework that is specifically for
+Firefox on desktop. New performance tests should be added to the newer framework
+`mozperftest </testing/perfdocs/mozperftest.html>`_ unless there are limitations
+there (highly unlikely) that make it absolutely necessary to add them to Talos. Talos is
+named after the `bronze automaton from Greek myth <https://en.wikipedia.org/wiki/Talos>`_.
+
+.. contents::
+   :depth: 1
+   :local:
+
+Talos tests are run in a similar manner to xpcshell and mochitests. They are started via
+the command :code:`mach talos-test`. A `python script <https://searchfox.org/mozilla-central/source/testing/talos>`_
+then launches Firefox, which runs the tests via JavaScript special powers. The test timing
+information is recorded in a text log file, e.g. :code:`browser_output.txt`, and then processed
+into the `JSON format supported by Perfherder <https://searchfox.org/mozilla-central/source/testing/mozharness/external_tools/performance-artifact-schema.json>`_.
+
+Talos bugs can be filed in `Testing::Talos <https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Talos>`_.
+
+Talos infrastructure is still mostly documented `on the Mozilla Wiki <https://wiki.mozilla.org/TestEngineering/Performance/Talos>`_.
+In addition, there are plans to surface all of the individual tests using PerfDocs.
+This work is tracked in `Bug 1674220 <https://bugzilla.mozilla.org/show_bug.cgi?id=1674220>`_.
+
+Examples of current Talos runs can be `found in Treeherder by searching for "Talos" <https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=Talos>`_.
+If none are immediately available, then scroll to the bottom of the page and load more test
+runs. The tests all share a group symbol starting with a :code:`T`, for example
+:code:`T(c d damp g1)` or :code:`T-gli(webgl)`.
+
+Running Talos Locally
+*********************
+
+Running tests locally is most likely only useful for debugging what is going on in a test,
+as the test output is only reported as raw JSON. The CLI is documented via:
+
+.. code-block:: bash
+
+    ./mach talos-test --help
+
+To quickly try out the :code:`./mach talos-test` command, the following can be run to do a
+single run of the DevTools' simple netmonitor test.
+
+.. code-block:: bash
+
+    # Run the "simple.netmonitor" test very quickly with 1 cycle, and 1 page cycle.
+    ./mach talos-test --activeTests damp --subtests simple.netmonitor --cycles 1 --tppagecycles 1
+
+
+The :code:`--print-suites` and :code:`--print-tests` are two helpful command flags to
+figure out what suites and tests are available to run.
+
+.. code-block:: bash
+
+    # Print out the suites:
+    ./mach talos-test --print-suites
+
+    # Available suites:
+    #  bcv                          (basic_compositor_video)
+    #  chromez                      (about_preferences_basic:tresize)
+    #  dromaeojs                    (dromaeo_css:kraken)
+    # ...
+
+    # Run all of the tests in the "bcv" test suite:
+    ./mach talos-test --suite bcv
+
+    # Print out the tests:
+    ./mach talos-test --print-tests
+
+    # Available tests:
+    # ================
+    #
+    # a11yr
+    # -----
+    # This test ensures basic a11y tables and permutations do not cause
+    # performance regressions.
+    #
+    # ...
+
+    # Run the tests in "a11yr" listed above
+    ./mach talos-test --activeTests a11yr
+
+Running Talos on Try
+********************
+
+Talos runs can be generated through the mach try fuzzy finder:
+
+.. code-block:: bash
+
+    ./mach try fuzzy
+
+The following is an example output at the time of this writing. Refine the query for the
+platform and test suites of your choosing.
+
+.. code-block::
+
+    | test-windows10-64-qr/opt-talos-bcv-swr-e10s
+    | test-linux64-shippable/opt-talos-webgl-e10s
+    | test-linux64-shippable/opt-talos-other-e10s
+    | test-linux64-shippable-qr/opt-talos-g5-e10s
+    | test-linux64-shippable-qr/opt-talos-g4-e10s
+    | test-linux64-shippable-qr/opt-talos-g3-e10s
+    | test-linux64-shippable-qr/opt-talos-g1-e10s
+    | test-windows10-64/opt-talos-webgl-gli-e10s
+    | test-linux64-shippable/opt-talos-tp5o-e10s
+    | test-linux64-shippable/opt-talos-svgr-e10s
+    | test-linux64-shippable/opt-talos-damp-e10s
+    > test-windows7-32/opt-talos-webgl-gli-e10s
+    | test-linux64-shippable/opt-talos-bcv-e10s
+    | test-linux64-shippable/opt-talos-g5-e10s
+    | test-linux64-shippable/opt-talos-g4-e10s
+    | test-linux64-shippable/opt-talos-g3-e10s
+    | test-linux64-shippable/opt-talos-g1-e10s
+    | test-linux64-qr/opt-talos-bcv-swr-e10s
+
+      For more shortcuts, see mach help try fuzzy and man fzf
+      select: <tab>, accept: <enter>, cancel: <ctrl-c>, select-all: <ctrl-a>, cursor-up: <up>, cursor-down: <down>
+      1379/2967
+    > talos
+
+At a glance
+***********
+
+-  Tests are defined in
+   `testing/talos/talos/test.py <https://searchfox.org/mozilla-central/source/testing/talos/talos/test.py>`__
+-  Treeherder abbreviations are defined in
+   `taskcluster/ci/test/talos.yml <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml>`__
+-  Suites are defined for production in
+   `testing/talos/talos.json <https://searchfox.org/mozilla-central/source/testing/talos/talos.json>`__
+
+Test lifecycle
+**************
+
+-  Taskcluster schedules `talos
+   jobs <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml>`__
+-  Taskcluster runs a Talos job on a hardware machine when one is
+   available - this is bootstrapped by
+   `mozharness <https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py>`__
+
+   -  mozharness downloads the build, talos.zip (found in
+      `talos.json <https://searchfox.org/mozilla-central/source/testing/talos/talos.json>`__),
+      and creates a virtualenv for running the test.
+   -  mozharness `configures the test and runs
+      it <https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running#How_Talos_is_Run_in_Production>`__
+   -  After the test is completed the data is uploaded to
+      `Perfherder <https://treeherder.mozilla.org/perf.html#/graphs>`__
+
+-  Treeherder displays a green (all OK) status and has a link to
+   `Perfherder <https://treeherder.mozilla.org/perf.html#/graphs>`__
+-  13 pushes later,
+   `analyze_talos.py <http://hg.mozilla.org/graphs/file/tip/server/analysis/analyze_talos.py>`__
+   is ran which compares your push to the previous 12 pushes and next 12
+   pushes to look for a
+   `regression <https://wiki.mozilla.org/TestEngineering/Performance/Talos/Data#Regressions>`__
+
+   -  If a regression is found, it will be posted on `Perfherder
+      Alerts <https://treeherder.mozilla.org/perf.html#/alerts>`__
+
+Test types
+**********
+
+There are two different species of Talos tests:
+
+-  Startup_: Start up the browser and wait for either the load event or the paint event and exit, measuring the time
+-  `Page load`_: Load a manifest of pages
+
+In addition we have some variations on existing tests:
+
+-  Heavy_: Run tests with the heavy user profile instead of a blank one
+-  `Web extension`_: Run tests with a web extension to see the perf impact extension have
+-  `Real-world WebExtensions`_: Run tests with a set of 5 popular real-world WebExtensions installed and enabled.
+
+Some tests measure different things:
+
+-  Paint_: These measure events from the browser like moz_after_paint, etc.
+-  ASAP_: These tests go really fast and typically measure how many frames we can render in a time window
+-  Benchmarks_: These are benchmarks that measure specific items and report a summarized score
+
+Startup
+=======
+
+`Startup
+tests <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test>`__
+launch Firefox and measure the time to the onload or paint events. We
+run this in a series of cycles (default to 20) to generate a full set of
+data. Tests that currently are startup tests are:
+
+-  `ts_paint <#ts_paint>`_
+-  tpaint_
+-  `tresize <#tresize>`_
+-  `sessionrestore <#sessionrestore>`_
+-  `sessionrestore_no_auto_restore <#sessionrestore_no_auto_restore>`_
+-  `sessionrestore_many_windows <#sessionrestore_many_windows>`_
+
+Page load
+=========
+
+Many of the talos tests use the page loader to load a manifest of pages.
+These are tests that load a specific page and measure the time it takes
+to load the page, scroll the page, draw the page etc. In order to run a
+page load test, you need a manifest of pages to run. The manifest is
+simply a list of URLs of pages to load, separated by carriage returns,
+e.g.:
+
+.. code-block:: none
+
+   https://www.mozilla.org
+   https://www.mozilla.com
+
+Example:
+`svgx.manifest <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svgx/svgx.manifest>`__
+
+Manifests may also specify that a test computes its own data by
+prepending a ``%`` in front of the line:
+
+.. code-block:: none
+
+   % https://www.mozilla.org
+   % https://www.mozilla.com
+
+Example:
+`v8.manifest <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/v8_7/v8.manifest>`__
+
+The file you created should be referenced in your test config inside of
+`test.py <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l607>`__.
+For example, open test.py, and look for the line referring to the test
+you want to run:
+
+.. code-block:: python
+
+   tpmanifest = '${talos}/page_load_test/svgx/svgx.manifest'
+   tpcycles = 1 # run a single cycle
+   tppagecycles = 25 # load each page 25 times before moving onto the next page
+
+Heavy
+=====
+
+All our testing is done with empty blank profiles, this is not ideal for
+finding issues for end users. We recently undertook a task to create a
+daily update to a profile so it is modern and relevant. It browses a
+variety of web pages, and have history and cache to give us a more
+realistic scenario.
+
+The toolchain is documented on
+`github <https://github.com/tarekziade/heavy-profile>`__ and was added
+to Talos in `bug
+1407398 <https://bugzilla.mozilla.org/show_bug.cgi?id=1407398>`__.
+
+Currently we have issues with this on windows (takes too long to unpack
+the files from the profile), so we have turned this off there. Our goal
+is to run this on basic pageload and startup tests.
+
+Web extension
+=============
+
+Web Extensions are what Firefox has switched to and there are different
+code paths and APIs used vs addons. Historically we don't test with
+addons (other than our test addons) and are missing out on common
+slowdowns. In 2017 we started running some startup and basic pageload
+tests with a web extension in the profile (`bug
+1398974 <https://bugzilla.mozilla.org/show_bug.cgi?id=1398974>`__). We
+have updated the Extension to be more real world and will continue to do
+that.
+
+Real-world WebExtensions
+========================
+
+We've added a variation on our test suite that automatically downloads,
+installs and enables 5 popular WebExtensions. This is used to measure
+things like the impact of real-world WebExtensions on start-up time.
+
+Currently, the following extensions are installed:
+
+-  Adblock Plus (3.5.2)
+-  Cisco Webex Extension (1.4.0)
+-  Easy Screenshot (3.67)
+-  NoScript (10.6.3)
+-  Video DownloadHelper (7.3.6)
+
+Note that these add-ons and versions are "pinned" by being held in a
+compressed file that's hosted in an archive by our test infrastructure
+and downloaded at test runtime. To update the add-ons in this set, one
+must provide a new ZIP file to someone on the test automation team. See
+`this comment in
+Bugzilla <https://bugzilla.mozilla.org/show_bug.cgi?id=1575089#c3>`__.
+
+Paint
+=====
+
+Paint tests are measuring the time to receive both the
+`MozAfterPaint <https://developer.mozilla.org/en-US/docs/Web/Events/MozAfterPaint>`__
+and OnLoad event instead of just the OnLoad event. Most tests now look
+for this unless they are an ASAP test, or an internal benchmark
+
+ASAP
+====
+
+We have a variety of tests which we now run in ASAP mode where we render
+as fast as possible (disabling vsync and letting the rendering iterate
+as fast as it can using \`requestAnimationFrame`). In fact we have
+replaced some original tests with the 'x' versions to make them measure.
+We do this with RequestAnimationFrame().
+
+ASAP tests are:
+
+-  `basic_compositor_video <#basic_compositor_video>`_
+-  `displaylist_mutate <#displaylist_mutate>`_
+-  `glterrain <#glterrain>`_
+-  `rasterflood_svg <#rasterflood_svg>`_
+-  `rasterflood_gradient <#rasterflood_gradient>`_
+-  `tsvgx <#tsvgx>`_
+-  `tscrollx <#tscrollx>`_
+-  `tp5o_scroll <#tp5o_scroll>`_
+-  `tabswitch <#tabswitch>`_
+-  `tart <#tart>`_
+
+Benchmarks
+==========
+
+Many tests have internal benchmarks which we report as accurately as
+possible. These are the exceptions to the general rule of calculating
+the suite score as a geometric mean of the subtest values (which are
+median values of the raw data from the subtests).
+
+Tests which are imported benchmarks are:
+
+-  `ARES6 <#ares6>`_
+-  `dromaeo <#dromaeo>`_
+-  `JetStream <#jetstream>`_
+-  `kraken <#kraken>`_
+-  `motionmark <#motionmark>`_
+-  `stylebench <#stylebench>`_
+
+Row major vs. column major
+==========================
+
+To get more stable numbers, tests are run multiple times. There are two
+ways that we do this: row major and column major. Row major means each
+test is run multiple times and then we move to the next test (and run it
+multiple times). Column major means that each test is run once one after
+the other and then the whole sequence of tests is run again.
+
+More background information about these approaches can be found in Joel
+Maher's `Reducing the Noise in
+Talos <https://elvis314.wordpress.com/2012/03/12/reducing-the-noise-in-talos/>`__
+blog post.
+
+Page sets
+*********
+
+We run our tests 100% offline, but serve pages via a webserver. Knowing
+this we need to store and make available the offline pages we use for
+testing.
+
+tp5pages
+========
+
+Some tests make use of a set of 50 "real world" pages, known as the tp5n
+set. These pages are not part of the talos repository, but without them
+the tests which use them won't run.
+
+-  To add these pages to your local setup, download the latest tp5n zip
+   from `tooltool <https://mozilla-releng.net/tooltool/>`__, and extract
+   it such that ``tp5n`` ends up as ``testing/talos/talos/tests/tp5n``.
+   You can also obtain it by running a talos test locally to get the zip
+   into ``testing/talos/talos/tests/``, i.e ``./mach talos-test --suite damp``
+-  see also `tp5o <#tp5o>`_.
+
+{documentation}
+
+Extra Talos Tests
+*****************
+
+.. contents::
+    :depth: 1
+    :local:
+
+about_newtab_with_snippets
+==========================
+
+.. note::
+
+   add test details
+
+File IO
+-------
+
+File IO is tested using the tp5 test set in the `xperf`_
+test.
+
+Possible regression causes
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+-  **nonmain_startup_fileio opt (with or without e10s) windows7-32** –
+   `bug
+   1274018 <https://bugzilla.mozilla.org/show_bug.cgi?id=1274018>`__
+   This test seems to consistently report a higher result for
+   mozilla-central compared to Try even for an identical revision due to
+   extension signing checks. In other words, if you are comparing Try
+   and Mozilla-Central you may see a false-positive regression on
+   perfherder. Graphs:
+   `non-e10s <https://treeherder.mozilla.org/perf.html#/graphs?timerange=604800&series=%5Bmozilla-central,e5f5eaa174ef22fdd6b6e150e8c450aa827c2ff6,1,1%5D&series=%5Btry,e5f5eaa174ef22fdd6b6e150e8c450aa827c2ff6,1,1%5D>`__
+   `e10s <https://treeherder.mozilla.org/perf.html#/graphs?series=%5B%22mozilla-central%22,%222f3af3833d55ff371ecf01c41aeee1939ef3a782%22,1,1%5D&series=%5B%22try%22,%222f3af3833d55ff371ecf01c41aeee1939ef3a782%22,1,1%5D&timerange=604800>`__
+
+Xres (X Resource Monitoring)
+----------------------------
+
+A memory metric tracked during tp5 test runs. This metric is sampled
+every 20 seconds. This metric is collected on linux only.
+
+`xres man page <https://linux.die.net/man/3/xres>`__.
+
+% CPU
+-----
+
+Cpu usage tracked during tp5 test runs. This metric is sampled every 20
+seconds. This metric is collected on windows only.
+
+Responsiveness
+--------------
+
+contact: :jimm, :overholt
+
+Measures the delay for the event loop to process a `tracer
+event <https://wiki.mozilla.org/Performance/Snappy#Current_Infrastructure>`__.
+For more details, see `bug
+631571 <https://bugzilla.mozilla.org/show_bug.cgi?id=631571>`__.
+
+The score on this benchmark is proportional to the sum of squares of all
+event delays that exceed a 20ms threshold. Lower is better.
+
+We collect 8000+ data points from the browser during the test and apply
+`this
+formula <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/output.py#l95>`__
+to the results:
+
+.. code-block:: python
+
+   return sum([float(x)*float(x) / 1000000.0 for x in val_list])
+
+tpaint
+======
+
+.. warning::
+
+   This test no longer exists
+
+-  contact: :davidb
+-  source:
+   `tpaint-window.html <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tpaint.html>`__
+-  type: Startup_
+-  data: we load the tpaint test window 20 times, resulting in 1 set of
+   20 data points.
+-  summarization:
+
+   -  subtest: `ignore first`_ **5** data points, then take the `median`_ of the remaining 15; `source:
+      test.py <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l190>`__
+   -  suite: identical to subtest
+
++-----------------+---------------------------------------------------+
+| Talos test name | Description                                       |
++-----------------+---------------------------------------------------+
+| tpaint          | twinopen but measuring the time after we receive  |
+|                 | the `MozAfterPaint and OnLoad event <#paint>`__.  |
++-----------------+---------------------------------------------------+
+
+Tests the amount of time it takes the open a new window. This test does
+not include startup time. Multiple test windows are opened in
+succession, results reported are the average amount of time required to
+create and display a window in the running instance of the browser.
+(Measures ctrl-n performance.)
+
+**Example Data**
+
+.. code-block:: none
+
+    [209.219, 222.180, 225.299, 225.970, 228.090, 229.450, 230.625, 236.315, 239.804, 242.795, 244.5, 244.770, 250.524, 251.785, 253.074, 255.349, 264.729, 266.014, 269.399, 326.190]
+
+Possible regression causes
+--------------------------
+
+-  None listed yet. If you fix a regression for this test and have some
+   tips to share, this is a good place for them.
+
+xperf
+=====
+
+-  contact: fx-perf@mozilla.com
+-  source: `xperf
+   instrumentation <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos>`__
+-  type: `Page load`_ (tp5n) / Startup_
+-  measuring: IO counters from windows (currently, only startup IO is in
+   scope)
+-  reporting: Summary of read/write counters for disk, network (lower is
+   better)
+
+These tests only run on windows builds. See `this active-data
+query <https://activedata.allizom.org/tools/query.html#query_id=zqlX+2Jn>`__
+for an updated set of platforms that xperf can be found on. If the query
+is not found, use the following on the query page:
+
+.. code-block:: javascript
+
+   {
+       "from":"task",
+       "groupby":["run.name","build.platform"],
+       "limit":2000,
+       "where":{"regex":{"run.name":".*xperf.*"}}
+   }
+
+Talos will turn orange for 'x' jobs on windows 7 if your changeset
+accesses files which are not predefined in the
+`allowlist <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos/xperf_allowlist.json>`__
+during startup; specifically, before the
+"`sessionstore-windows-restored <https://hg.mozilla.org/mozilla-central/file/0eebc33d8593/toolkit/components/startup/nsAppStartup.cpp#l631>`__"
+Firefox event. If your job turns orange, you will see a list of files in
+Treeherder (or in the log file) which have been accessed unexpectedly
+(similar to this):
+
+.. code-block:: none
+
+    TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
+    TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
+    TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
+    TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
+    TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
+    TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
+
+In the case that these files are expected to be accessed during startup
+by your changeset, then we can add them to the
+`allowlist <https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Talos>`__.
+
+Xperf runs tp5 while collecting xperf metrics for disk IO and network
+IO. The providers we listen for are:
+
+-  `'PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO',
+   'FILE_IO_INIT' <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l10>`__
+
+The values we collect during stackwalk are:
+
+-  `'FileRead', 'FileWrite',
+   'FileFlush' <https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l11>`__
+
+Notes:
+
+-  Currently some runs may `return all-zeros and skew the
+   results <https://bugzilla.mozilla.org/show_bug.cgi?id=1614805>`__
+-  Additionally, these runs don't have dedicated hardware and have a
+   large variability. At least 30 runs are likely to be needed to get
+   stable statistics (xref `bug
+   1616236 <https://bugzilla.mozilla.org/show_bug.cgi?id=1616236>`__)
+
+Build metrics
+*************
+
+These are not part of the Talos code, but like Talos they are benchmarks
+that record data using the graphserver and are analyzed by the same
+scripts for regressions.
+
+Number of constructors (num_ctors)
+==================================
+
+This test runs at build time and measures the number of static
+initializers in the compiled code. Reducing this number is helpful for
+`startup
+optimizations <https://blog.mozilla.org/tglek/2010/05/27/startup-backward-constructors/>`__.
+
+-  https://hg.mozilla.org/build/tools/file/348853aee492/buildfarm/utils/count_ctors.py
+
+   -  these are run for linux 32+64 opt and pgo builds.
+
+Platform microbenchmark
+***********************
+
+IsASCII and IsUTF8 gtest microbenchmarks
+========================================
+
+-  contact: :hsivonen
+-  source:
+   `TestStrings.cpp <https://dxr.mozilla.org/mozilla-central/source/xpcom/tests/gtest/TestStrings.cpp>`__
+-  type: Microbench_
+-  reporting: intervals in ms (lower is better)
+-  data: each test is run and measured 5 times
+-  summarization: take the `median`_ of the 5 data points; `source:
+   MozGTestBench.cpp <https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp#43-46>`__
+
+Test whose name starts with PerfIsASCII test the performance of the
+XPCOM string IsASCII function with ASCII inputs if different lengths.
+
+Test whose name starts with PerfIsUTF8 test the performance of the XPCOM
+string IsUTF8 function with ASCII inputs if different lengths.
+
+Possible regression causes
+--------------------------
+
+-  The --enable-rust-simd accidentally getting turned off in automation.
+-  Changes to encoding_rs internals.
+-  LLVM optimizations regressing between updates to the copy of LLVM
+   included in the Rust compiler.
+
+Microbench
+==========
+
+-  contact: :bholley
+-  source:
+   `MozGTestBench.cpp <https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp>`__
+-  type: Custom GTest micro-benchmarking
+-  data: Time taken for a GTest function to execute
+-  summarization: Not a Talos test. This suite is provides a way to add
+   low level platform performance regression tests for things that are
+   not suited to be tested by Talos.
+
+PerfStrip Tests
+===============
+
+-  contact: :davidb
+-  source:
+   https://dxr.mozilla.org/mozilla-central/source/xpcom/tests/gtest/TestStrings.cpp
+-  type: Microbench_
+-  reporting: execution time in ms (lower is better) for 100k function
+   calls
+-  data: each test run and measured 5 times
+-  summarization:
+
+PerfStripWhitespace - call StripWhitespace() on 5 different test cases
+20k times (each)
+
+PerfStripCharsWhitespace - call StripChars("\f\t\r\n") on 5 different
+test cases 20k times (each)
+
+PerfStripCRLF - call StripCRLF() on 5 different test cases 20k times
+(each)
+
+PerfStripCharsCRLF() - call StripChars("\r\n") on 5 different test cases
+20k times (each)
+
+Stylo gtest microbenchmarks
+===========================
+
+-  contact: :bholley, :SimonSapin
+-  source:
+   `gtest <https://dxr.mozilla.org/mozilla-central/source/layout/style/test/gtest>`__
+-  type: Microbench_
+-  reporting: intervals in ms (lower is better)
+-  data: each test is run and measured 5 times
+-  summarization: take the `median`_ of the 5 data points; `source:
+   MozGTestBench.cpp <https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp#43-46>`__
+
+Servo_StyleSheet_FromUTF8Bytes_Bench parses a sample stylesheet 20 times
+with Stylo’s CSS parser that is written in Rust. It starts from an
+in-memory UTF-8 string, so that I/O or UTF-16-to-UTF-8 conversion is not
+measured.
+
+Gecko_nsCSSParser_ParseSheet_Bench does the same with Gecko’s previous
+CSS parser that is written in C++, for comparison.
+
+Servo_DeclarationBlock_SetPropertyById_Bench parses the string "10px"
+with Stylo’s CSS parser and sets it as the value of a property in a
+declaration block, a million times. This is similar to animations that
+are based on JavaScript code modifying Element.style instead of using
+CSS @keyframes.
+
+Servo_DeclarationBlock_SetPropertyById_WithInitialSpace_Bench is the
+same, but with the string " 10px" with an initial space. That initial
+space is less typical of JS animations, but is almost always there in
+stylesheets or full declarations like "width: 10px". This microbenchmark
+was used to test the effect of some specific code changes. Regressions
+here may be acceptable if Servo_StyleSheet_FromUTF8Bytes_Bench is not
+affected.
+
+History of tp tests
+*******************
+
+tp
+==
+
+The original tp test created by Mozilla to test browser page load time.
+Cycled through 40 pages. The pages were copied from the live web during
+November, 2000. Pages were cycled by loading them within the main
+browser window from a script that lived in content.
+
+tp2/tp_js
+=========
+
+The same tp test but loading the individual pages into a frame instead
+of the main browser window. Still used the old 40 page, year 2000 web
+page test set.
+
+tp3
+===
+
+An update to both the page set and the method by which pages are cycled.
+The page set is now 393 pages from December, 2006. The pageloader is
+re-built as an extension that is pre-loaded into the browser
+chrome/components directories.
+
+tp4
+===
+
+Updated web page test set to 100 pages from February 2009.
+
+tp4m
+====
+
+This is a smaller pageset (21 pages) designed for mobile Firefox. This
+is a blend of regular and mobile friendly pages.
+
+We landed on this on April 18th, 2011 in `bug
+648307 <https://bugzilla.mozilla.org/show_bug.cgi?id=648307>`__. This
+runs for Android and Maemo mobile builds only.
+
+tp5
+===
+
+Updated web page test set to 100 pages from April 8th, 2011. Effort was
+made for the pages to no longer be splash screens/login pages/home pages
+but to be pages that better reflect the actual content of the site in
+question. There are two test page data sets for tp5 which are used in
+multiple tests (i.e. awsy, xperf, etc.): (i) an optimized data set
+called tp5o, and (ii) the standard data set called tp5n.
+
+tp6
+===
+
+Created June 2017 with recorded pages via mitmproxy using modern google,
+amazon, youtube, and facebook. Ideally this will contain more realistic
+user accounts that have full content, in addition we would have more
+than 4 sites- up to top 10 or maybe top 20.
+
+These were migrated to Raptor between 2018 and 2019.
+
+.. _geometric mean: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Data#geometric_mean
+.. _ignore first: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Data#ignore_first
+.. _median: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Data#median