diff options
Diffstat (limited to 'testing/perfdocs/generated')
| -rw-r--r-- | testing/perfdocs/generated/IndexedDB.rst | 2 | ||||
| -rw-r--r-- | testing/perfdocs/generated/debugging.rst | 4 | ||||
| -rw-r--r-- | testing/perfdocs/generated/mach-try-perf.rst | 2 | ||||
| -rw-r--r-- | testing/perfdocs/generated/metrics.rst | 82 | ||||
| -rw-r--r-- | testing/perfdocs/generated/raptor-metrics.rst | 206 | ||||
| -rw-r--r-- | testing/perfdocs/generated/raptor.rst | 435 | ||||
| -rw-r--r-- | testing/perfdocs/generated/standard-workflow.rst | 30 | ||||
| -rw-r--r-- | testing/perfdocs/generated/talos.rst | 5 |
8 files changed, 387 insertions, 379 deletions
diff --git a/testing/perfdocs/generated/IndexedDB.rst b/testing/perfdocs/generated/IndexedDB.rst index ec54211038..5d7d98e3de 100644 --- a/testing/perfdocs/generated/IndexedDB.rst +++ b/testing/perfdocs/generated/IndexedDB.rst @@ -51,7 +51,7 @@ How to add more tests? * Under the ``[test_name]`` section, specity the test parameters as a sequence of ``--browsertime.key=value`` arguments as a value of ``browsertime_args =`` * Under the ``[test_name]`` section, override any other values as needed -* Add test as a subtest to run for Desktop ``taskcluster/ci/test/browsertime-desktop.yml`` (maybe also for mobile) +* Add test as a subtest to run for Desktop ``taskcluster/kinds/test/browsertime-desktop.yml`` (maybe also for mobile) * Add test documentation to ``testing/raptor/raptor/perfdocs/config.yml`` * Generated files: diff --git a/testing/perfdocs/generated/debugging.rst b/testing/perfdocs/generated/debugging.rst index 8cc799e820..87165765f8 100644 --- a/testing/perfdocs/generated/debugging.rst +++ b/testing/perfdocs/generated/debugging.rst @@ -14,9 +14,9 @@ As of now, there is no easy way to do this. Raptor was not built for debugging f To debug a functional failure in Raptor you can follow these steps: #. If bug 1653617 has not landed yet, apply the patch. -#. Add the --verbose flag to the extra-options list `here <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor.yml#98-101>`__. +#. Add the --verbose flag to the extra-options list `here <https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/raptor.yml#98-101>`__. #. If the --setenv doesn't exist yet (`bug 1494669 <https://bugzilla.mozilla.org/show_bug.cgi?id=1494669>`_), then add your MOZ_LOG environment variables to give you additional logging `here <https://searchfox.org/mozilla-central/source/testing/raptor/raptor/webextension/desktop.py#42>`_. -#. If the flag does exist, then you can add the MOZ_LOG variables to the `raptor.yml <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor.yml>`_ configuration file. +#. If the flag does exist, then you can add the MOZ_LOG variables to the `raptor.yml <https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/raptor.yml>`_ configuration file. #. Push to try if you can't reproduce the failure locally. You can follow `bug 1655554 <https://bugzilla.mozilla.org/show_bug.cgi?id=1655554>`_ as we work on improving this workflow. diff --git a/testing/perfdocs/generated/mach-try-perf.rst b/testing/perfdocs/generated/mach-try-perf.rst index a637788624..6fcd0e70f2 100644 --- a/testing/perfdocs/generated/mach-try-perf.rst +++ b/testing/perfdocs/generated/mach-try-perf.rst @@ -39,7 +39,7 @@ The tool is built to be conservative about the number of tests to run, so if you at all relative to the existing GeckoView, and Fenix tasks, then you will need to make fixes in the associated taskcluster files (e.g. - taskcluster/ci/test/browsertime-mobile.yml). + taskcluster/kinds/test/browsertime-mobile.yml). Alternatively, set MOZ_FIREFOX_ANDROID_APK_OUTPUT to a path to an APK, and then run the command with --browsertime-upload-apk firefox-android. This option diff --git a/testing/perfdocs/generated/metrics.rst b/testing/perfdocs/generated/metrics.rst deleted file mode 100644 index 931456467c..0000000000 --- a/testing/perfdocs/generated/metrics.rst +++ /dev/null @@ -1,82 +0,0 @@ -################ -Metrics Gathered -################ - -.. contents:: - :depth: 2 - :local: - -**WARNING: This page is still being actively developed.** - -This document contains information about the metrics gathered in Browsertime tests, as well as detailed information of how they are gathered. - -Pageload Tests --------------- - -For browsertime pageload tests, there is a limited set of metrics that we collect (which can easily be expanded). Currently we divide these into two sets of metrics: (i) visual metrics, and (ii) technical metrics. These are gathered through two types of tests called warm and cold pageload tests. We have combined these two into a single "chimera" mode which you'll find in the Treeherder tasks. - -Below, you can find the process of how we run Warm, Cold, and Chimera pageload tests. - -Warm Pageload -============== - -In this pageload test type, we open the browser, then repeatedly navigate to the same page without restarting the browser in between cycles. - -* A new, or conditioned, browser profile is created -* The browser is started up -* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile -* A new tab is opened -* The test URL is loaded; measurements taken -* The tab is reloaded ``X`` more times (for ``X`` replicates); measurements taken each time - -NOTES: -- The measurements from the first page-load are not included in overall results metrics b/c of first load noise; however they are listed in the JSON artifacts -- The bytecode cache gets populated on the first test cycle, and subsequent iterations will already have the cache built to reduce noise. - -Cold Pageload -============== - -In this pageload test type, we open the browser, navigate to the page, then restart the browser before performing the next cycle. - -* A new, or conditioned, browser profile is created -* The browser is started up -* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile -* A new tab is opened -* The test URL is loaded; measurements taken -* The browser is shut down -* Entire process is repeated for the remaining browser cycles - -NOTE: The measurements from all browser cycles are used to calculate overall results - -Chimera Pageload -================ - -A new mode for pageload testing is called chimera mode. It combines the warm and cold variants into a single test. This test mode is used in our Taskcluster tasks. - -* A new, or conditioned, browser profile is created -* The browser is started up -* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile -* A new tab is opened -* The test URL is loaded; measurements taken for ``Cold pageload`` -* Navigate to a secondary URL (to preserve the content process) -* The test URL is loaded again; measurements taken for ``Warm pageload`` -* The desktop browser is shut down -* Entire process is repeated for the remaining browser cycles - -NOTE: The bytecode cache mentioned in Warm pageloads still applies here. - -Technical Metrics -================= - -These are metrics that are obtained from the browser. This includes metrics like First Paint, DOM Content Flushed, etc.. - -Visual Metrics -============== - -When you run Raptor Browsertime with ``--browsertime-visualmetrics``, it will record a video of the page being loaded and then process this video to build the metrics. The video is either produced using FFMPEG (with ``--browsertime-no-ffwindowrecorder``) or the Firefox Window Recorder (default). - - -Benchmarks ----------- - -Benchmarks gather their own custom metrics unlike the pageload tests above. Please ping the owners of those benchmarks to determine what they mean and how they are produced, or reach out to the Performance Test and Tooling team in #perftest on Element. diff --git a/testing/perfdocs/generated/raptor-metrics.rst b/testing/perfdocs/generated/raptor-metrics.rst new file mode 100644 index 0000000000..aa0429aed7 --- /dev/null +++ b/testing/perfdocs/generated/raptor-metrics.rst @@ -0,0 +1,206 @@ +################ +Metrics Gathered +################ + +.. contents:: + :depth: 2 + :local: + +**WARNING: This page is still being actively developed.** + +This document contains information about the metrics gathered in Browsertime tests, as well as detailed information of how they are gathered. + +Pageload Tests +-------------- + +For browsertime pageload tests, there is a limited set of metrics that we collect (which can easily be expanded). Currently we divide these into two sets of metrics: (i) visual metrics, and (ii) technical metrics. These are gathered through two types of tests called warm and cold pageload tests. We have combined these two into a single "chimera" mode which you'll find in the Treeherder tasks. + +Below, you can find the process of how we run Warm, Cold, and Chimera pageload tests. + +Warm Pageload +============== + +In this pageload test type, we open the browser, then repeatedly navigate to the same page without restarting the browser in between cycles. + +* A new, or conditioned, browser profile is created +* The browser is started up +* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile +* A new tab is opened +* The test URL is loaded; measurements taken +* The tab is reloaded ``X`` more times (for ``X`` replicates); measurements taken each time + +NOTES: +- The measurements from the first page-load are not included in overall results metrics b/c of first load noise; however they are listed in the JSON artifacts +- The bytecode cache gets populated on the first test cycle, and subsequent iterations will already have the cache built to reduce noise. + +Cold Pageload +============== + +In this pageload test type, we open the browser, navigate to the page, then restart the browser before performing the next cycle. + +* A new, or conditioned, browser profile is created +* The browser is started up +* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile +* A new tab is opened +* The test URL is loaded; measurements taken +* The browser is shut down +* Entire process is repeated for the remaining browser cycles + +NOTE: The measurements from all browser cycles are used to calculate overall results + +Chimera Pageload +================ + +A new mode for pageload testing is called chimera mode. It combines the warm and cold variants into a single test. This test mode is used in our Taskcluster tasks. + +* A new, or conditioned, browser profile is created +* The browser is started up +* Post-startup browser settle pause of 30 seconds or 1 second if using a conditioned profile +* A new tab is opened +* The test URL is loaded; measurements taken for ``Cold pageload`` +* Navigate to a secondary URL (to preserve the content process) +* The test URL is loaded again; measurements taken for ``Warm pageload`` +* The desktop browser is shut down +* Entire process is repeated for the remaining browser cycles + +NOTE: The bytecode cache mentioned in Warm pageloads still applies here. + +Technical Metrics +================= + +These are metrics that are obtained from the browser. This includes metrics like First Paint, DOM Content Flushed, etc.. + +Visual Metrics +============== + +When you run Raptor Browsertime with ``--browsertime-visualmetrics``, it will record a video of the page being loaded and then process this video to build the metrics. The video is either produced using FFMPEG (with ``--browsertime-no-ffwindowrecorder``) or the Firefox Window Recorder (default). + + +Benchmarks +---------- + +Benchmarks gather their own custom metrics unlike the pageload tests above. Please ping the owners of those benchmarks to determine what they mean and how they are produced, or reach out to the Performance Test and Tooling team in #perftest on Element. + +Metric Definitions +------------------ + +The following documents all available metrics that current alert in Raptor Browsertime tests. + +Contentful Speed Index +====================== +Similar to SpeedIndex, except that it uses the contentfulness of a frame to determine visual completeness instead of the histogram differences. The contentfulness is determined by calculating the number of edges found in the frame. A lower number of edges detected gives a smaller contentfulness value (and a smaller visually complete value). + + + * **Aliases**: ContentfulSpeedIndex + * **Tests using it**: + * **Custom**: `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +First Paint +=========== +Denotes the first time the browser performs a paint that has content in it (in ms). + + + * **Aliases**: First Contentful Composite, First Contentful Paint, fcp + * **Tests using it**: + * **Custom**: `browsertime <raptor.html#browsertime-c>`__, `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + * **Unittests**: `test-page-1 <raptor.html#test-page-1-u>`__, `test-page-2 <raptor.html#test-page-2-u>`__, `test-page-3 <raptor.html#test-page-3-u>`__, `test-page-4 <raptor.html#test-page-4-u>`__ + + +First Visual Change +=================== +The first visual change detected in the test (in ms). + + * **Aliases**: FirstVisualChange + * **Tests using it**: + * **Custom**: `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +Largest Contentful Paint +======================== +The time (in ms) at which the largest piece of content on the page was rendered/painted. + + + * **Aliases**: largestContentfulPaint, lcp + * **Tests using it**: + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +Last Visual Change +================== +The last visual change detected in the test (in ms). + + * **Aliases**: LastVisualChange + * **Tests using it**: + * **Custom**: `process-switch <raptor.html#process-switch-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +Load Time +========= +The time it took for the page to complete loading (in ms). + + * **Aliases**: loadtime + * **Tests using it**: + * **Custom**: `browsertime <raptor.html#browsertime-c>`__, `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + * **Unittests**: `test-page-1 <raptor.html#test-page-1-u>`__, `test-page-2 <raptor.html#test-page-2-u>`__, `test-page-3 <raptor.html#test-page-3-u>`__, `test-page-4 <raptor.html#test-page-4-u>`__ + + +Perceptual Speed Index +====================== +Similar to SpeedIndex, except that it uses the structural similarity index measure (ssim) to determine visual completeness. This technique compares the luminance, contrast, and structure of the frames (a given frame vs. a final frame) to determine the completeness. + + + * **Aliases**: PerceptualSpeedIndex + * **Tests using it**: + * **Custom**: `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +Speed Index +=========== +A metric used to denote the speed at which a page loaded. Lower values indicate faster pageloads. Units are in (Visually-Complete x Milliseconds). Calculated by using videos of the pageload which provide a measure of visual completeness. Visual completeness is calculated by comparing the histogram of a given frame to the final frame of the pageload. The SpeedIndex is calculated as the area between the curves of a constant line at y=1, and the graph of the visual completeness from 0ms to when visual completeness reaches 100% (or hits the y=1 line). + + + * **Aliases**: SpeedIndex + * **Tests using it**: + * **Custom**: `connect <raptor.html#connect-c>`__, `process-switch <raptor.html#process-switch-c>`__, `throttled <raptor.html#throttled-c>`__, `welcome <raptor.html#welcome-c>`__ + * **Desktop**: `amazon <raptor.html#amazon-d>`__, `bing-search <raptor.html#bing-search-d>`__, `buzzfeed <raptor.html#buzzfeed-d>`__, `cnn <raptor.html#cnn-d>`__, `docomo <raptor.html#docomo-d>`__, `ebay <raptor.html#ebay-d>`__, `espn <raptor.html#espn-d>`__, `expedia <raptor.html#expedia-d>`__, `facebook <raptor.html#facebook-d>`__, `fandom <raptor.html#fandom-d>`__, `google-docs <raptor.html#google-docs-d>`__, `google-mail <raptor.html#google-mail-d>`__, `google-search <raptor.html#google-search-d>`__, `google-slides <raptor.html#google-slides-d>`__, `imdb <raptor.html#imdb-d>`__, `imgur <raptor.html#imgur-d>`__, `instagram <raptor.html#instagram-d>`__, `linkedin <raptor.html#linkedin-d>`__, `microsoft <raptor.html#microsoft-d>`__, `netflix <raptor.html#netflix-d>`__, `nytimes <raptor.html#nytimes-d>`__, `office <raptor.html#office-d>`__, `openai <raptor.html#openai-d>`__, `outlook <raptor.html#outlook-d>`__, `paypal <raptor.html#paypal-d>`__, `pinterest <raptor.html#pinterest-d>`__, `reddit <raptor.html#reddit-d>`__, `samsung <raptor.html#samsung-d>`__, `tiktok <raptor.html#tiktok-d>`__, `tumblr <raptor.html#tumblr-d>`__, `twitch <raptor.html#twitch-d>`__, `twitter <raptor.html#twitter-d>`__, `weather <raptor.html#weather-d>`__, `wikia <raptor.html#wikia-d>`__, `wikipedia <raptor.html#wikipedia-d>`__, `yahoo-mail <raptor.html#yahoo-mail-d>`__, `youtube <raptor.html#youtube-d>`__ + * **Interactive**: `cnn-nav <raptor.html#cnn-nav-i>`__, `facebook-nav <raptor.html#facebook-nav-i>`__, `reddit-billgates-ama <raptor.html#reddit-billgates-ama-i>`__, `reddit-billgates-post-1 <raptor.html#reddit-billgates-post-1-i>`__, `reddit-billgates-post-2 <raptor.html#reddit-billgates-post-2-i>`__ + * **Live**: `booking-sf <raptor.html#booking-sf-l>`__, `discord <raptor.html#discord-l>`__, `fashionbeans <raptor.html#fashionbeans-l>`__, `google-accounts <raptor.html#google-accounts-l>`__, `imdb-firefox <raptor.html#imdb-firefox-l>`__, `medium-article <raptor.html#medium-article-l>`__, `people-article <raptor.html#people-article-l>`__, `reddit-thread <raptor.html#reddit-thread-l>`__, `rumble-fox <raptor.html#rumble-fox-l>`__, `stackoverflow-question <raptor.html#stackoverflow-question-l>`__, `urbandictionary-define <raptor.html#urbandictionary-define-l>`__, `wikia-marvel <raptor.html#wikia-marvel-l>`__ + * **Mobile**: `allrecipes <raptor.html#allrecipes-m>`__, `amazon <raptor.html#amazon-m>`__, `amazon-search <raptor.html#amazon-search-m>`__, `bild-de <raptor.html#bild-de-m>`__, `bing <raptor.html#bing-m>`__, `bing-search-restaurants <raptor.html#bing-search-restaurants-m>`__, `booking <raptor.html#booking-m>`__, `cnn <raptor.html#cnn-m>`__, `cnn-ampstories <raptor.html#cnn-ampstories-m>`__, `dailymail <raptor.html#dailymail-m>`__, `ebay-kleinanzeigen <raptor.html#ebay-kleinanzeigen-m>`__, `ebay-kleinanzeigen-search <raptor.html#ebay-kleinanzeigen-search-m>`__, `espn <raptor.html#espn-m>`__, `facebook <raptor.html#facebook-m>`__, `facebook-cristiano <raptor.html#facebook-cristiano-m>`__, `google <raptor.html#google-m>`__, `google-maps <raptor.html#google-maps-m>`__, `google-search-restaurants <raptor.html#google-search-restaurants-m>`__, `imdb <raptor.html#imdb-m>`__, `instagram <raptor.html#instagram-m>`__, `microsoft-support <raptor.html#microsoft-support-m>`__, `reddit <raptor.html#reddit-m>`__, `sina <raptor.html#sina-m>`__, `stackoverflow <raptor.html#stackoverflow-m>`__, `wikipedia <raptor.html#wikipedia-m>`__, `youtube <raptor.html#youtube-m>`__, `youtube-watch <raptor.html#youtube-watch-m>`__ + + +Youtube Playback Metrics +======================== +Metrics starting with VP9/H264 give the number of frames dropped, and painted. + + + * **Aliases**: H264, VP9 + * **Tests using it**: + * **Benchmarks**: `youtube-playback <raptor.html#youtube-playback-b>`__, `youtube-playback-hfr <raptor.html#youtube-playback-hfr-b>`__ + + diff --git a/testing/perfdocs/generated/raptor.rst b/testing/perfdocs/generated/raptor.rst index d32718e0ca..8182f94f90 100644 --- a/testing/perfdocs/generated/raptor.rst +++ b/testing/perfdocs/generated/raptor.rst @@ -23,7 +23,7 @@ Locally, Raptor can be invoked with the following command: browsertime debugging contributing - metrics + raptor-metrics .. contents:: :depth: 2 @@ -212,11 +212,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-ares6** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-ares6** - ❌ - ❌ @@ -453,11 +448,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-assorted-dom** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-assorted-dom** - ❌ - ❌ @@ -695,11 +685,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-jetstream2** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-jetstream2** - ❌ - ❌ @@ -932,11 +917,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-matrix-react-bench** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-matrix-react-bench** - ❌ - ❌ @@ -1010,7 +990,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **expose chrome trace**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false @@ -1273,11 +1253,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-1-3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-1-3** - ❌ - ❌ @@ -1351,7 +1326,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **cold**: true * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false @@ -1614,11 +1589,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-1-3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-1-3** - ❌ - ❌ @@ -1694,7 +1664,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-desktop.toml#16>`__ @@ -1905,11 +1875,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-animometer** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-animometer** - ❌ - ❌ @@ -1976,7 +1941,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **apps**: fenix, chrome-m, geckoview * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-mobile.toml#16>`__ @@ -2187,11 +2152,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-animometer** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-animometer** - ❌ - ❌ @@ -2262,7 +2222,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-desktop.toml#18>`__ @@ -2478,11 +2438,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-animometer-ramp** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-animometer-ramp** - ❌ - ❌ @@ -2554,7 +2509,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **apps**: fenix, chrome-m, geckoview * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-mobile.toml#18>`__ @@ -2770,11 +2725,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-animometer-ramp** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-animometer-ramp** - ❌ - ❌ @@ -2851,7 +2801,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-desktop.toml#17>`__ @@ -3062,11 +3012,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite** - ❌ - ❌ @@ -3134,7 +3079,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-mobile.toml#17>`__ @@ -3345,11 +3290,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite** - ❌ - ❌ @@ -3423,7 +3363,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **expose chrome trace**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false @@ -3686,11 +3626,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite-1-3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite-1-3** - ❌ - ❌ @@ -3764,7 +3699,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **cold**: true * **expected**: pass * **expose browser profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false @@ -4027,11 +3962,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite-1-3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite-1-3** - ❌ - ❌ @@ -4108,7 +4038,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-desktop.toml#19>`__ @@ -4324,11 +4254,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite-ramp** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp** - ❌ - ❌ @@ -4401,7 +4326,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true - * **gecko profile features**: stackwalk,js,cpu,java,processcpu + * **gecko profile features**: stackwalk,js,cpu,java,processcpu,memory * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-mobile.toml#19>`__ @@ -4617,11 +4542,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-motionmark-htmlsuite-ramp** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp** - ❌ - ❌ @@ -5016,11 +4936,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-speedometer** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-speedometer** - ❌ - ❌ @@ -5411,11 +5326,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-speedometer** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-speedometer** - ❌ - ❌ @@ -5812,11 +5722,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-speedometer3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-speedometer3** - ❌ - ❌ @@ -6209,11 +6114,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-speedometer3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-speedometer3** - ❌ - ❌ @@ -6453,11 +6353,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-stylebench** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-stylebench** - ❌ - ❌ @@ -6690,11 +6585,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-sunspider** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-sunspider** - ❌ - ❌ @@ -7190,11 +7080,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-unity-webgl** - ❌ - ❌ @@ -7468,11 +7353,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-unity-webgl** - ❌ - ❌ @@ -7706,11 +7586,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-wasm-chrome-wasm-godot** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-wasm-firefox-wasm-godot** - ❌ - ❌ @@ -8369,11 +8244,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-wasm-chrome-wasm-misc** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-wasm-firefox-wasm-misc** - ❌ - ❌ @@ -9039,11 +8909,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-benchmark-chrome-webaudio** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-benchmark-firefox-webaudio** - ❌ - ❌ @@ -9110,7 +8975,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: PerfTest Team - * **alert on**: H264.1080p30@1X_dropped_frames H264.1080p60@1X_dropped_frames H264.1440p30@1X_dropped_frames H264.144p15@1X_dropped_frames H264.2160p30@1X_dropped_frames H264.240p30@1X_dropped_frames H264.360p30@1X_dropped_frames H264.480p30@1X_dropped_frames H264.720p30@1X_dropped_frames H264.720p60@1X_dropped_frames VP9.1080p30@1X_dropped_frames VP9.1080p60@1X_dropped_frames VP9.1440p30@1X_dropped_frames VP9.1440p60@1X_dropped_frames VP9.144p30@1X_dropped_frames VP9.2160p30@1X_dropped_frames VP9.2160p60@1X_dropped_frames VP9.240p30@1X_dropped_frames VP9.360p30@1X_dropped_frames VP9.480p30@1X_dropped_frames VP9.720p30@1X_dropped_frames VP9.720p60@1X_dropped_frames + * **alert on**: `H264.1080p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.1080p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.1440p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.144p15@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.2160p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.240p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.360p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.480p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.720p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264.720p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.1080p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.1080p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.1440p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.1440p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.144p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.2160p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.2160p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.240p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.360p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.480p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.720p30@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `VP9.720p60@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__ * **alert threshold**: 2.0 * **apps**: firefox, geckoview, fenix, refbrow, chrome * **expected**: pass @@ -10264,7 +10129,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: PerfTest Team - * **alert on**: H2641080p60fps@1X_dropped_frames H264720p60fps@1X_dropped_frames + * **alert on**: `H2641080p60fps@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__, `H264720p60fps@1X_dropped_frames <raptor-metrics.html#youtube-playback-metrics>`__ * **alert threshold**: 2.0 * **apps**: firefox, geckoview, fenix, refbrow, chrome * **expected**: pass @@ -12246,7 +12111,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#30>`__ * **lower is better**: true @@ -12476,7 +12341,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#34>`__ * **lower is better**: true @@ -12706,7 +12571,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#38>`__ * **lower is better**: true @@ -12936,7 +12801,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#42>`__ * **lower is better**: true @@ -13166,7 +13031,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#46>`__ * **lower is better**: true @@ -13396,7 +13261,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#50>`__ * **lower is better**: true @@ -13626,7 +13491,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#54>`__ * **lower is better**: true @@ -13856,7 +13721,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#58>`__ * **lower is better**: true @@ -14086,7 +13951,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#62>`__ * **lower is better**: true @@ -14316,7 +14181,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#66>`__ * **lower is better**: true @@ -14546,7 +14411,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#70>`__ * **lower is better**: true @@ -14776,7 +14641,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#74>`__ * **lower is better**: true @@ -14999,7 +14864,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Used to run vanilla browsertime tests through raptor. For example: `./mach raptor --browsertime -t browsertime --browsertime-arg url=https://www.sitespeed.io --browsertime-arg iterations=3` - * **alert on**: fcp, loadtime + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__ * **alert threshold**: 2.0 * **apps**: firefox * **browser cycles**: 1 @@ -15027,7 +14892,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Verifies that connection time on a pageload is zero when a speculative connection has first been made * **alert change type**: absolute - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__ * **alert threshold**: 10.0 * **apps**: firefox * **browser cycles**: 3 @@ -15300,7 +15165,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 - * **gecko profile features**: js,stackwalk,cpu + * **gecko profile features**: js,stackwalk,cpu,memory * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#78>`__ * **lower is better**: true @@ -15552,7 +15417,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures process switch time * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **browser cycles**: 25 @@ -15789,7 +15654,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Pageload test using a throttled network environment - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox * **browser cycles**: 3 @@ -16359,11 +16224,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - autoland - mozilla-release - mozilla-beta - * - **browsertime-upload-chrome-upload** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-upload-firefox-upload** - ❌ - ❌ @@ -16590,11 +16450,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - autoland - mozilla-release - mozilla-beta - * - **browsertime-upload-chrome-upload-h3** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-upload-firefox-upload-h3** - ❌ - ❌ @@ -16663,7 +16518,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures pageload metrics for the first-install about:welcome page - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox * **browser cycles**: 25 @@ -16871,7 +16726,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false @@ -17447,7 +17302,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true @@ -17845,7 +17700,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false @@ -18084,7 +17939,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true @@ -18565,7 +18420,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **benchmark page**: false @@ -18788,7 +18643,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true @@ -19112,7 +18967,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false @@ -19423,7 +19278,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true @@ -19435,7 +19290,7 @@ Tests for page-load performance. The links direct to the actual websites that ar * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm7-linux-firefox-expedia.manifest + * **playback pageset manifest**: mitm8-darwin-firefox-expedia.manifest * **playback version**: 8.1.1 * **secondary url**: `<https://groups.expedia.com/Group-Rate/?locale=en_US&ol=1>`__ * **test url**: `<https://expedia.com/Hotel-Search?destination=New+York%2C+New+York&latLong=40.756680%2C-73.986470®ionId=178293&startDate=&endDate=&rooms=1&_xpid=11905%7C1&adults=2>`__ @@ -19709,14 +19564,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#54>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#55>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -20075,14 +19930,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#59>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#60>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -20473,14 +20328,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#64>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#65>`__ * **lower is better**: true * **page complete wait time**: 8000 * **page cycles**: 25 @@ -20798,14 +20653,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#69>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#70>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21121,14 +20976,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#74>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#75>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21444,14 +21299,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#78>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#79>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21843,14 +21698,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#84>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#85>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -22274,14 +22129,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#88>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#89>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -22579,14 +22434,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#100>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#101>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23080,14 +22935,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#105>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#106>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23404,19 +23259,19 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#110>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#111>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm7-linux-firefox-microsoft.manifest + * **playback pageset manifest**: mitm8-linux-firefox-microsoft.manifest * **playback version**: 8.1.1 * **secondary url**: `<https://support.microsoft.com/en-us>`__ * **test url**: `<https://www.microsoft.com/en-us/>`__ @@ -23728,14 +23583,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#114>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#116>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24047,14 +23902,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#119>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#121>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24333,14 +24188,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#123>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#125>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24572,7 +24427,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **benchmark page**: false @@ -24795,14 +24650,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#127>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#129>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25113,14 +24968,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#131>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#133>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25436,14 +25291,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#135>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#137>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25760,14 +25615,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#141>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#143>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26191,7 +26046,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **benchmark page**: false @@ -26414,7 +26269,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **benchmark page**: false @@ -26637,14 +26492,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#147>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#149>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26961,14 +26816,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#152>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#154>`__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -27287,14 +27142,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#165>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#167>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -27680,7 +27535,7 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome * **benchmark page**: false @@ -27903,14 +27758,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#170>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#172>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28142,14 +27997,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#175>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#177>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28648,14 +28503,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#179>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#181>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29046,14 +28901,14 @@ Tests for page-load performance. The links direct to the actual websites that ar **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 - * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#184>`__ + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#186>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29485,7 +29340,7 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests **Description**: Navigates to cnn main page, then to the world sub-page. * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari * **browser cycles**: 10 @@ -29727,7 +29582,7 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests **Description**: Navigates to facebook, then the sub-pages friends, marketplace, groups. * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari * **browser cycles**: 10 @@ -29969,7 +29824,7 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests **Description**: Navigates from the Bill Gates AMA to the Reddit members section. * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari * **browser cycles**: 10 @@ -30211,7 +30066,7 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests **Description**: Navigates the `thisisbillgates` user starting at the main user page, then to the posts, comments, hot, and top sections. * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari * **browser cycles**: 10 @@ -30453,7 +30308,7 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests **Description**: Navigates the `thisisbillgates` user starting at the main user page, then to the posts, comments, hot, and top sections. * **accept zero vismet**: true - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: firefox, chrome, safari * **browser cycles**: 10 @@ -30697,7 +30552,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -30765,7 +30620,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -30833,7 +30688,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -30901,7 +30756,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -30969,7 +30824,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31037,7 +30892,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31105,7 +30960,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31173,7 +31028,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31241,7 +31096,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31309,7 +31164,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31377,7 +31232,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31445,7 +31300,7 @@ A set of test pages that are run as live sites instead of recorded versions. The **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 @@ -31518,7 +31373,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -31615,7 +31470,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32189,7 +32044,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32296,7 +32151,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32428,7 +32283,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32560,7 +32415,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32692,7 +32547,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -32824,7 +32679,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -33303,7 +33158,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -33435,7 +33290,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -33567,7 +33422,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -33699,7 +33554,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -33831,7 +33686,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -34140,7 +33995,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -34505,7 +34360,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -34637,7 +34492,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -34735,7 +34590,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -34867,7 +34722,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -35000,7 +34855,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -35429,7 +35284,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -35929,7 +35784,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -36026,7 +35881,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -36455,7 +36310,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -36562,7 +36417,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -36694,7 +36549,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -37198,7 +37053,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -37627,7 +37482,7 @@ Page-load performance test suite on Android. The links direct to the actual webs **Owner**: PerfTest Team - * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__, `ContentfulSpeedIndex <raptor-metrics.html#contentful-speed-index>`__, `PerceptualSpeedIndex <raptor-metrics.html#perceptual-speed-index>`__, `SpeedIndex <raptor-metrics.html#speed-index>`__, `FirstVisualChange <raptor-metrics.html#first-visual-change>`__, `LastVisualChange <raptor-metrics.html#last-visual-change>`__, `largestContentfulPaint <raptor-metrics.html#largest-contentful-paint>`__ * **alert threshold**: 2.0 * **apps**: geckoview, fenix, refbrow, chrome-m * **benchmark page**: true @@ -37771,7 +37626,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un **Owner**: PerfTest Team - * **alert on**: fcp, loadtime + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__ * **alert threshold**: 2.0 * **apps**: firefox * **expected**: pass @@ -37793,7 +37648,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un **Owner**: PerfTest Team - * **alert on**: fcp, loadtime + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__ * **alert threshold**: 2.0 * **apps**: firefox * **expected**: pass @@ -37815,7 +37670,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un **Owner**: PerfTest Team - * **alert on**: fcp, loadtime + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__ * **alert threshold**: 2.0 * **apps**: firefox * **expected**: pass @@ -37837,7 +37692,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un **Owner**: PerfTest Team - * **alert on**: fcp, loadtime + * **alert on**: `fcp <raptor-metrics.html#first-paint>`__, `loadtime <raptor-metrics.html#load-time>`__ * **alert threshold**: 2.0 * **apps**: firefox * **expected**: pass diff --git a/testing/perfdocs/generated/standard-workflow.rst b/testing/perfdocs/generated/standard-workflow.rst index e874ecc7b8..5b1fd4c667 100644 --- a/testing/perfdocs/generated/standard-workflow.rst +++ b/testing/perfdocs/generated/standard-workflow.rst @@ -39,7 +39,35 @@ Some more information on this tool `can be found here <https://wiki.mozilla.org/ Understanding the Results ------------------------- -In the image above, the **base**, and **new** columns show the average value across all tests. The number of data points used here can be changed by clicking the **Use replicates** button at the top-right of the table (only for try pushes). Alternatively, you can navigate to the try runs to retrigger the tests, or use the "refresh"-like button mentioned above. Hovering over the values in these columns will show you the spread of the data along with the standard deviation in percentage form. +In the image above, the **base**, and **new** columns show the average value across all tests. You can navigate to the try runs to retrigger the tests, or use the "refresh"-like button mentioned above (only visible while logged in). Hovering over the values in these columns will show you the spread of the data along with the standard deviation in percentage form. The number of data points used here can also be changed by clicking the **Use replicates** button at the top-right of the table (only for try pushes). It causes the comparison to use the trials/replicates data from the perfherder data instead of the summary values. Only 1 summary value is generated per task, whereas there can be multiple replicates generated per task. Here's an example of where this data comes from in the ``PERFHERDER_DATA`` JSON (output in the performance task logs, or as ``perfherder-data.json`` file): + +.. code-block:: none + + ... # This type of data can be found in any PERFHERDER_DATA output + "subtests": [ + { + "alertThreshold": 2.0, + "lowerIsBetter": true, + "name": "Charts-chartjs/Draw opaque scatter/Async", + "replicates": [ # These are the trials/replicates (multiple per task) + 1.74, + 1.36, + 1.16, + 1.62, + 1.42, + 1.28, + 1.12, + 1.4, + 1.26, + 1.44, + 1.22, + 3.32 + ], + "unit": "ms", + "value": 1.542 # This is the summary value of those replicates (only 1 per task) + }, + ... + The **delta** column shows the difference between the two revisions' average in percentage. A negative value here means that the associated metric has decreased, and vice versa for positive values. diff --git a/testing/perfdocs/generated/talos.rst b/testing/perfdocs/generated/talos.rst index 3a730e1035..266f4028be 100644 --- a/testing/perfdocs/generated/talos.rst +++ b/testing/perfdocs/generated/talos.rst @@ -125,7 +125,7 @@ At a glance - Tests are defined in `testing/talos/talos/test.py <https://searchfox.org/mozilla-central/source/testing/talos/talos/test.py>`__ - Treeherder abbreviations are defined in - `taskcluster/ci/test/talos.yml <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml>`__ + `taskcluster/kinds/test/talos.yml <https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/talos.yml>`__ - Suites are defined for production in `testing/talos/talos.json <https://searchfox.org/mozilla-central/source/testing/talos/talos.json>`__ @@ -133,7 +133,7 @@ Test lifecycle ************** - Taskcluster schedules `talos - jobs <https://searchfox.org/mozilla-central/source/taskcluster/ci/test/talos.yml>`__ + jobs <https://searchfox.org/mozilla-central/source/taskcluster/kinds/test/talos.yml>`__ - Taskcluster runs a Talos job on a hardware machine when one is available - this is bootstrapped by `mozharness <https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py>`__ @@ -8000,6 +8000,7 @@ For the sample commands found below, note that the capitalization used is import repository, and this test pulls those in for testing locally through a toolchain artifact called talos-pdfs. * pdfpaint: True + * subtest_alerts: True * timeout: 1800 * tpmanifest: ${talos}/tests/pdfpaint/pdfpaint.manifest * tppagecycles: 1 |