From def92d1b8e9d373e2f6f27c366d578d97d8960c6 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Wed, 15 May 2024 05:34:50 +0200 Subject: Merging upstream version 126.0. Signed-off-by: Daniel Baumann --- testing/perfdocs/generated/browsertime.rst | 19 + testing/perfdocs/generated/mach-try-perf.rst | 4 - testing/perfdocs/generated/raptor.rst | 2148 +++++++++++++++++++------- testing/perfdocs/generated/talos.rst | 1255 ++++++++------- testing/perfdocs/generated/webextension.rst | 4 +- 5 files changed, 2339 insertions(+), 1091 deletions(-) (limited to 'testing/perfdocs') diff --git a/testing/perfdocs/generated/browsertime.rst b/testing/perfdocs/generated/browsertime.rst index beff3f0ad8..3cde0baca6 100644 --- a/testing/perfdocs/generated/browsertime.rst +++ b/testing/perfdocs/generated/browsertime.rst @@ -180,6 +180,25 @@ trigger explicit linter errors, :doc:`see condprof-addons linter docs `` option, you will be able to upload a locally built APK of Fenix to run in try. It will be used in the new revision push. See the ``--help`` for more information on this. - Future Work ----------- diff --git a/testing/perfdocs/generated/raptor.rst b/testing/perfdocs/generated/raptor.rst index 789fbb6883..d32718e0ca 100644 --- a/testing/perfdocs/generated/raptor.rst +++ b/testing/perfdocs/generated/raptor.rst @@ -45,10 +45,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 4 * **page timeout**: 270000 @@ -283,10 +284,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: PerfTest Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -523,10 +525,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 4 * **page timeout**: 2000000 @@ -764,10 +767,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 30 * **page timeout**: 2000000 @@ -1000,7 +1004,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Graphics Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 1 * **cold**: true * **expected**: pass @@ -1010,6 +1014,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1350,6 +1355,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1685,12 +1691,13 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1972,6 +1979,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2251,12 +2259,13 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose browser profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2548,6 +2557,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2837,13 +2847,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -3126,6 +3137,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -3405,7 +3417,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Graphics Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 1 * **cold**: true * **expected**: pass @@ -3415,6 +3427,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -3755,6 +3768,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -4090,13 +4104,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4389,6 +4404,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4678,10 +4694,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4733,8 +4750,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-geckoview-nofis** @@ -5076,6 +5093,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -5127,8 +5145,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-geckoview-nofis** @@ -5470,13 +5488,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Performance Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 5 * **cold**: true * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -5532,8 +5551,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** @@ -5563,12 +5582,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5594,12 +5613,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5873,6 +5892,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1 * **host from parent**: false + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -5928,8 +5948,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** @@ -5959,12 +5979,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5990,12 +6010,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -6266,10 +6286,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :emelio and Layout Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 140000 @@ -6504,10 +6525,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 55000 @@ -6744,6 +6766,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2000000 @@ -6957,10 +6980,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -7139,11 +7163,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ - ❌ - * - **browsertime-benchmark-safari-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ .. list-table:: **test-windows10-32-qr/opt** @@ -7243,6 +7262,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 420000 @@ -7421,11 +7441,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ - ❌ - * - **browsertime-benchmark-safari-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ .. list-table:: **test-windows10-32-qr/opt** @@ -7525,10 +7540,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :lth and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -7766,6 +7782,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -7979,6 +7996,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -8188,10 +8206,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :lth and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8426,6 +8445,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8641,6 +8661,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8852,10 +8873,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :padenot and Media Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 360000 @@ -9095,6 +9117,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9117,6 +9140,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9129,6 +9153,38 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **use live sites**: true * **Test Task**: + .. list-table:: **test-android-hw-a51-11-0-aarch64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-android-hw-a51-11-0-aarch64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis** + - ✅ + - ❌ + - ❌ + - ❌ + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -9158,6 +9214,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9359,6 +9416,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9560,6 +9618,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9761,6 +9820,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9961,6 +10021,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9982,11 +10043,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-mobile-fenix-youtube-playback-h264-sfr-nofis** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis** - ❌ - ❌ @@ -10003,11 +10059,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-mobile-fenix-youtube-playback-h264-sfr-nofis** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis** - ✅ - ❌ @@ -10220,6 +10271,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -10263,7 +10315,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - mozilla-release - mozilla-beta * - **browsertime-mobile-fenix-youtube-playback-hfr-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -10479,6 +10531,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -10680,6 +10733,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -10881,6 +10935,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -11082,6 +11137,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -11282,6 +11338,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11294,6 +11351,38 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **use live sites**: true * **Test Task**: + .. list-table:: **test-android-hw-a51-11-0-aarch64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-android-hw-a51-11-0-aarch64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis** + - ✅ + - ❌ + - ❌ + - ❌ + + .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -11498,6 +11587,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11714,6 +11804,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11930,6 +12021,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -12148,7 +12240,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small ArrayBuffers to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=1 * **custom data**: true @@ -12156,6 +12248,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12218,11 +12311,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12276,11 +12364,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12318,11 +12401,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ❌ - ❌ @@ -12376,11 +12454,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12397,7 +12470,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small ArrayBuffers to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=0 * **custom data**: true @@ -12405,6 +12478,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12467,11 +12541,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12525,11 +12594,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12567,11 +12631,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ❌ - ❌ @@ -12625,11 +12684,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12646,7 +12700,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Arrays to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Array --browsertime.atomic=1 * **custom data**: true @@ -12654,6 +12708,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12716,11 +12771,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12774,11 +12824,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12816,11 +12861,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ❌ - ❌ @@ -12874,11 +12914,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12895,7 +12930,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Arrays to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Array --browsertime.atomic=0 * **custom data**: true @@ -12903,6 +12938,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12965,11 +13001,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13023,11 +13054,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13065,11 +13091,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ❌ - ❌ @@ -13123,11 +13144,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13144,7 +13160,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Blobs to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Blob --browsertime.atomic=1 * **custom data**: true @@ -13152,6 +13168,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13214,11 +13231,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13272,11 +13284,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13314,11 +13321,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ❌ - ❌ @@ -13372,11 +13374,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13393,7 +13390,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Blobs to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Blob --browsertime.atomic=0 * **custom data**: true @@ -13401,6 +13398,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13463,11 +13461,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13521,11 +13514,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13563,11 +13551,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ❌ - ❌ @@ -13621,11 +13604,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13642,7 +13620,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large ArrayBuffers to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=1 * **custom data**: true @@ -13650,6 +13628,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13712,11 +13691,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13770,11 +13744,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13812,11 +13781,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ❌ - ❌ @@ -13870,11 +13834,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13891,7 +13850,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large ArrayBuffers to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=0 * **custom data**: true @@ -13899,6 +13858,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13961,11 +13921,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14019,11 +13974,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14061,11 +14011,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ❌ - ❌ @@ -14119,11 +14064,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14140,7 +14080,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large JS Arrays to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Array --browsertime.atomic=1 * **custom data**: true @@ -14148,6 +14088,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14210,11 +14151,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14268,11 +14204,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14310,11 +14241,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ❌ - ❌ @@ -14368,11 +14294,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14389,7 +14310,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large JS Arrays to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Array --browsertime.atomic=0 * **custom data**: true @@ -14397,6 +14318,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14459,11 +14381,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14517,11 +14434,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14559,11 +14471,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ❌ - ❌ @@ -14617,11 +14524,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14638,7 +14540,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large Blobs to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Blob --browsertime.atomic=1 * **custom data**: true @@ -14646,6 +14548,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14708,11 +14611,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14766,11 +14664,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14808,11 +14701,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ❌ - ❌ @@ -14866,11 +14754,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14887,7 +14770,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large Blobs to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Blob --browsertime.atomic=0 * **custom data**: true @@ -14895,6 +14778,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14957,11 +14841,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15015,11 +14894,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15057,11 +14931,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ❌ - ❌ @@ -15115,11 +14984,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15141,6 +15005,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **browser cycles**: 1 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fnbpaint, fcp, dcf, loadtime * **page cycles**: 1 @@ -15170,6 +15035,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -15365,13 +15231,14 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Generates a constant value that can be changed to induce a regression. * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, geckoview, fenix + * **apps**: firefox, chrome, safari, geckoview, fenix * **browser cycles**: 1 * **browsertime args**: --browsertime.constant_value=1500 * **custom data**: true * **expected**: pass * **fore window**: 1 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **max back window**: 3 * **min back window**: 2 @@ -15428,13 +15295,14 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures the speed of key-range based reads of IndexedDB * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: ``__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -15497,11 +15365,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15555,11 +15418,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15597,11 +15455,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ❌ - ❌ @@ -15655,11 +15508,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15676,12 +15524,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Runs the Grandprix benchmark * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.grandprix_iterations=25 * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **output timeout**: 2000000 * **page cycles**: 1 @@ -15705,11 +15554,12 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **browser cycles**: 25 * **browsertime args**: --pageCompleteWaitTime=1000 --pageCompleteCheckInactivity=true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -15916,6 +15766,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **expected**: pass * **fore window**: 1 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **max back window**: 3 * **min back window**: 2 @@ -15946,6 +15797,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -16138,7 +15990,8 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Owner**: PerfTest Team - **Description**: A pageload benchmark test that quickly runs through all of the pages available to produce a single overall metric. + **Description**: A pageload benchmark test that quickly runs through a selection of the pages available to produce a single overall metric for fcp, lcp, and loadtime. The metrics from all pages get summarized with the geomean, and mean. + * **alert threshold**: 2.0 * **apps**: firefox, chrome, chromium, safari @@ -16147,6 +16000,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **output timeout**: 2000000 * **page cycles**: 1 @@ -16348,12 +16202,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures http/2 file upload throughput * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:false --chrome.args disable-quic * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **output timeout**: 2000000 * **page cycles**: 1 @@ -16578,12 +16433,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures http/3 file upload throughput * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:true * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: false * **output timeout**: 2000000 * **page cycles**: 1 @@ -16813,6 +16669,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -17016,16 +16873,17 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm7-linux-firefox-amazon.manifest + * **playback pageset manifest**: mitm8-linux-firefox-amazon.manifest * **playback version**: 8.1.1 * **secondary url**: ``__ * **test url**: ``__ @@ -17105,8 +16963,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-amazon-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-amazon** @@ -17591,11 +17449,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -17988,11 +17847,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -18226,11 +18086,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -18315,8 +18176,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn** @@ -18699,32 +18560,33 @@ Tests for page-load performance. The links direct to the actual websites that ar -.. dropdown:: ebay - :class-container: anchor-id-ebay-d +.. dropdown:: docomo + :class-container: anchor-id-docomo-d **Owner**: PerfTest Team * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car - * **benchmark page**: true + * **apps**: firefox, chrome + * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm8-linux-firefox-ebay.manifest + * **playback pageset manifest**: mitm8-linux-firefox-docomo.manifest * **playback version**: 8.1.1 - * **secondary url**: ``__ - * **test url**: ``__ + * **secondary url**: ``__ + * **test url**: ``__ * **type**: pageload * **unit**: ms * **use live sites**: false * **Test Task**: - .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18733,19 +18595,30 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-ebay** + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-ebay** + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + .. list-table:: **test-linux1804-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18754,19 +18627,19 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-ebay** + * - **browsertime-tp7-chrome-docomo** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-ebay** + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-qr/opt** + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18775,19 +18648,30 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-ebay** + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-ebay** + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-shippable-qr/opt** + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18796,17 +18680,145 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-chrome-ebay** + * - **browsertime-tp7-chrome-docomo** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-firefox-ebay** - - ✅ - - ✅ + * - **browsertime-tp7-firefox-docomo** - ❌ - ❌ - * - **browsertime-tp6-live-chrome-ebay** + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + +.. dropdown:: ebay + :class-container: anchor-id-ebay-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome, safari, custom-car + * **benchmark page**: true + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: ``__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-ebay.manifest + * **playback version**: 8.1.1 + * **secondary url**: ``__ + * **test url**: ``__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-ebay** - ❌ - ❌ - ❌ @@ -18818,7 +18830,7 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18839,7 +18851,7 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-qr/opt** + .. list-table:: **test-linux1804-64-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -18860,7 +18872,80 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-shippable-qr/opt** + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-chrome-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-firefox-ebay** + - ✅ + - ✅ + - ❌ + - ❌ + * - **browsertime-tp6-live-chrome-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-ebay** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -19029,11 +19114,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19093,8 +19179,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-espn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-espn-nofis** @@ -19339,11 +19425,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19624,11 +19711,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19989,11 +20077,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -20386,11 +20475,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page complete wait time**: 8000 * **page cycles**: 25 @@ -20710,11 +20800,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21032,11 +21123,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21354,11 +21446,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21752,11 +21845,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21841,8 +21935,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-imdb-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-imdb** @@ -22182,11 +22276,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -22486,16 +22581,17 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm6-linux-firefox-instagram.manifest + * **playback pageset manifest**: mitm8-darwin-firefox-instagram.manifest * **playback version**: 8.1.1 * **secondary url**: ``__ * **test url**: ``__ @@ -22575,8 +22671,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-instagram-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-instagram** @@ -22986,11 +23082,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23309,11 +23406,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23632,11 +23730,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23950,11 +24049,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24235,11 +24335,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24466,31 +24567,33 @@ Tests for page-load performance. The links direct to the actual websites that ar -.. dropdown:: outlook - :class-container: anchor-id-outlook-d +.. dropdown:: openai + :class-container: anchor-id-openai-d **Owner**: PerfTest Team * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car - * **benchmark page**: true + * **apps**: firefox, chrome + * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm5-linux-firefox-live.manifest + * **playback pageset manifest**: mitm8-linux-firefox-openai.manifest * **playback version**: 8.1.1 - * **test url**: ``__ + * **secondary url**: ``__ + * **test url**: ``__ * **type**: pageload * **unit**: ms * **use live sites**: false * **Test Task**: - .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24499,19 +24602,30 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-outlook** + * - **browsertime-tp7-firefox-openai** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-outlook** + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + .. list-table:: **test-linux1804-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24520,19 +24634,19 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-outlook** + * - **browsertime-tp7-chrome-openai** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-outlook** + * - **browsertime-tp7-firefox-openai** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-qr/opt** + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24541,19 +24655,30 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-firefox-outlook** + * - **browsertime-tp7-firefox-openai** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-outlook** + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-linux1804-64-shippable-qr/opt** + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24562,17 +24687,144 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-chrome-outlook** + * - **browsertime-tp7-chrome-openai** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-firefox-outlook** - - ✅ - - ✅ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-chrome-outlook** + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + +.. dropdown:: outlook + :class-container: anchor-id-outlook-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome, safari, custom-car + * **benchmark page**: true + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: ``__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm5-linux-firefox-live.manifest + * **playback version**: 8.1.1 + * **test url**: ``__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-outlook** - ❌ - ❌ - ❌ @@ -24584,7 +24836,7 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24605,7 +24857,7 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-qr/opt** + .. list-table:: **test-linux1804-64-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24626,7 +24878,80 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-shippable-qr/opt** + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-chrome-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-firefox-outlook** + - ✅ + - ✅ + - ❌ + - ❌ + * - **browsertime-tp6-live-chrome-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-firefox-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-outlook** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -24790,11 +25115,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25112,11 +25438,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25435,11 +25762,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25524,8 +25852,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-reddit-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-reddit** @@ -25858,6 +26186,452 @@ Tests for page-load performance. The links direct to the actual websites that ar +.. dropdown:: samsung + :class-container: anchor-id-samsung-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: ``__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-samsung.manifest + * **playback version**: 8.1.1 + * **secondary url**: ``__ + * **test url**: ``__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + +.. dropdown:: tiktok + :class-container: anchor-id-tiktok-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: ``__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-tiktok.manifest + * **playback version**: 8.1.1 + * **secondary url**: ``__ + * **test url**: ``__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. dropdown:: tumblr :class-container: anchor-id-tumblr-d @@ -25865,11 +26639,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26188,11 +26963,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -26513,11 +27289,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26675,7 +27452,162 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-qr/opt** + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-bytecode-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-bytecode-firefox-twitter** + - ✅ + - ✅ + - ❌ + - ❌ + * - **browsertime-tp6-essential-chrome-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-custom-car-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-firefox-twitter** + - ✅ + - ✅ + - ❌ + - ❌ + * - **browsertime-tp6-live-chrome-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-bytecode-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-bytecode-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-chrome-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-chrome-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp6-bytecode-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-essential-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp6-live-firefox-twitter** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26701,7 +27633,7 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-macosx1015-64-shippable-qr/opt** + .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26742,7 +27674,34 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - .. list-table:: **test-windows10-32-qr/opt** + +.. dropdown:: weather + :class-container: anchor-id-weather-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: ``__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-weather.manifest + * **playback version**: 8.1.1 + * **secondary url**: ``__ + * **test url**: ``__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26751,24 +27710,14 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-bytecode-firefox-twitter** - - ❌ - - ❌ - - ❌ - - ❌ - * - **browsertime-tp6-essential-firefox-twitter** - - ❌ - - ❌ - - ❌ - - ❌ - * - **browsertime-tp6-live-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-windows10-32-shippable-qr/opt** + .. list-table:: **test-linux1804-64-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26777,34 +27726,51 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-bytecode-firefox-twitter** - - ❌ - - ❌ - - ❌ - - ❌ - * - **browsertime-tp6-essential-chrome-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-essential-firefox-twitter** + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-chrome-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-twitter** + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + .. list-table:: **test-macosx1015-64-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26813,24 +27779,35 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-bytecode-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-essential-firefox-twitter** + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-windows10-64-qr/opt** + .. list-table:: **test-windows10-32-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26839,24 +27816,35 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-bytecode-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-essential-firefox-twitter** + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - ❌ - .. list-table:: **test-windows10-64-shippable-qr/opt** + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -26865,32 +27853,44 @@ Tests for page-load performance. The links direct to the actual websites that ar - autoland - mozilla-release - mozilla-beta - * - **browsertime-tp6-bytecode-firefox-twitter** - - ✅ - - ✅ - - ❌ - - ❌ - * - **browsertime-tp6-essential-chrome-twitter** - - ❌ + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ - * - **browsertime-tp6-essential-custom-car-twitter** - - ❌ - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - * - **browsertime-tp6-essential-firefox-twitter** - - ✅ - - ✅ - ❌ - ❌ - * - **browsertime-tp6-live-chrome-twitter** + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** - ❌ - ❌ - ❌ - ❌ - * - **browsertime-tp6-live-firefox-twitter** + * - **browsertime-tp7-firefox-weather** - ❌ - ❌ - ❌ @@ -26905,11 +27905,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -27143,11 +28144,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -27232,8 +28234,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-wikipedia-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-wikipedia** @@ -27648,11 +28650,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28045,11 +29048,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28134,8 +29138,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-youtube-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-youtube** @@ -28483,11 +29487,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28724,11 +29729,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -28965,11 +29971,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 240000 @@ -29206,11 +30213,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -29447,11 +30455,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -29690,9 +30699,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29757,9 +30767,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29824,9 +30835,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29891,9 +30903,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29958,9 +30971,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30025,9 +31039,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30092,9 +31107,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30159,9 +31175,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30226,9 +31243,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30293,9 +31311,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30360,9 +31379,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30427,9 +31447,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30503,6 +31524,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30561,8 +31583,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-allrecipes-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-allrecipes-nofis** @@ -30599,6 +31621,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30682,8 +31705,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-amazon-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-amazon** @@ -31172,6 +32195,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31235,8 +32259,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-amazon-search-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-amazon-search-nofis** @@ -31278,6 +32302,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31361,8 +32386,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bild-de-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bild-de** @@ -31409,6 +32434,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31492,8 +32518,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bing-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bing** @@ -31540,6 +32566,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31623,8 +32650,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bing-search-restaurants-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bing-search-restaurants** @@ -31671,6 +32698,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31754,8 +32782,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-booking-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-booking** @@ -31802,6 +32830,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31885,8 +32914,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn** @@ -32280,6 +33309,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32363,8 +33393,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-ampstories-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn-ampstories** @@ -32411,6 +33441,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32494,8 +33525,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-dailymail-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-dailymail** @@ -32542,6 +33573,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32625,8 +33657,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-ebay-kleinanzeigen-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-ebay-kleinanzeigen** @@ -32673,6 +33705,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32756,8 +33789,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-ebay-kleinanzeigen-search-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-ebay-kleinanzeigen-search** @@ -32804,6 +33837,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32862,8 +33896,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-espn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-espn-nofis** @@ -33112,6 +34146,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33476,6 +34511,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -33559,8 +34595,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-facebook-cristiano-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-facebook-cristiano** @@ -33607,6 +34643,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33666,8 +34703,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-google-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-google-nofis** @@ -33704,6 +34741,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -33787,8 +34825,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-google-maps-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-google-maps** @@ -33835,6 +34873,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33919,8 +34958,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-google-search-restaurants-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-google-search-restaurants** @@ -33967,6 +35006,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -34050,8 +35090,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-imdb-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-imdb** @@ -34395,6 +35435,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -34479,8 +35520,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-instagram-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-instagram** @@ -34894,6 +35935,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -34952,8 +35994,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-microsoft-support-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-microsoft-support-nofis** @@ -34990,6 +36032,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35073,8 +36116,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-reddit-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-reddit** @@ -35418,6 +36461,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35491,8 +36535,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-sina-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-sina** @@ -35524,6 +36568,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35607,8 +36652,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-stackoverflow-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-stackoverflow** @@ -35655,6 +36700,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35738,8 +36784,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-wikipedia-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-wikipedia** @@ -36158,6 +37204,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36241,8 +37288,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-youtube-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-youtube** @@ -36586,6 +37633,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36644,8 +37692,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-youtube-watch-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-youtube-watch-nofis** @@ -36684,6 +37732,7 @@ Tests that perform a specific action (a scenario), i.e. idle application, idle a * **alert threshold**: 2.0 * **apps**: fenix, geckoview, refbrow * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fakeMeasure * **page cycles**: 1 @@ -36702,6 +37751,7 @@ Tests that perform a specific action (a scenario), i.e. idle application, idle a * **apps**: fenix, geckoview, refbrow * **browsertime args**: --browsertime.scenario_time=60000 --browsertime.background_app=false * **expected**: pass + * **link searchfox**: ``__ * **lower is better**: true * **measure**: fakeMeasure * **page cycles**: 1 @@ -36727,6 +37777,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36748,6 +37799,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36769,6 +37821,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36790,6 +37843,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: ``__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 diff --git a/testing/perfdocs/generated/talos.rst b/testing/perfdocs/generated/talos.rst index fe95ab6207..3a730e1035 100644 --- a/testing/perfdocs/generated/talos.rst +++ b/testing/perfdocs/generated/talos.rst @@ -1243,7 +1243,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -1439,32 +1438,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -2025,7 +1998,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -2353,7 +2325,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -2911,7 +2882,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 25 * unit: frame interval - * w7_counters: None * win_counters: None * Command @@ -3107,32 +3077,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -3258,7 +3202,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -3454,32 +3397,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -4127,32 +4044,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -4279,7 +4170,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -4475,32 +4365,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -4627,7 +4491,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -4823,32 +4686,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -4975,7 +4812,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -5171,32 +5007,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -5323,7 +5133,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -5519,32 +5328,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -5671,7 +5454,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -5867,32 +5649,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -6019,7 +5775,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -6215,32 +5970,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -6367,7 +6096,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -6563,7 +6291,7 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** + .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -6573,34 +6301,8 @@ For the sample commands found below, note that the capitalization used is import - mozilla-release - mozilla-beta * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - - .. list-table:: **test-windows10-64-shippable-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ✅ - - ✅ + - ✅ + - ✅ - ❌ - ❌ * - **talos-webgl-gli** @@ -6715,7 +6417,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -6911,32 +6612,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -7063,7 +6738,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -7259,32 +6933,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -7411,7 +7059,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -7607,32 +7254,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -7759,7 +7380,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -7955,32 +7575,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -8107,7 +7701,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 5 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -8303,32 +7896,6 @@ For the sample commands found below, note that the capitalization used is import - ❌ - .. list-table:: **test-windows10-64-ref-hw-2017-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-webgl** - - ❌ - - ❌ - - ❌ - - ❌ - * - **talos-webgl-gli** - - ✅ - - ✅ - - ❌ - - ❌ - * - **talos-webgl-swr** - - ❌ - - ❌ - - ❌ - - ❌ - - .. list-table:: **test-windows10-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -8425,11 +7992,18 @@ For the sample commands found below, note that the capitalization used is import * source: * type: `Page load`_ * reporting: time from *performance.timing.navigationStart* to *pagerendered* event in ms (lower is better) - * data: load a PDF 20 times + * data: loads a PDF 5 times + * description: + | Runs through a set of chunks. Each chunk runs 100 PDFs with 5 iterations each. + If --pdfPaintChunk is not used when running the test locally, all PDFs will be tested + by default with only 1 cycle each. The PDFs that are run are found in the Mozilla pdf.js + repository, and this test pulls those in for testing locally through a toolchain artifact + called talos-pdfs. * pdfpaint: True - * timeout: 600 + * timeout: 1800 * tpmanifest: ${talos}/tests/pdfpaint/pdfpaint.manifest - * tppagecycles: 20 + * tppagecycles: 1 + * tptimeout: 60000 * unit: ms * Command @@ -8448,158 +8022,768 @@ For the sample commands found below, note that the capitalization used is import - autoland - mozilla-release - mozilla-beta - * - **talos-other** + * - **talos-pdfpaint-1** - ❌ - ❌ - ❌ - ❌ - * - **talos-other-swr** + * - **talos-pdfpaint-1-swr** - ❌ - ❌ - ❌ - ❌ - - - .. list-table:: **test-linux1804-64-shippable-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** - - ✅ - - ✅ + * - **talos-pdfpaint-10** - ❌ - ❌ - * - **talos-other-profiling** - ❌ - ❌ + * - **talos-pdfpaint-10-swr** - ❌ - ❌ - * - **talos-other-swr** - - ✅ - - ✅ - ❌ - ❌ - - - .. list-table:: **test-macosx1015-64-shippable-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** - - ✅ - - ✅ + * - **talos-pdfpaint-2** - ❌ - ❌ - * - **talos-other-profiling** - ❌ - ❌ + * - **talos-pdfpaint-2-swr** - ❌ - ❌ - * - **talos-other-swr** - - ✅ - - ✅ - ❌ - ❌ - - - .. list-table:: **test-windows10-32-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** + * - **talos-pdfpaint-3** - ❌ - ❌ - ❌ - ❌ - * - **talos-other-swr** + * - **talos-pdfpaint-3-swr** - ❌ - ❌ - ❌ - ❌ - - - .. list-table:: **test-windows10-32-shippable-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** + * - **talos-pdfpaint-4** - ❌ - ❌ - ❌ - ❌ - * - **talos-other-profiling** + * - **talos-pdfpaint-4-swr** - ❌ - ❌ - ❌ - ❌ - * - **talos-other-swr** + * - **talos-pdfpaint-5** - ❌ - ❌ - ❌ - ❌ - - - .. list-table:: **test-windows10-64-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** + * - **talos-pdfpaint-5-swr** - ❌ - ❌ - ❌ - ❌ - * - **talos-other-swr** + * - **talos-pdfpaint-6** - ❌ - ❌ - ❌ - ❌ - - - .. list-table:: **test-windows10-64-shippable-qr/opt** - :widths: 30 15 15 15 15 - :header-rows: 1 - - * - **Test Name** - - mozilla-central - - autoland - - mozilla-release - - mozilla-beta - * - **talos-other** - - ✅ - - ✅ + * - **talos-pdfpaint-6-swr** - ❌ - ❌ - * - **talos-other-profiling** - ❌ - ❌ + * - **talos-pdfpaint-7** - ❌ - ❌ - * - **talos-other-swr** + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ✅ + - ✅ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ✅ + - ✅ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ❌ + - ❌ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **talos-pdfpaint-1** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-1-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-10-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-2-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-3-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-4-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-5-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-6-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-7-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-8-swr** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9** + - ✅ + - ✅ + - ❌ + - ❌ + * - **talos-pdfpaint-9-swr** - ✅ - ✅ - ❌ @@ -9062,7 +9246,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 10 * unit: score - * w7_counters: None * win_counters: None * Command @@ -9274,7 +9457,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 10 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -11146,7 +11328,6 @@ For the sample commands found below, note that the capitalization used is import * tpmozafterpaint: False * tppagecycles: 25 * unit: ms - * w7_counters: None * win_counters: None * Command @@ -11361,7 +11542,6 @@ For the sample commands found below, note that the capitalization used is import * tppagecycles: 1 * tptimeout: 10000 * unit: ms - * w7_counters: [] * win_counters: [] * xperf_counters: ['main_startup_fileio', 'main_startup_netio', 'main_normal_fileio', 'main_normal_netio', 'nonmain_startup_fileio', 'nonmain_normal_fileio', 'nonmain_normal_netio', 'mainthread_readcount', 'mainthread_readbytes', 'mainthread_writecount', 'mainthread_writebytes', 'time_to_session_store_window_restored_ms'] * xperf_providers: ['PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO', 'FILE_IO_INIT'] @@ -11524,7 +11704,6 @@ For the sample commands found below, note that the capitalization used is import * tppagecycles: 25 * tptimeout: 5000 * unit: ms - * w7_counters: ['% Processor Time'] * win_counters: ['% Processor Time'] * Command diff --git a/testing/perfdocs/generated/webextension.rst b/testing/perfdocs/generated/webextension.rst index e1883755a4..b688c22f32 100644 --- a/testing/perfdocs/generated/webextension.rst +++ b/testing/perfdocs/generated/webextension.rst @@ -21,7 +21,7 @@ Warm Page-Load ============== For warm page-load tests, the browser is just started up once; so the browser is warm on each page-load. -**Raptor warm page-load test process when running on Firefox/Chrome/Chromium desktop:** +**Raptor warm page-load test process when running on Firefox/Chrome desktop:** * A new browser profile is created * The desktop browser is started up @@ -45,7 +45,7 @@ Cold Page-Load ============== For cold page-load tests, the browser is shut down and restarted between page load cycles, so the browser is cold on each page-load. This is what happens for Raptor cold page-load tests: -**Raptor cold page-load test process when running on Firefox/Chrome/Chromium desktop:** +**Raptor cold page-load test process when running on Firefox/Chrome desktop:** * A new browser profile is created * The desktop browser is started up -- cgit v1.2.3