diff options
Diffstat (limited to 'testing/perfdocs/generated/raptor.rst')
| -rw-r--r-- | testing/perfdocs/generated/raptor.rst | 1960 |
1 files changed, 1507 insertions, 453 deletions
diff --git a/testing/perfdocs/generated/raptor.rst b/testing/perfdocs/generated/raptor.rst index 789fbb6883..d32718e0ca 100644 --- a/testing/perfdocs/generated/raptor.rst +++ b/testing/perfdocs/generated/raptor.rst @@ -45,10 +45,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/ares6.toml#16>`__ * **lower is better**: true * **page cycles**: 4 * **page timeout**: 270000 @@ -283,10 +284,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: PerfTest Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/assorted-dom.toml#15>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -523,10 +525,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/jetstream2.toml#16>`__ * **lower is better**: false * **page cycles**: 4 * **page timeout**: 2000000 @@ -764,10 +767,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/matrix-react-bench.toml#16>`__ * **lower is better**: true * **page cycles**: 30 * **page timeout**: 2000000 @@ -1000,7 +1004,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Graphics Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 1 * **cold**: true * **expected**: pass @@ -1010,6 +1014,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#12>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1350,6 +1355,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#11>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1685,12 +1691,13 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-desktop.toml#16>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -1972,6 +1979,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-mobile.toml#16>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2251,12 +2259,13 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose browser profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-desktop.toml#18>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2548,6 +2557,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-animometer-mobile.toml#18>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -2837,13 +2847,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-desktop.toml#17>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -3126,6 +3137,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-mobile.toml#17>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -3405,7 +3417,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Graphics Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 1 * **cold**: true * **expected**: pass @@ -3415,6 +3427,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#27>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -3755,6 +3768,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#26>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 600000 @@ -4090,13 +4104,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose chrome trace**: true * **expose gecko profiler**: true * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-desktop.toml#19>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4389,6 +4404,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gecko profile features**: stackwalk,js,cpu,java,processcpu * **gecko profile interval**: 1 * **gecko profile threads**: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-htmlsuite-mobile.toml#19>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4678,10 +4694,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#16>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -4733,8 +4750,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-geckoview-nofis** @@ -5076,6 +5093,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#16>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -5127,8 +5145,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer-mobile-geckoview-nofis** @@ -5470,13 +5488,14 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: Performance Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **browser cycles**: 5 * **cold**: true * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#18>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 600000 @@ -5532,8 +5551,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** @@ -5563,12 +5582,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5594,12 +5613,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5873,6 +5892,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1 * **host from parent**: false + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#18>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -5928,8 +5948,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** @@ -5959,12 +5979,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -5990,12 +6010,12 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-fenix-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ * - **browsertime-benchmark-speedometer3-mobile-geckoview-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -6266,10 +6286,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :emelio and Layout Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/stylebench.toml#16>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 140000 @@ -6504,10 +6525,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jandem and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/sunspider.toml#10>`__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 55000 @@ -6744,6 +6766,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/twitch-animation.toml#17>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2000000 @@ -6957,10 +6980,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :jgilbert and Graphics(gfx) Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-desktop.toml#14>`__ * **lower is better**: false * **page cycles**: 5 * **page timeout**: 420000 @@ -7139,11 +7163,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ - ❌ - * - **browsertime-benchmark-safari-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ .. list-table:: **test-windows10-32-qr/opt** @@ -7243,6 +7262,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-mobile.toml#14>`__ * **lower is better**: false * **page cycles**: 1 * **page timeout**: 420000 @@ -7421,11 +7441,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - ❌ - ❌ - ❌ - * - **browsertime-benchmark-safari-unity-webgl** - - ❌ - - ❌ - - ❌ - - ❌ .. list-table:: **test-windows10-32-qr/opt** @@ -7525,10 +7540,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :lth and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot.toml#11>`__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -7766,6 +7782,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-baseline.toml#14>`__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -7979,6 +7996,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-optimizing.toml#14>`__ * **lower is better**: true * **newtab per cycle**: true * **page cycles**: 5 @@ -8188,10 +8206,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :lth and SpiderMonkey Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc.toml#14>`__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8426,6 +8445,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-baseline.toml#13>`__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8641,6 +8661,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-optimizing.toml#13>`__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 1200000 @@ -8852,10 +8873,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr **Owner**: :padenot and Media Team * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **expected**: pass * **expose browser profiler**: true * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/webaudio.toml#10>`__ * **lower is better**: true * **page cycles**: 5 * **page timeout**: 360000 @@ -9095,6 +9117,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#24>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9117,6 +9140,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#52>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9129,6 +9153,38 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **use live sites**: true * **Test Task**: + .. list-table:: **test-android-hw-a51-11-0-aarch64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-android-hw-a51-11-0-aarch64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis** + - ✅ + - ❌ + - ❌ + - ❌ + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -9158,6 +9214,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#56>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9359,6 +9416,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#62>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9560,6 +9618,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#68>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9761,6 +9820,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#83>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -9961,6 +10021,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#98>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -9982,11 +10043,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-mobile-fenix-youtube-playback-h264-sfr-nofis** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis** - ❌ - ❌ @@ -10003,11 +10059,6 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - autoland - mozilla-release - mozilla-beta - * - **browsertime-mobile-fenix-youtube-playback-h264-sfr-nofis** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis** - ✅ - ❌ @@ -10220,6 +10271,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#102>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -10263,7 +10315,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr - mozilla-release - mozilla-beta * - **browsertime-mobile-fenix-youtube-playback-hfr-nofis** - - ❌ + - ✅ - ❌ - ❌ - ❌ @@ -10479,6 +10531,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#110>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -10680,6 +10733,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#116>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -10881,6 +10935,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#122>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -11082,6 +11137,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **gather cpuTime**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#137>`__ * **lower is better**: true * **page cycles**: 20 * **page timeout**: 2700000 @@ -11282,6 +11338,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#152>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11294,6 +11351,38 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **use live sites**: true * **Test Task**: + .. list-table:: **test-android-hw-a51-11-0-aarch64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-android-hw-a51-11-0-aarch64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis** + - ✅ + - ❌ + - ❌ + - ❌ + + .. list-table:: **test-linux1804-64-clang-trunk-qr/opt** :widths: 30 15 15 15 15 :header-rows: 1 @@ -11498,6 +11587,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#156>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11714,6 +11804,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#170>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -11930,6 +12021,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr * **expose browser profiler**: true * **gecko profile interval**: 1000 * **gecko profile threads**: MediaPlayback + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#184>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 2700000 @@ -12148,7 +12240,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small ArrayBuffers to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=1 * **custom data**: true @@ -12156,6 +12248,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#30>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12218,11 +12311,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12276,11 +12364,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12318,11 +12401,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ❌ - ❌ @@ -12376,11 +12454,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMab1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMab1** - ✅ - ✅ @@ -12397,7 +12470,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small ArrayBuffers to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=0 * **custom data**: true @@ -12405,6 +12478,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#34>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12467,11 +12541,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12525,11 +12594,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12567,11 +12631,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ❌ - ❌ @@ -12625,11 +12684,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMabN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMabN** - ✅ - ✅ @@ -12646,7 +12700,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Arrays to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Array --browsertime.atomic=1 * **custom data**: true @@ -12654,6 +12708,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#38>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12716,11 +12771,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12774,11 +12824,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12816,11 +12861,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ❌ - ❌ @@ -12874,11 +12914,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMar1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMar1** - ✅ - ✅ @@ -12895,7 +12930,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Arrays to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Array --browsertime.atomic=0 * **custom data**: true @@ -12903,6 +12938,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#42>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -12965,11 +13001,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13023,11 +13054,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13065,11 +13091,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ❌ - ❌ @@ -13123,11 +13144,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMarN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMarN** - ✅ - ✅ @@ -13144,7 +13160,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Blobs to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Blob --browsertime.atomic=1 * **custom data**: true @@ -13152,6 +13168,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#46>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13214,11 +13231,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13272,11 +13284,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13314,11 +13321,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ❌ - ❌ @@ -13372,11 +13374,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMbl1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMbl1** - ✅ - ✅ @@ -13393,7 +13390,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send many small Blobs to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=16384 --browsertime.chunk_size=1024 --browsertime.buffer_type=Blob --browsertime.atomic=0 * **custom data**: true @@ -13401,6 +13398,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#50>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13463,11 +13461,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13521,11 +13514,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13563,11 +13551,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ❌ - ❌ @@ -13621,11 +13604,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addMblN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addMblN** - ✅ - ✅ @@ -13642,7 +13620,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large ArrayBuffers to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=1 * **custom data**: true @@ -13650,6 +13628,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#54>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13712,11 +13691,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13770,11 +13744,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13812,11 +13781,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ❌ - ❌ @@ -13870,11 +13834,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAB1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAB1** - ✅ - ✅ @@ -13891,7 +13850,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large ArrayBuffers to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=ArrayBuffer --browsertime.atomic=0 * **custom data**: true @@ -13899,6 +13858,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#58>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -13961,11 +13921,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14019,11 +13974,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14061,11 +14011,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ❌ - ❌ @@ -14119,11 +14064,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkABN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkABN** - ✅ - ✅ @@ -14140,7 +14080,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large JS Arrays to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Array --browsertime.atomic=1 * **custom data**: true @@ -14148,6 +14088,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#62>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14210,11 +14151,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14268,11 +14204,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14310,11 +14241,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ❌ - ❌ @@ -14368,11 +14294,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkAR1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkAR1** - ✅ - ✅ @@ -14389,7 +14310,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large JS Arrays to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Array --browsertime.atomic=0 * **custom data**: true @@ -14397,6 +14318,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#66>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14459,11 +14381,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14517,11 +14434,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14559,11 +14471,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ❌ - ❌ @@ -14617,11 +14524,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkARN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkARN** - ✅ - ✅ @@ -14638,7 +14540,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large Blobs to IndexedDB in one transaction * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Blob --browsertime.atomic=1 * **custom data**: true @@ -14646,6 +14548,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#70>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14708,11 +14611,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14766,11 +14664,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14808,11 +14701,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ❌ - ❌ @@ -14866,11 +14754,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBL1** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBL1** - ✅ - ✅ @@ -14887,7 +14770,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Use add API to send a couple of large Blobs to IndexedDB independently * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.iterations=128 --browsertime.chunk_size=1048576 --browsertime.buffer_type=Blob --browsertime.atomic=0 * **custom data**: true @@ -14895,6 +14778,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#74>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -14957,11 +14841,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15015,11 +14894,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15057,11 +14931,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ❌ - ❌ @@ -15115,11 +14984,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-addkBLN** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-addkBLN** - ✅ - ✅ @@ -15141,6 +15005,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **browser cycles**: 1 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-custom.toml#18>`__ * **lower is better**: true * **measure**: fnbpaint, fcp, dcf, loadtime * **page cycles**: 1 @@ -15170,6 +15035,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-speculative.toml#18>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -15365,13 +15231,14 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Generates a constant value that can be changed to induce a regression. * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, geckoview, fenix + * **apps**: firefox, chrome, safari, geckoview, fenix * **browser cycles**: 1 * **browsertime args**: --browsertime.constant_value=1500 * **custom data**: true * **expected**: pass * **fore window**: 1 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-regression-test.toml#23>`__ * **lower is better**: true * **max back window**: 3 * **min back window**: 2 @@ -15428,13 +15295,14 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures the speed of key-range based reads of IndexedDB * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **custom data**: true * **expected**: pass * **gecko profile entries**: 131072000 * **gecko profile features**: js,stackwalk,cpu * **gecko profile threads**: GeckoMain,DOM Worker,IndexedDB + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#78>`__ * **lower is better**: true * **measure**: cpuTime * **output timeout**: 2000000 @@ -15497,11 +15365,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15555,11 +15418,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15597,11 +15455,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ❌ - ❌ @@ -15655,11 +15508,6 @@ Browsertime tests that use a custom pageload test script. These use the pageload - ❌ - ❌ - ❌ - * - **browsertime-indexeddb-chromium-getkeyrng** - - ❌ - - ❌ - - ❌ - - ❌ * - **browsertime-indexeddb-firefox-getkeyrng** - ✅ - ✅ @@ -15676,12 +15524,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Runs the Grandprix benchmark * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.grandprix_iterations=25 * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-grandprix.toml#16>`__ * **lower is better**: true * **output timeout**: 2000000 * **page cycles**: 1 @@ -15705,11 +15554,12 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium + * **apps**: firefox, chrome * **browser cycles**: 25 * **browsertime args**: --pageCompleteWaitTime=1000 --pageCompleteCheckInactivity=true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-process-switch.toml#19>`__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -15916,6 +15766,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **expected**: pass * **fore window**: 1 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-sample-python-support.toml#22>`__ * **lower is better**: true * **max back window**: 3 * **min back window**: 2 @@ -15946,6 +15797,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-throttled-pageload.toml#17>`__ * **lower is better**: true * **page cycles**: 1 * **page timeout**: 60000 @@ -16138,7 +15990,8 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Owner**: PerfTest Team - **Description**: A pageload benchmark test that quickly runs through all of the pages available to produce a single overall metric. + **Description**: A pageload benchmark test that quickly runs through a selection of the pages available to produce a single overall metric for fcp, lcp, and loadtime. The metrics from all pages get summarized with the geomean, and mean. + * **alert threshold**: 2.0 * **apps**: firefox, chrome, chromium, safari @@ -16147,6 +16000,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-tp6-bench.toml#19>`__ * **lower is better**: true * **output timeout**: 2000000 * **page cycles**: 1 @@ -16348,12 +16202,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures http/2 file upload throughput * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:false --chrome.args disable-quic * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#16>`__ * **lower is better**: false * **output timeout**: 2000000 * **page cycles**: 1 @@ -16578,12 +16433,13 @@ Browsertime tests that use a custom pageload test script. These use the pageload **Description**: Measures http/3 file upload throughput * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 1 * **browsertime args**: --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:true * **custom data**: true * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#31>`__ * **lower is better**: false * **output timeout**: 2000000 * **page cycles**: 1 @@ -16813,6 +16669,7 @@ Browsertime tests that use a custom pageload test script. These use the pageload * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-welcome.toml#17>`__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -17016,16 +16873,17 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#21>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm7-linux-firefox-amazon.manifest + * **playback pageset manifest**: mitm8-linux-firefox-amazon.manifest * **playback version**: 8.1.1 * **secondary url**: `<https://www.amazon.com/Acer-A515-46-R14K-Quad-Core-Processor-Backlit/dp/B08VKNVDDR/ref=sr_1_3?dchild=1&keywords=laptop&qid=1627047187&sr=8-3>`__ * **test url**: `<https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1>`__ @@ -17105,8 +16963,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-amazon-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-amazon** @@ -17591,11 +17449,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#27>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -17988,11 +17847,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#31>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -18226,11 +18086,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#36>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -18315,8 +18176,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn** @@ -18699,6 +18560,229 @@ Tests for page-load performance. The links direct to the actual websites that ar +.. dropdown:: docomo + :class-container: anchor-id-docomo-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp7/desktop/browsertime-tp7.toml#22>`__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-docomo.manifest + * **playback version**: 8.1.1 + * **secondary url**: `<https://www.docomo.ne.jp/iphone/?icid=CRP_menu_to_CRP_IPH>`__ + * **test url**: `<https://www.docomo.ne.jp/>`__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-docomo** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. dropdown:: ebay :class-container: anchor-id-ebay-d @@ -18706,11 +18790,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#40>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19029,11 +19114,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#45>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19093,8 +19179,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-espn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-espn-nofis** @@ -19339,11 +19425,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#50>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19624,11 +19711,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#54>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -19989,11 +20077,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#59>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -20386,11 +20475,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#64>`__ * **lower is better**: true * **page complete wait time**: 8000 * **page cycles**: 25 @@ -20710,11 +20800,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#69>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21032,11 +21123,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#74>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21354,11 +21446,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#78>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21752,11 +21845,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#84>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -21841,8 +21935,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-imdb-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-imdb** @@ -22182,11 +22276,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#88>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -22486,16 +22581,17 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#100>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 * **playback**: mitmproxy - * **playback pageset manifest**: mitm6-linux-firefox-instagram.manifest + * **playback pageset manifest**: mitm8-darwin-firefox-instagram.manifest * **playback version**: 8.1.1 * **secondary url**: `<https://www.instagram.com/nobelprize_org/>`__ * **test url**: `<https://www.instagram.com/>`__ @@ -22575,8 +22671,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-instagram-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-instagram** @@ -22986,11 +23082,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#105>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23309,11 +23406,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#110>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23632,11 +23730,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#114>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -23950,11 +24049,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#119>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24235,11 +24335,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#123>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24466,6 +24567,229 @@ Tests for page-load performance. The links direct to the actual websites that ar +.. dropdown:: openai + :class-container: anchor-id-openai-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp7/desktop/browsertime-tp7.toml#26>`__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-openai.manifest + * **playback version**: 8.1.1 + * **secondary url**: `<https://openai.com/blog>`__ + * **test url**: `<https://openai.com/>`__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-openai** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-openai** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. dropdown:: outlook :class-container: anchor-id-outlook-d @@ -24473,11 +24797,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#127>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -24790,11 +25115,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#131>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25112,11 +25438,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#135>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25435,11 +25762,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#141>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -25524,8 +25852,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-reddit-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-reddit** @@ -25858,6 +26186,452 @@ Tests for page-load performance. The links direct to the actual websites that ar +.. dropdown:: samsung + :class-container: anchor-id-samsung-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp7/desktop/browsertime-tp7.toml#31>`__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-samsung.manifest + * **playback version**: 8.1.1 + * **secondary url**: `<https://www.samsung.com/us/mobile>`__ + * **test url**: `<https://www.samsung.com/us>`__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-samsung** + - ❌ + - ❌ + - ❌ + - ❌ + + + +.. dropdown:: tiktok + :class-container: anchor-id-tiktok-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp7/desktop/browsertime-tp7.toml#27>`__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-tiktok.manifest + * **playback version**: 8.1.1 + * **secondary url**: `<https://www.tiktok.com/explore>`__ + * **test url**: `<https://www.tiktok.com/en>`__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-tiktok** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. dropdown:: tumblr :class-container: anchor-id-tumblr-d @@ -25865,11 +26639,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#147>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26188,11 +26963,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#152>`__ * **lower is better**: true * **measure**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **page cycles**: 25 @@ -26513,11 +27289,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#165>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -26898,6 +27675,229 @@ Tests for page-load performance. The links direct to the actual websites that ar +.. dropdown:: weather + :class-container: anchor-id-weather-d + + **Owner**: PerfTest Team + + * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint + * **alert threshold**: 2.0 + * **apps**: firefox, chrome + * **benchmark page**: false + * **browser cycles**: 25 + * **expected**: pass + * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp7/desktop/browsertime-tp7.toml#40>`__ + * **lower is better**: true + * **page cycles**: 25 + * **page timeout**: 60000 + * **playback**: mitmproxy + * **playback pageset manifest**: mitm8-linux-firefox-weather.manifest + * **playback version**: 8.1.1 + * **secondary url**: `<https://weather.com/en-CA/weather/monthly/l/43.67%2C-79.42>`__ + * **test url**: `<https://weather.com/en-CA/weather/today/l/43.67%2C-79.42>`__ + * **type**: pageload + * **unit**: ms + * **use live sites**: false + * **Test Task**: + + .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-linux1804-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-macosx1015-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-32-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. list-table:: **test-windows10-64-shippable-qr/opt** + :widths: 30 15 15 15 15 + :header-rows: 1 + + * - **Test Name** + - mozilla-central + - autoland + - mozilla-release + - mozilla-beta + * - **browsertime-tp7-chrome-weather** + - ❌ + - ❌ + - ❌ + - ❌ + * - **browsertime-tp7-firefox-weather** + - ❌ + - ❌ + - ❌ + - ❌ + + + .. dropdown:: wikia :class-container: anchor-id-wikia-d @@ -26905,11 +27905,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#170>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -27143,11 +28144,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#175>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -27232,8 +28234,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-wikipedia-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-wikipedia** @@ -27648,11 +28650,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: false * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#179>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28045,11 +29048,12 @@ Tests for page-load performance. The links direct to the actual websites that ar * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange, largestContentfulPaint * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari, custom-car + * **apps**: firefox, chrome, safari, custom-car * **benchmark page**: true * **browser cycles**: 25 * **expected**: pass * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.toml#184>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28134,8 +29138,8 @@ Tests for page-load performance. The links direct to the actual websites that ar - ❌ - ❌ * - **browsertime-tp6m-fenix-youtube-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-youtube** @@ -28483,11 +29487,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/interactive/browsertime-responsiveness.toml#24>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -28724,11 +29729,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/interactive/browsertime-responsiveness.toml#39>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -28965,11 +29971,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/interactive/browsertime-responsiveness.toml#63>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 240000 @@ -29206,11 +30213,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/interactive/browsertime-responsiveness.toml#79>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -29447,11 +30455,12 @@ Browsertime tests that interact with the webpage. Includes responsiveness tests * **accept zero vismet**: true * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: firefox, chrome, chromium, safari + * **apps**: firefox, chrome, safari * **browser cycles**: 10 * **expected**: pass * **gecko profile interval**: 1 * **interactive**: true + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/interactive/browsertime-responsiveness.toml#100>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 90000 @@ -29690,9 +30699,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#16>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29757,9 +30767,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#19>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29824,9 +30835,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#22>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29891,9 +30903,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#25>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -29958,9 +30971,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#28>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30025,9 +31039,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#31>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30092,9 +31107,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#34>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30159,9 +31175,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#37>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30226,9 +31243,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#40>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30293,9 +31311,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#43>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30360,9 +31379,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#46>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30427,9 +31447,10 @@ A set of test pages that are run as live sites instead of recorded versions. The * **alert on**: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange * **alert threshold**: 2.0 - * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome, chromium + * **apps**: geckoview, fenix, refbrow, chrome-m, firefox, chrome * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/live/browsertime-live.toml#49>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30503,6 +31524,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#21>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30561,8 +31583,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-allrecipes-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-allrecipes-nofis** @@ -30599,6 +31621,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#24>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -30682,8 +31705,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-amazon-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-amazon** @@ -31172,6 +32195,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#27>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31235,8 +32259,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-amazon-search-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-amazon-search-nofis** @@ -31278,6 +32302,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#30>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31361,8 +32386,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bild-de-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bild-de** @@ -31409,6 +32434,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#34>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31492,8 +32518,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bing-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bing** @@ -31540,6 +32566,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#37>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31623,8 +32650,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-bing-search-restaurants-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-bing-search-restaurants** @@ -31671,6 +32698,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#40>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31754,8 +32782,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-booking-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-booking** @@ -31802,6 +32830,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#43>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -31885,8 +32914,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn** @@ -32280,6 +33309,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#46>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32363,8 +33393,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-cnn-ampstories-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-cnn-ampstories** @@ -32411,6 +33441,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#49>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32494,8 +33525,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-dailymail-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-dailymail** @@ -32542,6 +33573,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#52>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32625,8 +33657,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-ebay-kleinanzeigen-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-ebay-kleinanzeigen** @@ -32673,6 +33705,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#55>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32756,8 +33789,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-ebay-kleinanzeigen-search-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-ebay-kleinanzeigen-search** @@ -32804,6 +33837,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#58>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -32862,8 +33896,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-espn-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-espn-nofis** @@ -33112,6 +34146,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#61>`__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33476,6 +34511,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#66>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -33559,8 +34595,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-facebook-cristiano-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-facebook-cristiano** @@ -33607,6 +34643,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#69>`__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33666,8 +34703,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-google-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-google-nofis** @@ -33704,6 +34741,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#74>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -33787,8 +34825,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-google-maps-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-google-maps** @@ -33835,6 +34873,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#77>`__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -33919,8 +34958,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-google-search-restaurants-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-google-search-restaurants** @@ -33967,6 +35006,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#82>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -34050,8 +35090,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-imdb-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-imdb** @@ -34395,6 +35435,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#85>`__ * **login**: true * **lower is better**: true * **page cycles**: 25 @@ -34479,8 +35520,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-instagram-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-instagram** @@ -34894,6 +35935,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#90>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -34952,8 +35994,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-microsoft-support-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-microsoft-support-nofis** @@ -34990,6 +36032,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#93>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35073,8 +36116,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-reddit-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-reddit** @@ -35418,6 +36461,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#96>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35491,8 +36535,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-sina-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-sina** @@ -35524,6 +36568,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#100>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35607,8 +36652,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-stackoverflow-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-stackoverflow** @@ -35655,6 +36700,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#103>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -35738,8 +36784,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-wikipedia-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-wikipedia** @@ -36158,6 +37204,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#106>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36241,8 +37288,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-fenix-youtube-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-geckoview-youtube** @@ -36586,6 +37633,7 @@ Page-load performance test suite on Android. The links direct to the actual webs * **benchmark page**: true * **browser cycles**: 15 * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.toml#109>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36644,8 +37692,8 @@ Page-load performance test suite on Android. The links direct to the actual webs - ❌ - ❌ * - **browsertime-tp6m-essential-fenix-youtube-watch-nofis** - - ❌ - - ❌ + - ✅ + - ✅ - ❌ - ❌ * - **browsertime-tp6m-essential-geckoview-youtube-watch-nofis** @@ -36684,6 +37732,7 @@ Tests that perform a specific action (a scenario), i.e. idle application, idle a * **alert threshold**: 2.0 * **apps**: fenix, geckoview, refbrow * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/scenario/idle.toml#14>`__ * **lower is better**: true * **measure**: fakeMeasure * **page cycles**: 1 @@ -36702,6 +37751,7 @@ Tests that perform a specific action (a scenario), i.e. idle application, idle a * **apps**: fenix, geckoview, refbrow * **browsertime args**: --browsertime.scenario_time=60000 --browsertime.background_app=false * **expected**: pass + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/scenario/idle.toml#16>`__ * **lower is better**: true * **measure**: fakeMeasure * **page cycles**: 1 @@ -36727,6 +37777,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/unittests/browsertime-tp6-unittest.toml#17>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36748,6 +37799,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/unittests/browsertime-tp6-unittest.toml#20>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36769,6 +37821,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/unittests/browsertime-tp6-unittest.toml#23>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 @@ -36790,6 +37843,7 @@ These tests aren't used in standard testing, they are only used in the Raptor un * **expected**: pass * **gecko profile entries**: 14000000 * **gecko profile interval**: 1 + * **link searchfox**: `<https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/unittests/browsertime-tp6-unittest.toml#26>`__ * **lower is better**: true * **page cycles**: 25 * **page timeout**: 60000 |