Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-11-23 21:01:08 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Backed out changeset c5618823c6b5 (bug 1716825) for causing Btime failures. CLOSED TREE

Browse Source
This commit is contained in:
Butkovits Atila 2021-08-07 12:08:15 +03:00
parent f434e44ced
commit 1147123b55
13 changed files with 17 additions and 138 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
taskcluster/ci/test/browsertime-desktop.yml
View File

@ -90,7 +90,6 @@ browsertime-tp6:
run-visual-metrics: true
treeherder-symbol: Btime(tp6)
raptor-subtests:
- [amazon-sec, amzn-sec]
- buzzfeed
- ebay
- espn

64
testing/perfdocs/generated/raptor.rst
View File

@ -2421,70 +2421,6 @@ Tests for page-load performance. The links direct to the actual websites that ar
* browsertime-tp6-live-firefox-amazon-fis-e10s: None
.. dropdown:: amazon-sec (BT, FF, CH, CU)
:container: + anchor-id-amazon-sec-d
* **alert on**: fcp, loadtime
* **alert threshold**: 2.0
* **apps**: firefox, chrome, chromium
* **browser cycles**: 25
* **expected**: pass
* **gecko profile entries**: 14000000
* **gecko profile interval**: 1
* **lower is better**: true
* **measure**: fnbpaint, fcp, dcf, loadtime
* **page cycles**: 25
* **page timeout**: 60000
* **playback**: mitmproxy
* **playback pageset manifest**: mitm6-linux-firefox-amazon-sec.manifest
* **playback version**: 6.0.2
* **secondary url**: `<https://www.amazon.com/Acer-A515-46-R14K-Quad-Core-Processor-Backlit/dp/B08VKNVDDR/ref=sr_1_3?dchild=1&keywords=laptop&qid=1627047187&sr=8-3>`__
* **test url**: `<https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1>`__
* **type**: pageload
* **unit**: ms
* **use live sites**: false
* **Test Task**:
* test-linux1804-64-clang-trunk/opt
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-linux1804-64-qr/opt
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-linux1804-64-shippable-qr/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: mozilla-beta, trunk
* browsertime-tp6-firefox-amazon-sec-fis-e10s: mozilla-central
* test-macosx1014-64-shippable-qr/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: trunk
* browsertime-tp6-firefox-amazon-sec-fis-e10s: mozilla-central
* test-macosx1015-64-shippable-qr/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: mozilla-beta, trunk
* browsertime-tp6-firefox-amazon-sec-fis-e10s: mozilla-central
* test-macosx1015-64-shippable/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-windows10-32-qr/opt
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-windows10-32-shippable-qr/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-fis-e10s: None
* test-windows10-64-qr/opt
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-windows10-64-ref-hw-2017/opt
* browsertime-tp6-firefox-amazon-sec-e10s: None
* test-windows10-64-shippable-qr/opt
* browsertime-tp6-chrome-amazon-sec-e10s: None
* browsertime-tp6-chromium-amazon-sec-e10s: None
* browsertime-tp6-firefox-amazon-sec-e10s: mozilla-beta, trunk
* browsertime-tp6-firefox-amazon-sec-fis-e10s: mozilla-central
.. dropdown:: bing-search (BT, FF, CH, CU)
:container: + anchor-id-bing-search-d

5
testing/performance/hooks_recording.py
View File

@ -75,10 +75,7 @@ def before_runs(env):
recording_file,
)
add_option(
env, "browsertime.test_url", test_site.get("test_url"), overwrite=True
)
add_option(env, "browsertime.secondary_url", test_site.get("secondary_url"))
add_option(env, "browsertime.url", test_site.get("test_url"), overwrite=True)
add_option(env, "browsertime.screenshot", "true")
add_option(env, "browsertime.testName", test_site.get("name"))

16
testing/performance/pageload_sites.json
View File

@ -142,12 +142,6 @@
"name": "amazon",
"test_url": "https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1"
},
{
"login": false,
"name": "amazon-sec",
"test_url": "https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1",
"secondary_url": "https://www.amazon.com/Acer-A515-46-R14K-Quad-Core-Processor-Backlit/dp/B08VKNVDDR/ref=sr_1_3?dchild=1&keywords=laptop&qid=1627047187&sr=8-3"
},
{
"login": false,
"name": "bing-search",
@ -183,16 +177,16 @@
"name": "facebook",
"test_url": "https://www.facebook.com"
},
{
"login": false,
"name": "fandom",
"test_url": "https://www.fandom.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase"
},
{
"login": true,
"name": "google",
"test_url": "https://www.google.com/search?hl=en&q=barack+obama&cad=h"
},
{
"login": false,
"name": "fandom",
"test_url": "https://www.fandom.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase"
},
{
"login": false,
"name": "google-docs",

18
testing/performance/perftest_record.js
View File

@ -5,23 +5,11 @@
"use strict";
async function test(context, commands) {
let testUrl = context.options.browsertime.test_url;
let secondaryUrl = context.options.browsertime.secondary_url;
let rootUrl = context.options.browsertime.url;
let testName = context.options.browsertime.testName;
// Wait for browser to settle
await commands.wait.byTime(1000);
await commands.measure.start(testUrl);
commands.screenshot.take("test_url_" + testName);
if (secondaryUrl !== null) {
// Wait for browser to settle
await commands.wait.byTime(1000);
await commands.measure.start(secondaryUrl);
commands.screenshot.take("secondary_url_" + testName);
}
await commands.measure.start(rootUrl);
commands.screenshot.take(testName);
// Wait for browser to settle
await commands.wait.byTime(1000);

15
testing/raptor/browsertime/browsertime_pageload.js
View File

@ -6,8 +6,7 @@
module.exports = async function(context, commands) {
context.log.info("Starting a browsertime pageload");
let test_url = context.options.browsertime.test_url;
let secondary_url = context.options.browsertime.secondary_url;
let url = context.options.browsertime.url;
let page_cycles = context.options.browsertime.page_cycles;
let page_cycle_delay = context.options.browsertime.page_cycle_delay;
let post_startup_delay = context.options.browsertime.post_startup_delay;
@ -19,20 +18,14 @@ module.exports = async function(context, commands) {
await commands.wait.byTime(post_startup_delay);
for (let count = 0; count < page_cycles; count++) {
if (count !== 0 && secondary_url !== undefined) {
context.log.info("Navigating to secondary url:" + secondary_url);
await commands.navigate(secondary_url);
} else {
context.log.info("Navigating to about:blank");
await commands.navigate("about:blank");
}
context.log.info("Navigating to about:blank");
await commands.navigate("about:blank");
context.log.info("Navigating to primary url:" + test_url);
context.log.info("Cycle %d, waiting for %d ms", count, page_cycle_delay);
await commands.wait.byTime(page_cycle_delay);
context.log.info("Cycle %d, starting the measure", count);
await commands.measure.start(test_url);
await commands.measure.start(url);
}
context.log.info("Browsertime pageload ended.");
return true;

6
testing/raptor/raptor/browsertime/base.py
View File

@ -221,11 +221,7 @@ class Browsertime(Perftest):
browsertime_script.extend(
["--browsertime.page_cycles", str(test.get("page_cycles", 1))]
)
browsertime_script.extend(["--browsertime.test_url", test["test_url"]])
if test.get("secondary_url"):
browsertime_script.extend(
["--browsertime.secondary_url", test.get("secondary_url")]
)
browsertime_script.extend(["--browsertime.url", test["test_url"]])
# Raptor's `pageCycleDelay` delay (ms) between pageload cycles
browsertime_script.extend(["--browsertime.page_cycle_delay", "1000"])

3
testing/raptor/raptor/perfdocs/config.yml
View File

@ -10,7 +10,6 @@ suites:
description: "Tests for page-load performance. The links direct to the actual websites that are being tested. (WX: WebExtension, BT: Browsertime, FF: Firefox, CH: Chrome, CU: Chromium)"
tests:
amazon: "BT, FF, CH, CU"
amazon-sec: "BT, FF, CH, CU"
bing-search: "BT, FF, CH, CU"
buzzfeed: "BT, FF, CH, CU"
cnn: "BT, FF, CH, CU"
@ -186,4 +185,4 @@ suites:
raptor-tp6-unittest-amazon-firefox: "FF"
raptor-tp6-unittest-facebook-firefox: "FF"
raptor-tp6-unittest-google-firefox: "FF"
raptor-tp6-unittest-youtube-firefox: "FF"
raptor-tp6-unittest-youtube-firefox: "FF"

6
testing/raptor/raptor/tests/tp6/desktop/browsertime-tp6.ini
View File

@ -27,12 +27,6 @@ use_live_sites = false
[amazon]
test_url = https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1
[amazon-sec]
test_url = https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1
secondary_url = https://www.amazon.com/Acer-A515-46-R14K-Quad-Core-Processor-Backlit/dp/B08VKNVDDR/ref=sr_1_3?dchild=1&keywords=laptop&qid=1627047187&sr=8-3
playback_version = 6.0.2
playback_pageset_manifest = mitm6-linux-firefox-amazon-sec.manifest
[bing-search]
test_url = https://www.bing.com/search?q=barack+obama

9
testing/raptor/raptor/tooltool-manifests/playback/mitm6-linux-firefox-amazon-sec.manifest
View File

@ -1,9 +0,0 @@
[
{
"size": 20179208,
"visibility": "public",
"digest": "3980b2e1eb978afb6c869a65b7673d2d47410e926ee9003622cdcf51d64b5d04ea92aa4f73b34da9b2cf0b6fcd19b8555895c8ff668b6c9a0955507495050186",
"algorithm": "sha512",
"filename": "mitm6-linux-firefox-amazon-sec.zip"
}
]

6
testing/raptor/test/conftest.py
View File

@ -51,11 +51,7 @@ def raptor(options):
@pytest.fixture
def mock_test():
return {
"name": "raptor-firefox-tp6",
"test_url": "/dummy/url",
"secondary_url": "/dummy/url-2",
}
return {"name": "raptor-firefox-tp6", "test_url": "/dummy/url"}
@pytest.fixture(scope="session")

4
testing/raptor/test/test_raptor.py
View File

@ -236,10 +236,8 @@ def test_cmd_arguments(ConcreteBrowsertime, browsertime_options, mock_test):
browsertime_options["browsertime_geckodriver"],
"--browsertime.page_cycles",
"1",
"--browsertime.test_url",
"--browsertime.url",
mock_test["test_url"],
"--browsertime.secondary_url",
mock_test["secondary_url"],
"--browsertime.page_cycle_delay",
"1000",
"--browsertime.post_startup_delay",

2
tools/lint/perfdocs/framework_gatherers.py
View File

@ -238,8 +238,6 @@ class RaptorGatherer(FrameworkGatherer):
description[key] = description[key].replace("<", "\<")
description[key] = description[key].replace(">", "\>")
result += f" * **{sub_title}**: `<{description[key]}>`__\n"
elif key == "secondary_url":
result += f" * **{sub_title}**: `<{description[key]}>`__\n"
elif key in ["playback_pageset_manifest"]:
result += (
f" * **{sub_title}**: "