Bug 1773621 - Remove panda.tv from list of sites tested on WebPageTest. r=perftest-reviewers,AlexandruIonescu

A week ago we received a notification that we had a test that the WPT chrome tests were perma failing on [[ https://bugzilla.mozilla.org/show_bug.cgi?id=1773621 | bugzilla ]]
After going through the fail logs I realized it was because of website "panda.tv" directing to a unable to connect page message, after some digging it was not returning proper data because panda.tv has not been a company since March 2019(bankruptcy filing), why this only is causing an issue now I believe is because of some kind of update from WPT as I can see a noticeable UI difference on the test results page from before and after the failures started.

My resolution was to remove Panda.tv from our test list and that seems to have resolved the issue.

I also updated the error message to display which website is causing the issue so that if this happens again I don't need to go through each and every webpagetest result to know which of the 40 websites are having an issue.

Differential Revision: https://phabricator.services.mozilla.com/D149642
This commit is contained in:
andrej 2022-06-20 14:16:41 +00:00
parent 0e4b106ace
commit b5f60d8890
4 changed files with 5 additions and 3 deletions

View File

@ -359,7 +359,9 @@ class WebPageTest(Layer):
self.error(f"Fail {wpt_run['data']['url']}")
return
if value not in wpt_run["data"][statistic][view].keys():
raise WPTDataProcessingError(f"{value} not found wpt results")
raise WPTDataProcessingError(
f"{value} not found {wpt_run['data']['url']}"
)
desired_values[f"{value}.{view}.{statistic}"] = int(
wpt_run["data"][statistic][view][value]
)

View File

@ -68,6 +68,7 @@ def init_placeholder_wpt_data(fvonly=False, invalid_results=False):
"standardDeviation": views,
"median": views,
"runs": {"1": {"firstView": {"browserVersion": 101.1}}},
"url": "testurl.ca",
},
"webPagetestVersion": 21.0,
}
@ -87,6 +88,7 @@ def init_mocked_request(status_code, **kwargs):
"ec2-us-east-1": {"PendingTests": {"Queued": 3}, "Label": "California"},
"jsonUrl": "mock_test.com",
"summary": "Just a pageload test",
"url": "testurl.ca",
},
"statusCode": status_code,
}

View File

@ -70,7 +70,6 @@ module.exports = {
"netflix.com",
"microsoft.com",
"instagram.com",
"panda.tv",
"google.com.hk",
"csdn.net",
"bing.com",

View File

@ -70,7 +70,6 @@ module.exports = {
"netflix.com",
"microsoft.com",
"instagram.com",
"panda.tv",
"google.com.hk",
"csdn.net",
"bing.com",