Bug 1633889 - Add test url to test description r=sparky,perftest-reviewers

Differential Revision: https://phabricator.services.mozilla.com/D84133
This commit is contained in:
Myeongjun Go 2020-07-25 01:01:10 +00:00
parent 1e96614739
commit c2457707a7
3 changed files with 155 additions and 120 deletions

View File

@ -13,67 +13,67 @@ Desktop
-------
Tests for page-load performance. (WX: WebExtension, BT: Browsertime, FF: Firefox, CH: Chrome, CU: Chromium)
* amazon (BT)
* apple (BT)
* bing-search (BT)
* ebay (BT)
* facebook (BT)
* facebook-redesign (BT)
* fandom (BT)
* google-docs (BT)
* google-mail (BT)
* google-search (BT)
* google-sheets (BT)
* google-slides (BT)
* imdb (BT)
* imgur (BT)
* instagram (BT)
* linkedin (BT)
* microsoft (BT)
* netflix (BT)
* office (BT)
* outlook (BT)
* paypal (BT)
* pinterest (BT)
* raptor-tp6-amazon (WX, FF, CH, CU)
* raptor-tp6-apple (WX, FF, CH, CU)
* raptor-tp6-bing (WX, FF, CH, CU)
* raptor-tp6-docs (WX, FF, CH, CU)
* raptor-tp6-ebay (WX, FF, CH, CU)
* raptor-tp6-facebook (WX, CH, CU)
* raptor-tp6-fandom (WX, FF, CH, CU)
* raptor-tp6-google (WX, FF, CH, CU)
* raptor-tp6-google-mail (WX, FF, CH, CU)
* raptor-tp6-imdb (WX, FF, CH, CU)
* raptor-tp6-imgur (WX, FF, CH, CU)
* raptor-tp6-instagram (WX, FF, CH, CU)
* raptor-tp6-linkedin (WX, FF, CH, CU)
* raptor-tp6-microsoft (WX, FF, CH, CU)
* raptor-tp6-netflix (WX, FF, CH, CU)
* raptor-tp6-office (WX, FF, CH, CU)
* raptor-tp6-outlook (WX, FF, CH, CU)
* raptor-tp6-paypal (WX, FF, CH, CU)
* raptor-tp6-pinterest (WX, FF, CH, CU)
* raptor-tp6-reddit (WX, FF, CH, CU)
* raptor-tp6-sheets (WX, FF, CH, CU)
* raptor-tp6-slides (WX, FF, CH, CU)
* raptor-tp6-tumblr (WX, FF, CH, CU)
* raptor-tp6-twitch (WX, FF, CH, CU)
* raptor-tp6-twitter (WX, FF, CH, CU)
* raptor-tp6-wikipedia (WX, FF, CH, CU)
* raptor-tp6-yahoo-mail (WX, FF, CH, CU)
* raptor-tp6-yahoo-news (WX, FF, CH, CU)
* raptor-tp6-yandex (WX, FF, CH, CU)
* raptor-tp6-youtube (WX, FF, CH, CU)
* reddit (BT)
* tumblr (BT)
* twitch (BT)
* twitter (BT)
* wikipedia (BT)
* yahoo-mail (BT)
* yahoo-news (BT)
* yandex (BT)
* youtube (BT)
* `amazon (BT) <https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1>`__
* `apple (BT) <https://www.apple.com/macbook-pro/>`__
* `bing-search (BT) <https://www.bing.com/search?q=barack+obama>`__
* `ebay (BT) <https://www.ebay.com/>`__
* `facebook (BT) <https://www.facebook.com>`__
* `facebook-redesign (BT) <https://www.facebook.com>`__
* `fandom (BT) <https://www.fandom.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase>`__
* `google-docs (BT) <https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing>`__
* `google-mail (BT) <https://mail.google.com/>`__
* `google-search (BT) <https://www.google.com/search?hl=en&q=barack+obama&cad=h>`__
* `google-sheets (BT) <https://docs.google.com/spreadsheets/d/1jT9qfZFAeqNoOK97gruc34Zb7y_Q-O_drZ8kSXT-4D4/edit?usp=sharing>`__
* `google-slides (BT) <https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit?usp=sharing>`__
* `imdb (BT) <https://www.imdb.com/title/tt0084967/?ref_=nv_sr_2>`__
* `imgur (BT) <https://imgur.com/gallery/m5tYJL6>`__
* `instagram (BT) <https://www.instagram.com/>`__
* `linkedin (BT) <https://www.linkedin.com/in/thommy-harris-hk-385723106/>`__
* `microsoft (BT) <https://www.microsoft.com/en-us/>`__
* `netflix (BT) <https://www.netflix.com/title/80117263>`__
* `office (BT) <https://office.live.com/start/Word.aspx?omkt=en-US>`__
* `outlook (BT) <https://outlook.live.com/mail/inbox>`__
* `paypal (BT) <https://www.paypal.com/myaccount/summary/>`__
* `pinterest (BT) <https://pinterest.com/>`__
* `raptor-tp6-amazon (WX, FF, CH, CU) <https://www.amazon.com/s?k=laptop&ref=nb_sb_noss_1>`__
* `raptor-tp6-apple (WX, FF, CH, CU) <https://www.apple.com/macbook-pro/>`__
* `raptor-tp6-bing (WX, FF, CH, CU) <https://www.bing.com/search?q=barack+obama>`__
* `raptor-tp6-docs (WX, FF, CH, CU) <https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing>`__
* `raptor-tp6-ebay (WX, FF, CH, CU) <https://www.ebay.com/>`__
* `raptor-tp6-facebook (WX, CH, CU) <https://www.facebook.com>`__
* `raptor-tp6-fandom (WX, FF, CH, CU) <https://www.fandom.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase>`__
* `raptor-tp6-google (WX, FF, CH, CU) <https://www.google.com/search?hl=en&q=barack+obama&cad=h>`__
* `raptor-tp6-google-mail (WX, FF, CH, CU) <https://mail.google.com/>`__
* `raptor-tp6-imdb (WX, FF, CH, CU) <https://www.imdb.com/title/tt0084967/?ref_=nv_sr_2>`__
* `raptor-tp6-imgur (WX, FF, CH, CU) <https://imgur.com/gallery/m5tYJL6>`__
* `raptor-tp6-instagram (WX, FF, CH, CU) <https://www.instagram.com/>`__
* `raptor-tp6-linkedin (WX, FF, CH, CU) <https://www.linkedin.com/in/thommy-harris-hk-385723106/>`__
* `raptor-tp6-microsoft (WX, FF, CH, CU) <https://www.microsoft.com/en-us/>`__
* `raptor-tp6-netflix (WX, FF, CH, CU) <https://www.netflix.com/title/80117263>`__
* `raptor-tp6-office (WX, FF, CH, CU) <https://office.live.com/start/Word.aspx?omkt=en-US>`__
* `raptor-tp6-outlook (WX, FF, CH, CU) <https://outlook.live.com/mail/inbox>`__
* `raptor-tp6-paypal (WX, FF, CH, CU) <https://www.paypal.com/myaccount/summary/>`__
* `raptor-tp6-pinterest (WX, FF, CH, CU) <https://pinterest.com/>`__
* `raptor-tp6-reddit (WX, FF, CH, CU) <https://www.reddit.com/r/technology/comments/9sqwyh/we_posed_as_100_senators_to_run_ads_on_facebook/>`__
* `raptor-tp6-sheets (WX, FF, CH, CU) <https://docs.google.com/spreadsheets/d/1jT9qfZFAeqNoOK97gruc34Zb7y_Q-O_drZ8kSXT-4D4/edit?usp=sharing>`__
* `raptor-tp6-slides (WX, FF, CH, CU) <https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit?usp=sharing>`__
* `raptor-tp6-tumblr (WX, FF, CH, CU) <https://www.tumblr.com/dashboard>`__
* `raptor-tp6-twitch (WX, FF, CH, CU) <https://www.twitch.tv/videos/326804629>`__
* `raptor-tp6-twitter (WX, FF, CH, CU) <https://twitter.com/BarackObama>`__
* `raptor-tp6-wikipedia (WX, FF, CH, CU) <https://en.wikipedia.org/wiki/Barack_Obama>`__
* `raptor-tp6-yahoo-mail (WX, FF, CH, CU) <https://mail.yahoo.com/>`__
* `raptor-tp6-yahoo-news (WX, FF, CH, CU) <https://www.yahoo.com/lifestyle/police-respond-noise-complaint-end-playing-video-games-respectful-tenants-002329963.html>`__
* `raptor-tp6-yandex (WX, FF, CH, CU) <https://yandex.ru/search/?text=barack%20obama&lr=10115>`__
* `raptor-tp6-youtube (WX, FF, CH, CU) <https://www.youtube.com>`__
* `reddit (BT) <https://www.reddit.com/r/technology/comments/9sqwyh/we_posed_as_100_senators_to_run_ads_on_facebook/>`__
* `tumblr (BT) <https://www.tumblr.com/dashboard>`__
* `twitch (BT) <https://www.twitch.tv/videos/326804629>`__
* `twitter (BT) <https://twitter.com/BarackObama>`__
* `wikipedia (BT) <https://en.wikipedia.org/wiki/Barack_Obama>`__
* `yahoo-mail (BT) <https://mail.yahoo.com/>`__
* `yahoo-news (BT) <https://www.yahoo.com/lifestyle/police-respond-noise-complaint-end-playing-video-games-respectful-tenants-002329963.html>`__
* `yandex (BT) <https://yandex.ru/search/?text=barack%20obama&lr=10115>`__
* `youtube (BT) <https://www.youtube.com>`__
Live
----
@ -84,59 +84,59 @@ Mobile
------
Page-load performance test suite on Android. (WX: WebExtension, BT: Browsertime, GV: Geckoview, RB: Refbrow, FE: Fenix, F68: Fennec68)
* allrecipes (BT)
* amazon (BT)
* amazon-search (BT)
* bbc (BT)
* bing (BT)
* bing-search-restaurants (BT)
* booking (BT)
* cnn (BT)
* cnn-ampstories (BT)
* ebay-kleinanzeigen (BT)
* ebay-kleinanzeigen-search (BT)
* espn (BT)
* facebook (BT)
* facebook-cristiano (BT)
* google (BT)
* google-maps (BT)
* google-search-restaurants (BT)
* imdb (BT)
* instagram (BT)
* jianshu (BT)
* microsoft-support (BT)
* raptor-tp6m-allrecipes (WX, GV, RB, FE, F68)
* raptor-tp6m-amazon (WX, GV, RB, FE, F68)
* raptor-tp6m-amazon-search (WX, GV, RB, FE, F68)
* raptor-tp6m-bbc (WX, GV, RB, FE, F68)
* raptor-tp6m-bing (WX, GV, RB, FE, F68)
* raptor-tp6m-bing-restaurants (WX, GV, RB, FE, F68)
* raptor-tp6m-booking (WX, GV, RB, FE, F68)
* raptor-tp6m-cnn (WX, GV, RB, FE, F68)
* raptor-tp6m-cnn-ampstories (WX, GV, RB, FE, F68)
* raptor-tp6m-ebay-kleinanzeigen (WX, GV, RB, FE, F68)
* raptor-tp6m-espn (WX, GV, RB, FE, F68)
* raptor-tp6m-facebook (WX, GV, RB, FE, F68)
* raptor-tp6m-facebook-cristiano (WX, GV, RB, FE, F68)
* raptor-tp6m-google (WX, GV, RB, FE, F68)
* raptor-tp6m-google-maps (WX, GV, RB, FE, F68)
* raptor-tp6m-google-restaurants (WX, GV, RB, FE, F68)
* raptor-tp6m-imdb (WX, GV, RB, FE, F68)
* raptor-tp6m-instagram (WX, GV, RB, FE, F68)
* raptor-tp6m-jianshu (WX, GV, RB, FE, F68)
* raptor-tp6m-microsoft-support (WX, GV, RB, FE, F68)
* raptor-tp6m-reddit (WX, GV, RB, FE, F68)
* raptor-tp6m-stackoverflow (WX, GV, RB, FE, F68)
* raptor-tp6m-web-de (WX, GV, RB, FE, F68)
* raptor-tp6m-wikipedia (WX, GV, RB, FE, F68)
* raptor-tp6m-youtube (WX, GV, RB, FE, F68)
* raptor-tp6m-youtube-watch (WX, GV, RB, FE, F68)
* reddit (BT)
* stackoverflow (BT)
* web-de (BT)
* wikipedia (BT)
* youtube (BT)
* youtube-watch (BT)
* `allrecipes (BT) <https://www.allrecipes.com/>`__
* `amazon (BT) <https://www.amazon.com>`__
* `amazon-search (BT) <https://www.amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone>`__
* `bbc (BT) <https://www.bbc.com/news/business-47245877>`__
* `bing (BT) <https://www.bing.com/>`__
* `bing-search-restaurants (BT) <https://www.bing.com/search?q=restaurants>`__
* `booking (BT) <https://www.booking.com/>`__
* `cnn (BT) <https://cnn.com>`__
* `cnn-ampstories (BT) <https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other>`__
* `ebay-kleinanzeigen (BT) <https://m.ebay-kleinanzeigen.de>`__
* `ebay-kleinanzeigen-search (BT) <https://m.ebay-kleinanzeigen.de/s-anzeigen/auf-zeit-wg-berlin/zimmer/c199-l3331>`__
* `espn (BT) <http://www.espn.com/nba/story/_/page/allstarweekend25788027/the-comparison-lebron-james-michael-jordan-their-own-words>`__
* `facebook (BT) <https://m.facebook.com>`__
* `facebook-cristiano (BT) <https://m.facebook.com/Cristiano>`__
* `google (BT) <https://www.google.com>`__
* `google-maps (BT) <https://www.google.com/maps?force=pwa>`__
* `google-search-restaurants (BT) <https://www.google.com/search?q=restaurants+near+me>`__
* `imdb (BT) <https://m.imdb.com/>`__
* `instagram (BT) <https://www.instagram.com>`__
* `jianshu (BT) <https://www.jianshu.com/>`__
* `microsoft-support (BT) <https://support.microsoft.com/en-us>`__
* `raptor-tp6m-allrecipes (WX, GV, RB, FE, F68) <https://www.allrecipes.com/>`__
* `raptor-tp6m-amazon (WX, GV, RB, FE, F68) <https://www.amazon.com>`__
* `raptor-tp6m-amazon-search (WX, GV, RB, FE, F68) <https://www.amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone>`__
* `raptor-tp6m-bbc (WX, GV, RB, FE, F68) <https://www.bbc.com/news/business-47245877>`__
* `raptor-tp6m-bing (WX, GV, RB, FE, F68) <https://www.bing.com/>`__
* `raptor-tp6m-bing-restaurants (WX, GV, RB, FE, F68) <https://www.bing.com/search?q=restaurants>`__
* `raptor-tp6m-booking (WX, GV, RB, FE, F68) <https://www.booking.com/>`__
* `raptor-tp6m-cnn (WX, GV, RB, FE, F68) <https://cnn.com>`__
* `raptor-tp6m-cnn-ampstories (WX, GV, RB, FE, F68) <https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other>`__
* `raptor-tp6m-ebay-kleinanzeigen (WX, GV, RB, FE, F68) <https://m.ebay-kleinanzeigen.de>`__
* `raptor-tp6m-espn (WX, GV, RB, FE, F68) <http://www.espn.com/nba/story/_/page/allstarweekend25788027/the-comparison-lebron-james-michael-jordan-their-own-words>`__
* `raptor-tp6m-facebook (WX, GV, RB, FE, F68) <https://m.facebook.com>`__
* `raptor-tp6m-facebook-cristiano (WX, GV, RB, FE, F68) <https://m.facebook.com/Cristiano>`__
* `raptor-tp6m-google (WX, GV, RB, FE, F68) <https://www.google.com>`__
* `raptor-tp6m-google-maps (WX, GV, RB, FE, F68) <https://www.google.com/maps?force=pwa>`__
* `raptor-tp6m-google-restaurants (WX, GV, RB, FE, F68) <https://www.google.com/search?q=restaurants+near+me>`__
* `raptor-tp6m-imdb (WX, GV, RB, FE, F68) <https://m.imdb.com/>`__
* `raptor-tp6m-instagram (WX, GV, RB, FE, F68) <https://www.instagram.com>`__
* `raptor-tp6m-jianshu (WX, GV, RB, FE, F68) <https://www.jianshu.com/>`__
* `raptor-tp6m-microsoft-support (WX, GV, RB, FE, F68) <https://support.microsoft.com/en-us>`__
* `raptor-tp6m-reddit (WX, GV, RB, FE, F68) <https://www.reddit.com>`__
* `raptor-tp6m-stackoverflow (WX, GV, RB, FE, F68) <https://stackoverflow.com/>`__
* `raptor-tp6m-web-de (WX, GV, RB, FE, F68) <https://web.de/magazine/politik/politologe-glaubt-grossen-koalition-herbst-knallen-33563566>`__
* `raptor-tp6m-wikipedia (WX, GV, RB, FE, F68) <https://en.m.wikipedia.org/wiki/Main_Page>`__
* `raptor-tp6m-youtube (WX, GV, RB, FE, F68) <https://m.youtube.com>`__
* `raptor-tp6m-youtube-watch (WX, GV, RB, FE, F68) <https://www.youtube.com/watch?v=COU5T-Wafa4>`__
* `reddit (BT) <https://www.reddit.com>`__
* `stackoverflow (BT) <https://stackoverflow.com/>`__
* `web-de (BT) <https://web.de/magazine/politik/politologe-glaubt-grossen-koalition-herbst-knallen-33563566>`__
* `wikipedia (BT) <https://en.m.wikipedia.org/wiki/Main_Page>`__
* `youtube (BT) <https://m.youtube.com>`__
* `youtube-watch (BT) <https://www.youtube.com/watch?v=COU5T-Wafa4>`__
Scenario
--------

View File

@ -3,6 +3,7 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
from __future__ import absolute_import
import collections
import os
import re
@ -29,6 +30,7 @@ class FrameworkGatherer(object):
self._yaml_path = yaml_path
self._suite_list = {}
self._test_list = {}
self._urls = {}
self._manifest_path = ""
self._manifest = None
@ -117,7 +119,7 @@ class RaptorGatherer(FrameworkGatherer):
return self._suite_list
def _get_subtests_from_ini(self, manifest_path):
def _get_subtests_from_ini(self, manifest_path, suite_name):
"""
Returns a list of (sub)tests from an ini file containing the test definitions.
@ -126,7 +128,17 @@ class RaptorGatherer(FrameworkGatherer):
"""
test_manifest = TestManifest([manifest_path], strict=False)
test_list = test_manifest.active_tests(exists=False, disabled=False)
subtest_list = {subtest["name"]: subtest["manifest"] for subtest in test_list}
subtest_list = {}
for subtest in test_list:
subtest_list[subtest["name"]] = subtest["manifest"]
self._urls[subtest["name"]] = {
"type": suite_name,
"url": subtest["test_url"],
}
self._urls = collections.OrderedDict(
sorted(self._urls.items(), key=lambda t: len(t[0]))
)
return subtest_list
@ -152,13 +164,34 @@ class RaptorGatherer(FrameworkGatherer):
if not self._test_list.get(suite_name):
self._test_list[suite_name] = {}
for i, manifest_path in enumerate(manifest_paths, 1):
subtest_list = self._get_subtests_from_ini(manifest_path)
subtest_list = self._get_subtests_from_ini(manifest_path, suite_name)
self._test_list[suite_name].update(subtest_list)
return self._test_list
def build_test_description(self, title, test_description=""):
return ["* " + title + " (" + test_description + ")"]
def build_test_description(self, title, test_description="", suite_name=""):
matcher = set()
for name, val in self._urls.items():
if title == name and suite_name == val["type"]:
matcher.add(val["url"])
break
if len(matcher) == 0:
for name, val in self._urls.items():
if title in name and suite_name == val["type"]:
matcher.add(val["url"])
break
return [
"* `"
+ title
+ " ("
+ test_description
+ ") "
+ "<"
+ matcher.pop()
+ ">`__"
]
class MozperftestGatherer(FrameworkGatherer):

View File

@ -98,7 +98,9 @@ class Generator(object):
documentation.extend(
self._verifier._gatherer.framework_gatherers[
yaml_content["name"]
].build_test_description(test_name, tests[test_name])
].build_test_description(
test_name, tests[test_name], suite_name
)
)
documentation.append("")