This patch adds documentation to the BasePerfComparator, and also adds some documentation for Comparators in general to the mach-try-perf Firefox source docs.
Differential Revision: https://phabricator.services.mozilla.com/D177760
This patch adds the ability to run manual logins for our websites since it can be simpler, and quicker in some cases. At the same time, a bug with the options handling is fixed.
Differential Revision: https://phabricator.services.mozilla.com/D164590
This patch attempts to address a high frequency of failing imgur raptor-browsertime tests on macos.
Additionally, the secondary url is replaced as it gives a 404
Differential Revision: https://phabricator.services.mozilla.com/D168106
This patch adds an FAQ (Frequently Asked Questions) section to the mach try perf docs. It also does a small cleanup to move fxrecord into the `testing/performance` folder, re-organize the linting configuration file, fix file naming, and captializes the `mozperftest` and `fxrecord` title names in the side-bar. Lastly, it adds a warning to the `mozperftest` docs to direct people who make it there to the `mach try perf` page.
Differential Revision: https://phabricator.services.mozilla.com/D167555
This patch adds docs for the mach try perf tool, and also allows images to be used in all parts of perfdocs (only PNGs though).
Differential Revision: https://phabricator.services.mozilla.com/D163514
This patch adds a binary for the current latest mitmproxy version. Recordings are forward compatible so it should be safe to start replaying recordings on mitm8, with re-recordings happening in a future date.
Only desktop recordings will be addressed here, and android platforms at a later date.
Differential Revision: https://phabricator.services.mozilla.com/D159798
The last time we re-recorded the google-docs site, it had been updated to the canvas version. There is no reason to keep the separate google-docs-canvas page load test around.
Differential Revision: https://phabricator.services.mozilla.com/D157507
The cookie prompt handling code was initially added under `test_cmds` but it got in conflict with the interactive sites code (the prompts are for pageload sites while the interactive sites are a different type).
This patch is adding a new section in the json that is specially for handling the cookie prompts for pageload sites. We have to handle prompts for the login sites. but we don't currently encounter those prompts on CI and locally the login code doesn't work ATM. So this affects local site recording from Europe.
Differential Revision: https://phabricator.services.mozilla.com/D153748
A week ago we received a notification that we had a test that the WPT chrome tests were perma failing on [[ https://bugzilla.mozilla.org/show_bug.cgi?id=1773621 | bugzilla ]]
After going through the fail logs I realized it was because of website "panda.tv" directing to a unable to connect page message, after some digging it was not returning proper data because panda.tv has not been a company since March 2019(bankruptcy filing), why this only is causing an issue now I believe is because of some kind of update from WPT as I can see a noticeable UI difference on the test results page from before and after the failures started.
My resolution was to remove Panda.tv from our test list and that seems to have resolved the issue.
I also updated the error message to display which website is causing the issue so that if this happens again I don't need to go through each and every webpagetest result to know which of the 40 websites are having an issue.
Differential Revision: https://phabricator.services.mozilla.com/D149642
This next patch in the series utilizes the same login-logic in Mozperftest and makes it available to the `pageload_test` method so that we can now automate the logging into of accounts during perftest recordings.
Additional logic is also added to account for if the site requires login, if we are running on CI or locally (and if on CI, accounting for the SCM level), and removal of the verbose flags so secrets do not leak.
Differential Revision: https://phabricator.services.mozilla.com/D147775
This patch adds the --proxy-perftest-page option. With this option, we'll be able to specify which test pages we want to record rather than modifying the pageload_sites.json. Note that the login fields will not be taken into consideration with this option.
You can use the flag as follows (seperate multiple pages with a comma): --proxy-perftest-page microsft,linkedin,netflix
Furthermore, some changes were made to strengthen the conditions around when a login site can be tested (only when RAPTOR_LOGINS is defined locally, or if we are in CI).
Differential Revision: https://phabricator.services.mozilla.com/D137468
This patchs adds login logic for our various websites. The logic can handle all of our existing test pages that need logins. Browsertime needed an update in mozperftest for this work (to get the new mouse code). The logic for getting the login information is also implemented here for local and CI runs. Websites with 2FA cannot be tested with this code until we disable 2FA, the auto-login works on those sites though.
Differential Revision: https://phabricator.services.mozilla.com/D133138
