This patch changes the external storage to accomodate condprof creation
and subsequently performance testing tasks are enabled on the A55.
Differential Revision: https://phabricator.services.mozilla.com/D220067
This patch moves the pageload result/output handling out of the results.py/output.py files, and into the browsertime_pageload.py support script. Unit tests, and the tests themselves are modified for this. The move is mostly a copy-paste from the result/output files, with some additional changes for handling the args passed into the test.
At the same time, the code where the results were previously handled is replaced with an exception to ensure that future tests use the support script instead of the results code. The output code is still being used by other custom tests that use `custom_data`, but then have their data parsed by the pageload-specific code. A warning is added in that case to dissuade future users of using the code.
Differential Revision: https://phabricator.services.mozilla.com/D219875
This patch adds some more documentation to the perftest in a nutshell docs. This includes the regression workshop docs, some information about compare view, and some information about getting profiles from our tests.
Differential Revision: https://phabricator.services.mozilla.com/D219187
Temporarily turn on sp3 tests for the win10-64 platform until we
completely phase it out.
We will only run it on the shippable platform.
Differential Revision: https://phabricator.services.mozilla.com/D218648
This patch switches the builder from the intel osx 10.15 builders to the
Apple Silicon builders. 10.15 Is no longer a supported build target for
CaR (Chromium-as-Release) so let's use this opportunity to just go to the latest version
builders we have.
Additionally, this patch removes the custom CaR osx 10.15 tasks as that
is no longer a supported platform for the application.
Differential Revision: https://phabricator.services.mozilla.com/D218368
This patch adds some documentation about the --tests option, and also updates a bit of the other documentation for mach-try-perf.
Differential Revision: https://phabricator.services.mozilla.com/D216934
This patch adds a small delay to the URL navigation in talos tests to resolve some permafailure timeouts on windows 11. At the same time some test timeouts, and max runtimes are adjusted to handle the new delay.
Differential Revision: https://phabricator.services.mozilla.com/D216388
These changes will make it easier to run performance tests locally, this will reduce the barrier to entry for people poking around and help improve our reach!
Differential Revision: https://phabricator.services.mozilla.com/D214367
This patch adds a new location for a performance testing guide called "Performance Testing in a Nutshell". It'll contain an overview of all the information needed for understanding performance testing at Mozilla.
Differential Revision: https://phabricator.services.mozilla.com/D214282
The video under test is a simple recording of solid colors, where the
first frame is red, second frame is blue, subsequent frames alternate
between green and purple, and the final frame is yellow. This allows us
to identify what frame is being displayed and when in a browser agnostic
way.
This is achieved by updating the visualmetrics python script in
browsertime to identify frames which match a given color range, called
key color frames. The script will produce a list of which frames appear
when that we can use to estimate the playback latency. Note that the
annotated videos produced by the jobs will also be annotated to make it
clear which frames it believes are key color frames.
Until the tests have proven stable, they have been set at tier 3.
The status of the merge can be seen at:
https://github.com/sitespeedio/browsertime/pull/2119
Differential Revision: https://phabricator.services.mozilla.com/D210242
This patch re-records the bing-search pageload test. It was pointed out
that the previous re-recording was triggering two FCP events. Upon
inspecting the profile it was found that the page was some how loading
twice.
This re-recording does not seem to have this issue.
Differential Revision: https://phabricator.services.mozilla.com/D212851
This patch adds a `--power-test` flag that can be used to do power usage testing on mobile devices that are hooked up to a USB power meter. It is gathered for all tests when it's enabled, and a helper method is added to make it easier for support-scripts to gather it. An update is also done to browsertime to gather the power usage data. At the same time, a bit of documentation about this flag is added to our Raptor Browsertime docs.
Differential Revision: https://phabricator.services.mozilla.com/D212107
Alexandru Ionescu is no longer on the team and so we're updating the owners of all the perf modules he was either an owner or a co-owner of
Differential Revision: https://phabricator.services.mozilla.com/D210090
This patch adds a new layer that will allow us to run custom/adhoc bash scripts in mozperftest. Currently, the layer can only handle bash scripts. At the same time, a new flavor is added for this, and additional script parsing capabilities are added.
Differential Revision: https://phabricator.services.mozilla.com/D211740
As part of our okr goals of being competative in pageload we are running essential pageload tests on chrome on android to have an idea of how we compare to chrome on android
Differential Revision: https://phabricator.services.mozilla.com/D209981
This patch re-records the following sites:
* google docs
* google search
* cnn
* ebay
* idmb
* reddit
* bing-search
to see if this helps with an osx 10.15 intermittent failure.
Differential Revision: https://phabricator.services.mozilla.com/D210175
This patch updates our documents to include a section with instructions
on how a user can run a browsertime or raptor-browsertime test locally
with safari tech preview
Differential Revision: https://phabricator.services.mozilla.com/D209878
This patch adds the ability to run Raptor performance tests
(specifically just speedometer 3 for now) on Safari Technology Preview.
Differential Revision: https://phabricator.services.mozilla.com/D209611
This patch updates perfdocs with talos doc changes, and makes a modification to the perfdocs-verify task to trigger it on reviewbot again.
Differential Revision: https://phabricator.services.mozilla.com/D209228
This patch updates perfdocs with talos doc changes, and makes a modification to the perfdocs-verify task to trigger it on reviewbot again.
Differential Revision: https://phabricator.services.mozilla.com/D209228
This patch adds a couple sentences to our standard-workflow docs for mach-try-perf to explain the use replicates button.
Differential Revision: https://phabricator.services.mozilla.com/D208877
Grab the correct folder after extracting the x64 chromedrivers.
Previously we were always using the 32 bit chromedrivers in CI and only
recently added the 64 bit chromedrivers for win x64 platforms.
Additionally, since we don't make use of the 32 bit windows platform
anymore, it is removed for the chrome platform.
Differential Revision: https://phabricator.services.mozilla.com/D208001
Olivia Hall