This patch renames the power usage metrics to be only `powerUsage` and removes the `Support`/`Pageload` suffixes. At the same time, it makes sure that only one type of these measuerments is ever recorded.
Differential Revision: https://phabricator.services.mozilla.com/D227379
This patch changes the names of the cpu time metric to only be `cpuTime`, and removes the cpuTimePageload, and cpuTimeSupport metric. Also, only one version of it will ever be gathered (benchmark one if it exists, otherwise the pageload one).
Differential Revision: https://phabricator.services.mozilla.com/D227276
This patch enables power testing for pageload tests on windows 11. It also fixes a couple issues related to gathering additional metrics for pageloads.
Differential Revision: https://phabricator.services.mozilla.com/D227188
This patch adds the ability to gather a power profile for support measurements. This power profile is produced in the browsertime results directory.
Differential Revision: https://phabricator.services.mozilla.com/D227187
This patch moves the power_profile_* artifacts from the upload directory to the browsertime results directory when the test is finished. It also adds a new method that can be used in other areas to generate the correct path to the result directory.
Differential Revision: https://phabricator.services.mozilla.com/D227186
This patch fixes an issue where the iteration index was undefined because it wasn't being passed properly (missing in one of the function calls).
Differential Revision: https://phabricator.services.mozilla.com/D226552
This patch enables performance pageload tests to now use mitmproxy
11.0.0, only on the Mac/Windows platform.
This will run on a subset of tests as the binary doesn't properly run on
old recordings (e.g. mitm 6) but seems to run fine on recordings mitm7
and greater.
Additionally, the mitmproxy version for android is bumped from 10 -> 11.
Although 10.x.x was only recently landed, 11 was released shortly after
so we might as well keep all platforms on the same version where
possible.
Differential Revision: https://phabricator.services.mozilla.com/D224383
This patch makes available the mitm 10 binary for android, and at the
moment only runs it on recordings using version 8 as there are errors on
older versions so we might as well wait until we record those.
These subset of sites are bild-de, allrecipes, instagram, and sina
Differential Revision: https://phabricator.services.mozilla.com/D223829
This patch adds Windows 11 power testing to our benchmark support tooling. It also removes the condition that prevented desktop from using the --power-test option.
At the same time some modifications are made to Chrome tasks running on Windows 11 to be able to use the xpcshell binary for the power usage testing.
Differential Revision: https://phabricator.services.mozilla.com/D222181
This patch moves the windows power profiling code into separate methods so they can be used in benchmarks to gather power usage during the benchmark itself.
Differential Revision: https://phabricator.services.mozilla.com/D222180
This patch adds tooling for producing android power usage measurements in benchmark-like tests. It makes use of the usb-power-profiling module that comes with Browsertime. Some changes are needed to pass the Browsertime node-modules path to the user-scripts.
Differential Revision: https://phabricator.services.mozilla.com/D222179
This patch changes how cpuTime is produced, and gathered for benchmark tests. It makes use of the new SupportMeasurements class.
The cpuTime for desktop on all browsers is produced using the `os` node module, and only the increase in user+system time is reported over the duration of the benchmark test. This measurement isn't process-specific.
On Android, the cpuTime is produced by using the shell `ps` command. We take all the processes for a given package, and then report the sum of the increase in cpuTime used by those processes at the end of the test (similar to desktop).
Measurements are parsed by the BaseSupportPython class in the `add_additional_metrics` method.
Differential Revision: https://phabricator.services.mozilla.com/D221728
This patch reworks the additional metrics parsing for support classes into a more generic method. It separates the metric parsing into multiple methods as well to make it simpler for other support classes to use them directly.
At the same time, these changes make better use of the saved data for building the measurements, and adds functionality that allows callers to specify if they want cold, or warm measurements (in case these need to be separate).
Some comments are modified as well to better reflect inputs, and what the methods are meant to be used for.
Differential Revision: https://phabricator.services.mozilla.com/D222178
This patch fixes an issue with the support class import tooling. Instead of residing in the utils.py file, it'll now reside in it's own file to resolve circular dependency issues related to using utils.py code in support classes. A bug in the flatten code is also fixed here (adding a base case for it).
Differential Revision: https://phabricator.services.mozilla.com/D222177
We have motionmark 1.3 tests running already and we are no longer
interested in motionmark 1.0 so this patch removes it from our testing.
Differential Revision: https://phabricator.services.mozilla.com/D221533
As part of out expanding performance metrics for resource usage we are adding the cpuTime metric to track CPU time in various performance tests on mobile and desktop tests for firefox browsers only
Differential Revision: https://phabricator.services.mozilla.com/D217146
This patch adds the first step in consolidating our support measurement tooling for custom scripts into a single location. A new file is added that is imported into the speedometer3 tests and produces a new wallclock time metric for sp3 on all platforms, and browsers.
Two methods are exported from the support_measurements.js file to make is simple to integrate into existing scripts (a start, and stop command).
Differential Revision: https://phabricator.services.mozilla.com/D221080
As part of out expanding performance metrics for resource usage we are adding the cpuTime metric to track CPU time in various performance tests on mobile and desktop tests for firefox browsers only
Differential Revision: https://phabricator.services.mozilla.com/D217146
This patch changes the external storage to accomodate condprof creation
and subsequently performance testing tasks are enabled on the A55.
Differential Revision: https://phabricator.services.mozilla.com/D220067
This patch moves the pageload result/output handling out of the results.py/output.py files, and into the browsertime_pageload.py support script. Unit tests, and the tests themselves are modified for this. The move is mostly a copy-paste from the result/output files, with some additional changes for handling the args passed into the test.
At the same time, the code where the results were previously handled is replaced with an exception to ensure that future tests use the support script instead of the results code. The output code is still being used by other custom tests that use `custom_data`, but then have their data parsed by the pageload-specific code. A warning is added in that case to dissuade future users of using the code.
Differential Revision: https://phabricator.services.mozilla.com/D219875
Mobile unit tests use --no-insall and raptor tests use --noinstall, this patch modifies the flag to be --no-install so that developers use one flag and are not confused from 2 difference but VERY similar flags
Differential Revision: https://phabricator.services.mozilla.com/D219817
