Commit Graph

300 Commits

Author SHA1 Message Date
Gregory Mierzwinski
d36b68fefc Bug 1552548 - Set screen brightness to 50% on reference phones for power tests. r=perftest-reviewers,stephendonner
This patch sets the screen brightness on the reference Moto G5 and Pixel 2 phones to 50% before starting power tests and resets it to its original value once the test is complete.

Differential Revision: https://phabricator.services.mozilla.com/D31662

--HG--
extra : moz-landing-system : lando
2019-05-18 14:01:05 +00:00
Stephen Donner
6d86a346d7 Bug 1551324: Hook up remaining Fennec and Fenix jobs in CI for scn-power-idle tests. r=perftest-reviewers,rwood,sparky
Differential Revision: https://phabricator.services.mozilla.com/D31025

--HG--
extra : moz-landing-system : lando
2019-05-18 00:10:05 +00:00
Henrik Skupin
25e8969644 Bug 1548845 - [raptor] Don't filter-out tests which are white-listed for "use_live_sites". r=perftest-reviewers,stephendonner,rwood
Depends on D31681

Differential Revision: https://phabricator.services.mozilla.com/D31682

--HG--
extra : moz-landing-system : lando
2019-05-17 20:58:12 +00:00
Henrik Skupin
53ac5b9613 Bug 1548845 - [raptor] Log discarded tests in filter_live_sites. r=perftest-reviewers,rwood
Depends on D30532

Differential Revision: https://phabricator.services.mozilla.com/D31681

--HG--
extra : moz-landing-system : lando
2019-05-17 20:45:13 +00:00
Henrik Skupin
bc20797022 Bug 1548845 - [raptor] Fix local import of filter module. r=perftest-reviewers,rwood
To not collide with the built-in "filter" method, the local
filter module should be named as filters.

Differential Revision: https://phabricator.services.mozilla.com/D30532

--HG--
rename : testing/raptor/raptor/filter.py => testing/raptor/raptor/filters.py
extra : moz-landing-system : lando
2019-05-17 20:16:12 +00:00
Henrik Skupin
4e8e65b9a9 Bug 1548845 - [raptor] Allow tests to specify the alertChangeType. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30531

--HG--
extra : moz-landing-system : lando
2019-05-17 20:14:25 +00:00
Henrik Skupin
a092be21b1 Bug 1548845 - [raptor] Use a multiplier for page timeout when using live sites. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30530

--HG--
extra : moz-landing-system : lando
2019-05-17 20:14:23 +00:00
Rob Wood
8fdc5e82c4 Bug 1551972 - Fix Fenix cold page load by clearing app data before subsequent subtests; #perftest-reviewers r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D31632

--HG--
extra : moz-landing-system : lando
2019-05-17 16:55:05 +00:00
Mihai Alexandru Michis
724354e338 Backed out 5 changesets (bug 1548845) for failing new youtube playback raptor tests. CLOSED TREE
Backed out changeset 934d2f88195d (bug 1548845)
Backed out changeset 609f489bdc8c (bug 1548845)
Backed out changeset a2544ca8c593 (bug 1548845)
Backed out changeset 152615db9db6 (bug 1548845)
Backed out changeset 6b3a8394727f (bug 1548845)

--HG--
rename : testing/raptor/raptor/filters.py => testing/raptor/raptor/filter.py
2019-05-17 16:17:06 +03:00
Henrik Skupin
85b0987a4f Bug 1548845 - [raptor] Integrate Youtube video playback performance suite as benchmark test. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30483

--HG--
extra : moz-landing-system : lando
2019-05-17 11:12:53 +00:00
Henrik Skupin
992c3cdb7d Bug 1548845 - [raptor] Fix local import of filter module. r=perftest-reviewers,rwood
To not collide with the built-in "filter" method, the local
filter module should be named as filters.

Differential Revision: https://phabricator.services.mozilla.com/D30532

--HG--
rename : testing/raptor/raptor/filter.py => testing/raptor/raptor/filters.py
extra : moz-landing-system : lando
2019-05-17 11:12:40 +00:00
Henrik Skupin
11d84bf86a Bug 1548845 - [raptor] Allow tests to specify the alertChangeType. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30531

--HG--
extra : moz-landing-system : lando
2019-05-17 11:12:20 +00:00
Henrik Skupin
9eeae6d8c8 Bug 1548845 - [raptor] Use a multiplier for page timeout when using live sites. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30530

--HG--
extra : moz-landing-system : lando
2019-05-17 11:12:02 +00:00
Edwin Gao
d05d708389 Bug 1552000 - move root-required tuning commands to its appropriate location for Raptor Android r=acreskey
Changes:

- move the I/O scheduler modification subroutine under the block that checks for `root` availability of device.

This change should not be visible on CI systems; all devices there are already rooted. Where this change makes an impact is for developers running Raptor Android tests on non-rooted devices.

Differential Revision: https://phabricator.services.mozilla.com/D31339

--HG--
extra : moz-landing-system : lando
2019-05-16 11:57:31 +00:00
shindli
d1ef75d446 Backed out changeset 90a51dbd6ee1 (bug 1528236) for causing permafailures in raptor tests CLOSED TREE 2019-05-15 00:48:21 +03:00
Henrik Skupin
dceb981485 Bug 1528236 - [raptor] Use opt-out mechanism for setting test specific preferences. r=perftest-reviewers,stephendonner
Due to missing class abstractions for Gecko- and non-Gecko based
browsers it's currently sub-optimal to define when preferences
as defined by tests will be set.

Given that by default we run Gecko-based applications other
browsers should opt-out from setting test preferences.

Differential Revision: https://phabricator.services.mozilla.com/D30923

--HG--
extra : moz-landing-system : lando
2019-05-14 19:32:27 +00:00
Henrik Skupin
22364e41eb Bug 1528236 - [raptor] Fix root permission failures for device tuning. r=perftest-reviewers,stephendonner
Differential Revision: https://phabricator.services.mozilla.com/D30922

--HG--
extra : moz-landing-system : lando
2019-05-14 19:29:53 +00:00
Henrik Skupin
1fb0816fdd Bug 1528236 - [raptor] Environment variables should not be set via extra args. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29333

--HG--
extra : moz-landing-system : lando
2019-05-14 19:32:15 +00:00
Henrik Skupin
33461df3ea Bug 1528236 - [raptor] Update profile building code to align with mochitests. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29332

--HG--
extra : moz-landing-system : lando
2019-05-14 19:32:13 +00:00
Henrik Skupin
d93887ea36 Bug 1528236 - [raptor] Use constructor for initialization of raptor instances. r=perftest-reviewers,rwood
The patch tries to reduce the amount of initialization code
to call when creating a new Raptor class.

Initializing the ADBDevice already in the constructor of the
RaptorAndroid class is currently not possible because it would
mean that a device is immediately created but the "adb" binary
is not available for python test jobs. This has to be fixed later.

Differential Revision: https://phabricator.services.mozilla.com/D29331

--HG--
extra : moz-landing-system : lando
2019-05-14 19:32:11 +00:00
Henrik Skupin
c1ba84a470 Bug 1528236 - [raptor] Allow Raptor to run the tests all at once. r=perftest-reviewers,stephendonner,rwood
Differential Revision: https://phabricator.services.mozilla.com/D31047

--HG--
extra : moz-landing-system : lando
2019-05-14 19:32:01 +00:00
Henrik Skupin
d529cab0f1 Bug 1528236 - [raptor] Remove test root folder during clean-up. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29330

--HG--
extra : moz-landing-system : lando
2019-05-14 19:31:57 +00:00
Henrik Skupin
908e40dddf Bug 1528236 - [raptor] Refactor handling of remote profiles. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29329

--HG--
extra : moz-landing-system : lando
2019-05-14 19:30:11 +00:00
alexandru.ionescu
40ff099e27 Bug 1549669 - [raptor] Handle "--disable-e10s" argument to avoid causing fennec tests to fail r=perftest-reviewers,whimboo,ahal,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30171

--HG--
extra : moz-landing-system : lando
2019-05-14 11:50:00 +00:00
Stephen Donner
2c218a311e Bug 1548689: Hook idle-power-usage tests up to CI. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30456

--HG--
extra : moz-landing-system : lando
2019-05-13 20:33:15 +00:00
Stephen Donner
4a55597f03 Bug 1550828: String fixes and minor cleanup for power.py and Raptor's mach_commands.py. r=bc
Differential Revision: https://phabricator.services.mozilla.com/D30775

--HG--
extra : moz-landing-system : lando
2019-05-11 08:13:29 +00:00
Florin Strugariu
3dc303757f Bug 1549948 Fenix jobs are failing with "activity class does not exist" r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D30359

--HG--
extra : moz-landing-system : lando
2019-05-13 07:30:08 +00:00
Edwin Gao
1b56450c60 Bug 1549843 - implement jitter reduction for Motorola G5 r=jmaher,rwood
Changes:

- implement cpu, gpu, memory bus and other kernel tweaks for Motorola G5 to complement D30116

Differential Revision: https://phabricator.services.mozilla.com/D30715

--HG--
extra : moz-landing-system : lando
2019-05-10 21:15:20 +00:00
Andreea Pavel
e14f62f15c Merge mozilla-central to autoland on a CLOSED TREE 2019-05-10 13:01:38 +03:00
Florin Strugariu
9d23aa6093 Bug 1549949 Errors on launching activities are causing timeouts r=perftest-reviewers,davehunt
Differential Revision: https://phabricator.services.mozilla.com/D30343

--HG--
extra : moz-landing-system : lando
2019-05-10 09:32:00 +00:00
Andreea Pavel
f5f94453ae Backed out changeset ba9d948d404c (bug 1544470) for breaking raptor-chromium a=backout 2019-05-10 11:35:42 +03:00
Chris Hartjes
80fbae9239 Bug 1544470 - Added in code that can be used to take a snapshot of CPU usage on Android devices; r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D27954

--HG--
extra : moz-landing-system : lando
2019-05-09 20:39:43 +00:00
Edwin Gao
1298ea44c8 Bug 1547135 - reduce tp6m test result jitter for Android (Pixel 2) r=jmaher,acreskey,rwood
All changes are reversible upon reboot, with exception of the animation setting which is restored to default settings once Raptor concludes testing.

All values are for Pixel 2. Motorola G5 patch will be available separately.

Changes:
- set cpu/gpu to `performance` mode, and override minimum frequency with the maximum available frequency
- tweak msm-bus related kernel settings
- set display to be permanently on when connected over USB
- disable animation for duration of test
- turn off thermal throttling

Differential Revision: https://phabricator.services.mozilla.com/D30116

--HG--
extra : moz-landing-system : lando
2019-05-09 16:35:27 +00:00
Florin Strugariu
70d07476cb Bug 1541041 Page timeouts are ignored during cold page-load test suites r=rwood,davehunt
Differential Revision: https://phabricator.services.mozilla.com/D26309

--HG--
extra : moz-landing-system : lando
2019-05-08 18:29:56 +00:00
Ionut Goldan
ab23804ea9 Bug 1548577 - Add missing chromium counterparts r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D30188

--HG--
extra : moz-landing-system : lando
2019-05-08 19:39:10 +00:00
Stephen Donner
d229732882 Bug 1548987: pypi.python.org should now just be pypi.org in mach_commands.py. r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D29985

--HG--
extra : moz-landing-system : lando
2019-05-08 17:50:21 +00:00
Stephen Donner
e52c3cef40 Bug 1546741. Land initial idle-power-usage-measurement test for Android. r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D29654

--HG--
extra : moz-landing-system : lando
2019-05-08 19:21:12 +00:00
Rob Wood
046bef30c3 Bug 1545186 - Raptor test definitions and taskcluster configs for cold page-load tests on GeckoView Example App; #perftest r=davehunt
Differential Revision: https://phabricator.services.mozilla.com/D29207

--HG--
extra : moz-landing-system : lando
2019-05-03 15:53:27 +00:00
Ivan Yung
9914e7d5ec Bug 1519884: Improve help text for --test argument r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D29541

--HG--
extra : moz-landing-system : lando
2019-05-01 19:26:41 +00:00
Rob Wood
0678c2e034 Bug 1522972 - Raptor cold-load tests on Firefox and Chromium desktop; #perftest r=davehunt
Differential Revision: https://phabricator.services.mozilla.com/D29032

--HG--
extra : moz-landing-system : lando
2019-05-01 14:52:59 +00:00
Henrik Skupin
fc6a577886 Bug 1547069 - [raptor] Always take a screenshot on page load timeouts. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29178

--HG--
extra : moz-landing-system : lando
2019-04-30 07:59:28 +00:00
Henrik Skupin
3b68194ffa Bug 1547069 - [raptor] Refactoring screen capturing code in web extension. r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29177

--HG--
extra : moz-landing-system : lando
2019-04-30 07:59:11 +00:00
Henrik Skupin
e1bb9a3e44 Bug 1547069 - [raptor] Reduce CPU overhead when waiting for results. r=perftest-reviewers,rwood
The method checkForResult is called each and every 5 milliseconds while the Raptor
tests are running. This actually consumes a lot of CPU which can decrease the
performance of the website under test.

It's totally enough to check for results 4 times in a minute.

Differential Revision: https://phabricator.services.mozilla.com/D29176

--HG--
extra : moz-landing-system : lando
2019-04-30 07:58:56 +00:00
Henrik Skupin
f45dee943e Bug 1547069 - [raptor] Generalize code in waitForResult(). r=perftest-reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D29175

--HG--
extra : moz-landing-system : lando
2019-04-30 07:58:35 +00:00
Henrik Skupin
fdfa7dd694 Bug 1544686 - [raptor] Allow CLI arguments to override Gecko profiler settings from test manifest. r=perftest reviewers,rwood
Differential Revision: https://phabricator.services.mozilla.com/D28047

--HG--
extra : moz-landing-system : lando
2019-04-29 13:35:51 +00:00
Henrik Skupin
27e49c37df Bug 1544686 - [raptor] Make Gecko profiler threads customizable in the harness. r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D28046

--HG--
extra : moz-landing-system : lando
2019-04-23 15:37:30 +00:00
Henrik Skupin
f957b3ea28 Bug 1544686 - [raptor] Fix Gecko profiler settings. r=rwood
When the Gecko profiler is enabled the interval and entries as specified
in manifest files aren't correctly set. Instead the default values from
the extension are always used.

Also with WebRender enabled the threads are "WR", and "Renderer", but
not "WR,Renderer".

Differential Revision: https://phabricator.services.mozilla.com/D28045

--HG--
extra : moz-landing-system : lando
2019-04-23 15:37:29 +00:00
Florin Strugariu
d3d1d11735 Bug 1544356 4 - 281% raptor-tp6 regression r=davehunt
Differential Revision: https://phabricator.services.mozilla.com/D28598

--HG--
extra : moz-landing-system : lando
2019-04-26 12:49:09 +00:00
Cosmin Sabou
8f9aff6a55 Bug 1513467 - Disable raptor-tp6-amazon-firefox due to frequent failures. r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D29090

--HG--
extra : moz-landing-system : lando
2019-04-27 02:06:28 +00:00
Henrik Skupin
b77e627ac4 Bug 1546611 - [raptor] Fix None checks when validating test manifests. r=davehunt
Differential Revision: https://phabricator.services.mozilla.com/D28642

--HG--
extra : moz-landing-system : lando
2019-04-26 12:48:02 +00:00