Csoregi Natalia
2d34e224ae
Backed out changeset 448679382b06 (bug 1649497) for linting failures on framework_gatherers.py. CLOSED TREE
2020-07-14 20:20:48 +03:00
Myeongjun Go
62777a18b5
Bug 1649497 - [perfdocs] Port mozperftest into Performance Testing r=sparky,perftest-reviewers
...
Differential Revision: https://phabricator.services.mozilla.com/D82207
2020-07-14 16:55:56 +00:00
Tarek Ziadé
7dfca9db55
Bug 1644778 - add a mozperftest to measure browser.xhtml DOM size, r=tarek,dhouse,sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D79152
2020-07-13 15:44:38 +00:00
Bob Clary
13b8cbabeb
Bug 1652339 - set mozdevice version requirements to exclude version 4.0.0 on trunk, beta r=marionette-reviewers,perftest-reviewers,whimboo,sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D83278
2020-07-13 11:36:39 +00:00
Jim Porter
5c759e3542
Bug 1631806 - [mozproxy] Add a command line interface r=tarek,Bebe
...
Differential Revision: https://phabricator.services.mozilla.com/D76017
2020-07-08 13:41:24 +00:00
Tarek Ziadé
44eb9c7f7e
Bug 1651048 - remove unused transformer r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D82521
2020-07-07 12:48:47 +00:00
Tarek Ziadé
93ae761b53
Bug 1651044 - Fixed a test in mozperftest r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D82515
2020-07-07 12:34:32 +00:00
Tarek Ziadé
ee86c32c2e
Bug 1648360 - complex metrics in --perfherder-metrics r=sparky
...
This patch adds the ability to define complex metrics
Differential Revision: https://phabricator.services.mozilla.com/D82161
2020-07-06 14:28:37 +00:00
xuanqi xu
a4676c048b
Bug 1640875 - Add compare function to notebook layer. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D80384
2020-07-06 15:05:06 +00:00
Coroiu Cristina
50d2261646
Backed out changeset 65acd64b9e0e (bug 1640875) for mpu failures
2020-07-03 15:59:28 +03:00
xuanqi xu
aff7d7822c
Bug 1640875 - Add compare function to notebook layer. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D80384
2020-07-02 16:14:23 +00:00
Yue Zhang
765e406782
Bug 1643469 - Swap NotebookLogger for MachLogger. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D81156
2020-06-30 13:14:38 +00:00
Tarek Ziadé
9294a5bfc1
Bug 1649176 - add batch attribute to add --test-date r=sparky
...
Right now setup_perftest_test_date adds --test-date yesterday to all perftest
runs. we want that only for the ones doing batches
Differential Revision: https://phabricator.services.mozilla.com/D81562
2020-06-29 20:12:42 +00:00
Gregory Mierzwinski
91291fe122
Bug 1646697 - Modify how the multi-commit tasks work. r=mozperftest-reviewers,tarek
...
This patch adds a `--test-date` option to mozperftest that lets us specify the date that we are testing on. This lets us trigger multi-commit tests on past dates, and also ensures that retriggers of the multi-commit tests use the same day that they were pushed on (giving us reproducibility).
Differential Revision: https://phabricator.services.mozilla.com/D80173
2020-06-18 22:37:35 +00:00
Gregory Mierzwinski
b1676248ea
Bug 1646368 - Add performance tuning feature to android layer. r=mozperftest-reviewers,tarek
...
This patch ports the performance tuning functionality from Raptor to mozperftest and adds a small test for it.
Differential Revision: https://phabricator.services.mozilla.com/D80168
2020-06-18 22:37:27 +00:00
Sylvestre Ledru
4b7c742212
Bug 1646405 - update the code using pyyaml to use the new path r=ahal
...
Differential Revision: https://phabricator.services.mozilla.com/D80025
2020-06-18 19:40:28 +00:00
Yue Zhang
f9a1c0c02d
Bug 1640951 - Implement a jsonschema for results coming from transformer merge functions. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D78622
2020-06-18 16:48:05 +00:00
Jim Porter
c9a16c40ed
Bug 1631806 - [part 1] Update perftests to use new API and fix --proxy
option; r=tarek
...
This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:
* `policy.json` wasn't being logged correctly, since mozlog was treating the
data as a format string
* Proxy prefs weren't being set correctly for the profile
Differential Revision: https://phabricator.services.mozilla.com/D79937
2020-06-18 03:59:54 +00:00
Mihai Alexandru Michis
6387c1ed23
Backed out changeset fa227b47dac7 (bug 1631806) for causing awsy failures regarding mitmproxy.
...
CLOSED TREE
2020-06-18 00:27:47 +03:00
Jim Porter
da72965e62
Bug 1631806 - [part 1] Update perftests to use new API and fix --proxy
option; r=tarek
...
This patch fixes the tests in testing/performance/ that hadn't been updated to
the new API, as well as fixing a couple issues with the `--proxy` option:
* `policy.json` wasn't being logged correctly, since mozlog was treating the
data as a format string
* Proxy prefs weren't being set correctly for the profile
Differential Revision: https://phabricator.services.mozilla.com/D79937
2020-06-17 09:41:24 +00:00
Tarek Ziadé
6ccb7d2797
Bug 1644912 - remove python.ini r=sparky
...
We're using a custom test runner.
Differential Revision: https://phabricator.services.mozilla.com/D79247
2020-06-11 12:19:59 +00:00
Gregory Mierzwinski
83d3f92844
Bug 1644993 - Add a non-standard prefix field to mozperftest data. r=mozperftest-reviewers,tarek
...
This patch adds a prefix field to the mozperftest perfherder data. This is a non-standard data field so the performance-artifact-schema.json file isn't modified to include it.
Differential Revision: https://phabricator.services.mozilla.com/D79296
2020-06-11 14:38:53 +00:00
Gregory Mierzwinski
9a6913c3d3
Bug 1644706 - Make a multi-commit test for app-link in CI. r=mozperftest-reviewers,tarek
...
This patch adds a new method to test multiple Fenix builds within one test run.
Depends on D78926
Differential Revision: https://phabricator.services.mozilla.com/D79107
2020-06-10 22:10:35 +00:00
Tarek Ziadé
afbe6e3cff
Bug 1644437 - add --test-iterations r=sparky
...
Adds a new option to allow running the whole test several times,
and a `before_iterations` hook
Differential Revision: https://phabricator.services.mozilla.com/D78926
2020-06-10 22:11:05 +00:00
Tarek Ziadé
a677101fc4
Bug 1644368 - improved scriptinfo r=sparky
...
Move scriptinfo to where it belongs and clean it
Differential Revision: https://phabricator.services.mozilla.com/D78870
2020-06-10 22:44:19 +00:00
Bogdan Tara
633c636ab1
Backed out 3 changesets (bug 1644706, bug 1644437, bug 1644368) for python related bustage CLOSED TREE
...
Backed out changeset 2a6645398c5a (bug 1644706)
Backed out changeset e695f5748e04 (bug 1644437)
Backed out changeset d615591507b9 (bug 1644368)
2020-06-11 01:04:33 +03:00
Gregory Mierzwinski
8d81e0549f
Bug 1644706 - Make a multi-commit test for app-link in CI. r=mozperftest-reviewers,tarek
...
This patch adds a new method to test multiple Fenix builds within one test run.
Depends on D78926
Differential Revision: https://phabricator.services.mozilla.com/D79107
2020-06-10 21:18:57 +00:00
Tarek Ziadé
b3c9f0eacc
Bug 1644437 - add --test-iterations r=sparky
...
Adds a new option to allow running the whole test several times,
and a `before_iterations` hook
Differential Revision: https://phabricator.services.mozilla.com/D78926
2020-06-10 21:44:37 +00:00
Tarek Ziadé
9fe5ec323a
Bug 1644368 - improved scriptinfo r=sparky
...
Move scriptinfo to where it belongs and clean it
Differential Revision: https://phabricator.services.mozilla.com/D78870
2020-06-10 20:57:06 +00:00
Tarek Ziadé
9458a80c0f
Bug 1644586 - remove get_nested_values r=sparky
...
unused
Differential Revision: https://phabricator.services.mozilla.com/D78988
2020-06-10 15:03:19 +00:00
Gregory Mierzwinski
de3c9ae058
Bug 1643685 - Use name instead of test_name when finding the metric test name. r=mozperftest-reviewers,tarek
...
This patch fixes the test name being used for the App-link test in CI (regression). The patch also adds a new feature to ScriptInfo which checks to make sure that the test doesn't have any extra unknown keys in it. If it does, a failure is raised. This change is to attempt to help prevent these kinds of regressions from happening again. A new test is added for this functinality as well.
Differential Revision: https://phabricator.services.mozilla.com/D78516
2020-06-08 21:02:53 +00:00
Tarek Ziadé
7489c5233a
Bug 1642779 - implement the 'androidlog' test r=acreskey,sparky
...
Add a layer that scans the android logcat
Differential Revision: https://phabricator.services.mozilla.com/D78268
2020-06-08 10:47:52 +00:00
Yue Zhang
fcaa3e19c7
Bug 1639692
- Re-enable custom transform discovery. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D76653
2020-06-05 21:09:32 +00:00
Gregory Mierzwinski
7a9ed37810
Bug 1643103 - Fix how system layers work with the test layers. r=mozperftest-reviewers,tarek
...
This patch fixes how the system and browser/test layers are run. With this fix, the system layer no longer fully runs its setup and teardown stages before the browser layer has started and finished. Now the setup/teardown happens before/after the full test layer run.
Depends on D78016
Differential Revision: https://phabricator.services.mozilla.com/D78128
2020-06-05 13:42:14 +00:00
Tarek Ziadé
18f9ccd8f2
Bug 1642615 - Renamed the browser layer to test r=sparky
...
This patch renames the browser layer to test, since
it could be something else than a browser
Differential Revision: https://phabricator.services.mozilla.com/D78016
2020-06-04 18:31:54 +00:00
Tarek Ziadé
9bcf786abb
Bug 1643621 - add moz.build r=Bebe
...
forgot to add this file, so mozperftest docs are added in the sphinx sparse profile
Differential Revision: https://phabricator.services.mozilla.com/D78465
2020-06-05 09:46:15 +00:00
xuanqi xu
33b538ca48
Bug 1636534 - Added test for notebook layer and perftestnotebook. r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D77386
2020-06-02 21:35:03 +00:00
xuanqi xu
2b9bf62319
Bug 1636534 - Separate perftestetl from perftestnotebook r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D75684
2020-06-02 21:34:49 +00:00
Tarek Ziadé
7c486e2ce0
Bug 1641834 - started a Sphinx doc r=sparky,acreskey
...
Initial Sphinx structure with minimal content.
This is mostly to get started in writing doc
Differential Revision: https://phabricator.services.mozilla.com/D77440
2020-06-03 08:17:06 +00:00
Tom Prince
581eba974b
Bug 1641971
: Pass mach perftest
options explicitly, rather than inpsect parameters; r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
Narcis Beleuzu
479c45bbea
Backed out 7 changesets (bug 1641971
) for breaking Gecko Decision Task opt . CLOSED TREE
...
Backed out changeset a039e60a26d2 (bug 1641971
)
Backed out changeset e7d4c490c31a (bug 1641971
)
Backed out changeset f2322981b294 (bug 1641971
)
Backed out changeset d123b08d2c20 (bug 1641971
)
Backed out changeset 57772c0b9b3e (bug 1641971
)
Backed out changeset 11a887f73dbb (bug 1641971
)
Backed out changeset e9c7a59d4bc5 (bug 1641971
)
2020-06-03 05:30:51 +03:00
Tom Prince
11aedca9b2
Bug 1641971
: Pass mach perftest
options explicitly, rather than inpsect parameters; r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
Gregory Mierzwinski
55c95e297a
Bug 1637994 - Add a transformer for extracting times/durations from logcat logs. r=acreskey,mozperftest-reviewers,tarek
...
This patch adds a transformer that is capable of extracting times/entries or durations for logcat logs. Here, we also implement the ability for users to pass options to the transformers so that they can be more generic (through transformer-options). The logcat transformer makes use of this, and also shows how a custom function can be used within transformers. A new test file is added for the transformer, as well as a test in test_perfherder.py so we can test the full pipeline.
Differential Revision: https://phabricator.services.mozilla.com/D77094
2020-05-29 12:40:40 +00:00
Gregory Mierzwinski
115d1b4294
Bug 1641566 - Update in-tree browsertime to latest. r=tarek,perftest-reviewers,Bebe
...
Differential Revision: https://phabricator.services.mozilla.com/D77257
2020-05-29 12:24:33 +00:00
Tarek Ziadé
3c6bbe84ba
Bug 1641549 - Run the metrics layer separately r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D77292
2020-05-29 12:21:52 +00:00
Tarek Ziadé
77da205d14
Bug 1640627 - added android logs options r=sparky
...
--android-capture-logcat - captures the logcat
--android-clear-logcat - clears the logcat on start
--android-capture-adb - captures the adb commands
Differential Revision: https://phabricator.services.mozilla.com/D77038
2020-05-28 16:42:37 +00:00
Tarek Ziadé
c0c06d6f1f
Bug 1641351 - raise on node error in BT r=acreskey
...
We forgot to check that the node command succeeded.
This patch fixes it.
Differential Revision: https://phabricator.services.mozilla.com/D77148
2020-05-27 21:43:38 +00:00
Tarek Ziadé
bddb8551ed
Bug 1640747 - Add test-level support for handling exceptions while running the tests r=acreskey
...
This patch adds the `on_exception` hook, that will let developers
drive the runner behavior when a layer fails.
Differential Revision: https://phabricator.services.mozilla.com/D76794
2020-05-26 13:35:00 +00:00
Tarek Ziadé
92a2c142f7
Bug 1640649 - add --android-timeout r=sparky
...
This patch surfaces the timeout value for ADBDevice.
It also adds the ability to run a single test with
mach perftest-test and a new --skip-lint option
to skip black/flake8
Differential Revision: https://phabricator.services.mozilla.com/D76791
2020-05-26 14:42:58 +00:00
Tarek Ziadé
af3ea56177
Bug 1639321 - increase the coverage to 85 r=sparky
...
Increase the coverage to 85%
Differential Revision: https://phabricator.services.mozilla.com/D76048
2020-05-26 14:44:14 +00:00