Commit Graph

40 Commits

Author SHA1 Message Date
Rob Wood
c70099b77d Bug 1478057 - Report the mean of all measurements per pageload, not separate values per measurement type; r=igoldan
MozReview-Commit-ID: H0lYbZPFzMY

--HG--
extra : rebase_source : bcb5b801eeec0a585f49f9e4e638beecdd1ee7e2
2018-07-27 11:10:30 -04:00
Rob Wood
b5fdd1ee41 Bug 1478443 - Fix Raptor webaudio benchmark (add webaudio back to the raptor webext manifest); r=ahal
MozReview-Commit-ID: FkrvLVKtKBh

--HG--
extra : rebase_source : 1de846f0f78d5fafea8e169d7b37f0d6a5a276ee
2018-07-25 14:09:21 -04:00
Rob Wood
1d37fa795b Bug 1436827 - add new raptor 'gdocs' pageload test suite (for google docs/slides/sheets); r=igoldan
MozReview-Commit-ID: EfhBJPgpyRc

--HG--
extra : rebase_source : 73e37a56af23f1a287e84fba4de4bf35016acba9
2018-07-20 11:50:48 -04:00
Ionut Goldan
f1e3db10e4 Bug 1472803 - Rebase on latest m-i, resolve conflicts, address PR requests r=rwood
MozReview-Commit-ID: CnXZ6ucXbe1

--HG--
extra : rebase_source : 5fda301911496faaf6308b090cf09eeb79f7b8e7
2018-07-23 14:21:24 +03:00
Ionut Goldan
0a75f53524 Bug 1472803 - Add dedicated parse method r=rwood
MozReview-Commit-ID: FEqq34xW02S

--HG--
extra : rebase_source : eda946c6f143d0709f28ff2373ebea2c123b059c
2018-07-19 14:17:31 +03:00
Ionut Goldan
0eb2e670a3 Bug 1472803 - Add TaskCluster configs for Raptor's SunSpider; tweak new raptor ini r=jmaher,rwood
MozReview-Commit-ID: AKFMkb20vyS

--HG--
extra : rebase_source : a3ae3dea637c7b3bfe6e3bb75889cb74faf5a1e0
2018-07-19 12:11:42 +03:00
Ionut Goldan
bdf791d659 Bug 1472803 - Run SunSpider in browser mode r=jmaher
MozReview-Commit-ID: GgWLVx5RveF

--HG--
extra : rebase_source : 3c00a1b0feb6831e3355e70022ae57e2d10be09b
2018-07-13 16:38:19 +03:00
Rob Wood
15993592c1 Bug 1454053 - re-record tp6 pages for raptor; r=igoldan
MozReview-Commit-ID: Esy7cWeADdy

--HG--
rename : testing/raptor/raptor/playback/mitmproxy-playback-set.manifest => testing/raptor/raptor/playback/mitmproxy-recordings-raptor-tp6.manifest
extra : rebase_source : 99437134717bb9df35a7b70c493c3166319aad44
2018-07-24 09:33:10 -04:00
Joel Maher
04f65a0f76 Bug 1473334 - run the webaudio benchmark in raptor instead of AWFY. r=rwood
port the webaudio benchmark from arewefastyet.com to in-tree raptor.

Differential Revision: https://phabricator.services.mozilla.com/D2224

--HG--
extra : moz-landing-system : lando
2018-07-19 19:03:43 +00:00
Kirk Steuber
da107c225e Bug 1420514 - Replace app.update.enabled with app.update.disabledForTesting in the test harness r=ato,chutten,jmaher,lina
MozReview-Commit-ID: GCGKQ5QCC7o

--HG--
extra : rebase_source : ba135c7d285987fd030fef4e18a5aa701a43cdba
extra : intermediate-source : e6c89a0ff0c7a7e7cd1c028e49660bab0416a531
extra : source : 646080a591c0d2365d6e8f4a493607fa3fff86c9
2018-05-22 09:14:19 -07:00
Andreea Pavel
1e8d2e6f7d Backed out 7 changesets (bug 1420514) for breaking bc on different files on a CLOSED TREE
Backed out changeset 5a46a221b6e8 (bug 1420514)
Backed out changeset 48ed6dfe8772 (bug 1420514)
Backed out changeset 01bf643c77e5 (bug 1420514)
Backed out changeset 5791b17ccb82 (bug 1420514)
Backed out changeset 416e398d942f (bug 1420514)
Backed out changeset de4a4592dbd9 (bug 1420514)
Backed out changeset d3399edf0197 (bug 1420514)
2018-07-12 21:22:32 +03:00
Kirk Steuber
35d8c30dbb Bug 1420514 - Replace app.update.enabled with app.update.disabledForTesting in the test harness r=ato,chutten,jmaher,lina
MozReview-Commit-ID: GCGKQ5QCC7o

--HG--
extra : rebase_source : e38cf587711192c3020cd9ac6a39cdbd97d2600b
extra : source : 646080a591c0d2365d6e8f4a493607fa3fff86c9
2018-05-22 09:14:19 -07:00
Joel Maher
27356a6aa3 Bug 1473907 - adjust raptor default timeout while waiting for browser closure from 75 seconds to 15 seconds. r=rwood
Differential Revision: https://phabricator.services.mozilla.com/D2002

--HG--
extra : moz-landing-system : lando
2018-07-06 15:22:11 +00:00
Joel Maher
bab63d85df Bug 1471908 - 'OSError' when raptor shutsdown the browser on win32/64. r=rwood
raptor has two calls to proc.wait() and this isn't needed.  We can monitor the browser_proc and timeout if we go too long.  Also this fixes the timeout that we use since it was never used to timeout the browser process.

Differential Revision: https://phabricator.services.mozilla.com/D1972

--HG--
extra : moz-landing-system : lando
2018-07-05 18:11:57 +00:00
Rob Wood
49994d39f4 Bug 1467824 - Port raptor-speedometer google chrome to linux64; r=jmaher
MozReview-Commit-ID: Ak2dq9zI9Nw

--HG--
extra : rebase_source : dd1df4e91c32072e4bf8e95e274b0777ef7ca604
2018-07-04 16:20:37 -04:00
Rob Wood
04c2aebe5f Bug 1468865 - Port raptor firefox tp6 to win; r=jmaher
MozReview-Commit-ID: 78VGzPkhkww

--HG--
extra : rebase_source : bffe3cfdd3384ebbff41c01347dde03f656935a7
2018-06-26 17:56:20 -04:00
Andrew Halberstadt
9435736ade Bug 1471620 - Skip python-tests locally that don't run with python 3 in CI r=davehunt
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).

MozReview-Commit-ID: 3OBr9yLSlSq

--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
2018-06-27 11:10:02 -04:00
Joel Maher
7b326fba08 Bug 1469280 - Fix raptor speedometer score calculation. r=rwood
ensure that raptor summarizes data properly from all benchmarks

Differential Revision: https://phabricator.services.mozilla.com/D1815
2018-06-26 10:53:05 +00:00
Joel Maher
9c7f5155eb Bug 1466853 - Port stylebench and motionmark benchmarks test from talos to raptor. r=rwood
stylebench and motionmark run in talos, we want these running in raptor

Differential Revision: https://phabricator.services.mozilla.com/D1814
2018-06-25 20:28:58 +00:00
Rob Wood
c69268f5dd Bug 1468535 - Port raptor-speedometer firefox to linux64; r=jmaher
MozReview-Commit-ID: 2guFxLN3yLq

--HG--
extra : rebase_source : ffddd40608baaccb4a44eb142e43ad991cc5ab73
2018-06-22 11:24:38 -04:00
Gurzau Raul
9862c7afdc Backed out changeset 96496ebab2d9 (bug 1466853) for StyleBench failures on a CLOSED TREE 2018-06-22 02:51:38 +03:00
Joel Maher
2495332bb2 Bug 1466853 - Port stylebench and motionmark benchmarks test from talos to raptor. r=rwood
Currently stylebench and motionmark run on talos, these are ideal benchmarks to run on raptor

Differential Revision: https://phabricator.services.mozilla.com/D1752
2018-06-21 21:00:41 +00:00
Gurzau Raul
977ed7b22b Backed out changeset 683a682e52b2 (bug 1466853) for flake fail on checkouts/gecko/testing/raptor/raptor/benchmark.py on a CLOSED TREE 2018-06-21 23:57:30 +03:00
Joel Maher
63ddbd28ee Bug 1466853 - Port stylebench and motionmark benchmarks test from talos to raptor. r=rwood
Currently stylebench and motionmark run on talos, these are ideal benchmarks to run on raptor

Differential Revision: https://phabricator.services.mozilla.com/D1752
2018-06-21 20:34:51 +00:00
Rob Wood
685ce41033 Bug 1467827 - Port raptor firefox tp6 to linux; r=jmaher
MozReview-Commit-ID: AQuWSuw7cFw

--HG--
extra : rebase_source : 7c74a2a31c0fdac89b9fd0950e5a53cd64e2d7f5
2018-06-13 14:04:46 -07:00
Rob Wood
978d9b823c Bug 1460743 - Add 'speedometer' benchmark to raptor for google chrome; r=jmaher
Includes a temporary downloading/installing Chrome 'hack' just so we can get going on try, until 'official' chrome builds are available in CI. Also some small raptor fixes/changes to support speedometer on chrome.

MozReview-Commit-ID: 8ZBlOg0BOE4

--HG--
extra : rebase_source : 8271a7b9efd6b1d30e363eb47876d82f243d5ca7
2018-06-04 15:42:44 -04:00
Rob Wood
ee248fc775 Bug 1460741 - Add 'speedometer' benchmark to raptor for firefox; r=jmaher
MozReview-Commit-ID: 6eTJhUJv3y9

--HG--
extra : rebase_source : 5649ab3a3f228e9fc6966bad254b9ff4d8b2f1bd
2018-05-15 14:50:48 -04:00
Rob Wood
02be8e9f99 Bug 1461676 - Expand raptor firefox tp6 to include the other pages; r=jmaher
MozReview-Commit-ID: Kzn9sRz8rip

--HG--
extra : rebase_source : e858f3a4e2b4b82ab72d90d47de32834a610b69d
2018-05-31 16:09:09 -04:00
Rob Wood
736da4ed3d Bug 1462783 - Artifact raptor test results in treeherder; r=ahal
MozReview-Commit-ID: FYXU2nVhn3n

--HG--
extra : rebase_source : 9ae83608d6bd7f94736ade618312099d02d8bce5
2018-05-24 10:50:39 -04:00
Rob Wood
7844966513 Bug 1450069 - Have the raptor control server shutdown the browser; r=ahal
Having the control server shutdown the browser via the browser pid is a better (and cross-browser) solution.

MozReview-Commit-ID: 19Gwg5TwIIy

--HG--
extra : rebase_source : d1321889d784e526d2f5ec8a593000d97b7f0927
2018-05-23 18:45:51 -04:00
Rob Wood
4f36739d0a Bug 1462776 - Add subtest support to raptor; r=ahal
Some minor changes to allow running subtests in Raptor. With this patch (and the other commit in this series) you can run raptor tests that have mutliple subtests. For example, you can specify a raptor test INI and every test inside that INI will be run and ultimately reported as subtests. This will allow, for example, the raptor tp6 test to have multiple testpages. Also just one of the subtests itself can be specified to be run instead of the entire INI.

MozReview-Commit-ID: Fto7wNOzRft

--HG--
extra : rebase_source : 37d02fb43354fa0a799c7af9ea41c16670f2244c
2018-05-23 15:14:41 -04:00
Rob Wood
c2edc6eaf3 Bug 1462776 - Run single raptor test or multiple subtests based on test name cmd line arg; r=ahal
MozReview-Commit-ID: 3hVDkEgqCD

--HG--
extra : rebase_source : 1a5108ee6e94323b98b8ad484585dad69a473f7c
2018-05-18 15:33:26 -04:00
Rob Wood
4fdc780fad Bug 1461414 - Add a post-browser startup delay and a pagecycles delay; r=jmaher
MozReview-Commit-ID: DWawXf6qWsF

--HG--
extra : rebase_source : a4918025bd75ab42cf8bc625cb75d8226ccee5af
2018-05-16 14:52:55 -04:00
Rob Wood
f49fd4e88b Bug 1449195 - Process raptor test results and format for output; r=jmaher
MozReview-Commit-ID: JJ6shYJ2cG5

--HG--
extra : rebase_source : 249fe22f6bd5ddd648ac5b906038e97e6a420c6d
2018-05-03 12:00:57 -04:00
Andrew Halberstadt
970f7e41b1 Bug 1458571 - Use base testing/profiles in raptor, r=rwood
This gets raptor to use the newly created "perf" profile that talos
also uses. There is a single pref that raptor sets that we can't set
in talos. To that end, this also creates a "raptor" specific profile.

This means to set a pref in talos and raptor, edit:
testing/profiles/perf/user.js

To set a pref in raptor only, edit:
testing/profiles/raptor/user.js

The performance of extensions can now be tested by dropping the
extension into:
testing/profiles/perf/extensions

MozReview-Commit-ID: LEJeytmmiFF

--HG--
extra : rebase_source : 0d2a6b18868f8cc6ff198ef868ad0324b57b1dc2
2018-05-04 11:19:49 -04:00
Rob Wood
b75b06de95 Bug 1459251 - Running |mach raptor-test| with no test specified results in a traceback; r=ahal
MozReview-Commit-ID: 9yggrydEoJv

--HG--
extra : rebase_source : 2c4367350b1b3ee496e620c4190e4fd32963a951
2018-05-04 13:46:03 -04:00
Rob Wood
6930542f6f Bug 1455872 - Raptor fixes for production; r=ahal
MozReview-Commit-ID: KVqypuqHjF9

--HG--
extra : rebase_source : 16a8a7ff6097e050b820c5d341ccee5d676df5bd
2018-05-01 14:02:39 -04:00
Rob Wood
8b76a9ff3c Bug 1455107 - Integrate raptor into mach; r=gbrown
MozReview-Commit-ID: 84vIqU2NWkE

--HG--
extra : rebase_source : ac5454192271d19b4da448e6be2f97dbb3420828
2018-04-23 16:43:30 -04:00
Rob Wood
a71d99bc3e Bug 1449199 - Mitmproxy integration with raptor for OSX; r=jmaher
MozReview-Commit-ID: 1Nnny3OynQw

--HG--
rename : testing/raptor/raptor/tests/raptor-firefox-tp7.ini => testing/raptor/raptor/tests/raptor-firefox-tp6.ini
extra : rebase_source : 75f109508bcfd015d82c5995a7c8ff1644a50029
2018-04-24 09:25:47 -04:00
Rob Wood
4af496d7f2 Bug 1454466 - Move raptor in-tree; r=ahal
MozReview-Commit-ID: AkqbROfvmbu

--HG--
extra : rebase_source : bf30030a333852d6d81c4278e91b1ac9fef237fe
2018-04-16 17:15:38 -04:00