Commit Graph

230 Commits

Author SHA1 Message Date
Andrew Halberstadt
3d3df5c730 Bug 1445944 - [mozrunner] Create a base BlinkRuntimeRunner and add a ChromeRunner to the runners list r=rwood
This allows consumers to bootstrap Chrome with mozrunner. For now the profile implementation
is just an empty class but this will be expanded in a future commit.

MozReview-Commit-ID: 1Z14FudH0JJ

--HG--
extra : rebase_source : b593965a6bd725b133adf42ff31d61726bcff520
2018-04-12 22:29:17 -04:00
Andrew Halberstadt
07297de09a Bug 1445944 - [mozrunner] Convert mozrunner unittests to the pytest format r=davehunt
This will make it easier to add the ChromeRunner tests in the next couple of
commits.

MozReview-Commit-ID: 2Nfz92FStSX

--HG--
rename : testing/mozbase/mozrunner/tests/mozrunnertest.py => testing/mozbase/mozrunner/tests/conftest.py
extra : rebase_source : 27837679b744e7f765fdb2d4f43d4bab14fb2dc0
2018-04-12 15:14:03 -04:00
Andrew Halberstadt
11c250e87b Bug 1445944 - [mozrunner] Remove ability to specify the 'wrap_command' function on an Application context r=rwood
This is a leftover artifact from the B2G days that isn't being used anymore.

MozReview-Commit-ID: FZoTwHltmAG

--HG--
extra : rebase_source : 455450f360fa222522f365118d1e687528e08b69
2018-04-12 12:08:05 -04:00
Andreea Pavel
7a4b9a3f56 Merge mozilla-inbound to mozilla-central. a=merge
--HG--
extra : rebase_source : 66bd87105d99036ada5008499ff0eaea579b531a
2018-04-06 13:20:21 +03:00
Geoff Brown
146ff5998a Bug 1440714 - Convert Android mach commands to adb.py; r=bc 2018-04-05 12:50:29 -06:00
Henrik Skupin
4c98f9ad50 Bug 1451310 - [mozrunner] Catch exceptions when starting the client process. r=ahal
If a ProcessHandler instance has been created, but mozprocess fails
to start the child process, a dangling process_handler instance is
attached to the runner instance. This should be avoided, and a
RunnerNotStartedError has to be thrown.

MozReview-Commit-ID: LgNFVaT9qVs

--HG--
extra : rebase_source : c06aef08d7619ac9d3fe94ad29bdae06f0f79364
2018-04-04 15:04:42 +02:00
Henrik Skupin
14f784eac8 Bug 1451246 - [mozrunner] Always encode key and value of env variables as UTF-8 if those are of unicode type. r=gbrown
Without encoding the key and value of environment variables as UTF-8
for non-interactive sessions "subprocess.Popen()" and specifically
"os.execvpe()" will fail.

MozReview-Commit-ID: 9lO562XnDZx

--HG--
extra : rebase_source : bba542648b6050d0e9f628c95b658c3a546d2b5d
2018-04-04 10:36:58 +02:00
Andrew Halberstadt
61fa56c010 Bug 1328273 - Log mozrunner.utils failure messages at the error level, r=jgraham
MozReview-Commit-ID: LSzpGJeyY77

--HG--
extra : rebase_source : b9eef524597a29cfa912fb14c5eab0d58f74d430
2018-04-04 16:46:32 -04:00
Henrik Skupin
9726fd9d3b Bug 1451680 - Release mozrunner 6.15. r=gbrown
DONTBUILD

MozReview-Commit-ID: 6gRUgVbNjwA

--HG--
extra : rebase_source : ea47169db17f4e4b7a3d464d298f684d984926aa
2018-04-05 12:12:07 +02:00
Geoff Brown
b4dddb3c35 Bug 1440719 - Set MOZ_CRASHREPORTER_SHUTDOWN during tests, other than browser-chrome and non-e10s; r=ted 2018-03-28 12:43:52 -06:00
Andrew Halberstadt
c252fe40f7 Bug 1447401 - [mozrunner] Remove b2g related runners and devices, r=jmaher
MozReview-Commit-ID: 4TYYSoviSEy

--HG--
extra : rebase_source : a6fd7356ad4c320c00c1f9ddad9b3924086c4954
2018-03-20 10:43:39 -04:00
Henrik Skupin
f5e866d199 Bug 1438679 - [mozrunner] DeviceRunner has to override returncode and wait() to check for remote process status. r=gbrown
Without returncode and wait() being overridden the default
implementation of the Runner class takes precedence and will
run the check for the adb command but not the remote process.
This always returns 0 because adb runs or forks itself as daemon.

Instead the remote process has to be checked for existence.

MozReview-Commit-ID: GvuAaMSxBT2

--HG--
extra : rebase_source : e84b52fdc9ce48617102650d6d0ae73e90899538
2018-02-22 23:08:49 +01:00
Csoregi Natalia
393c7ab6c0 Backed out 2 changesets (bug 1438679) for xpcshell failures on Android. CLOSED TREE
Backed out changeset 0f79e397a320 (bug 1438679)
Backed out changeset 5685421d86eb (bug 1438679)
2018-02-26 21:57:40 +02:00
Henrik Skupin
37160bb41e Bug 1438679 - [mozrunner] DeviceRunner has to override returncode and wait() to check for remote process status. r=gbrown
Without returncode and wait() being overridden the default
implementation of the Runner class takes precedence and will
run the check for the adb command but not the remote process.
This always returns 0 because adb runs or forks itself as daemon.

Instead the remote process has to be checked for existence.

MozReview-Commit-ID: GvuAaMSxBT2

--HG--
extra : rebase_source : 9dc790a343d8f53bf00589c68f16e51a17d1b057
2018-02-22 23:08:49 +01:00
James Willcox
a47a1c6678 Bug 1291387 - Make mochitest and reftest work against TestRunnerActivity r=gbrown,jchen
Pass --appname org.mozilla.geckoview.test to 'mach mochitest' or
'mach reftest'. This runs the tests without e10s currently.

MozReview-Commit-ID: 7TIvA3zRCw2
2018-02-22 09:18:37 -06:00
Jim Chen
5e9435a262 Bug 1434469 - Add ACCESS_COARSE_LOCATION to list of permissions; r=snorp
Add ACCESS_COARSE_LOCATION to the Fennec and GeckoView list of
permissions. For completeness, also add ACCESS_COARSE_LOCATION to JS
modules that handle runtime permissions.

MozReview-Commit-ID: 8UHaiJcRnq

--HG--
extra : rebase_source : 5a74d4138d6d7b4bf6cf70724f695ff06201c38c
2018-02-13 13:01:18 -05:00
Geoff Brown
cd78045019 Bug 1434423 - Check for installation of correct test application when --app is passed to mach; r=jmaher 2018-02-09 14:34:20 -07:00
Andrew Halberstadt
f354075c7a Bug 1434430 - [flake8] Fix blank 'except' statements r=rwood
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.

This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`.  If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.

Of course, being specific is often better than a catch all.

MozReview-Commit-ID: FKx80MLO4RN

--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
2018-01-31 14:32:08 -05:00
Geoff Brown
3e48fc1177 Bug 1433308 - Follow-up to fix lint error; r=me,a=test-only 2018-01-25 17:19:44 -07:00
Geoff Brown
a993a26609 Bug 1433308 - Update 'mach android-emulator' configurations; r=jmaher
Various updates to emulator command lines. Use -skip-adb-auth. Use -verbose
instead of trying to specify debug categories. Use more -memory and -cores
where applicable. Use -ranchu and -selinux permissive where applicable.
2018-01-25 17:01:23 -07:00
Geoff Brown
6b6a2f5071 Bug 1424005 - Add support for an Android 7.0 x86_64 avd in 'mach android-emulator'; r=me, a=test-only
This enables 'mach android-emulator --version x86-7.0'. Works great for me!
2017-12-07 18:23:39 -07:00
Mike Hommey
2b2a874b47 Bug 1420355 - Statically link DMD. r=njn
--HG--
extra : rebase_source : 8e7cf975d096116b666532f3fe8aa5a7f61b5725
2017-11-28 08:10:48 +09:00
Csoregi Natalia
4ce8d0124c Backed out 7 changesets (bug 1420355) for mass failures on OS X and Android. r=backout on a CLOSED TREE
Backed out changeset a7ed89e13a4c (bug 1420355)
Backed out changeset fd6702e6e0a0 (bug 1420355)
Backed out changeset 0479dda078a2 (bug 1420355)
Backed out changeset e69357ccca9e (bug 1420355)
Backed out changeset 3742a4b69ba2 (bug 1420355)
Backed out changeset 451cd087922f (bug 1420355)
Backed out changeset d80b5c4e1dd0 (bug 1420355)
2017-11-29 03:08:46 +02:00
Mike Hommey
7c62087a43 Bug 1420355 - Statically link DMD. r=njn
--HG--
extra : rebase_source : 46800c9c0c5006a5a32f11abc209da27e65ae0f5
2017-11-28 08:10:48 +09:00
William Lachance
8eef61942a Bug 1420935 - Bump mozrunner version to 6.14 r=ahal
MozReview-Commit-ID: 2E3FotN9yJg

--HG--
extra : rebase_source : 86fbea075d2780682395633b9a82fa2b09078297
2017-11-27 11:15:23 -05:00
Tom Prince
a45ea20b93 Bug 1413575: Adjust in-tree references to mozbase documentation to point in-tree. r=ahal
MozReview-Commit-ID: rctbByyjTA

--HG--
extra : rebase_source : a682eb6d77b79640ac9ec20e9d64340613bcfed7
2017-11-20 11:43:13 -07:00
Ryan VanderMeulen
b16410f51c Merge inbound to m-c. a=merge 2017-11-10 16:13:15 -05:00
Geoff Brown
bb5c47c068 Bug 1416006 - Offer to update host-utils from mach; r=jmaher 2017-11-10 07:03:40 -07:00
Henrik Skupin
de0aa97084 Bug 1414882 - Remove unused Device.wait_for_port() in mozrunner. r=ahal
This method has not a single caller and as such doesn't seem to
be necessary anymore.

MozReview-Commit-ID: qhNK3EBc6Q

--HG--
extra : rebase_source : 2978829739f0bc465f98b8f6b727c27a03a42b11
2017-11-09 20:46:54 +01:00
Attila Craciun
7bcf901c3b Merge inbound to mozilla-central r=merge a=merge 2017-10-25 12:30:28 +03:00
Geoff Brown
5facbdc255 Bug 1410714 - Help mach tests find adb when running Android tests; r=jmaher 2017-10-24 15:17:25 -06:00
Steve Armand
fa0b6e7c7e Bug 1397849 - Enable py2 and py3 linter on testing/mozbase. r=ahal
MozReview-Commit-ID: GnaVLhtO4un

--HG--
extra : rebase_source : 8839a423c5db505469b813957649b1605ed5737f
2017-09-25 22:57:18 -04:00
Geoff Brown
cec74e4961 Bug 1390606 - (follow-up) Remove extra diagnostics; r=me,test-only 2017-10-20 15:30:59 -06:00
Geoff Brown
c4c7c9357a Bug 1390606 - Check <sdk>/emulator for emulator exe; r=nalexander 2017-10-18 15:30:48 -06:00
Mike Hommey
37d01456dc Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : e0b1dcdabe798af478e054cde0df65facf25ea21
2017-09-28 11:00:09 +09:00
Sebastian Hengst
59aaffbcda Backed out changeset 9264d5789f76 (bug 1397849) for import failures, e.g. in mochitests and xpcshell tests. r=backout 2017-10-04 15:25:07 +02:00
Steve Armand
50c9b712c0 Bug 1397849 - Enable py2 and py3 linter on testing/mozbase. r=ahal
MozReview-Commit-ID: GnaVLhtO4un

--HG--
extra : rebase_source : 19f47b7f5854b7ee25d9779622ce4cd734ac15e5
2017-09-25 22:57:18 -04:00
Sebastian Hengst
2e58d81866 Backed out changeset ff0705eda4bd (bug 1403366) 2017-10-04 01:26:56 +02:00
Mike Hommey
5f2f5b4e64 Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : e0b1dcdabe798af478e054cde0df65facf25ea21
2017-09-28 11:00:09 +09:00
Sebastian Hengst
6a0c7a5682 Backed out changeset 28b00bdf83a3 (bug 1403366) 2017-09-29 17:19:35 +02:00
Mike Hommey
8142d74974 Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : 59ba89dbd8de9c0b9361872f3f45504a46f454a2
2017-09-28 11:00:09 +09:00
Myk Melez
87ae2b846a Bug 1398345 - find llvm-symbolizer, get memory size on Mac; r=ahal 2017-09-11 09:04:57 -07:00
Eric Rahm
0938982c90 Bug 1389598 - Part 4: Remove remaining gonk refs. r=froydnj
--HG--
extra : rebase_source : 063c7f95dda063eafabfa1921366bd1957b8fe73
2017-08-11 17:45:18 -07:00
William Lachance
b16d5c2ff2 Bug 1390208 - Remove metro-specific code from testing/mozbase r=ahal
MozReview-Commit-ID: 5SPYXXTlFFt

--HG--
extra : rebase_source : 59d06862b4ff2408da69ac595511fdfff3cfdc0e
2017-08-14 13:12:07 -04:00
Christian Holler
b8ee76ebd8 Bug 1373256 - Changes to support -fsanitize=integer in the codebase. r=froydnj
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.

It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.

Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.

MozReview-Commit-ID: C5ofhfJdpCS

--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
2017-04-09 12:59:26 +02:00
Carsten "Tomcat" Book
d8e080c10c Backed out changeset 07e0c4e4ae85 (bug 1373256) for breaking android tests 2017-06-28 14:47:55 +02:00
Christian Holler
9924ab3046 Bug 1373256 - Changes to support -fsanitize=integer in the codebase. r=froydnj
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.

It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.

Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.

MozReview-Commit-ID: C5ofhfJdpCS

--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
2017-04-09 12:59:26 +02:00
Andrew Halberstadt
c9beaa56b4 Bug 1373294 - Fix E305 (two blank lines after method or class) in files enabled by flake8 linter, r=jmaher
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.

These files were modified by running:
autopep8 --select E305 --in-place -r <dir>

on the affected directories. I did it one dir at a time and verified the result after each.

MozReview-Commit-ID: FmlsfiKIbtr

--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
2017-06-15 12:10:59 -04:00
Ting-Yu Chou
5dd0d99f75 Bug 1333003 part 6 - Fix test scripts to run ASan on Windows. r=ted
MozReview-Commit-ID: 6F9UXgTRb2Y

--HG--
extra : rebase_source : c618544db62381087f4190d3b12c57c340606b15
2017-03-10 12:08:34 +08:00
Geoff Brown
7cc08b99a6 Bug 1352084 - Improve debug logging for 'mach android-emulator'; r=jmaher
Various minor improvements to aid debugging:
 - recommend --verbose on most common failure
 - in verbose mode, display platform
 - in verbose mode, display file creation date of binaries
 - in verbose mode, display sdk binary versions
 - remind of x86 vs arm emulator and need for corresponding apk
2017-04-04 09:45:19 -06:00