Commit Graph

2129 Commits

Author SHA1 Message Date
James Graham
e8bc04493f Bug 1695263 - Skip moztest resolve tests with Python 2, r=firefox-build-system-reviewers,mhentges
wpt no longer supports this, so the test resolver doesn't work.
Since mozbuild always used Py 3 this doesn't seem to be a problem

Differential Revision: https://phabricator.services.mozilla.com/D106765
2021-03-03 10:03:05 +00:00
Geoff Brown
c6f86c79ea Bug 1694275 - Handle exceptions in mozprocess callbacks; r=jmaher
Catch and report any exceptions raised in mozprocess callbacks. This allows the
ProcessReader thread to continue and mozprocess to function normally following
an exception raised in client code.

Differential Revision: https://phabricator.services.mozilla.com/D106536
2021-02-25 21:02:14 +00:00
Nicolas B. Pierron
0c635c7669 Bug 1692253 - Create a thread to handle ADB process timeout. r=jmaher,gbrown
This change add the ability to yield the output as it is produced, by
creating a thread responsible for killing the process if it takes too
long.

This way we can loop with blocking I/O on stdout in python, while the
process is running concurrently. The loop ends when the process exit
either by itself or after being killed by the timeout.

Differential Revision: https://phabricator.services.mozilla.com/D105037
2021-02-25 11:47:26 +00:00
Geoff Brown
413323460d Bug 1694275 - Add comment to mozprocess warning about shortcomings; r=jmaher
The python subprocess module keeps getting better, while mozprocess is poorly
maintained and subject to various problems (like this bug!), so this patch
adds a comment to mozprocess warning about shortcomings.

Differential Revision: https://phabricator.services.mozilla.com/D106198
2021-02-24 21:34:04 +00:00
Joel Maher
f5334a8b26 Bug 1692570 - Allow failure jobs to handle timeouts and crashes for browser-chrome, mochitest, xpcshell. r=releng-reviewers,bhearsum
Allow failure jobs to handle timeouts and crashes for browser-chrome, mochitest, xpcshell.

Differential Revision: https://phabricator.services.mozilla.com/D105057
2021-02-22 19:38:12 +00:00
Csoregi Natalia
fc82ffc2b0 Backed out changeset 7f3db661fd19 (bug 1692570) for mochitest failures on runtests.py. CLOSED TREE 2021-02-22 18:36:32 +02:00
Joel Maher
a9ba91f56d Bug 1692570 - Allow failure jobs to handle timeouts and crashes for browser-chrome, mochitest, xpcshell. r=releng-reviewers,bhearsum
Allow failure jobs to handle timeouts and crashes for browser-chrome, mochitest, xpcshell.

Differential Revision: https://phabricator.services.mozilla.com/D105057
2021-02-22 15:55:59 +00:00
Aaron Klotz
4cef0c49f7 Bug 1581971: Part 5 - Update android_device.py to support installing and running lldb-server; r=gbrown,jmaher
In `verify_android_device`, when `debugger` is true, we setup `lldb-server`. We
also add a new method, `run_lldb_server`, for actually starting `lldb-server`.

Both cases call into a new function, `_setup_or_run_lldb_server`.

For setup functionality, this function copies `lldb-server` to the device and
then sets up its perms within the target app's data directory.

For run functionality, this function sets some environment variables and
(re)creates directories and log files as needed. Finally it starts the server
and returns to the caller the absolute path to the socket file on the device.
The client provides that path to `lldb` in order to connect to the server.

Both cases use `ADBDevice.batch_execute` to run several commands. The
`LLDB_SERVER_INSTALL_COMMANDS_SCRIPT` and `LLDB_SERVER_START_COMMAND_SCRIPT`
commands are intended to produce the same result as the `start_lldb_server.sh`
script that is provided by Android SDK.

Differential Revision: https://phabricator.services.mozilla.com/D94383
2021-02-19 21:07:19 +00:00
Aaron Klotz
06c23d6183 Bug 1581971: Part 4 - Change Android's priority debugger to lldb; r=ahal
`./mach run --debug` is being enabled for `lldb` only, so it should be
the preferred debugger instead of `gdb`.

Differential Revision: https://phabricator.services.mozilla.com/D94382
2021-02-19 21:07:19 +00:00
Aaron Klotz
ea70bcd068 Bug 1581971: Part 3 - ADBDevice changes to support mach run --debug; r=gbrown,jmaher
- We make `pidof` public and we change it to guarantee that its resulting list
  of pids consists of integers.
- We update `forward` and its dependencies to return the value of the port
  that was assigned by `adb` so that clients may learn which port to use when
  forwarding from `'tcp:0'`

Differential Revision: https://phabricator.services.mozilla.com/D94381
2021-02-19 21:07:19 +00:00
Gregory Mierzwinski
57fa6fcce0 Bug 1693042 - Decrease default timeout for talos tests to 10 minutes. r=jmaher
This patch decreases the default timeout to 10 minutes (from 2 hours) and also fixes an issue in mozcrash.

Differential Revision: https://phabricator.services.mozilla.com/D105356
2021-02-16 22:45:48 +00:00
Joel Maher
43d0329455 Bug 1678641 - add support to mozinfo for apple_silicon. r=releng-reviewers,aki
Differential Revision: https://phabricator.services.mozilla.com/D97767
2021-02-12 21:26:34 +00:00
Florin Strugariu
efc1960325 Bug 1672719 - Stop reporting confidence/replayed/not-replayed to Perfherder and remove all asociated code r=perftest-reviewers,AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D103619
2021-02-02 14:18:57 +00:00
Makoto Kato
071043917d Bug 1688521 - moznetwork supports multiple IP address on Windows using PowerShell. r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D102894
2021-02-01 09:01:34 +00:00
Joel Maher
c6c9ae717e Bug 1683001 - xpcshell support for --run-failures. r=aki
add support to xpcshell harness + mozharess for --run-failures

Differential Revision: https://phabricator.services.mozilla.com/D101662
2021-01-22 21:14:27 +00:00
Geoff Brown
fc9db06c35 Bug 1685878 - Remove remoteautomation.py; r=jmaher
remoteautomation.py is an old collection of code used by android mochitest and android reftest;
it survived the removal of automation.py. This patch removes remoteautomation.py, moving the
majority of the functionality to a new class in mozdevice. Some features are simplified or
removed, and the remainder moved into the remote mochitest/reftest harnesses.

Differential Revision: https://phabricator.services.mozilla.com/D102239
2021-01-20 17:00:18 +00:00
Gregory Mierzwinski
40f4b63085 Bug 1686327 - Rewrite raptor gecko-profiling code. r=julienw
This patch rewrites some parts of the GeckoProfiler code to make it clearer and easier to maintain. It also changes how the profiles get organized into separate folders for each type. Furthermore, the archives no longer have the full directory path in them. To do this, we also have to update browsertime.

Differential Revision: https://phabricator.services.mozilla.com/D102043
2021-01-20 15:40:35 +00:00
Tooru Fujisawa
c496e41b54 Bug 1686647 - Use integer division when performing binary search. r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D101705
2021-01-14 17:31:10 +00:00
Henrik Skupin
04e2940102 Bug 1668243 - [rust-mozversion] Release version 0.4.1. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D101590
2021-01-13 16:42:01 +00:00
Henrik Skupin
6c2057c69c Bug 1668243 - [rust-mozrunner] Release version 0.12.1. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D101589
2021-01-13 16:41:54 +00:00
Henrik Skupin
425ed3a57c Bug 1668243 - [rust-mozprofile] Release version 0.7.1. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D101588
2021-01-13 16:41:46 +00:00
Henrik Skupin
e3692a7e5f Bug 1668243 - [rust-mozdevice] Release version 0.3.1. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D101587
2021-01-13 15:55:18 +00:00
Rob Wu
5bcd69dce6 Bug 1658733 - Fix typo, use --no-system-changes in message r=jmaher DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D101472
2021-01-12 18:20:55 +00:00
Henrik Skupin
8f891b15aa Bug 1638962 - [geckodriver] Remove broken geckodriver-test mach command. r=webdriver-reviewers,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D100901
2021-01-06 12:36:17 +00:00
Geoff Brown
13ad2a5fbe Bug 1643448 - Update test handling of ACCESS_WIFI_STATE; r=geckoview-reviewers,bc,agi
Add ACCESS_WIFI_STATE to the gve manifest, same as androidTest.
Remove ACCESS_WIFI_STATE from the list of pm grants, since it is not of
type dangerous.

Differential Revision: https://phabricator.services.mozilla.com/D100727
2021-01-05 20:14:23 +00:00
Bob Clary
110347760c Bug 1681439 - [mozdevice] Supported unrooted devices with undebuggable apps. r=jmaher,gbrown
* temporarily turn off verbose during get_top_activity to prevent spamming logs.

* Update version_codes with R/30.

* Do not attempt to use su or run-as with builtin commands.

* Do not clean the test root during ADBDeviceFactory initialization
  if test_root was not specified.

* Make ADBDevice.package_dir a read-only property determined
  solely by the value of the internal ADBDevice._run_as_package
  property.

* Use new internal method ADBDevice._try_test_root_candidates to
  centralize checking if one of a list of test root candidates can be
  used.

* Use new method ADBDevice.is_package_debuggable to check if
  a package is debuggable and cache the result.

* run_as_package setter changes:

  When setting the ADBDevice.run_as_package property to None, make
  sure both the old and new values of the test root are cleaned.

  If the package is not debuggable, attempt to use one of
  /storage/emulated/0/Android/data/<app_name>/test_root,
  /sdcard/test_root or /mnt/sdcard/test_root regardless of whether
  secure storage is in effect for the version of Android. If neither
  is permitted, this will result in an ADBError which will be easily
  diagnosable.

* test_root getter

  When first attempting to access ADBDevice.test_root, attempt to use
  /data/local/tmp/test_root as the test root. If the device is not
  rooted, also include /sdcard/test_root, /storage/sdcard/test_root
  and /mnt/sdcard/test_root in the set of paths to check.

* test_root setter

  Cache the detected test root in _initial_test_root so it can be
  restored without re-running _try_test_root.

Differential Revision: https://phabricator.services.mozilla.com/D100392
2021-01-05 01:10:14 +00:00
Florin Strugariu
6a150994ae Bug 1679453 - Create a mozproxy recording mode r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D98017
2021-01-05 09:47:53 +00:00
Florin Strugariu
c3bdd440d2 Bug 1679453 - Update mitmproxy recording scripts r=AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D100154
2021-01-05 09:21:15 +00:00
James Graham
2f3983ae70 Bug 1683158 - Update mozrunner version number for release, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D100036
2020-12-17 18:00:49 +00:00
James Graham
94bef72056 Bug 1683156 - Update mozlog version number for release, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D100035
2020-12-17 18:01:17 +00:00
Bob Clary
28542c1e10 Bug 1681096 - python3 - pylint --py3k - W1639: filter built-in referenced when not iterating (filter-builtin-not-iterating) r=marionette-reviewers,perftest-reviewers,gbrown,maja_zf,whimboo
Differential Revision: https://phabricator.services.mozilla.com/D99091
2020-12-15 15:50:31 +00:00
Bob Clary
fbefc79bcb Bug 1681096 - python3 - pylint --py3k - W1641: Implementing __eq__ without also implementing __hash__ (eq-without-hash) r=marionette-reviewers,aki,gbrown,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D99092
2020-12-11 12:21:32 +00:00
Bob Clary
286a2dd0d3 Bug 1681096 - python3 - pylint --py3k - W1638: range built-in referenced when not iterating (range-builtin-not-iterating) r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D99088
2020-12-08 14:54:03 +00:00
Bob Clary
21d25f887d Bug 1681096 - python3 - pylint --py3k - W1629: __nonzero__ method defined (nonzero-method) r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D99083
2020-12-09 20:03:29 +00:00
Bob Clary
f3712b2973 Bug 1681096 - python3 - pylint --py3k - W1612: unicode built-in referenced (unicode-builtin) r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D99082
2020-12-08 16:28:08 +00:00
Bob Clary
841da37d77 Bug 1681096 - python3 - pylint --py3k - W1636: map built-in referenced when not iterating (map-builtin-not-iterating) r=perftest-reviewers,aki,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D99079
2020-12-09 08:05:21 +00:00
Bob Clary
05d29ba882 Bug 1681096 - python3 - pylint --py3k - W1633: round built-in referenced (round-builtin) r=marionette-reviewers,perftest-reviewers,aki,gbrown,whimboo
Differential Revision: https://phabricator.services.mozilla.com/D99067
2020-12-09 15:10:53 +00:00
Bob Clary
f8a258e442 Bug 1681096 - python3 - pylint --py3k - W1623: Assigning to a class's __metaclass__ attribute (metaclass-assignment) r=perftest-reviewers,gbrown
Differential Revision: https://phabricator.services.mozilla.com/D99065
2020-12-09 20:01:10 +00:00
Bob Clary
48ee50e6e3 Bug 1681096 - python3 - pylint --py3k - W1645: Exception.message removed in Python 3 (exception-message-attribute) r=marionette-reviewers,aki,gbrown,maja_zf
Differential Revision: https://phabricator.services.mozilla.com/D99059
2020-12-08 22:44:00 +00:00
Bob Clary
d841c6145f Bug 1681096 - python3 - pylint --py3k - W1656: dict.values referenced when not iterating (dict-values-not-iterating) r=perftest-reviewers,jmaher,AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D99057
2020-12-14 07:23:20 +00:00
Bob Clary
c35f5ec5c8 Bug 1681096 - python3 - pylint --py3k - W1655: dict.keys referenced when not iterating (dict-keys-not-iterating) r=marionette-reviewers,perftest-reviewers,jmaher,maja_zf
Differential Revision: https://phabricator.services.mozilla.com/D99056
2020-12-11 21:13:12 +00:00
Bob Clary
191a877dd1 Bug 1681096 - python3 - pylint --py3k - W1620: Calling a dict.iter*() method (dict-iter-method) r=marionette-reviewers,perftest-reviewers,jmaher,jgraham,AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D99049
2020-12-09 09:02:22 +00:00
Bob Clary
c8acffecc0 Bug 1681160 - python3 - mozbase/mozhttpd r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D99040
2020-12-14 20:10:53 +00:00
Bob Clary
7ce119064d Bug 1681109 - python3 - use six.moves.urllib instead of urllib in testing/mozbase/mozproxy/mozproxy/backends/mitm/scripts/alternate-server-replay.py. r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D98968
2020-12-11 19:45:55 +00:00
Bob Clary
23295f7122 Bug 1681106 - python3 - testing/mozbase/mozpower/tests/test_intelpowergadget.py thread.isAlive -> thread.is_alive. r=perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D98964
2020-12-14 19:50:00 +00:00
Bob Clary
9e125ea710 Bug 1681096 - python3 - pylint --py3k - W1619: from __future__ import division r=marionette-reviewers,perftest-reviewers,gbrown
Differential Revision: https://phabricator.services.mozilla.com/D98938
2020-12-11 16:05:25 +00:00
James Graham
bf778457d6 Bug 1678663 - Allow mozrunner.Runner instances to avoid cleanup in __del__, r=whimboo
The change in behaviour of the gc in Python 3 means that __del__ is
being called in cases where it previously wasn't, and this can have
unfortunate side effects where __del__ is being used to clean up
resources.

In particular with web-platform-tests running on Android, we end up
closing the test application unexpectedly. That causes widespread test
failures.

In general using __del__ for this kind of cleanup is problematic
because it's not defined exactly when, or if, it's called. It's much
better to require explcit cleanup (and better still to make that
scope-based, using a context manager, although this patch doesn't do
so).

Therefore we add an option to only do explicit cleanup. This is
currently opt-in to make the breakage as small as possible, but
ultimately ought to be the default and only behaviour. We also change
the wpt android runner to use this behaviour.

Differential Revision: https://phabricator.services.mozilla.com/D99256
2020-12-14 15:17:04 +00:00
James Graham
fe8c2be576 Bug 1678663 - Change wptrunner multiprocessing start method to spawn, r=karlcow,bc
This makes Linux match Windows and macOS, and avoids a py3-only bug
where mozrunner desctructors can end up being called incorrectly in
the child process, so causing tests to fail.

To do this correctly turns out to require using an explicit context
which we pass down to everywhere we use multiprocessing. That's
because wptrunner isn't usually invoked directly but from a frontend
like `wpt run` or `mach`. Therefore spawned processed don't go through
the same entrypoint as the initial process and so don't set the
context themselves.

This unfortunately requires backwards-incompatible changes in mozlog,
so that will need to be released for this to land upstream.

Differential Revision: https://phabricator.services.mozilla.com/D99247
2020-12-14 14:10:53 +00:00
James Graham
62ed408fdf Bug 1678663 - Fix TypeError under Python 3, r=bc
data[bytes] must exist but can be None; comaping None with an integer
is an error in Py3

Differential Revision: https://phabricator.services.mozilla.com/D99248
2020-12-14 14:10:51 +00:00
Bob Clary
5da968a1bf Bug 1677422 - python3 - [mozlog] use cgi.escape for Python2, html.escape for Python3. r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D99395
2020-12-11 11:43:14 +00:00