Add test package mach support for gtest and hook into the custom retrigger
action. Some existing custom retrigger features, like setting gecko prefs,
are not (easily) applicable to gtest, which doesn't use mozprofile; for
this reason, use a separate action context with items suitable for gtest.
Differential Revision: https://phabricator.services.mozilla.com/D67384
--HG--
extra : moz-landing-system : lando
Convert the gtest option parser from optparse to argparse. mochitest, reftest,
and other suites use argparse. Using argparse will simplify the integration
of gtest with the custom retrigger action.
Differential Revision: https://phabricator.services.mozilla.com/D66429
--HG--
extra : moz-landing-system : lando
Whereas:
- desktop tests don't make this check;
- the check for directory existence has been troublesome and almost never useful;
- bug classification of this condition has been troublesome;
- if a startup crash actually did occur before crashreporter init, there would still be an indication in logcat and possibly a tombstone, and the "No test summary found" check would definitely be triggered;
Let's stop checking for minidumps directory creation.
Differential Revision: https://phabricator.services.mozilla.com/D54755
--HG--
extra : moz-landing-system : lando
Improve gtest diagnostics in an effort to improve our understanding
of this intermittent failure.
Differential Revision: https://phabricator.services.mozilla.com/D50654
--HG--
extra : moz-landing-system : lando
In Android gtest, instead of pushing the gtest libxul.so to the
(assumed and potentially incorrect) location of the test application's
libraries, push libxul.so to /data/local/gtest, and direct the linker
to use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D34942
--HG--
extra : moz-landing-system : lando
A possible explanation for the infrequent observation of an incomplete log is that
the log is still being written when the harness detects that the process has completed.
Waiting for one extra polling interval may avoid this.
Differential Revision: https://phabricator.services.mozilla.com/D38577
--HG--
extra : moz-landing-system : lando
Including the traceback info will include the exception name, like "ADBError" which
should trigger the android-em task retry; it also provides more debugging info.
Differential Revision: https://phabricator.services.mozilla.com/D36116
--HG--
extra : moz-landing-system : lando
This allows me to run 'mach gtest "APZ*"' and other short runs of gtest
that were previously failing.
Differential Revision: https://phabricator.services.mozilla.com/D34764
--HG--
extra : moz-landing-system : lando
Increasing the timeout avoids a perma-fail on linux64/ccov and leaves room for the
test suite to grow on all platforms.
Differential Revision: https://phabricator.services.mozilla.com/D30083
--HG--
extra : moz-landing-system : lando
Add basic support for 'mach gtest' on Android.
Handling of Android-only and desktop-only options is awkward; I hope to
re-visit this after bug 1519369.
Differential Revision: https://phabricator.services.mozilla.com/D28129
--HG--
extra : moz-landing-system : lando
Adds remotegtests.py, like rungtests.py, but for Android. Unlike some other test harnesses, remotegtests is not a sub-class of rungtests: There wasn't much code that could be re-used.
The code to wait for the test app and collect and display the log is derived from and similar to existing code in remoteautomation.py: I didn't want to increase reliance on remoteautomation; I don't mind a bit of code duplication.
There is a TODO item related to libxul.so. gtest runs in the browser, but with tests compiled into a gtest-specific version of libxul.so. remotegtests.py patches the installed app by pushing the libxul.so to the installation directory of the app -- root required and fragile, but works fine on the emulator. I think the "right" way to do this is to create a gtest-specific apk; I plan to file a follow-up bug and petition build folks to help sort that out.
I have been running this locally with:
mach gtest (builds gtests, then errors out)
mach python testing/gtest/remotegtests.py --libxul=/home/gbrown/objdirs/x86_64/dist/bin/gtest/libxul.so
Work-in-progress on 'mach gtest' will make this much simpler.
Differential Revision: https://phabricator.services.mozilla.com/D26835
--HG--
extra : moz-landing-system : lando
Desktop gtest creates minidumps in the current working directory. That is
problematic on Android, since the test app's cwd may not be writable, or
may not be readable by the test harness. This patch allows the test harness
to specify an alternate minidump path with environment variable
MOZ_GTEST_MINIDUMPS_PATH.
Differential Revision: https://phabricator.services.mozilla.com/D26833
--HG--
extra : moz-landing-system : lando
On Android, update mozilla gtest logging so that logging appears in the Android logcat.
Also, when MOZ_GTEST_LOG_PATH is defined in the environment, create the named file
and direct logging to that file. Android gtest will use this to collect gtest logging from
the device and copy it to the test log.
Differential Revision: https://phabricator.services.mozilla.com/D26610
--HG--
extra : moz-landing-system : lando
Before this change, testing/gtest had 19 errors. These errors have been resolved.
Bug 1506611 - Fixed flake8 errors in testing/gtest
Differential Revision: https://phabricator.services.mozilla.com/D15074
--HG--
extra : moz-landing-system : lando
GTest supports the TEST macro, which just runs a bunch of tests, and the
TEST_F macro, which enables writing custom test fixtures to perform
per-test setup and teardown. The benchmarking framework we have has a
corresponding macro for the former, but no corresponding macro for the
latter. If there's significant work to be done during setup and
teardown, we certainly don't want to be doing that work during the
actual benchmarking! So let's introduce a MOZ_GTEST_BENCH_F macro that
works similarly to the TEST_F macro.
Because the default "fast" variant uses fork() on !windows, death tests
are dangerous, as they themselves say. There are race conditions
involving locks that lead to dead locks in the death test process while
disabling the crash reporter (currently, but this could happen for
different code, even the tested code itself).
See https://bugzilla.mozilla.org/show_bug.cgi?id=1419196#c7 for details.
Using the "threadsafe" variant creates new processes for each death
test. This is notably slower, but can't dead-lock because of some random
lock being held by some random other thread at the moment fork occurred.
--HG--
extra : rebase_source : 56bf678bc9a6588751520549d57db7293134e1f8
