gecko-dev/testing/xpcshell
Chris Manchester 160e0a36f6 Bug 1282522 - Ensure stderr is printed in case of a shutdown crash in a passing xpcshell test. r=gps
There were two assumptions preventing this output from being logged, both
related to the case a test passes and xpcshell returns 0. The first was
that we would not find crash dumps in this case, and would therefore not
need to log the full output of the test process (in the case xpcshell
returned non-zero or a test failed, we would log this output prior to checking
for crashes). The second was that if a test was eligible to retry, we wouldn't
need to store a test's output at all, because this output would only relate to
a failure that we would consider non-fatal.

The first assumption does not hold because it's possible to fatally assert
at shutdown in tests spawning child processes without causing a test failure
or non-zero exit code.

The second assumption followed from the first, and is violated when the first
is violated, because in this case we would consider a found crash fatal even
when a test was eligible to retry.

This patch reverses these assumptions and logs the full output of a test that
passes but produces crash dumps. It's not clear that the existing code intended
for a crash to ever be considered fatal when a test was eligible to retry, but
to change this criteria now would reduce our effective test coverage by
ignoring crashes that are now considered fatal, so after this patch we continue
to consider this scenario fatal. If it is determined these crashes are related
to these tests running in parallel with other tests, or they are not relevant
for some other reason, these tests should be run sequentially, or this criteria
should be changed.


MozReview-Commit-ID: 2PaFSGx2MVR

--HG--
extra : rebase_source : 34c0d1f13f4256928906729b1f3667bc395b2c56
2016-09-12 23:02:44 -07:00
..
example Bug 1286877 - do not set c-basic-offset for python-mode; r=gps 2016-07-14 10:16:42 -06:00
moz-http2 Bug 1296280 (part 0b) - Make debugging node h2 server work better r=mcmanus 2016-09-07 09:43:47 -07:00
node-http2 Bug 1296280 (part 0a) - Update to node-http2 3.3.6 r=mcmanus 2016-09-16 08:54:03 -07:00
b2g_xpcshell_venv.sh
dbg-actors.js Bug 1203159 - Rewrite DevTools resource URLs. r=ochameau 2015-10-21 12:48:53 -05:00
head.js Bug 717566 - Remove use of parseFromStream in xpcshell head.js and android robocop_head.js. r=jmaher 2016-08-20 14:18:55 -04:00
mach_commands.py Bug 1272851 - Use objdir for temporary directory for mach xpcshell-test; r=ted 2016-05-26 11:56:20 -07:00
mach_test_package_commands.py Bug 1300163 - Reuse the 'e10s' value from the mozharness config on interactive loaners, r=armenzg 2016-09-02 14:42:36 -04:00
moz.build Bug 1286877 - do not set c-basic-offset for python-mode; r=gps 2016-07-14 10:16:42 -06:00
README
remotexpcshelltests.py Merge m-c to autoland, a=merge 2016-08-16 22:07:30 -07:00
runtestsb2g.py Bug 1207747 - Ensure that logging command line options are always available when running xpcshell tests, r=chmanchester 2015-10-08 10:28:15 +01:00
runxpcshelltests.py Bug 1282522 - Ensure stderr is printed in case of a shutdown crash in a passing xpcshell test. r=gps 2016-09-12 23:02:44 -07:00
selftest.py Bug 1303804 - Fix Promise test that relies on non-release feature (async stacks), r=froydnj 2016-09-19 15:21:25 -07:00
xpcshell.eslintrc Bug 1251011 - Enable ESLint "no-undef" rule for PSM. r=keeler r=mossop 2016-03-16 16:50:33 -07:00
xpcshellcommandline.py Bug 1278649 - Add code coverage to xpcshell tests. r=chmanchester 2016-06-08 09:41:04 -04:00

Simple xpcshell-based test harness

converted from netwerk/test/unit

Some documentation at http://developer.mozilla.org/en/docs/Writing_xpcshell-based_unit_tests
See also http://wiki.mozilla.org/SoftwareTesting:Tools:Simple_xpcshell_test_harness