Adapt check_for_crashes() for latest mozcrash changes, which returns the number of crashes for both log_crashes()
and check_for_crashes() now. Also the runner should accumulate the number of crashes so it will be known at any
time how many times the process has been crashed.
MozReview-Commit-ID: Dl9FlH1TalH
--HG--
extra : rebase_source : b27895482fcad099cf4fcfc01a65fe0fbc5243e3
Currently check_for_crashes() behaves differently compared to log_crashes(), whereby it only returns a
boolean if a crash has been detected but not the amount of crash reports found. We should make sure that
both methods behave the same. Given that this change might affect consumers, we should have a major version
bump for the new release.
MozReview-Commit-ID: LiPaozJL5NF
--HG--
extra : rebase_source : d4392207399a1383a20e037bcf73f44bf3c36c7d
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
--HG--
extra : rebase_source : be7a2504af6853abb1bc532a058738f33d8dcbee
added testing/mozbase to tools/lint/flake8.lint
fixed a first batch of PEP8 errors/warnings
at first the commad autopep8 -i --max-line-length 99 -r -j 8 .
has been used to fix simpler problems, run from testing/mozbase
some of the issues can not easily fixed :
- undefined 'names' in code for example isLinux - isLinux and isBsd "fixed" with # noqa
- undefined 'message' resolved with return fmt.format(...
- undefined 'structured' resolved replacing those with mozlog
- long comments - some remaining - addressed with # noqa
- package level import everything - addressed with # flake8: noqa
restored testing/mozbase/mozdevice/mozdevice/Zeroconf.py
fixed issues reported on mozreview
fixed ')' in testing/mozbase/mozprocess/mozprocess/qijo.py imports
finally fixed multiline string at testing/mozbase/manifestparser/tests/test_manifestparser.py:114
^^^ and again, but now with ./mach python-test --path-only testing/mozbase/manifestparser/tests/test_manifestparser.py passing
fixed testing/mozbase/manifestparser/tests/test_convert_directory.py assert
fixed this error:
10:15:21 INFO - return lambda line: stack_fixer_module.fixSymbols(line)
10:15:21 INFO - TypeError: fixSymbols() takes exactly 2 arguments (1 given)
fixed two spaces lint error even of # noqa comments
restored assignement to lambda with # noqa to silence the lint error
global noqa for testing/mozbase/manifestparser/tests/test_filters.py
stupid is/is not error...
MozReview-Commit-ID: 1FpJF54GqIi
--HG--
extra : rebase_source : 3cf0277fb36a296e3506aeacc2ff05e1b03f9eac
As part of converting WPT to run from a source checkout in automation,
I ran into problems where mozinfo.json wasn't being loaded, causing
several tests to fail. This is because mozinfo.json is only found
from a source directory or from a parent of the WPT meta directory.
Before, mozinfo.json was provided by the "common" tests zip file
and extracted to the base of the destination directory. WPT files
were extracted to a "web-platform" sub-directory. So mozinfo.json
was in a parent directory of "web-platform/meta" and was loaded
automatically.
Now that we want to run WPT out of the source directory, there is
no mozinfo.json and the source directory and no mozinfo.json can be
loaded.
This commit teaches mozinfo.json to look in a MOZINFO_PATH environment
variable for the path to mozinfo.json. This allows mozinfo.json
to exist anywhere on the filesystem. This provides a lot more
flexibility for loading the file.
MozReview-Commit-ID: 1I1cxS1IeyY
--HG--
extra : rebase_source : b06de2b53d3fb26df2c7b1c6d034644c20e47577
This allows the logs to work with the structured reftest viewer.
MozReview-Commit-ID: CY71vSdDjLP
--HG--
extra : rebase_source : 6b83d98aff1c5e73ac0a802b5a83b8be95adf59a
As part of converting WPT to run from a source checkout in automation,
I ran into problems where mozinfo.json wasn't being loaded, causing
several tests to fail. This is because mozinfo.json is only found
from a source directory or from a parent of the WPT meta directory.
Before, mozinfo.json was provided by the "common" tests zip file
and extracted to the base of the destination directory. WPT files
were extracted to a "web-platform" sub-directory. So mozinfo.json
was in a parent directory of "web-platform/meta" and was loaded
automatically.
Now that we want to run WPT out of the source directory, there is
no mozinfo.json and the source directory and no mozinfo.json can be
loaded.
This commit teaches mozinfo.json to look in a MOZINFO_PATH environment
variable for the path to mozinfo.json. This allows mozinfo.json
to exist anywhere on the filesystem. This provides a lot more
flexibility for loading the file.
MozReview-Commit-ID: 1I1cxS1IeyY
--HG--
extra : rebase_source : 6fb60257b6a20f6bd79461f1bf1f4f5be1f28072
Add the `pytest_mozlog` module (from https://github.com/davehunt/pytest-mozlog)
to mozbase/mozlog/mozlog, so that we can use it in the logging of the
Marionette harness tests.
Add an entry point for the plugin to mozlog/setup.py so that
when mozlog is installed, pytest will register the pytest_mozlog
plugin.py module as the plugin 'mozlog'.
Make the following modifications to pytest_mozlog/plugin.py:
- Disable stdout logging by default in pytest-mozlog
In the pytest-mozlog plugin, disable logging to stdout by
default when setting up the mozlog logger, so that the
default output seen when using the plugin is just the
Pytest terminal output. The user can specify a mozlog
logger to write to stdout by using the '--log-* -' option,
in which case both Pytest and pytest-mozlog will log to stdout.
- Record error msg and lineno in pytest-mozlog logs
Extract the error message and line number for tests with failures/
errors from the Pytest test report, and pass these to the Mozlog
logs output by the pytest-mozlog plugin. For skipped tests,
pass the skip reason, and for unexpected passes allow Mozlog to
report its default message "expected FAIL".
- Report collected item count in pytest-mozlog
Add/modify Pytest hooks in the pytest-mozlog plugin so that the
suite_start event sent to the logger contains the correct list
of tests to be run, i.e. items collected by Pytest, instead
of an empty list. This allows mozlog to log the correct number
of tests to be run.
- Log skipped/xfailed tests with corresponding status (not 'PASS')
Log skipped tests with status 'SKIP' and expected failures
that do indeed fail with status 'FAIL'/expected 'FAIL'
so that these can be distinguished from true passing tests.
- Pass parsed arguments directly to mozlog.commandline.setup_logging
Pass the entire Pytest config's argument namespace to `setup_logging`
instead of copying over only the non-None formatters/formatter option
arguments. Use the new `allow_unused_options` parameter to stop
Mozlog from raising an error about options for unused formatters.
MozReview-Commit-ID: 4WpprytS5cI
--HG--
extra : rebase_source : 409de7a89e860471870e3adfd7634bc06f6e1873
Add a parameter `allow_unused_options` to the `setup_logging` and
`setup_handlers` methods in mozlog.commandline. When this parameter
is True, do not raise a ValueError in `setup_handlers` if the
command line arguments include formatter options for unused formatters.
(Default value is False.)
This allows parsed arguments from e.g. the pytest-mozlog plugin
to be passed directly to `setup_logging`, without filtering out
unused formatter options.
MozReview-Commit-ID: 8el0oaT4y1X
--HG--
extra : rebase_source : 0e969a783b1754885e6c25d35c552134773f1b5e
In mozdevice, it's possible to pass in the path to the adb binary (rather than requiring it to be on the path). The
mochitest android options don't provide any means to set this however (even though the b2g options do). This patch
adds that option in.
This will be used by the mach environment on interactive loaners.
MozReview-Commit-ID: 4lsKGpizfH7
--HG--
extra : rebase_source : 07ad43c8fff3eb83a1eaab9ad0532cd85c4532c7
When running the command for starting Fennec, quotation marks aren't
processed properly when the 'am' portion of the command is represented
with one string token per argument; the args must be joined into one string
instead.
Also add log message about command being run in BaseRunner.
MozReview-Commit-ID: KZLnOdu9UGq
--HG--
extra : rebase_source : c8072c003afec3aba18f307b8cc64332736e55fc
This fixes an error when attempting to run xpcshell-test with a test path from an interactive loaner.
MozReview-Commit-ID: 20kg5zKplhT
--HG--
extra : rebase_source : cda8e17dd6ad7b606711630e90df8d5e65c8bda3
Ensure that the first line of the log for failing tests is identical to
one that would be produced by the tbplformatter from mozlog. This means
that treeherder will be able to sucessfully cross-match error lines.
Also make the data structures used for storing screenshots identical
between reftests and web-platform-tests so that we can start to move
toward removing the reftest-specific codepath here.
MozReview-Commit-ID: FZQXLjj9Ejv
--HG--
extra : rebase_source : 3e9bf70ff7915817b8c02032c8da0d0a659e4661
This fixes a regression from bug 1288827. It happened because I moved the logic that finds
the application path a little later on in the test harness. But there was an instance where
it was being used in the android mach command before that point.
As it turned out, we don't really *need* that value there. This patch grabs the same value
from build_obj.substs which is already an argument to the function.
MozReview-Commit-ID: 3IsI4VzEIIF
--HG--
extra : rebase_source : ceb718b265c7886b831e0378436baebf2b4e7598
This removes the unnecessary setting of c-basic-offset from all
python-mode files.
This was automatically generated using
perl -pi -e 's/; *c-basic-offset: *[0-9]+//'
... on the affected files.
The bulk of these files are moz.build files but there a few others as
well.
MozReview-Commit-ID: 2pPf3DEiZqx
--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
Any exception which gets thrown by a log handler while test results are getting generated, should
not cause test harnesses to stop immediately. To achive that the exception details are written to
stderr and not propagated up the stack.
MozReview-Commit-ID: ChyYxApYSGx
--HG--
extra : rebase_source : 9fc3fe597061bedb1df2f5b8de1daa4bd127ea1e
There are two identical pairs of screenshots (test/reference pair) which are
png/base64 raw files generated from mozlog's HTML formatter. One pair is stored
in the img element to present the visual result; the other pair is stored as a
hyperlink source in the visual result's title.
After part1 patch, we may have one more pair. It appears that the hyperlinks of
the visual result's titles could be eliminated since they are visually closed to
the visual results, and clicking the visual results provides the exact same
function.
DONTBUILD (NPOTB)
MozReview-Commit-ID: 4CLfYXX8g69
--HG--
extra : rebase_source : d4d3b4e08b66e737d75a2ca21b6e84b344a29fc8
Current mozlog (v3.2) doesn't support screenshot logs exported from wptrunner.
Add this support so we could run css test with --log-html to see more detail
information, such as screenshots of test/reference pages.
DONTBUILD (NPOTB)
MozReview-Commit-ID: AUJwYfvNfda
--HG--
extra : rebase_source : 1d3d1fcdc396638d256336d3dfaf5ba5bad35168
