As an `INFO` line under the top crash message, output MozCrashReason in the format `Mozilla crash reason: SOME_CRASH_REASON()`. This doesn't affect the top line (for bug filing) but issues some extra info from the .extra file, which may be useful to report assertion failures on Android. The format used looks something like this:
```
20:43:14 WARNING - PROCESS-CRASH | remoteautomation.py | application crashed [@ BackgroundEventTarget::Dispatch(already_AddRefed<nsIRunnable>, unsigned int)]
20:43:14 INFO - Mozilla crash reason: MOZ_CRASH()
```
Differential Revision: https://phabricator.services.mozilla.com/D59751
--HG--
extra : moz-landing-system : lando
The list of crash dumps in the dump directory appears as something like `['uuid.dmp, uuid-browser.dmp']`. When iterating over the list of files, the files are always sorted alphabetically, meaning any `uuid-<something>.dmp` goes before `uuid.dmp`. We only really care that `uuid.dmp` comes first, as the rest are just additional data not necessarily related to the crashing process, so iterating the list in reverse should put the right .dmp file first.
Differential Revision: https://phabricator.services.mozilla.com/D61166
--HG--
extra : moz-landing-system : lando
Adds `parse_extra_file(path)` to simply read some json .extra file and return a json dict, or an empty dict if it raises a `ValueError`. In patches for Bug 1572244 this function is used.
Differential Revision: https://phabricator.services.mozilla.com/D59749
--HG--
extra : moz-landing-system : lando
Changes:
`distro` and `platform` have differing behavior when presented with edgier cases such as Arch Linux.
In some cases, the output of `mozinfo.info` when run on Arch Linux comes out to:
```
(u'Arch Linux', '', '')
```
This causes problems for things that rely on mozinfo downstream because the `os_version` value is not set, and during the comparison stage of manifests, manifestparser attempts to run `somevalue > unset value` that then causes mach to error.
Differential Revision: https://phabricator.services.mozilla.com/D60462
--HG--
extra : moz-landing-system : lando
This patch will check the exit code of the ProcessHandler() that
calls the tooltool script, and will also retry when it fails.
Differential Revision: https://phabricator.services.mozilla.com/D60397
--HG--
extra : moz-landing-system : lando
The issue here is that the sort order of 2 or more elements with the same key
is indeterminate: One run (say, one test chunk) may order them one way and another
run (another test chunk) may order them another. The chunk_by_manifest() filter
relies on sort order in two places, in both cases using list length as the key
for sorting; as equal list lengths are not uncommon, multiple runs can assign
very different lists of tests to the chunks, even though all the inputs are the
same. Extending keys with unique data provides a simple fix.
Differential Revision: https://phabricator.services.mozilla.com/D59227
--HG--
extra : moz-landing-system : lando
To work around the lack of Exception.message in Python 3, I've switched adb.py to using str(e) instead of e.message where we either output or check for a substring in the exception message. This doesn't not work for cases where we want to do an equality comparison but fortunately adb.py does not use exact comparisons.
Differential Revision: https://phabricator.services.mozilla.com/D59130
--HG--
extra : moz-landing-system : lando
AVD names were updated in bug 1599534, but this case - auto-selecting an avd for an arm
build - was missed.
Differential Revision: https://phabricator.services.mozilla.com/D58694
--HG--
extra : moz-landing-system : lando
When we run a suite more than once we can get more than one result for
a test. Iterate over all results when writing the summary rather than
erroring out in this case.
Probably a better solution would be to only write the result from the
current run, but we don't necessarily know what that is, since it's
not tracked by the SummaryHandler.
Differential Revision: https://phabricator.services.mozilla.com/D58133
--HG--
extra : moz-landing-system : lando
Join <topsrcdir> with MANIFEST_PATH in android_device.py. I was concerned about TOOLTOOL_PATH
also, but that is already consistently joined with <topsrcdir> before use. I think these
are the only cases that needed attention.
Differential Revision: https://phabricator.services.mozilla.com/D58548
--HG--
extra : moz-landing-system : lando
Changes:
Change assert from `assertEquals` to `assertEqual` since the former has been technically deprecated as of 2.7.
Add python3 compatibility in `setup.py`.
This should be the last item to bring full python3 compatibility to manifestparser.
Differential Revision: https://phabricator.services.mozilla.com/D58208
--HG--
extra : moz-landing-system : lando
There was only a single use of 'install-to-subdir' in all our manifests (which
was removed in the previous commit). All the comments around it suggest that it
was a giant hack. Removing it cleans up a lot of complexity and saves me the
trouble of needing to support it in the new sourcedir based TestResolver.
Differential Revision: https://phabricator.services.mozilla.com/D57170
--HG--
extra : moz-landing-system : lando
While the 'prefs' key can only exist in the DEFAULT section of manifests, this
allows parent manifests to propagate their prefs down to included ones.
Differential Revision: https://phabricator.services.mozilla.com/D57635
--HG--
extra : moz-landing-system : lando
Since we made backwards incompatible changes, let's do a major version bump.
Differential Revision: https://phabricator.services.mozilla.com/D57478
--HG--
extra : moz-landing-system : lando
Perusing the changelog, the differences since last version bump are mostly
centered around ironing Python 3 issues. Nothing that should be backwards
incompatible, but larger changes than a dot release might warrant. So bumping
the minor version.
Differential Revision: https://phabricator.services.mozilla.com/D57854
--HG--
extra : moz-landing-system : lando
Previously the [DEFAULT] section of a manifest would simply overwrite whatever
values were passed down from the parent. This patch ensures we use
'combine_fields' so things like 'skip-if' and 'support-files' are properly
merged.
Differential Revision: https://phabricator.services.mozilla.com/D57410
--HG--
extra : moz-landing-system : lando
There aren't any manifests using '[parent:<manifest>]' in mozilla-central.
Depends on D57406
Differential Revision: https://phabricator.services.mozilla.com/D57407
--HG--
extra : moz-landing-system : lando
This was probably a remnant from Mozmill. I don't see any uses of it in mozilla-central
anywhere (or even comm-central for that matter).
Depends on D57405
Differential Revision: https://phabricator.services.mozilla.com/D57406
--HG--
extra : moz-landing-system : lando
Changes:
Fix syntax in the test file to work under python3.
Assorted changes to the code style, and eliminating unnecessary warnings.
Differential Revision: https://phabricator.services.mozilla.com/D56803
--HG--
extra : moz-landing-system : lando
Previously, when running |mach mochitest path/to/manifest.ini|, any tests in
manifests that that one includes would not be run.
This fixes that behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D57164
--HG--
extra : moz-landing-system : lando
Changes:
Import the appropriate version of `StringIO` instead of `BytesIO` depending on the version of python, and use it in `manifestparser.py`.
This is required for `test_convert_directory.py` to pass on both python versions. Changes to the test was not required.
Differential Revision: https://phabricator.services.mozilla.com/D56865
--HG--
extra : moz-landing-system : lando
We've long handled chunks by defining the total number of chunks in our CI
configuration, and then passing that value down into the test harnesses at task
runtime (via the '--this-chunk' and '--total-chunks' parameters). The test
harness then runs an algorithm to determine which tests should be run in "this"
chunk.
There are several problems with this approach, but by far the biggest is that
we can't use test information in our scheduling algorithms. The information
simply isn't available yet. This patch switches things around such that we
determine which tests go in which tasks during the taskgraph generation. This
means we have perfect information around which tasks are running which tests,
and if e.g a ccov or machine learning algorithm deems a particular test
important, we can make sure to *only* schedule the tasks that contain that
test.
I'm planning to enable this a couple suites at a time so we don't accidentally
stop running tests. This specifically only enables this mode for
'mochitest-media', 'mochitest-browser-chrome' and 'mochitest-devtools-chrome'.
I chose these suites because they are the ones that are already using the
'chunk_by_runtime' algorithm.
Differential Revision: https://phabricator.services.mozilla.com/D52729
--HG--
extra : moz-landing-system : lando
Ensures child (aka included) manifests always run in the same chunk as their parent.
Differential Revision: https://phabricator.services.mozilla.com/D55284
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Otherwise, Valgrind is liable to see false positives from mismatched
`new` where the `delete` has been inlined to `free` or vice versa.
Differential Revision: https://phabricator.services.mozilla.com/D55553
--HG--
extra : moz-landing-system : lando
Simple addition of one item to list of granted permissions, for Android P+; see bug 1553515.
Differential Revision: https://phabricator.services.mozilla.com/D55561
--HG--
extra : moz-landing-system : lando
Ensures child (aka included) manifests always run in the same chunk as their parent.
Differential Revision: https://phabricator.services.mozilla.com/D55284
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Changes:
Replace `<list>.items()` calls with `six.iteritems()`.
Remove `try/except` handling of `unittest` import as we have standardized on python2.7 in CI.
Use `six` to handle metaclass changes while python2/3 intercompatibility is required.
Differential Revision: https://phabricator.services.mozilla.com/D54376
--HG--
extra : moz-landing-system : lando
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.
This change simultaneously:
1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k
The chunks look more balanced from my unscientific glance (especially on Windows).
Differential Revision: https://phabricator.services.mozilla.com/D53702
--HG--
extra : moz-landing-system : lando
