This implements a chunk_by_manifest algorithm. It is similar to chunk_by_slice
in that it tries to make an even number of tests run in each chunk. However,
unlike chunk_by_slice it will guarantee that tests in the same manifest will
all run in the same chunk. This makes it suitable to use with run-by-manifest.
This means the chunks won't be perfect (as manifests are differnet sizes). It
is also prone to more randomization, similar to chunk-by-runtime.
In fact, this algorithm is nearly identical to the chunk-by-runtime one, so it
was refactored out to a base class.
MozReview-Commit-ID: HI2ByxW0i8V
--HG--
extra : rebase_source : e066c034b85222d26bafe6873a80366d5bd9df9e
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
Currently manifestparser will only look for line continuations *after* looking for a key. This means
that line continuations cannot contain key separators. For example, this:
[test]
foo=
bar=baz
gets treated as:
{'name': 'test', 'foo': '', 'bar': 'baz'}
Here, bar=baz will be treated as a new key/value pair despite the indentation. This patch switches
the order around, so we look for a continuation first. Now, it is only treated as a continuation if
the indent is greater than the indent of the preceding key.
So this manifest:
[test]
foo=bar
baz=fleem
is a continuation and results in:
{'name': 'test', 'foo': 'bar\nbaz=fleem'}
But this manifest:
[test]
foo=bar
baz=fleem
is not a continuation, and yields:
{'name': 'test', 'foo': 'bar', 'baz': 'fleem'}
MozReview-Commit-ID: FAMP5TUIo9q
--HG--
extra : rebase_source : 624c53cfe0565374c1224dd86a3fffc8831279d3
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
The subsuite is added conditionally because we only have the capability of
running source-check tasks on linux at the moment. Once taskcluster support
for windows and mac has matured a bit and the taskcluster configs support
source-check there, we should apply the subuite unconditionally.
MozReview-Commit-ID: Kk9Irz3fn14
--HG--
extra : rebase_source : b9266a06583083c36477d4e93f5462ee614cdb71
This refactors some of the error logic into a new IniParseError exception. In addition to checking for a 'name' property, we also
check for a 'path' property. This is because some file-like objects coming out of the build system use a 'path' attribute instead
of the standard 'name'.
MozReview-Commit-ID: EXXl9gt1MQk
--HG--
extra : rebase_source : a130824e6724fdef5cf0bd627b19db68ec5fe683
Previously manifestparser only supported inline comments on conditional keys, such as
skip-if, fails-if, subsuite-if, etc. But on any other key name, inline comments weren't
supported. This was a bad situation because people saw these skip-if comments, and
naturally assumed inline comments worked everywhere. Then they were surprised when things
broke in mysterious ways.
This patch removes the special-casing for skip-if and friends and allows inline comments
everywhere. A caveat to this, is that ' #' is no longer a valid substring in a value.
MozReview-Commit-ID: Hr0BIwzTgaJ
--HG--
extra : rebase_source : f9a042da2137f200434140a7e472df0799f6606d
It turns out there are shockingly few cases of manifestparser manifests that actually use the ';'
character as a comment. Because we will soon allow inline comments, deprecating the use of ';' will
ensure that values are allowed to have semicolons in them.
Even without inline comments, might as well enforce consistency across manifests.
MozReview-Commit-ID: AEPPQFdNXG0
--HG--
extra : rebase_source : 3540fa385f328bffb020c0a6debc4d2b3a90ed39
The mozbase unittests don't use mozunit, so their output is confusing in the log.
This makes mozbase output consistent with the rest of the python unittests.
MozReview-Commit-ID: AIs5mza8Rn6
--HG--
extra : rebase_source : 10f65e612f5b3cebb921c47699f5a8be7cd2ba5a
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
--HG--
extra : rebase_source : be7a2504af6853abb1bc532a058738f33d8dcbee
added testing/mozbase to tools/lint/flake8.lint
fixed a first batch of PEP8 errors/warnings
at first the commad autopep8 -i --max-line-length 99 -r -j 8 .
has been used to fix simpler problems, run from testing/mozbase
some of the issues can not easily fixed :
- undefined 'names' in code for example isLinux - isLinux and isBsd "fixed" with # noqa
- undefined 'message' resolved with return fmt.format(...
- undefined 'structured' resolved replacing those with mozlog
- long comments - some remaining - addressed with # noqa
- package level import everything - addressed with # flake8: noqa
restored testing/mozbase/mozdevice/mozdevice/Zeroconf.py
fixed issues reported on mozreview
fixed ')' in testing/mozbase/mozprocess/mozprocess/qijo.py imports
finally fixed multiline string at testing/mozbase/manifestparser/tests/test_manifestparser.py:114
^^^ and again, but now with ./mach python-test --path-only testing/mozbase/manifestparser/tests/test_manifestparser.py passing
fixed testing/mozbase/manifestparser/tests/test_convert_directory.py assert
fixed this error:
10:15:21 INFO - return lambda line: stack_fixer_module.fixSymbols(line)
10:15:21 INFO - TypeError: fixSymbols() takes exactly 2 arguments (1 given)
fixed two spaces lint error even of # noqa comments
restored assignement to lambda with # noqa to silence the lint error
global noqa for testing/mozbase/manifestparser/tests/test_filters.py
stupid is/is not error...
MozReview-Commit-ID: 1FpJF54GqIi
--HG--
extra : rebase_source : 3cf0277fb36a296e3506aeacc2ff05e1b03f9eac
This fixes an error when attempting to run xpcshell-test with a test path from an interactive loaner.
MozReview-Commit-ID: 20kg5zKplhT
--HG--
extra : rebase_source : cda8e17dd6ad7b606711630e90df8d5e65c8bda3
This requires a change to how we process test manifests in the build system:
now, whenever we see a support file mentioned in a manifest, we require that
file isn't already in that test's support files, but if we see a support file
that was already seen in some other test, the entry is ignored, but it is not
an error. As a result of this change, several duplicate support-files entries
needed to be removed.
MozReview-Commit-ID: G0juyxzcaB8
--HG--
rename : testing/mozbase/manifestparser/tests/test_default_skipif.py => testing/mozbase/manifestparser/tests/test_default_overrides.py
We need to go through an abstraction layer when manifestparser manifests
are read as a part of reading moz.build files in case moz.build reading
is using mercurial as its filesystem. This adds an optional finder member,
which will be used if present for IO that happens when reading manifests,
This needs to be optional, because the manifestparser can be used as a
standalone package that isn't distributed with mozpack.
--HG--
extra : commitid : 7S8mdpexvgW
Profiling the manifestparser revealed that 25% of its total time is spent in a single
call to os.path.normpath, called on each test path in each manifest. For the manifests
in mozilla-central, these calls almost always return their input. To save time, this
patch modifies the manifestparser to not call normpath on paths that don't contain
'..'. This will change the parsed result of manifests containing no-op path
components ('//' or '/./') to include those components in their output.
--HG--
extra : commitid : D7g4GX6VLuC
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
--HG--
extra : rebase_source : d75905da1ca021a15a9538117d3866425f73d962
extra : source : 3c34fd480729e3b6684fba747ff61078f672ce16
With --chunk-by-runtime enabled, test runtime data collected from automation is used to try and make all chunks take the same amount of time. So far only data for mochitest browser-chrome is added.
--HG--
extra : rebase_source : e6f8327286ee3e69036f7781211f618b870823b5
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
--HG--
extra : rebase_source : 68a0931c6a8ee1df4f5c09d67c396490774aa856
With this chunking strategy, the runtimes of tests are taken into account, such that each chunk
takes roughly the same amount of time to finish. Tests belonging to the same manifest will not get
split up.
The algorithm works by sorting every manifest from slowest to fastest. Each manifest is popped off
and its tests are added to the fastest chunk to date until no manifests are left. Total runtimes of
the chunks are re-calculated after every addition.
--HG--
extra : rebase_source : 55eb880256549cb387e659cddbad0f20eaca9f3b
A filter is a callable that accepts an iterable of tests and a dictionary of values (e.g mozinfo.info) and returns an iterable of tests. Note filtering can mean modifying tests in addition to removing them. For example, this implements a "timeout-if" tag in the manifest:
from manifestparser import expression
import mozinfo
def timeout_if(tests, values):
for test in tests:
if 'timeout-if' in test:
timeout, condition = test['timeout-if'].split(',', 1)
if expression.parse(condition, **values):
test['timeout'] = timeout
yield test
tests = mp.active_tests(filters=[timeout_if], **mozinfo.info)
--HG--
extra : rebase_source : adead90910811e71e8ea2bb862f2b8e92f2c1bee
Simple refactor that moves logic out of manifestparser.py and into cli.py, expression.py and ini.py.
--HG--
extra : rebase_source : dd454973cdb3bcb7ec29dd2e1c0c594e3b3fb817
