This is a new module that will provide a place to store some common
abstractions around the 'blessings' module. The main entrypoint is:
from mozterm import Terminal
term = Terminal()
If blessings is available, this will return a blessings.Terminal()
object. If it isn't available, or something went wrong on import,
this will return a NullTerminal() object, which is a drop-in
replacement that does no formatting.
MozReview-Commit-ID: 6c63svm4tM5
--HG--
extra : rebase_source : 9ab221774d92a418d9b098d79bb2c88f75d937f8
This patch introduces a new report formatter for the mozlint
framework used by "./mach lint" that respects Unix output conventions,
popularised by grep(1), compilers, and preprocessors.
The output format looks like this:
testing/marionette/driver.js:1153:48: no-unused-vars error: 'resp' is defined but never used.
Many editors (ed, Emacs, vi, Acme) recognise this format, allowing
users to interact with the output like a hyperlink to jump to the
specified location in a file.
MozReview-Commit-ID: 3IyiFmNbtMY
--HG--
extra : rebase_source : a2628a999c187a1c43f3cc5d32e6db835028eca4
This also migrates the vcs.py test to mozversioncontrol and adds a new task for
it.
MozReview-Commit-ID: 9jTRkjNupVA
--HG--
extra : rebase_source : 400f27498e00ea45234ad7c951770b098e916b8e
Currently line linters (linters that open a file and process it line by line,
by applying a regex for example), don't handle directories. If a directory is
passed in, it will try to 'open' it, which fails. Directories can get hit if
the linter has a directory in its include directive or if the user passes in
--no-filter.
This patch modifies LineLinters so that if a directory is detected, we search
for all relevant files under that directory. If 'extensions' is used, we'll
look for only files with appropriate extensions. Otherwise we assume the
linter wants every file.
MozReview-Commit-ID: D9lzTNuQTob
--HG--
extra : rebase_source : 0b952c06eae28b67b687813ff7e75b231b2dd4d3
This switches most tests over to use pytest as the runner instead of unittest (taking
advantage of the fact that pytest can run unittest based tests).
There were a couple tests that had failures when swithing to pytest:
config/tests/unit-expandlibs.py
xpcom/idl-parser/xpidl/runtests.py
For these tests, I added a runwith='unittest' argument so that they still run the
same way as before. Once we fix them to use pytest, the unittest logic in mozunit.py
can be deleted.
MozReview-Commit-ID: Gcsz6z8MeOi
--HG--
extra : rebase_source : 3c762422ce0af54cbbe7d9fc20085a2d1ebe7057
This allows .flake8 files to override one another, and fixes a pretty bad known
bug with our flake8 implementation. For example, say we have a .flake8 file at:
/foo/.flake8
Before this patch, if we ran |mach lint foo/bar|, the configuration defined in
that .flake8 file wouldn't get picked up. It would only work if running the
specific directory that contains it, e.g |mach lint foo|.
This change additionally allows multiple .flake8 files to be used. So if
there's one defined at both:
/.flake8
/foo/.flake8
Then running |mach lint foo/bar| will first apply the root .flake8, then the
one under /foo (overriding earlier configuration).
This bug still doesn't make flake8 configuration perfect though. Any directory
containing a .flake8 file still needs to be explicitly listed in the "include"
section of /tools/lint/flake8.yml. Otherwise in the example above, if running
|mach lint /|, it wouldn't be able to find /foo/.flake8. This is a hard problem
and is likely best solved by fixing flake8's upstream configuration handling.
Unfortunately this means we still can't switch from a whitelist to a blacklist.
MozReview-Commit-ID: 3DZAi1QHYYo
--HG--
extra : rebase_source : 51298c5847f6c2792581d9b312c87b70fa716ee1
There's a persistent test failure in automation that seems to have to do
with shutting down the `multiprocessing.Manager` that's used to get a
`Queue` to submit jobs to worker processes. After toying around with fixing
that I decided it would be simpler to just use concurrent.futures here,
since we already have it in-tree and it fits the use case here better
than using raw multiprocessing.
MozReview-Commit-ID: 8DdSvs2qp0q
--HG--
extra : rebase_source : 0b1cbb96bd3016778e4974a311722a8882f87216
While --fix previously worked with eslint, it is now more official (will show
up in the |mach lint --help|). ESlint is still the only thing that implements
it, but we can implement it for flake8 using the `autopep8` module.
--edit is a new concept that will open an editor for each failing file to let
you fix the errors manually. For now it is very naive (just opens the file),
and is only really useful if you have an editor integration for the linter(s).
But in the future I'd like to have editor-specific implementations for this.
For example, with vim, we can use -q to pass in an error file that will start
the editor pre-populated with a list of all errors that can then be easily
jumped to. Other editors may just open up to the line containing the error.
--fix and --edit can be used in conjunction with one another. Doing that means
only errors that can't be fixed automatically will show up in your editor.
MozReview-Commit-ID: 5JJJhMIrMIB
--HG--
extra : rebase_source : 2f78a77a91133d7fcc5620ecd5caa500decbce1b
While --fix previously worked with eslint, it is now more official (will show
up in the |mach lint --help|). ESlint is still the only thing that implements
it, but we can implement it for flake8 using the `autopep8` module.
--edit is a new concept that will open an editor for each failing file to let
you fix the errors manually. For now it is very naive (just opens the file),
and is only really useful if you have an editor integration for the linter(s).
But in the future I'd like to have editor-specific implementations for this.
For example, with vim, we can use -q to pass in an error file that will start
the editor pre-populated with a list of all errors that can then be easily
jumped to. Other editors may just open up to the line containing the error.
--fix and --edit can be used in conjunction with one another. Doing that means
only errors that can't be fixed automatically will show up in your editor.
MozReview-Commit-ID: 5JJJhMIrMIB
--HG--
extra : rebase_source : 889c825dcde5e950e35bc64539b299988f82ac68
We should have CI Lint YAML files in the tree.
MozReview-Commit-ID: IMOKGhxKFJW
--HG--
extra : rebase_source : 8c695dcb3a81b6683bc76ba663feeacb3126691e
We should have CI Lint YAML files in the tree.
MozReview-Commit-ID: IMOKGhxKFJW
--HG--
extra : rebase_source : a2e47f5c5517e999fe21bac4f86491611bd0cece
We should have CI Lint YAML files in the tree.
MozReview-Commit-ID: IMOKGhxKFJW
--HG--
extra : rebase_source : 0ef71f24141a531833d48f2a305183dd808f00f5
Create a test for version control related functionality.
MozReview-Commit-ID: GXd27O69GNg
--HG--
extra : rebase_source : 56ce4a38b591fd62f05fbaed0ff05d56ec127422
The underlying commands to mozlint's vcs.py had a few problems. This:
1. Ensures deleted files aren't considered in both hg and git
2. Automatically determines the default remote repo and branch when using --outgoing with git
3. Excludes metadata from output of the git command used with --outgoing
4. Adds --cached to the git command for --workdir, which lints all staged files
MozReview-Commit-ID: EBtM3MCiCDs
--HG--
extra : rebase_source : 982b24acd81c86e383b28b8a71bcd51a041e60f4
The old method had a bunch of 'if vcs == git' statements and the like. This
made it hard to follow code paths for a given vcs system. I think this refactor
makes things cleaner to read. It shouldn't be changing any functionality.
MozReview-Commit-ID: EBtM3MCiCDs
--HG--
extra : rebase_source : 9c0fda3f4f824351bae852af25bcd2240b1b1024
The rev option is inherently broken. It does let you lint files touched by any
revision, but it doesn't update those files to that revision first. Instead,
they get linted at whatever the working directory is and their results are
bogus. Even if we did some magic to update the files to the proper revision
with in-memory version control magic, the config files would still be out of
date.
Plus, the new --outgoing option does pretty much the only thing --rev was good
for. Rather than cause confusion, I think it's better to just remove the
option.
MozReview-Commit-ID: 2y2UnfIkvsR
--HG--
extra : rebase_source : 9b5c142270c98905d71ebb89d1620e91914c0b47
This was a regression from bug 1288432. The 'extensions' config in mozlint required a
leading period, but eslint requires them without the period (and this got copied over
to the linter definition). The result was mozlint filtering out any files (not dirs)
that were passed in.
This just modifies mozlint to strip out the period so both are acceptable.
MozReview-Commit-ID: CbNynYzrbGz
--HG--
extra : rebase_source : 51c740cb1d2febaee3ae46784f83381cda5e5eaa
Rather than using .lint.py files that contain a LINTER object, linter definitions are now in
standalone .yml files. In the case of external linters that need to run python code, the payload
is now of the form:
<module path>:<object path>
The <module path> is the import path to the module, and <object path> is the callable object to
use within that module. It is up to the consumer of mozlint to ensure the <module path> lives on
sys.path. For example, if an external lint's function lives in package 'foo', file 'bar.py' and
function 'lint', the payload would read:
foo.bar:lint
This mechanism was borrowed from taskcluster.
MozReview-Commit-ID: AIsfbVmozy4
--HG--
rename : python/mozlint/test/linters/badreturncode.lint.py => python/mozlint/test/linters/badreturncode.yml
rename : python/mozlint/test/linters/explicit_path.lint.py => python/mozlint/test/linters/explicit_path.yml
rename : python/mozlint/test/linters/external.lint.py => python/mozlint/test/linters/external.yml
rename : python/mozlint/test/linters/invalid_exclude.lint.py => python/mozlint/test/linters/invalid_exclude.yml
rename : python/mozlint/test/linters/invalid_extension.lnt => python/mozlint/test/linters/invalid_extension.ym
rename : python/mozlint/test/linters/invalid_include.lint.py => python/mozlint/test/linters/invalid_include.yml
rename : python/mozlint/test/linters/invalid_type.lint.py => python/mozlint/test/linters/invalid_type.yml
rename : python/mozlint/test/linters/missing_attrs.lint.py => python/mozlint/test/linters/missing_attrs.yml
rename : python/mozlint/test/linters/missing_definition.lint.py => python/mozlint/test/linters/missing_definition.yml
rename : python/mozlint/test/linters/raises.lint.py => python/mozlint/test/linters/raises.yml
rename : python/mozlint/test/linters/regex.lint.py => python/mozlint/test/linters/regex.yml
rename : python/mozlint/test/linters/string.lint.py => python/mozlint/test/linters/string.yml
rename : python/mozlint/test/linters/structured.lint.py => python/mozlint/test/linters/structured.yml
extra : rebase_source : bda3926712234123355c5af71c6453ce869b19fc
Back when the class was written, for the packaging code, it made sense
that the default was True. But now that it's used all over the place,
and that the vast majority of uses are with find_executables=False, it
makes more sense for that to be the default.
--HG--
extra : rebase_source : ff813735fc0d53093f348f20eb77ee03e9b09d4e
This is an educated guess in an attempt to solve a hard to reproduce intermittent. This
also makes sure the original SIGINT handler is restored after running the linters.
MozReview-Commit-ID: HRo0GRlSN5L
--HG--
extra : rebase_source : 7b8bf4b5818d741b5ab6a38c8c56f4eb53a801c8
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
This fixes a UnicodeDecodeError when sys.stdout's encoding can't handle unicode.
MozReview-Commit-ID: 3INna8MRje5
--HG--
extra : rebase_source : bb8f3b5cbdbb6601b6946d5fbb7396848f22b5c6
This replaces the "return_code" property on the LintRoller object with a list of "failed"
linters. This is a bit more useful as it lets us know exactly which linters had a problem
(whereas previously we just knew *something* went wrong). This patch pushes determining
the return code back into cli.py, which I think is fine.
In addition, we now pass the list of failed linters into the formatter. This allows us to
clarify exactly how many linters hit a failure which is a lot better than a seemingly
"successful" summary message.
Finally I also removed the "no files to lint" message because I've seen several people
confuse it for an error. I'll probably add it back as a debug log message when we switch
to using mozlog for output.
MozReview-Commit-ID: 4wyCeOZdOf8
--HG--
extra : rebase_source : 23810a8ab8dd9cbbad6b9e965ccff7214f947fbc
If a linter returns a status code instead of a list of results, mozlint will throw away
that status code and move on to the next linter. We should make sure that status code
bubbles all the way up to the cli.
MozReview-Commit-ID: 9fXpmtlUUT1
--HG--
extra : rebase_source : 84c80d1b24d7233d0c4c6e38b23e775110776d1f
This avoids printing harmless (but confusing) errors to the log. For instance, git users
will see a '.hg not found!' error in the output even though not finding an hg repo is
expected in that case.
MozReview-Commit-ID: DBPOabcV7PA
--HG--
extra : rebase_source : a40de185cd4bf0c58917065d1331e0e663b92b94
