Previously, using --workdir or --outgoing could miss errors when modifying a
support file since those could affect unmodified files.
This patch allows linters to define support-files in their .yml configuration.
If using --outgoing or --workdir and a file matching one of those patterns was
modified, we'll lint the entire tree.
MozReview-Commit-ID: CuGLYwQwiWr
--HG--
extra : rebase_source : 00d4107c41404f5e6ab05e0106d5cd377e25652f
Since I left the next two patches to bitrot, I realized that a path I had added
didn't exist anymore. We should definitely error out if non-existant paths are
specified, otherwise the lists will become outdated and it will be possible to
accidentally disable linting on some files.
I discovered a few instances of this already in our existing definitions.
MozReview-Commit-ID: 8jsTKLI0nFE
--HG--
extra : rebase_source : acceb0b129fc472fb456ff527e4c8c52228edd59
This crate contains a parser generator as a Rust crate. The parser generator is used to generate
BinSource-auto.h, BinSource-auto.cpp, BinToken.h. As of this changeset, to limit yak shaving,
the parser generator is not part of the build system. Making it part of the build system
is delegated to bug 1439645.
MozReview-Commit-ID: 1lODDSIsz8W
--HG--
extra : rebase_source : 2b09675167c12e33f5951ea00dc6df54dad11832
We no longer build .xpts in dist/bin/components and list them in an
interfaces file. Instead, they are build as intermediate files, and a
new XPTInfo.cpp is generated from *.xpt.
MozReview-Commit-ID: J05oYFK3adc
--HG--
extra : rebase_source : ad2bebfcb7877430b78441af1f41aac30c1607a9
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
For Automated submission to AMO we need unique language pack versions for every submitted build.
This will produce versions that are not valid for AMO on nightly (61.0a1buildid2018...) but that's ok, we're not ready to
submit these automatically on the nightly channel yet, they do still install fine into Firefox manually.
On beta/release/esr the version will however be ok for AMO "60.0buildid2018..." and will be reasonably unique per push.
This patch has the intent that, outside of automation, users who don't set a buildid get a langpack version without that
specified (rather than a buildid being automatically generated for them).
The releng scriptworker (addonscript) that will do the publishing to AMO will additionally sanity check that the string
'buildid' is present in the version part, so we don't accidentally break this functionality in production.
This patch is also intended to be uplifted to Gecko 60 to support ESR60 needs.
MozReview-Commit-ID: KuvwMyD6bwd
--HG--
extra : rebase_source : 04ecd87a15d5640f510b1f6123ba9467ccc4592a
In some cases we need to export other environment variables, so make it
easier to track them.
MozReview-Commit-ID: 82OOlRTdhH0
--HG--
extra : rebase_source : 1256098d1bdae9ab13bfe5a149ee33d509ead3f4
Also remove temporary string from preferences.ftl (search-input). This was added
to avoid removing the string when running migrations against mozilla-central
MozReview-Commit-ID: IDgTNoANPyV
--HG--
extra : rebase_source : a2fe06f2214dfb6189351cd9a0eef4e41869b639
The source filename should only be used as the destination filename if
we don't have a target_basename entry for the file. For most cases
there is no difference, but in browser/components/places/jar.mn, the
same source file (bookmarkProperties.xul) is used to two different
destination files (bookmarkProperties.xul and bookmarkProperties2.xul).
MozReview-Commit-ID: LCwncq4wxtU
--HG--
extra : rebase_source : 0a86da0879d4e516013c664fbc0eb422d4be9efa
The forces on the system are such that I really need to be able to
FORCE a few RecursiveMake targets in order to make Android and Gradle
use GENERATED_FILES and LOCALIZED_GENERATED_FILES. Over time, I hope
to avoid FORCE, but that time is not now.
MozReview-Commit-ID: 453FpnihSRK
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/generated-files-force/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/generate-bar.py => python/mozbuild/mozbuild/test/backend/data/generated-files-force/generate-bar.py
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/generated-files-force/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/non-localized-input
rename : python/mozbuild/mozbuild/test/frontend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/frontend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/localized-generated-files-force/moz.build
extra : rebase_source : 6820ac4b377ac596cafbdf065112738376cc473e
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
There a few pieces needed here to properly handle KeyboardInterrupts.
1. All in-progress work needs to abort. Ideally the underlying linters will be
able to catch KeyboardInterrupt, and return partial results (like the flake8
linter does). Linters may alternatively allow the KeyboardInterrupt to
propagate up. Mozlint will catch and handle this appropriately, though any
results found will be lost. The only change to this behaviour was fixing a bug
in the flake8 linter.
2. Any unstarted jobs need to be canceled. In concurrent.futures, there are two
different queues. First, jobs are placed on the work queue, which is just a list
maintained by the parent process. As workers become available, jobs are moved
off the work queue, and onto the call queue (which is a multiprocessing.Queue).
Jobs that live on the work queue can be canceled with 'future.cancel()', whereas
jobs that live on the call queue cannot. The number of extra jobs that are stored
on the call queue is determined by this variable:
https://hg.mozilla.org/mozilla-central/file/deb7714a7bcd/third_party/python/futures/concurrent/futures/process.py#l86
In this patch, the parent process' sigint handler (which will be called on Ctrl-C)
is responsible for canceling all the jobs on the work queue. For the jobs on the
call queue, the best we can do is set a global variable that tells workers to
abort early.
3. Idle workers should exit gracefully. When there are no more jobs left, workers
will block on the call queue (either waiting for more jobs, or waiting for the
executor to send the shutdown signal). If a KeyboardInterrupt is received while a
worker is blocking, it isn't possible to intercept that anywhere (due to quirks
of how concurrent.futures is implemented). The InterruptableQueue class was
created to solve this problem. It will return None instead of propagating
KeyboardInterrupt. A None value will wake the worker up and tell it to gracefully
shutdown. This way, we avoid cryptic tracebacks in the output.
With all of these various pieces solved, pressing Ctrl-C appears to always exit
gracefully, sometimes even printing partial results.
MozReview-Commit-ID: 36Pe3bbUKmk
--HG--
extra : rebase_source : d4c312ee5cc3679eeee1407c5521aed679f84ad4
extra : source : a93a00141bf62f6bc9e30934c0e56f6b2e434bf0
This commit doesn't change any behaviour, just attempts to make this a little
more readable. The workers will call '_collect_results' for each WorkItem they
process (either because it is finished or because it was canceled).
This also differentiates between setup failures and run failures.
MozReview-Commit-ID: 36Pe3bbUKmk
--HG--
extra : rebase_source : 873167512b745ccdc52de7e7f1ecf66b094e063d
