This is slightly ugly, but is unfortunately necessary due to do the nature of
l10n repacks. Hopefully this can go away once we move to bundling lancpack
add-ons rather than repacking in the future.
MozReview-Commit-ID: JZUblVsEbZI
--HG--
extra : rebase_source : 60c9ced2184a52f52c7f2a8820021b14b1a66abf
This one looks to be pretty straight-forward. It irritates me that
the jar.mn entry doesn't explicitly say that the result is coming from
the object directory, like
locale/browser/bookmarks.html (!bookmarks.html)
but that's for another day.
MozReview-Commit-ID: Cw8E0VJhSxv
--HG--
extra : rebase_source : a1045a5b564b0094b562729bc7234e69ec7a786d
To vendor a Python package, run ``mach vendor python [PACKAGE]``, where
``[PACKAGE]`` is one or more package names along with a version number in the
format ``pytest==3.5.1``. The package will be installed, transient dependencies
will be determined, and a ``requirements.txt`` file will be generated with the
full list of dependencies. The requirements file is then used with ``pip`` to
download and extract the source distributions of all packages into the
``third_party/python`` directory.
If you're familiar with ``Pipfile`` you can also directly modify this in the in
the top source directory and then run ``mach vendor python`` for your changes
to take effect. This allows advanced options such as specifying alternative
package indexed (see below), and
`PEP 508 specifiers <https://www.python.org/dev/peps/pep-0508/>`_.§
MozReview-Commit-ID: CRWoFamUy7V
--HG--
extra : rebase_source : e71ab6ecc74a168faa9df7cc4c2273cd8bf9e0d2
To vendor a Python package, run ``mach vendor python [PACKAGE]``, where
``[PACKAGE]`` is one or more package names along with a version number in the
format ``pytest==3.5.1``. The package will be installed, transient dependencies
will be determined, and a ``requirements.txt`` file will be generated with the
full list of dependencies. The requirements file is then used with ``pip`` to
download and extract the source distributions of all packages into the
``third_party/python`` directory.
If you're familiar with ``Pipfile`` you can also directly modify this in the in
the top source directory and then run ``mach vendor python`` for your changes
to take effect. This allows advanced options such as specifying alternative
package indexed (see below), and
`PEP 508 specifiers <https://www.python.org/dev/peps/pep-0508/>`_.§
MozReview-Commit-ID: CRWoFamUy7V
--HG--
extra : rebase_source : e71ab6ecc74a168faa9df7cc4c2273cd8bf9e0d2
This relies on the fact that providing multiple --version-script
combines them all, so we effectively create a new symbol version
that has no global symbol, but hides the std:🧵:_M_start_thread
symbols.
This version script trick happens to work with BFD ld, gold, and lld.
The downside is that when providing multiple --version-script's, ld
doesn't want any of them to have no version at all. So for the libraries
that do already have a version script (through SYMBOLS_FILE), we use a
version where there used to be none, using the library name as the
version. Practically speaking, this binds the libraries a little closer
than they used to be, kind of non-flat namespace on OSX (which is the
default there), meaning the dynamic linker will actively want to use
symbols from those libraries instead of a system library that might
happen to have the same symbol name.
--HG--
extra : rebase_source : a7f672c35609d993849385ddb874ba791b34f929
We wrap STL headers so as to ensure our inline infallible operator new
definitions are used, but for some reason, STL wrappers were disabled
in whatever code we used gyp for first, and that ended up spreading as
more things used gyp, and now with gn, whose handling cargo culted
STL disabling from the gyp code.
We remove all this disabling, because it turns out it causes no build
problem anymore.
--HG--
extra : rebase_source : 0dd5dc3dbf613a519db74072ebda04d05ea1214c
This relies on the fact that providing multiple --version-script
combines them all, so we effectively create a new symbol version
that has no global symbol, but hides as much std::* stuff as possible.
The added symbol script could use `extern "C++"` syntax and demangled
symbols but there is no guarantee the demangled symbols won't change.
Plus, it's not possible to match demangled symbols that have a return
type: they contain a space, and the only way to match that is to use
double quotes, which doesn't allow globs at the same time.
This version script trick happens to work with BFD ld, gold, and lld.
The downside is that when providing multiple --version-script's, ld
doesn't want any of them to have no version at all. So for the libraries
that do already have a version script (through SYMBOLS_FILE), we use a
version where there used to be none, using the library name as the
version. Practically speaking, this binds the libraries a little closer
than they used to be, kind of non-flat namespace on OSX (which is the
default there), meaning the dynamic linker will actively want to use
symbols from those libraries instead of a system library that might
happen to have the same symbol name.
--HG--
extra : rebase_source : 78adb64b90e75ebad203b8a647b305c9d7198d16
This also removes the workerbootstrap test extension, which is no longer used,
and contains the last references to the Worker and ChromeWorker bootstrap
globals.
MozReview-Commit-ID: 8YWReXMqX5W
--HG--
extra : rebase_source : b0aa59b2b5e6a08f4be803e828bd507f894e4a19
AMO needs to enter the application version for every Firefox release at this time, and in doing so they don't usually enter .sec versions, as these versions are exposed to addon devs in UX where they can specify outside of the xpi what versions of Firefox they are compatible with.
Language packs however set min version to things like 59.0.2 which AMO doesn't know about.
AMO will also fail to validate an .xpi with an unknown min version.
This code logic is slightly compounded by the fact that SeaMonkey uses these codepaths as well, so we need to account for it here.
Differential Revision: https://phabricator.services.mozilla.com/D1112
--HG--
extra : rebase_source : 5bf74d235bdf651714984b7fbe0e79d2d3f61b6e
extra : histedit_source : 91a49a4c0452f217d4e3de534bfdd817dd2350cc
Since MozbuildObject.from_environment() reads from mozinfo.json, tup
picks up that file as a dependency for anything that imports buildconfig
(eg: all generated files). Using FileAvoidWrite when creating
mozinfo.json will help avoid unnecessary work after re-running configure
in the tup backend.
MozReview-Commit-ID: EEOPQYJA1MV
--HG--
extra : rebase_source : 136a0579090776dc55ea5cee870574f13cc27c58
The build system knows at build-backend time where to find each IDL
file; making xpidl-process.py rediscover this by requiring
xpidl-process.py to search through directories to find input IDL files
is silly. To rememdy this, we're going to modify things so full paths
are passed into the script. Those paths can then be used directly, with
no searching.
The previous patch required us to pass a single -I argument pointing at
$(DIST)/idl so IDL include statements would work correctly. This patch
lifts that limitation and explicitly points xpidl-process.py at the
locations of all the IDL source directories to search for included IDL
files. Invocations of xpidl-process.py no longer depend on IDL files
being copied to the objdir.
Building on the last patch, we can change the build process to pass in
the directories where the input IDL files can be found. It is
convenient to pass in just the relative source directory paths, to
encourage people to not look in the object directory and to make the
command lines slightly shorter.
xpidl-process.py still assumes that included IDL files can be found by
looking in a single directory. We add a single -I argument to the
invocation of xpidl-process.py to accommodate this short-sightedness.
The current IDL build setup assumes that all IDL files can be found in a
single directory. This setup requires that all IDL files be copied to a
single directory, which is suboptimal in terms of disk I/O and also
complicates things like generating IDL files at build time.
As a first step in moving away from this state of affairs,
xpidl-process.py needs to be taught that the input IDL files could
potentially be found in multiple directories. The current setup can
just specify $(DIST)/idl as the lone directory to examine. Future
patches will change this to examine multiple directories.
This method is only called in one place, and it doesn't pass
allow_existing. Whatever ugly thing this keyword was working around
doesn't exist anymore, so let's get rid of it.
We no longer want to update mtimes of FileAvoidWrites so that downstream
rules aren't triggered if the files aren't changed. Since the .stub file
target of GENERATED_FILES are always touched, make won't continually
rebuild them.
MozReview-Commit-ID: GxrFgCJTYk
--HG--
extra : rebase_source : f4412af1dc29142b76f7695627ba3354baf84edd
The make backend was treating the first output of a GENERATED_FILES rule
specially, since it was the target of the rule containing the script
invocation. We want the outputs of GENERATED_FILES rules to be
FileAvoidWrite so that we avoid triggering downstream rules if the
outputs are unchanged, but if the target of the script invocation is
FileAvoidWrite, then make may continually re-run the script during a
no-op build.
The solution here is to use a stub file as the target of the script
invocation which will always be touched when the script runs. Since
nothing else in the build depends on the stub, we don't need to
FileAvoidWrite it. All actual outputs of the script can be
FileAvoidWrite, and make can properly avoid work for files that haven't
changed.
MozReview-Commit-ID: 3GejZw2tpqu
--HG--
extra : rebase_source : 2b9be82f893e89a4c2f254f05b1e8b9a0f9c631b
Some GENERATED_FILES entries don't have .scripts associated with them
(notably midl on Windows builds). In this case, we don't want to
generate dependencies automatically since they will be handled by the
Makefiles.
MozReview-Commit-ID: AXmN2Unk9AY
--HG--
extra : rebase_source : 1f06672add87c46ae199189fcae27b721e008f9e
Some commands produce a large number of output files, such as
make-system-wrappers.py, which has over 1000 outputs. The GeneratedFile
handler in the tup backend displayed all the outputs, which makes the
build output unreadable, and breaks 'tup graph'. This patch displays
only the first 3 outputs and truncates the rest.
MozReview-Commit-ID: 5AnrmMe0Nyx
--HG--
extra : rebase_source : 1a6766be36aef4603c1e5333cfc13af006369966
This gets raptor to use the newly created "perf" profile that talos
also uses. There is a single pref that raptor sets that we can't set
in talos. To that end, this also creates a "raptor" specific profile.
This means to set a pref in talos and raptor, edit:
testing/profiles/perf/user.js
To set a pref in raptor only, edit:
testing/profiles/raptor/user.js
The performance of extensions can now be tested by dropping the
extension into:
testing/profiles/perf/extensions
MozReview-Commit-ID: LEJeytmmiFF
--HG--
extra : rebase_source : 0d2a6b18868f8cc6ff198ef868ad0324b57b1dc2
This moves all of the global prefs that were previously defined
in testing/talos/talos/config.py, into a new "perf" profile under
testing/profiles/perf/user.js.
This perf profile will be shared with raptor, so changes to one
framework will result in changes to the other.
MozReview-Commit-ID: JRxZEDlPu6b
--HG--
extra : rebase_source : 38f61eb6f9dd3e8dd9e0425ffe32dbdf845fcf65
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : source : 22f2dc60e6d859d3ca411826c77002d87c1a49bd
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : source : 8309212d820bcca29aa95b7892d39940437f2aa8
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : rebase_source : d0c26a006bb4dbc429be5eedad7825d4412dc2a4
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : rebase_source : 44aee637ea9b828b43b82e8639ddc3cc7f68c797
Unfortunately, FileAvoidWrite causes us to re-generate .xpt files every time we
build if a dependency of the .xpt files doesn't affect their output.
The easiest solution is to stop performing this option until we get a better
build backend like `tup` which can handle problems like this.
This works around a situation observed with old hg versions (hg 4.2?)
with mozilla-unified. I don't know why we haven't witnessed it more
generally, since the sort order was textual and should have caused
issues.
MozReview-Commit-ID: DBtfRJ3NJGR
--HG--
extra : rebase_source : b8605e34341e2c3a40f424688ecef1dbac4dc58e
Enabling this flag on rules that produce many small outputs (especially
header files that may trigger many other rules) can help reduce
incremental build time. It probably doesn't make sense to have this
enabled for .o files and linker rules, because those are less likely to
be unchanged when an input file changes, and are also larger so the time
to diff them can be more significant.
MozReview-Commit-ID: BbJaMCqPU6z
--HG--
extra : rebase_source : 31c8be252c26d3aa85e4390cf27f3ec027e88c00
This fixes `./mach run --temp-profile` after a clobber.
MozReview-Commit-ID: 7xoH5RCSpXx
--HG--
extra : rebase_source : b26f54b6ac4b0d9020fb803f9fb8c51ea51f511d
This commit also removes "DEFAULT_APP", which is unused.
MozReview-Commit-ID: 5YYaC5LJqUn
--HG--
extra : rebase_source : 45f0f8f11698890fae0dcca71174f88dbdb412c8
This commit also removes "DEFAULT_APP", which is unused.
MozReview-Commit-ID: 5YYaC5LJqUn
--HG--
extra : rebase_source : 11a5264758ab3b2d8faef1e50a666981988457f2
Now that bug 1454771 has simplified the TupBackend class, it is much
easier to pass in the self.dry_run flag into BackendTupfile to avoid
writing out Tupfiles when --dry-run is used.
MozReview-Commit-ID: 4WDgXNyYuiQ
--HG--
extra : rebase_source : e0997a65f4aecd0927f77cfb13fd6c5063fd4c36
`mach vendor rust` currently removes third_party/rust before `cargo update`,
which prevents modifying vendored crates for local testing and try pushes.
It's also unnecessary. So this patch removes the code that removes the dir.
MozReview-Commit-ID: IE0FZ3of8Py
--HG--
extra : rebase_source : 5236e2decd95f4c83b34430d5423b80e84acdf07
The old message is ambiguous as to why it failed. The new one tells
you why and hopefully gives you enough info to fix it.
MozReview-Commit-ID: 9cBpYLpCFmt
--HG--
extra : rebase_source : cf0899e8c5f5330d31c645cba27d1bd3fb19e814
Some requirements.txt are very large and result in a lot of package already
installed messages. Would be nice to hide this.
MozReview-Commit-ID: FQecuePM0zZ
--HG--
extra : rebase_source : 58eaa7324775cfaa39077871be0be0ef39ad7c11
The mozconfig output parsing code already handles shell quoted strings
to some extent (except for bug 818377), because that's what `set`
outputs. By quoting environment variable values, we avoid a bunch of
problems with "weird" values.
--HG--
extra : rebase_source : 7a45b99c3fee807da395fc42352296b000b7535f
Work around the fact that tup is bad at handling out-of-order rules by
delaying them based on what outputs we've already seen in rule().
MozReview-Commit-ID: G2tyeQr7MTh
--HG--
extra : rebase_source : d854b8b1633243177e03698c12271b11e06b0963
The xpidl-process.py invocation now takes a --bindings-conf parameter,
and the final generated .cpp file uses a new python script and was
moved from xpcom/typelib/xpt/XPTInfo.cpp to
xptcom/reflect/xptinfo/xptdata.cpp
MozReview-Commit-ID: C3vK3VzgG6Q
--HG--
extra : rebase_source : d4a6b87a9fdde76d60aaf3876e8200e551ec958b
This patch adds a python script based on the old typelib.py script which takes
in a parsed XPIDL file, and generates a json-based XPT file to use as a build
intermediate. I did my best to keep the generated format simple.
This crate contains a parser generator as a Rust crate. The parser generator is used to generate
BinSource-auto.h, BinSource-auto.cpp, BinToken.h. As of this changeset, to limit yak shaving,
the parser generator is not part of the build system. Making it part of the build system
is delegated to bug 1439645.
MozReview-Commit-ID: 1lODDSIsz8W
--HG--
extra : rebase_source : 2b09675167c12e33f5951ea00dc6df54dad11832
We no longer build .xpts in dist/bin/components and list them in an
interfaces file. Instead, they are build as intermediate files, and a
new XPTInfo.cpp is generated from *.xpt.
MozReview-Commit-ID: J05oYFK3adc
--HG--
extra : rebase_source : ad2bebfcb7877430b78441af1f41aac30c1607a9
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
For Automated submission to AMO we need unique language pack versions for every submitted build.
This will produce versions that are not valid for AMO on nightly (61.0a1buildid2018...) but that's ok, we're not ready to
submit these automatically on the nightly channel yet, they do still install fine into Firefox manually.
On beta/release/esr the version will however be ok for AMO "60.0buildid2018..." and will be reasonably unique per push.
This patch has the intent that, outside of automation, users who don't set a buildid get a langpack version without that
specified (rather than a buildid being automatically generated for them).
The releng scriptworker (addonscript) that will do the publishing to AMO will additionally sanity check that the string
'buildid' is present in the version part, so we don't accidentally break this functionality in production.
This patch is also intended to be uplifted to Gecko 60 to support ESR60 needs.
MozReview-Commit-ID: KuvwMyD6bwd
--HG--
extra : rebase_source : 04ecd87a15d5640f510b1f6123ba9467ccc4592a
In some cases we need to export other environment variables, so make it
easier to track them.
MozReview-Commit-ID: 82OOlRTdhH0
--HG--
extra : rebase_source : 1256098d1bdae9ab13bfe5a149ee33d509ead3f4
The source filename should only be used as the destination filename if
we don't have a target_basename entry for the file. For most cases
there is no difference, but in browser/components/places/jar.mn, the
same source file (bookmarkProperties.xul) is used to two different
destination files (bookmarkProperties.xul and bookmarkProperties2.xul).
MozReview-Commit-ID: LCwncq4wxtU
--HG--
extra : rebase_source : 0a86da0879d4e516013c664fbc0eb422d4be9efa
The forces on the system are such that I really need to be able to
FORCE a few RecursiveMake targets in order to make Android and Gradle
use GENERATED_FILES and LOCALIZED_GENERATED_FILES. Over time, I hope
to avoid FORCE, but that time is not now.
MozReview-Commit-ID: 453FpnihSRK
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/generated-files-force/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/generate-bar.py => python/mozbuild/mozbuild/test/backend/data/generated-files-force/generate-bar.py
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/generated-files-force/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-force/non-localized-input
rename : python/mozbuild/mozbuild/test/frontend/data/generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/generated-files-force/moz.build
rename : python/mozbuild/mozbuild/test/frontend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/frontend/data/localized-generated-files-force/moz.build
extra : rebase_source : 6820ac4b377ac596cafbdf065112738376cc473e
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
This cleans up a few things, including simplifying the look of
backend.mk by keeping the relsrcdir in MERGE_RELATIVE_FILE similar to
the source path in the tree. Before, the locales/ floated around,
which is hard to understand but doesn't matter, since it's stripped by
MERGE_RELATIVE_FILE.
This also tests both relative and topsrcdir-absolute paths.
MozReview-Commit-ID: 1v3y9xGiNfL
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/locales/en-US/localized-input
extra : rebase_source : 302d7cb638974fc5ec71513f47ce98222c5e3bb6
`mach clobber python` currently purges Python files with known
extensions globally.
The python/ and third_party/python/ directories may also contain random
ignored/untracked files. Let's purge those as well.
Note: if someone has untracked but not ignored files, this will delete
them. This is possibly unwanted. But people shouldn't have untracked
but not ignored files sitting around in VCS. We don't run this command
by default, so I think it is safe to be aggressive in our purging of
these untracked files.
MozReview-Commit-ID: 8ql8QR6lP6j
--HG--
extra : rebase_source : 644eccee25913502ed4daa55e54aec9618ebe547
`mach clobber python` is supposed to remove autogenerated Python files.
Let's add .pyd files (compiled C extensions on Windows) to the list
for good measure.
MozReview-Commit-ID: EbHvBYB7hj3
--HG--
extra : rebase_source : 2736c3d1077c6f371df20b3854840612693117f8
There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/non-localized-input
extra : rebase_source : 816b6f220758f2bb3bdd3ec81a2cb02269c6de5b
Without linking support in the tup backend, we don't produce a js
binary, so we can't copy it via OBJDIR_FILES yet.
MozReview-Commit-ID: AxqhHi84HIg
--HG--
extra : rebase_source : 1011bc6296be81646004a6bd6572193c34f367d8
Starting with Rust 1.24, the default codegen-units limit is 16,
with jobserver control to avoid overprovisioning. Remove our
previous fixed limit of 4 threads for debug builds.
For release, retain codegen-units=1 to make sure we get the
most complete optimization results.
Thanks to Simon Sapin for the suggestion.
MozReview-Commit-ID: FmYF4DcmBvt
--HG--
extra : rebase_source : 307ad8fad2874636adb3ce95a5cd47339e83f40c
The Proguard dependency is now managed by Gradle.
MozReview-Commit-ID: EOvKSE5z28P
--HG--
extra : rebase_source : 760b117f500cc639cc8c24e9c02933990f358dd7
The moz.build Java JAR definitions are, of course, broken, but they
will be removed soon enough.
MozReview-Commit-ID: KIxqLDwd9I7
--HG--
extra : rebase_source : 8312b3f125793f73d3e835d1c0a5c7cabd4ebc0c
I choose to clean a bunch of ANDROID_* moz.build cruft here, too,
since it's just passing dependencies between moz.build and
Makefile.in. The replacement for all of this is to just use
GENERATED_FILES in moz.build, but it'll still take some work to get to
that. (Why does this stuff exist? GENERATED_FILES didn't exist and
was resisted when I built this stuff.)
MozReview-Commit-ID: D3GJqJNL0Ih
--HG--
extra : rebase_source : 07351f9d3702cfc42c58bd317885d07882c45c3a
This is the easy stuff -- everything but mobile/android/base/Makefile.in.
MozReview-Commit-ID: 5x2z97AHUrR
--HG--
extra : rebase_source : 531fd41d367cad071b209b85ca5b5602fd7cbf7b
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
--HG--
extra : rebase_source : ac4fea057bf6e731b0f26a1b6902f17a7362076d
The invalid variable test for #if{,n}def was only checking that the
first character in the variable was alphanumeric or underscore, not
the other characters.
More generally, preprocessor instructions were also cut out such that
whitespaces before and after arguments were part of the arguments.
Subtly, some legitimate strings end with what, in ISO-8859-1, is
considered as whitespaces, and because the preprocessor largely works
on byte strings (str), and because the regexps are using re.U, those
characters (e.g. 0xa0) that can legitimately appear in byte strings of
UTF-8 encoding, are treated as whitespaces. So we remove the re.U from
the instruction regexp, so that only plain ascii whitespaces only are
stripped out.
There's one place in layout/tools/reftest/manifest.jsm that was using
a broken pattern, making the test never true, which, once fixed, unveils
broken tests, so the branch that was never used is removed.
--HG--
extra : rebase_source : b695dec025c55aee0e50f2a0047278fe9c849c9e
Depending on the compiler you use when --enable-stdcxx-compat, the
compiler can know about different libstdc++.so libraries that are not
suitable for your target. This will manifest as an assertion in the
current libstdcxx.py file. And then, when you change the assertion to
actually print out useful information, you will see things like:
/bin/ld: skipping incompatible /builds/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.9.4/../../../libstdc++.so when searching for -lstdc++
/bin/ld: skipping incompatible /builds/worker/workspace/build/src/clang/bin/../lib/libstdc++.so when searching for -lstdc++
which libstdcxx.py misinterprets as candidates for libstdc++.so.
This patch attempts to remedy both situations by providing a more
informative error message when things go sideways and also filtering out
error messages from the linker. You could argue that perhaps
--enable-stdcxx-compat shouldn't be getting set for such builds, but
this change seems reasonable enough on its own.
The invalid variable test for #if{,n}def was only checking that the
first character in the variable was alphanumeric or underscore, not
the other characters.
More generally, preprocessor instructions were also cut out such that
whitespaces before and after arguments were part of the arguments.
There's one place in layout/tools/reftest/manifest.jsm that was using
a broken pattern, making the test never true, which, once fixed, unveils
broken tests, so the branch that was never used is removed.
--HG--
extra : rebase_source : d1fe8a299203a29c0906ff99054c326acd135000
Right now, the "restart flow" that combines |mach watch| with the
Quick-Restart Firefox for Desktop shortcut key is frustrated by
inconsistencies writing the .purgecaches sentinels for the
application. This commit uses recent work from
https://bugzilla.mozilla.org/show_bug.cgi?id=1368699 to write the
sentinels each time |mach watch| updates the object directory.
MozReview-Commit-ID: 62Aa85oT1SE
--HG--
extra : rebase_source : 746bbe5c6f1555e8b729cbbbc1f8ca57110ae9ba
IntelliJ should still work, but we're committed to Android Studio at
this point.
MozReview-Commit-ID: 3BaXB4dh4vA
--HG--
extra : rebase_source : 0bf39a7d8daddc9a5c74182cf266f5d01d17acc8
extra : source : 901b304603d9d4816856d560c61387501efceadc
This adds a new `post_build` step to each `BuildBackend`
implementation, and uses it to write .purgecaches after every |mach
build| invocation -- including after |mach build TARGET| invocations.
This approach should be more robust than the existing recursive-Make
based solution, which seems to not write the .purgecaches files in
some situations.
In addition, the recursive-Make solution does not generalize to other
backends, in particular Tup. It is possible that the Tup backend will
handle writing the .purgecaches sentinel as part of its regular build
process, but discussions with mshal suggest that there's no convenient
way for Tup to write .purgecaches only when something *changes* during
the build. That is, Tup can achieve the behaviour implemented by this
patch, but it's not easier to do better by not writing .purgecaches
when the caches do not in fact need to be purged.
I elected to bake in the special knowledge of
--enable-application=browser and macOS here since this whole process
is special. If we need to generalize, we could add a moz.configure
option specifying the purgecaches directories, but it doesn't seem
worth it right now.
The ideal approach would be to determine FINAL_TARGET from the
application directory, but that is determined by DIST_SUBDIR. In
addition, it's not clear how to present that information to the
post-build step in a build-backend agnostic manner.
This will require tweaking as we migrate the macOS bundle handling to
moz.build, especially in browser/app. See
https://bugzilla.mozilla.org/show_bug.cgi?id=1223748, which could
improve this significantly.
MozReview-Commit-ID: 63KZy18D23i
--HG--
extra : rebase_source : e973d065cd91e965f4103ed2732858e2e7a9c546