Toolchains that are used for local development need to be built on a level-3
branch to installable via `mach bootstrap`. Add an attribute to track the fact
that a toolchain is used that way, and:
- ensure that everything installed via `mach boostrap` has that attribute set
- ensure that everything with that attribute set is built on trunk projects
We could additionally verify that attribute is only set on things used by
bootstrap, but bootstrap doesn't currently have an exhaustive list of things
that it might install, making that difficult.
Differential Revision: https://phabricator.services.mozilla.com/D77706
Also update documentation to suggest using the `GeneratedFile` template rather than directly referencing `GENERATED_FILES` where possible.
Differential Revision: https://phabricator.services.mozilla.com/D77496
At the same time, because it's now simpler to do so, set the right data
file name for big-endians, even though we don't have or produce it
(bug #1264836). Also remove USE_ICU, which is redundant with
JS_HAS_INTL_API, and actively break the build at configure time when
using --without-intl-api with Firefox because this hasn't actually
worked for close to 3 years (since bug 1402048).
Differential Revision: https://phabricator.services.mozilla.com/D77416
The architecture of `mach` does not make this easy to do from within one process, not least of all due to persistent global, mutable state. :( There may be another way to do this, I'm not saying it's completely impossible (and obviously with a significant enough refactor there's a way to make this happen), but subprocessing is a foolproof way to accomplish the same.
Differential Revision: https://phabricator.services.mozilla.com/D77374
l10n-merge creates a full merge dir for a while now, let's
simplify the build logic to only read from that directory
during repacks and langpacks.
Differential Revision: https://phabricator.services.mozilla.com/D77023
Arrays are mutable, so appending each substring at a time and joining at the endis much faster than re-allocating
a new string on each loop.
Differential Revision: https://phabricator.services.mozilla.com/D76945
The need for --disable-install-strip in the mac mozconfigs comes from a
discrepancy in how stripping is handled between platforms. On Windows,
there is no stripping. On non-Mac unix, `strip` removes local symbols as
well as debug info and debug symbols. On Mac, it actually removes too
much, and one has to pass flags to remove both local symbols (`-x`) and
debug symbols (`-S`). Debug info is already in a separate file
(`.dSYM`).
For profiling reasons, we do ship e.g. nightlies with local symbols but
not debug info or symbols (or at least that's the intent). On Windows,
again, nothing to do. On non-Mac unix, we pass `--strip-debug` to
`strip` so that it keeps local symbols. That's where the discrepancy
comes in for Mac: the build system doesn't handle this at all, so the
mozconfigs contain --disable-install-strip to avoid stripping.
The build system should be doing what it's expected to be doing from the
start, without mozconfigs opting into anything.
AFAIK, we only really need the local symbols, so we can `strip -S` on
Mac when profiling is enabled, rather than `strip -x -S`. This also
significantly reduces the size of the installer for nightlies.
And while we're here, move the logic out of old-configure and into
python configure.
Differential Revision: https://phabricator.services.mozilla.com/D76789
`StaticAnalysisMonitor` uses `clang-tidy` config dictionary in order to
match different options for the checkers that are being used. For this `StaticAnalysisMonitor`
modifies this dictionary. The dictionary not being a deep copy of the original one, the
modifications are echoed in the initial dictionary thus having the potential to lead to
unexpected behavior from `clang-tidy` when generating the checkers list.
Differential Revision: https://phabricator.services.mozilla.com/D76834
Changes:
Current in-tree code will simply print `{path}` as many times as it reads a path.
This change will ensure that paths that have been read are output to the screen, which is much more useful.
Differential Revision: https://phabricator.services.mozilla.com/D76729
At the beginning of the Python 3 migration (circa bug 1602540), we made an update to the interface of `mozpack/files.py` in the direction of aligning with Python 3's built-in `file` support; namely, that opening a file in text mode returns a stream of `str` (text), and that opening a file in binary mode returns a stream of `bytes`. This was deemed to be more trouble than it was worth. This patch undoes all of those changes to the interface in favor of moving back to the Python 2 style, where all files are bytestreams.
Differential Revision: https://phabricator.services.mozilla.com/D75424
The module `distutils.sysconfig` isn't installed by default on Ubuntu -- the package `python3-distutils` exposes it. That's fine, except we unconditionally import it in `virtualenv.py`, which gets up indirectly being imported whenever you run any `mach` command, which will cause `bootstrap` to break before it even has the chance to install it.
`python3-pip` seems to rely on `python3-distutils` being installed so `bootstrap` will install it, so all we need to do is make sure it doesn't import `virtualenv` in any circumstance unless it's necessary (when surfacing an error in the module would be appropriate).
Differential Revision: https://phabricator.services.mozilla.com/D75833
Create a new MachCommandCondition, "is_firefox_or_thunderbird" which is then used
to allow mach valgrind-test work for Thunderbird builds.
Differential Revision: https://phabricator.services.mozilla.com/D73153
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).
The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.
Differential Revision: https://phabricator.services.mozilla.com/D75463
In bug 1637665, we changed the OS X codepath for `mach run` to consult
`MOZ_WIDGET_TOOLKIT` as a substitute for "browserness". Unfortunately,
this caused tests that only set `MOZ_MACBUNDLE_NAME` to fail. Rather
than set `MOZ_WIDGET_TOOLKIT` in those tests, we probably should have
been checking `MOZ_MACBUNDLE_NAME` in the first place, since
`MOZ_MACBUNDLE_NAME` is also set only for the browser and not for the JS
shell. We can also remove the xulrunner codepath, as xulrunner was
removed from the tree several years ago.
Differential Revision: https://phabricator.services.mozilla.com/D75315
We need to skip the xulrunner/browser code (do we even support xulrunner
anymore?) for `--enable-application=js` builds. `MOZ_WIDGET_TOOLKIT` is
used elsewhere to check for "browserness".
Differential Revision: https://phabricator.services.mozilla.com/D75140
The official decision appears to be that we want people to ask questions in the build channel on chat.mozilla.org for queries that require build peer approval, as opposed to emailing specific people directly, filing bugs, etc. Rather than the vague "consult a build peer" suggestion currently in the code, specify exactly what we expect people to do in mozbuild.
Differential Revision: https://phabricator.services.mozilla.com/D74963
The current error message leaves you with basically no recourse besides filing a bug if you're already at the latest HEAD. Meanwhile, `mach clobber` will fix it but in doing so you're taking a very blunt sledgehammer to the problem. Instead, I've updated this error message to tell you you can `mach clobber python`. I also removed the explicit reference to "artifact builds" because you can encounter this error outside of artifact builds as well. Finally, I added another reminder that `mach bootstrap` and `mach artifact` don't work for old revisions of central because I keep getting bugs about it and more screaming about how it's unsupported can't hurt.
Differential Revision: https://phabricator.services.mozilla.com/D74732
|./mach try| subcommands are now compatible with both python 2 and 3.
Hand-tested with many combinations of subcommand and subcommand flags.
Updates tryselect unit tests to use Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D73398
Python 3 doesn't allow strings to be written to files opened in binary mode
(it requires a byte array in that case). As it happens, we should really be
opening these Eclipse config files in text mode since it seems on Windows the
files use Windows line ending characters. So rather than change the strings
to byte arrays, this patch simply changes the code to open the files in text
mode.
Differential Revision: https://phabricator.services.mozilla.com/D74318
When handling bug 1632429, I found some tests that worked on Python 2, but not Python 3.
They were marked accordingly as "expected failures". However, my system version of Python
is 3.8, while CI (and a non-trivial number of devs, probably) use 3.6.
Some of these tests marked as xfail were actually still working on versions of Python until 3.8.
The failure of this test was due to a change in default tarfile format. Explicitly setting this
format makes the tests pass in all relevant python versions.
Differential Revision: https://phabricator.services.mozilla.com/D74337
