The module `distutils.sysconfig` isn't installed by default on Ubuntu -- the package `python3-distutils` exposes it. That's fine, except we unconditionally import it in `virtualenv.py`, which gets up indirectly being imported whenever you run any `mach` command, which will cause `bootstrap` to break before it even has the chance to install it.
`python3-pip` seems to rely on `python3-distutils` being installed so `bootstrap` will install it, so all we need to do is make sure it doesn't import `virtualenv` in any circumstance unless it's necessary (when surfacing an error in the module would be appropriate).
Differential Revision: https://phabricator.services.mozilla.com/D75833
Create a new MachCommandCondition, "is_firefox_or_thunderbird" which is then used
to allow mach valgrind-test work for Thunderbird builds.
Differential Revision: https://phabricator.services.mozilla.com/D73153
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).
The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.
Differential Revision: https://phabricator.services.mozilla.com/D75463
In bug 1637665, we changed the OS X codepath for `mach run` to consult
`MOZ_WIDGET_TOOLKIT` as a substitute for "browserness". Unfortunately,
this caused tests that only set `MOZ_MACBUNDLE_NAME` to fail. Rather
than set `MOZ_WIDGET_TOOLKIT` in those tests, we probably should have
been checking `MOZ_MACBUNDLE_NAME` in the first place, since
`MOZ_MACBUNDLE_NAME` is also set only for the browser and not for the JS
shell. We can also remove the xulrunner codepath, as xulrunner was
removed from the tree several years ago.
Differential Revision: https://phabricator.services.mozilla.com/D75315
We need to skip the xulrunner/browser code (do we even support xulrunner
anymore?) for `--enable-application=js` builds. `MOZ_WIDGET_TOOLKIT` is
used elsewhere to check for "browserness".
Differential Revision: https://phabricator.services.mozilla.com/D75140
The official decision appears to be that we want people to ask questions in the build channel on chat.mozilla.org for queries that require build peer approval, as opposed to emailing specific people directly, filing bugs, etc. Rather than the vague "consult a build peer" suggestion currently in the code, specify exactly what we expect people to do in mozbuild.
Differential Revision: https://phabricator.services.mozilla.com/D74963
The current error message leaves you with basically no recourse besides filing a bug if you're already at the latest HEAD. Meanwhile, `mach clobber` will fix it but in doing so you're taking a very blunt sledgehammer to the problem. Instead, I've updated this error message to tell you you can `mach clobber python`. I also removed the explicit reference to "artifact builds" because you can encounter this error outside of artifact builds as well. Finally, I added another reminder that `mach bootstrap` and `mach artifact` don't work for old revisions of central because I keep getting bugs about it and more screaming about how it's unsupported can't hurt.
Differential Revision: https://phabricator.services.mozilla.com/D74732
|./mach try| subcommands are now compatible with both python 2 and 3.
Hand-tested with many combinations of subcommand and subcommand flags.
Updates tryselect unit tests to use Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D73398
Python 3 doesn't allow strings to be written to files opened in binary mode
(it requires a byte array in that case). As it happens, we should really be
opening these Eclipse config files in text mode since it seems on Windows the
files use Windows line ending characters. So rather than change the strings
to byte arrays, this patch simply changes the code to open the files in text
mode.
Differential Revision: https://phabricator.services.mozilla.com/D74318
When handling bug 1632429, I found some tests that worked on Python 2, but not Python 3.
They were marked accordingly as "expected failures". However, my system version of Python
is 3.8, while CI (and a non-trivial number of devs, probably) use 3.6.
Some of these tests marked as xfail were actually still working on versions of Python until 3.8.
The failure of this test was due to a change in default tarfile format. Explicitly setting this
format makes the tests pass in all relevant python versions.
Differential Revision: https://phabricator.services.mozilla.com/D74337
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)
In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:
```
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```
In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.
That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
Differential Revision: https://phabricator.services.mozilla.com/D73484
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.). Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.
Differential Revision: https://phabricator.services.mozilla.com/D73747
Combined with the upcoming upgrade of the build environment to Debian 8,
which is also going to upgrade the Gtk+3 requirement to 3.14, of the
major Linux distros Firefox currently supports running on, this removes
runtime support for:
- Fedora 20 and earlier (EOLed in 2015),
- OpenSUSE 13.1 and earlier (EOLed in 2016),
- Debian 7 (EOLed in 2018),
- Ubuntu 14.04 (EOLed in 2019).
Some of them might actually be supported in practice because the Gtk+3
requirement might be more relaxed than 3.14, but figuring that out is not
worth the effort.
Differential Revision: https://phabricator.services.mozilla.com/D73783
We currently check that the binaries we ship are not using symbol
versions of system libraries that would not be available on some older
systems. In some cases, however, we may get dependencies on newer symbol
versions in the form of weak symbols, that are checked for at runtime.
This happens with __cxa_thread_atexit_impl when building against a glibc
newer than 2.18, and the supporting code in Rust libstd actually checks
at runtime whether the weak symbol is resolved before using it.
Differential Revision: https://phabricator.services.mozilla.com/D73782
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.). Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.
Differential Revision: https://phabricator.services.mozilla.com/D73747
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.
Also delete `mach analyze` which depends on the existence of a Tup backend.
Differential Revision: https://phabricator.services.mozilla.com/D73911
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.
Also delete `mach analyze` which depends on the existence of a Tup backend.
Differential Revision: https://phabricator.services.mozilla.com/D73911
