We need to skip the xulrunner/browser code (do we even support xulrunner
anymore?) for `--enable-application=js` builds. `MOZ_WIDGET_TOOLKIT` is
used elsewhere to check for "browserness".
Differential Revision: https://phabricator.services.mozilla.com/D75140
The official decision appears to be that we want people to ask questions in the build channel on chat.mozilla.org for queries that require build peer approval, as opposed to emailing specific people directly, filing bugs, etc. Rather than the vague "consult a build peer" suggestion currently in the code, specify exactly what we expect people to do in mozbuild.
Differential Revision: https://phabricator.services.mozilla.com/D74963
The current error message leaves you with basically no recourse besides filing a bug if you're already at the latest HEAD. Meanwhile, `mach clobber` will fix it but in doing so you're taking a very blunt sledgehammer to the problem. Instead, I've updated this error message to tell you you can `mach clobber python`. I also removed the explicit reference to "artifact builds" because you can encounter this error outside of artifact builds as well. Finally, I added another reminder that `mach bootstrap` and `mach artifact` don't work for old revisions of central because I keep getting bugs about it and more screaming about how it's unsupported can't hurt.
Differential Revision: https://phabricator.services.mozilla.com/D74732
|./mach try| subcommands are now compatible with both python 2 and 3.
Hand-tested with many combinations of subcommand and subcommand flags.
Updates tryselect unit tests to use Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D73398
Python 3 doesn't allow strings to be written to files opened in binary mode
(it requires a byte array in that case). As it happens, we should really be
opening these Eclipse config files in text mode since it seems on Windows the
files use Windows line ending characters. So rather than change the strings
to byte arrays, this patch simply changes the code to open the files in text
mode.
Differential Revision: https://phabricator.services.mozilla.com/D74318
When handling bug 1632429, I found some tests that worked on Python 2, but not Python 3.
They were marked accordingly as "expected failures". However, my system version of Python
is 3.8, while CI (and a non-trivial number of devs, probably) use 3.6.
Some of these tests marked as xfail were actually still working on versions of Python until 3.8.
The failure of this test was due to a change in default tarfile format. Explicitly setting this
format makes the tests pass in all relevant python versions.
Differential Revision: https://phabricator.services.mozilla.com/D74337
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)
In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:
```
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```
In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.
That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
Differential Revision: https://phabricator.services.mozilla.com/D73484
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.). Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.
Differential Revision: https://phabricator.services.mozilla.com/D73747
Combined with the upcoming upgrade of the build environment to Debian 8,
which is also going to upgrade the Gtk+3 requirement to 3.14, of the
major Linux distros Firefox currently supports running on, this removes
runtime support for:
- Fedora 20 and earlier (EOLed in 2015),
- OpenSUSE 13.1 and earlier (EOLed in 2016),
- Debian 7 (EOLed in 2018),
- Ubuntu 14.04 (EOLed in 2019).
Some of them might actually be supported in practice because the Gtk+3
requirement might be more relaxed than 3.14, but figuring that out is not
worth the effort.
Differential Revision: https://phabricator.services.mozilla.com/D73783
We currently check that the binaries we ship are not using symbol
versions of system libraries that would not be available on some older
systems. In some cases, however, we may get dependencies on newer symbol
versions in the form of weak symbols, that are checked for at runtime.
This happens with __cxa_thread_atexit_impl when building against a glibc
newer than 2.18, and the supporting code in Rust libstd actually checks
at runtime whether the weak symbol is resolved before using it.
Differential Revision: https://phabricator.services.mozilla.com/D73782
We currently generate absolute paths in all of our XPIDL-generated
source files, which is not so great for several reasons (deterministic
generation of files across machines, Searchfox analysis logic, shared
compilation caches, etc.). Let's generate paths that still indicate
where you should be looking, but are identical across compilations,
objdirs, etc.
Differential Revision: https://phabricator.services.mozilla.com/D73747
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.
Also delete `mach analyze` which depends on the existence of a Tup backend.
Differential Revision: https://phabricator.services.mozilla.com/D73911
It's not maintained and doesn't work as-is. If we want to revive it we can grab it from source control later, but in the meantime it's just confusing and it comes up in search results even though we never look for it.
Also delete `mach analyze` which depends on the existence of a Tup backend.
Differential Revision: https://phabricator.services.mozilla.com/D73911
This warning dates from bug 910487, which was 7 years ago. Since joining Mozilla I have *always* gotten this warning, and as far as I can tell since I never had a pre-2019 version of Visual Studio on my dev machine, the VS90COMNTOOLS variable was *never* set. Moreover, the "hack" is written in such a way that it does nothing *unless* you have `VS{100,110,120}COMNTOOLS` set, which I never have on my machine either, as you might expect since I only have the one version of Visual Studio installed.
The [latest public build documentation](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites) recommends that you install the Community edition of Visual Studio 2019, and as of 2019 the variable that's being used is `VS160COMNTOOLS`, so the only way someone would get value out of this hack is if they're using a substantially older version of Visual Studio than we recommend anyway.
Since 1) I *suspect* the hack is not doing anything for the large majority, if not all, of the people currently running builds on Windows on a day-to-day basis and 2) even if the hack continues to do something useful under some hypothetical scenarios, the content of the hack as well as the corresponding warning is so outdated that it should be updated anyway, I propose deleting it entirely.
Differential Revision: https://phabricator.services.mozilla.com/D72925
pip-compile can annotate each requirement with a reason why it is included (e.g.: transitive dependency, or depended-on directly).
When annotating direct dependencies, it notes it as "via -r <direct path to requirements.in file>".
Since we were using a temporary directory, the path of the directory was being included, making the output non-deterministic.
This change ensures that we run pip-compile in the same working directory as the temporary requirements file, enabling
the annotations to be deterministic: "via -r requirements-mach-vendor-python.in".
Differential Revision: https://phabricator.services.mozilla.com/D72181
`os.path.realpath` in Python 3.8 now always uppercases Windows drive letter, while it was just an alias of `os.path.abspath` in Windows. This patch uses `.realpath()` consistently to get `topobjdir` to fix the incompatibility from the behavior change.
Differential Revision: https://phabricator.services.mozilla.com/D72188
Split out jsreftest and jittest files into their own packages, removing them
from the common package.
This speeds up extracting files from the common test archive for
non-jsreftest/jittest suites.
Also, remove some files from the web-platform test archive that are already
present in the common archive.
Differential Revision: https://phabricator.services.mozilla.com/D72192
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.
This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.
This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly
Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS
Differential Revision: https://phabricator.services.mozilla.com/D72076
As we migrate from makefiles to moz.build, we want to warn developers if
variables only used by moz.build are accidentally defined in a makefile.
This was the purpose of `MOZBUILD_VARIABLES` in recursivemake.py, though it became out-of-date.
This patch defines `MOZBUILD_VARIABLES` off the official list from `mozbuild.frontend.context.VARIABLES`, and
removes unused (?) code from makefiles accordingly
Note that the following variables use to be in `MOZBUILD_VARIABLES`, but aren't there any more because
they aren't in `mozbuild.frontend.context.VARIABLES`:
* CMSRCS
* CMMSRCS
* EXTRA_JS_MODULES
* EXTRA_PP_COMPONENTS
* EXTRA_PP_JS_MODULES
* HOST_CSRCS
* HOST_CMMSRCS
* HOST_EXTRA_LIBS
* JAVA_JAR_TARGETS
* LIBS
* MAKE_FRAMEWORK
* MODULE
* NO_DIST_INSTALL
* NO_INTERFACES_MANIFEST
* PARALLEL_DIRS
* PREF_JS_EXPORTS
* RESOURCE_FILES
* SHARED_LIBRARY_LIBS
* TEST_DIRS
* TOOL_DIRS
* XPCSHELL_TESTS
Differential Revision: https://phabricator.services.mozilla.com/D72076
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.
The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.
Differential Revision: https://phabricator.services.mozilla.com/D71456
