Configure now detects VCS info. Configure now detects Watchman.
We can combine the two so configure can detect if Mercurial
is configured with Watchman enabled.
This commit does two things:
1) collects the Mercurial config so it is available to downstream checks
2) examines the config for presence and state of the fsmonitor
extension
We don't yet do anything with the fsmonitor state. But it should be
useful soon. Also, the return value is kinda wonky. This will almost
certainly be improved as soon as there is an actual consumer.
MozReview-Commit-ID: HyHZ2X8VI0h
--HG--
extra : rebase_source : d245d316cc8a27b2827b7824204549b91465bd34
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.
The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.
MozReview-Commit-ID: Jbugvf0lioM
--HG--
extra : rebase_source : 38b4af7b668830589126e8a83f5d5ab73914d07c
In bug 1181040, we added ${var}_IS_SET variables for
mk_add_options-defined variables. In the two years since, that has never
been used for anything else than MOZ_PGO_IS_SET, and the only use for
that has now been removed, so remove those ${var}_IS_SET variables.
--HG--
extra : rebase_source : 2fc9abe0c3badbf06f3858fcf326237e67891fee
Note this will only enable it on try, autoland and mozilla-inbound,
which are the only branches where sccache are enabled at the moment.
Enabling on more builds (or not) is the subject of bug 1373460.
Also note that bug 1181040, that ensured PGO builds weren't using
sccache mentions that back then, link times went up when using sccache
(with -Z7) vs. without, but that was presumably with MSVC 2013. Try
suggests link times are the same using sccache now (still using -Z7,
pending bug 1318370).
--HG--
extra : rebase_source : 9f9d87714f3c841b790eb7d692ea0968b1755b70
We want most builds to be actually using sccache, so we include
mozconfig.cache from mozconfig.common. However, since the --with-ccache
configure option doesn't exist on non-compile jobs (e.g. artifact
builds), we move to using the CCACHE environment variable instead, which
allows us to unset it in mozconfig.no-compile.
And since mozconfig.no-compile is always included where no_sccache is
set, we can remove that variable.
--HG--
extra : rebase_source : a8c743de1fd7a3c0fbc53f7c233df36585897767
Some gaia-related code was also found and removed as part of the
cleanup.
MozReview-Commit-ID: DEjVSljzzu1
--HG--
extra : rebase_source : 58c4c34df44a258d90029853f29ea01338bd142b
Bug 1365859 introduced a dependency on the Visual Studio binary 'fxc'
to generate Shader bytecode. This was unavailable when compiling for
Windows on Linux as part of a MinGW build.
This commit adds a configure check for fxc, and also searches for
fxc2, which was written (https://github.com/tomrittervg/fxc2) to be
a tiny application that wraps D3DCompileFromFile and can produce
similar (but not exactly the same) output as fxc.
fxc2 is compiled using MinGW for Windows, and runs under wine, so
we need to check for wine also.
Finally, fxc outputs some include information fxc2 doesn't, so
we will just change that assert to not take effect.
MozReview-Commit-ID: 8LVxuODi6cV
--HG--
extra : rebase_source : 9116d266663284d6594e34aa53bd37eae01ba67f
The |mach try| implementation isn't really testing specific so figured while doing this refactor
anyway we could take the opportunity to move it to /tools. This renames the "autotry" module to
"syntax" as one of the 'selectors' under the newly created tryselect module.
MozReview-Commit-ID: DmnGU5sKuCk
--HG--
rename : testing/tools/autotry/autotry.py => tools/tryselect/selectors/syntax.py
extra : rebase_source : 52f2a61d97107b8d50e3894ea0f5cd9c24d4108d
The values that we need to find in the registry can be inconsistent across
different installations, so we retrieve values from both views in our search
for a valid SDK. This also ensures this works for 32-bit and 64-bit python.
Mercurial installs on Windows often provide both a native hg.exe
and a hg shell script. This allows them to work from both Windows
and UNIX-like shells.
While the which Python package searches for .exe and other common
executable extensions on Windows automatically, it doesn't prefer
these results to an extension-less value. This meant that "hg"
was resolved to the shell script. When we went to run it, it
failed because it isn't a valid Win32 application.
We work around the problem by preferring "hg.exe" over "hg"
for the binary name.
Ideally this would be platform specific. But the harm should be
minimal.
--HG--
extra : rebase_source : 1354c739de1ff738812ad100176ad7e9d08fadde
This is now required as of 00ef8018730c.
Landing on a CLOSED TREE.
--HG--
extra : rebase_source : 410e4a677cdc1f1238f778a00078ac6e6de420bd
extra : amend_source : e81cf510948bedd76ac77790eb96aae1fe3cda1c
For reasons unknown to me, Windows CI is periodically failing to find
the Mercurial binary.
In addition, we've also reimplemented various VCS logic throughout
the build system. There is room to cut down on code complexity by
e.g. recording VCS info in configure instead of determining it
at run-time.
Also, for forensic purposes it is sometimes desirable to know which
VCS tool is in use by a build and which version of that tool is being
used.
This commit adds VCS type detection, binary searching, and version
resolution to configure.
substs now contains VCS_CHECKOUT_TYPE, HG, and GIT, which can be
consulted by downstream consumers.
If the Mercurial or Git versions could not be resolved, all variables
are not set. Otherwise, VCS_CHECKOUT_TYPE and one of HG or GIT is set.
If MOZ_AUTOMATION is set, we require that the VCS info be resolved.
This helps prevents weirdness in automation due to unexpected
environment configuration.
MozReview-Commit-ID: AMLy0Hfx5rD
--HG--
extra : rebase_source : edef9165d32dc47308a14b0fbabce3c1d3d28176
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
developer_options (!--enable-release) implies to search for and prefer
the gold linker. As part of porting gold detection to moz.configure
in d0e782180741 (bug 1351109), the logic inadvertently changed to
require gold when either --enable-gold or !--enable-release were
present.
This commit relaxes the requirement to find gold when only --enable-gold
is true.
MozReview-Commit-ID: HTBicWNUkvy
--HG--
extra : rebase_source : dd3938a7914f5db6c315fb775e7cc5ea177bf600
check_prog, when used with a `when` argument, doesn't work the same way
as putting it under a `with only_when()` block, while it should. The
difference comes from the fact that `with only_when()` applies the
`when` to every option and depends used in the block (which check_prog
calls a bunch of).
So, we "manually" apply the `when` to all option and depends in
check_prog. An alternative solution would be to put the whole function
under a `with only_when()` block, but that would mean reindenting the
whole function.
Either way, as a consequence, this requires the `when` to have a
dependency on --help for "non-trivial" functions, which fortunately,
there's only one of.
--HG--
extra : rebase_source : d91eca9e303c7062394d92a526983714ef3e612f
LLVM_CONFIG, per the contents of toolkit/moz.configure, is tied to
--enable-stylo, but it currently is set on all types of builds. It
currently happens to work, but it's actually not meant to, and sure
enough, the fix for bug 1374727 exacerbates that.
So we create a new mozconfig.stylo file that enables stylo and sets
LLVM_CONFIG, such that only build types that do enable stylo have
LLVM_CONFIG set.
--HG--
extra : rebase_source : 01277a79951888046c0b8e29c61cfc3b049ee0f0
For reasons unknown to me, Windows CI is periodically failing to find
the Mercurial binary.
In addition, we've also reimplemented various VCS logic throughout
the build system. There is room to cut down on code complexity by
e.g. recording VCS info in configure instead of determining it
at run-time.
Also, for forensic purposes it is sometimes desirable to know which
VCS tool is in use by a build and which version of that tool is being
used.
This commit adds VCS type detection, binary searching, and version
resolution to configure.
substs now contains VCS_CHECKOUT_TYPE, HG, and GIT, which can be
consulted by downstream consumers.
If the Mercurial or Git versions could not be resolved, all variables
are not set. Otherwise, VCS_CHECKOUT_TYPE and one of HG or GIT is set.
If MOZ_AUTOMATION is set, we require that the VCS info be resolved.
This helps prevents weirdness in automation due to unexpected
environment configuration.
MozReview-Commit-ID: AMLy0Hfx5rD
--HG--
extra : rebase_source : edef9165d32dc47308a14b0fbabce3c1d3d28176
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
This enables apm logging by setting the apm_debug_dump variable in gyp.mozbuild.
Prior to this change, some files were including apm_data_dumper.h with logging
enabled and some were not.
This also removes the AEC* C interface and calls into webrtc::Trace directly.
Whatever historical reasons for having a C interface into these calls no
longer seems to apply. In addition reserving a buffer for the base file name
and then ensuring it was null terminated caused an ASAN "stack-buffer-overflow"
while testing. This was because it was not handling an empty base file
name properly. This would not normally happen if AEC logging was enabled through
about:webrtc, but it still seems safer to use std::string.
MozReview-Commit-ID: Ikz5xO74syA
--HG--
extra : rebase_source : 8e0c59117135fadb75f4a7e6be5588af1404533d
When investigating why a particular build got a particularly high number
of cache misses, it is useful to know exactly *what* had cache misses,
and you can't get the missing information after the fact.
We used to have some stats stored in a log file uploaded as artifact
with the old python sccache per bug 1005495 (and before that inline in
the build log), and we actually still have remains of that in the form
of a EXTRA_UPLOAD_FILES in build/mozconfig.cache.
Let's reuse that, and get some useful logs from sccache. Ideally sccache
would give better logs, but those are sufficient for now. We'll figure
things out eventually. https://github.com/mozilla/sccache/issues/151
--HG--
extra : rebase_source : a2e7093697a5b720ab28f155560211d6c0a4347e
Rustc 1.20 gained support for make jobserver. But when rustc is invoked
through sccache, sccache needs to have inherited the make jobserver file
descriptor to pass it down to rustc. But what will be running sccache
first during the build is not really deterministic. It might as well end
up being a script invoking the preprocessor that ends up doing it. And
we can't add the necessary + prefix to all commands that potentially can
end up invoking sccache.
So we manually start the sccache server at the beginning of the build,
through preflight.
Note this works on automation, but not for local builds, but few people
are using sccache locally for rust (yet). Let's figure that out in a
followup.
--HG--
extra : rebase_source : eda5853d7247be247a1bec1a5d7e84b3846fd2c1
This patch does two things:
- add a Gradle-only ANDROID_COMPILE_SDK_VERSION substitution;
- uses it while uniformizing all of the Gradle Android SDK version
configurations.
The approach is fairly standard (and we were using it already); see,
for example
https://medium.com/@ali.muzaffar/gradle-configure-variables-for-all-android-project-modules-in-one-place-5a6e56cd384e
This will make bumping the Gradle configuration versions forward
easier.
MozReview-Commit-ID: 1j5siCvR5qt
--HG--
extra : rebase_source : 07afb00de0e4a72af4026eb19ff4f2530c119336
This is just a small piece in the larger project of minimizing the set
of global AC_DEFINEs. None of these are relevant to C/C++ code;
they're mobile/android specific, so we should get them out of
mozilla-config.h.
In addition, this replaces hard-coded values with values from the
build system throughout mobile/android.
MozReview-Commit-ID: 9IJlsm38LFK
--HG--
extra : rebase_source : 0e55c0967cbb7f0df0d297eb330fc615e5759346
Enable stylo in Firefox Nightly builds for macOS so we can
get testing feedback until we have addressed issues blocking
enabling it for developer builds.
MozReview-Commit-ID: 3y0QT4oiBpt
--HG--
extra : rebase_source : 531162da334a6d124dcd7328f45d677cb4933551
When cross-compiling, rust-bindgen needs the -isysroot
flag we pass to the C++ compiler to find the correct
headers. Add a new BINDGEN_CFLAGS environment variable
for passing this and other relevant options, and reformat
its contents in toolchain.configure so we can use autoconf-
style template substitution to poke it into a bindgen.toml
file to be read by build scripts like build_gecko.rs.
Set this variable from the macosx/cross-mozconfig.common
to the same extra flags we pass to CXX so automation
builds work correctly with --enable-stylo.
MozReview-Commit-ID: 7wabObiFtVb
--HG--
extra : rebase_source : eeba30e3d64112da65e2e6830ef5fc1b54965529
The PT_PHDR segment is optional, but the Android toolchain decides to
create one in some cases, and places it first. When that happens, the
work around for bug 1233963 fails, because the fake phdr section has not
been adjusted yet (it only happens when we see a PT_LOAD).
So we adjust the fake phdr section when we see a PT_PHDR segment (and
avoid re-updating it when we see a subsequent PT_LOAD).
--HG--
extra : rebase_source : 2190ec2f20ba9d144b8828874f9f8d70dd5ad2f6
Somehow, with the Android toolchain, we end up with
non-empty-but-really-empty DT_INIT_ARRAYs.
In practical terms, they are arrays with no relocations, and content
that is meaningless:
$ objdump -s -j .init_array libnss3.so
libnss3.so: file format elf32-little
Contents of section .init_array:
1086e0 00000000 ....
$ readelf -r libnss3.so | grep 1086e0
$ objdump -s -j .init_array libplugin-container-pie.so
libplugin-container-pie.so: file format elf32-little
Contents of section .init_array:
4479c ffffffff 00000000 ffffffff 00000000 ................
$ readelf -r libplugin-container-pie.so | grep 4479c
Because so far, elfhack expected meaningful DT_INIT_ARRAYs, it bailed out
early in that case.
--HG--
extra : rebase_source : 217aacb42fdfabb466ed1f8149dfaeb4a595eda8