We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
We only enable libstdc++ compat on Android for the host parts, and
in practice, the target part has it not enabled at the moment because
libstdc++ can't be found. But making the clang toolchain capable of
building for x86 changes the deal for Android x86.
We need to add mappings for target.cpu to MSVC's arm64 name for
determining various paths, and we need to add an extra case to
get_vc_paths so the compiler can find all the necessary DLLs.
This patch adds JaCoCo as a dependency for the geckoview androidTest configurations, as well as
the `mach android archive-geckoview-coverage-artifacts` command, and the `--enable-java-coverage`
mozconfig flag.
MozReview-Commit-ID: 36jNAzK44g3
--HG--
extra : rebase_source : 9edc37913a3929ad045270c601c77791d122e363
JS, at least, doesn't need bindgen at this point. If JS does start
requiring bindgen under certain build configurations, we'll have to
figure out what to do about the bindgen dependency at that point:
require bindgen always, or just require bindgen for the
configuration(s).
Currently, --enable-lto just implies --enable-linker=lld, which
essentially only works on Linux, and assumes one can't do lto with
anything other than lld. Which is not true. As a matter of fact, even
ld.bfd can do lto, as long as the gold plugin for llvm is available,
or when doing lto with GCC instead of clang.
Anyways, to allow more lto setups, we adapt the --enable-linker setup
to:
- work on macOS, which it currently doesn't, and add support for the mac
linker (ld64), which, unfortunately, doesn't have a clean way to be
detected, so work around that.
- default to lld if lto is enable, no linker was explicitly given, the
compiler is clang *and* the build target is not macOS.
--HG--
extra : rebase_source : 1dab2ad6230d18e7f4285943e76352e23b987d4e
Because we have no linking configure test in python configure (yet), we
just make old-configure tests use LINKER_LDFLAGS, and make those flags
added to LDFLAGS by old-configure at the same time.
--HG--
extra : rebase_source : 80ab7c5021c7ddd1b53d58ef76cd4372a524d3cb
The intent is for the build system to soon require Node.js to build
Firefox. But we aren't ready to make Node.js a build requirement
just yet.
The goal of this commit is to implement configure detection for
Node.js so that we can a) work out detection bugs b) give people a
means to validate system compatibility *before* we throw the switch to
require Node.js.
This commit introduces configure logic for finding a Node.js
executable, resolving its version, and validating its suitability.
By default, if Node.js cannot be found or there is an error resolving
its version, we print some warning messages and move on.
If --enable-nodejs is used (not the default), errors are raised
if Node.js cannot be found or its version isn't suitable.
Once we require Node.js, the added code can likely be simplified.
When writing the code, I went out of my way to make failures as
non-fatal as possible. e.g. normally we'd say that failures to run
`node --version` would be fatal. I'm purposefully trying to not have
this configure check break anyone's environment, even if failure
occurs. Again, the goal is to introduce the configure checks first
in a non-fatal way such that we can debug failures so the flag day
transition is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D1818
--HG--
extra : moz-landing-system : lando
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : a17aa496bf3d4af4d1349d69a637c686c6817d0f
When I originally implemented bug 1458161, this is how it was done, but
it was suggested to use a configure-time check. This turned out to not
be great, because the rust compiler changes regularly, and we don't run
the configure tests when the version changes. When people upgraded their
rust compiler to 1.27, the code subsequently failed to build because the
features were still set for the previous version they had installed.
--HG--
extra : rebase_source : 1b5f7a02ad8495d68cd29289f7beea59b8912183
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : c788ef4f7da9949b81df2f0577af6f6039ea63d8
When we're running using pipenv, we have more than one virtual environment. This means the current check to see if python matches the initial virtual environment gives a false positive when we're in a secondary virtual environment. This patch changes the condition to check if the current python path exists within the virtual environments root directory.
MozReview-Commit-ID: AAONwLWsigL
--HG--
extra : rebase_source : c0ac94448ee4545417b5116e58b51c6187cdb175
We used to do that before bug 1455767. Taking a step back, what we want
(and what this change implements) is to:
- allow to specify a linker to use explicitly
- check that using the corresponding linker flags makes us use that
linker
- if no linker is specified, and developer options are enabled, we want
to try to use gold if it's available, otherwise, we want to detect
what kind of linker the default linker is
- if --disable-gold is explicitly given, we don't want to automatically
use it.
--HG--
extra : rebase_source : 8d89c54bd5e555984d815beb8fdd3f5f75dae31e
We used to do that before bug 1455767. Taking a step back, what we want
(and what this change implements) is to:
- allow to specify a linker to use explicitly
- check that using the corresponding linker flags makes us use that
linker
- if no linker is specified, and developer options are enabled, we want
to try to use gold if it's available, otherwise, we want to detect
what kind of linker the default linker is
--HG--
extra : rebase_source : 655089f29c6403f6c31a977dcf5c6fd1e9941121
--enable-release not being passed means developer options are enabled,
which is generally speaking not desirable for builds meant to be
shipped. This is somewhat alleviated for Firefox by MOZILLA_OFFICIAL
implying --enable-release (as well as MOZ_AUTOMATION), but that doesn't
apply to e.g. standalone js builds (even some of the standalone js jobs
on our automation don't set MOZ_AUTOMATION for some reason).
A reasonable thing to do is just to default builds for release/beta
milestones to --enable-release, but still allow --disable-release
to enable the developer options.
--HG--
extra : rebase_source : 770d51b10a9cd17c63972435bb61eed10345ea71
Can be used:
ac_add_options --enable-linker=lld-7
or
ac_add_options --enable-linker=lld
MozReview-Commit-ID: GfDevGeooU4
--HG--
extra : rebase_source : ee2aad5f8f721a6fe5840affde6b29a3b940fb91
This option does nothing if it's provided, and things will probably
break in some most peculiar manner if somebody uses --without-pthreads.
Given that neither outcome is useful, we should remove the option.
OOM rust crashes are currently not identified as such in crash reports
because rust libstd handles the OOMs and panics itself.
There are unstable ways to hook into this, which unfortunately are under
active changes in rust 1.27, but we're currently on 1.24 and 1.27 is not
released yet. The APIs didn't change between 1.24 and 1.26, so it's
fine-ish to use them as long as we limit their use to those versions.
As long as the Firefox versions we ship (as opposed to downstream) use
the "right" version of rust, we're good to go.
The APIs are in their phase of stabilization, so there shouldn't be too
many variants of the code to support.
--HG--
extra : rebase_source : 08a85aa102b24380b1f6764effffcc909ef3191b
We previously did not require Python 3.5+ everywhere because we failed
to detect Python 3.5 on Windows in CI. That's because CI isn't using
start-shell.bat and it hasn't yet updated PATH to include
%MOZILLABUILD%\python3.
We shouldn't need to teach CI to have PATH contain everything in
MozillaBuild. This commit teaches moz.configure to automatically use
MozillaBuild's Python 3 if we're running in MozillaBuild.
Since we can now detect Python 3 everywhere in CI, we make Python 3.5+
required on all build configurations.
MozReview-Commit-ID: BwgWGeYMyPM
--HG--
extra : rebase_source : b52f604b01c73b0493b142f3f71aa26c99a42b91
But only if we are:
a) not running in CI
b) running in CI on Linux
We will ideally make the requirement global. But Python 3.5 is not
yet available in CI on macOS. And we're not finding the MozillaBuild
copy in configure.
This was previously announced in November at
https://groups.google.com/d/msg/mozilla.dev.platform/rJrPh1QYXrQ/hqRrQsJ_BgAJ.
MozReview-Commit-ID: IyPCAcL3gop
--HG--
extra : rebase_source : f9e3db043a1ce9c1a903c943663f22245febf101
We want this on for "normal" developer builds, but we want to leave it
off when the build is for some kind of profiling, i.e. when the
optimization level is higher than "normal" builds.
This will tell us which functions PGO is spending its time in,
which functions are being optimized, and other interesting
statistics. The output is a handful of pages of text in our
current PGO builds and it seems useful for understanding the PGO
black box.
MozReview-Commit-ID: HUbAhltQrdL
--HG--
extra : rebase_source : eb188ebf84ee9ce6c7fe8bf69b6b156ee9f14956
Require the current stable Rust release so new features can
be used in development.
MozReview-Commit-ID: 4NQNk3RfBkF
--HG--
extra : rebase_source : 9d88e6fdb823bd2e2ca8ac9940b1fafd420eebdc
The Proguard dependency is now managed by Gradle.
MozReview-Commit-ID: EOvKSE5z28P
--HG--
extra : rebase_source : 760b117f500cc639cc8c24e9c02933990f358dd7
The heuristic simply fails for clang as shipped in Android NDKs: those
clang binaries can't target any non-Android host.
MozReview-Commit-ID: 6AhOJxE3boW
--HG--
extra : rebase_source : 3716d4ff7ce2f1e796489e02e312d7554a5b1e6c
The heuristic simply fails for clang as shipped in Android NDKs: those
clang binaries can't target any non-Android host.
MozReview-Commit-ID: 6AhOJxE3boW
--HG--
extra : rebase_source : 2c8b59c15a7a7e9803f5447fc8f41fc843b2e647
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
Android headers can have a <sys/quota.h> header but with its functions
ifdef'ed out. Check for that by checking for the presence of quotactl().
MozReview-Commit-ID: 26ILnbP7flO
--HG--
extra : rebase_source : fe78c114189a362cdad3d79ce67868b045328392
NDK headers are grouped into a "sysroot" directory, which doesn't
contain architecture-specific bits, and a "system" directory, which
contains only the architecture-specific bits. Previously, both
directories are the same, under platforms/android-*/arch-*/. However,
with unified headers in NDK r16, the two are different, so we need to
support that in the Android build scripts.
Unified headers also rely on the build system defining the
__ANDROID_API__ macro, so we add support for that as well.
MozReview-Commit-ID: 9zBNQC3BRFl
--HG--
extra : rebase_source : 9ded282343df64d9cc4abcf7d7c6b03ac3423ff0
Android headers can have a <sys/quota.h> header but with its functions
ifdef'ed out. Check for that by checking for the presence of quotactl().
MozReview-Commit-ID: 26ILnbP7flO
--HG--
extra : rebase_source : 147e4be9eb98326fd4946b7b3695c573c7108347
NDK headers are grouped into a "sysroot" directory, which doesn't
contain architecture-specific bits, and a "system" directory, which
contains only the architecture-specific bits. Previously, both
directories are the same, under platforms/android-*/arch-*/. However,
with unified headers in NDK r16, the two are different, so we need to
support that in the Android build scripts.
Unified headers also rely on the build system defining the
__ANDROID_API__ macro, so we add support for that as well.
MozReview-Commit-ID: 9zBNQC3BRFl
--HG--
extra : rebase_source : 36c9d4d5625fecbbe2485668811f85f9d94372d5
It's been more than two weeks since the 1.23 stable release, and
we're making official builds with that toolchain release, so begin
requiring that version so new language features can be used in
development.
MozReview-Commit-ID: E6WuP41ceTn
--HG--
extra : rebase_source : 75850dd9edbf8e3f9beab394e4af7fad76ce3b17
Will also address Bug 1377553 and part of Bug 1419607
MozReview-Commit-ID: AUCqBxEGpAl
--HG--
extra : rebase_source : 5547e2c8fbf4e2e87182b8720d8352c131e4ec65
We currently use a 32-bit Rust toolchain for win32 builds, but this can lead
to OOM situations. This patch makes win32 builds use a 64-bit Rust toolchain,
which requires a little bit of extra configuration because rustc needs to
be able to find a link.exe that produces 64-bit binaries for building
things like build scripts, which are host binaries.
We will now generate a batch file that sets LIB to the paths to 64-bit
libraries and invokes the x64-targeting link.exe, and add a section to the
.cargo/config file to instruct cargo to use that batch file as the linker
when producing 64-bit binaries.
MozReview-Commit-ID: 9vKBbm7Gvra
--HG--
extra : rebase_source : 273a99be71914167664482c2bdb26c840ec6867b
At the same time, we make it actually do something on spidermonkey
builds. We also add an environment variable alternative, that we use
in mozconfig.stdcxx, allowing for mozconfig.no-compile to override it
and avoid configure failures on e.g. artifact builds.
--HG--
extra : rebase_source : b68d362025e0c99f9184a03391c652ec2c9357ad
Bug 1325632 added some facility to add target compiler flags. This
change extends it to add allow adding host compiler flags as well.
--HG--
extra : rebase_source : 424b405a1d8f9a4778ff75c3308c9622f050e194
We currently use a 32-bit Rust toolchain for win32 builds, but this can lead
to OOM situations. This patch makes win32 builds use a 64-bit Rust toolchain,
which requires a little bit of extra configuration because rustc needs to
be able to find a link.exe that produces 64-bit binaries for building
things like build scripts, which are host binaries.
We will now generate a batch file that sets LIB to the paths to 64-bit
libraries and invokes the x64-targeting link.exe, and add a section to the
.cargo/config file to instruct cargo to use that batch file as the linker
when producing 64-bit binaries.
MozReview-Commit-ID: 9vKBbm7Gvra
--HG--
extra : rebase_source : 366dd966cafe4f07b8e59fc170d2db2dada32627
We need MOZ_UI_LOCALE even when building the JS shell so
config/config.mk variable assignments don't run into issues. But it
doesn't make any sense to configure a UI locale for the JS shell. So
make --enable-ui-locale a normal `option`, but give it a `default`,
which is the value shell-only builds will always see.
--HG--
extra : amend_source : 047759dd6ec446d9d6f8f5992ed9cf6628ce859e
This has the virtue of not executing python three times during configure
just to read the same value of milestone.txt and munge it. We can also
remove milestone.py as a happy side effect, so all the milestone
computations can be done in init.configure.
Bump the minimum required Rust version now that 1.22.1
has been in stable release for more than two weeks.
Version 1.22.0 works fine everywhere but macOS 10.13,
but 1.22.1 was released the same day, so it's no harder
to upgrade to.
Also update the base-toolchain builds to ensure we
continue to build with this version.
MozReview-Commit-ID: GlRWUNE07G0
--HG--
extra : rebase_source : f37585db5633e6e64b02bc94c2516b5ab18c3e15
Will also address Bug 1377553 and part of Bug 1419607
MozReview-Commit-ID: AUCqBxEGpAl
--HG--
extra : rebase_source : f7582d7089f0f4582a02aeaef090dc0701df994d
--enable-ion was only used by --enable-simulator and related options, so
there wasn't much point in making two separate commits.
This translation is a little more verbose than the original
old-configure code, but I think it is more readable and easier to
follow. We also don't port over --enable-simulator=no, as there doesn't
seem to be much point in doing so.
And statically link logalloc.
Statically linking is the default, except when building with
--enable-project=memory, allowing to use the generated libraries from
such builds with Firefox.
--HG--
extra : rebase_source : efe9edce8db6a6264703e0105c2192edc5ca8415
And statically link logalloc.
Statically linking is the default, except when building with
--enable-project=memory, allowing to use the generated libraries from
such builds with Firefox.
--HG--
extra : rebase_source : efe9edce8db6a6264703e0105c2192edc5ca8415
rustup installs rustc+cargo to ~/.cargo/bin by default, so make configure look
there to avoid the annoying case where someone installed rust and cargo
(possibly via `mach bootstrap`) but forgot to add ~/.cargo/bin to their PATH.
MozReview-Commit-ID: GZOcFdFmzA5
--HG--
extra : rebase_source : 9dae473b5804f096992cae7f90df4c87bb4e5af4
extra : source : 9324e5e56038a1e548b0fc4d94d9df445734ff1e
The first step of making --enable-jemalloc the default everywhere is to
at least allow to build with it everywhere. Which currently probably
fails on a few platforms, but they're not going to be fixed if they're
explicitly rejected at configure time.
--HG--
extra : rebase_source : f29383e2d73986f3e5b033ac82c0d520bacd4fa6
Hopefully, the bug we worked around by disabling jemalloc on 32-bit OSX
is gone. We're not shipping 32-bit binaries for OSX anyways.
--HG--
extra : rebase_source : 148a80a7ab006d3be81fb931cbbb4ad2c81690c3
The first step of making --enable-jemalloc the default everywhere is to
at least allow to build with it everywhere. Which currently probably
fails on a few platforms, but they're not going to be fixed if they're
explicitly rejected at configure time.
--HG--
extra : rebase_source : 0905861d5a2cc62f5b37c5ee811e56e1e063a133
Hopefully, the bug we worked around by disabling jemalloc on 32-bit OSX
is gone. We're not shipping 32-bit binaries for OSX anyways.
--HG--
extra : rebase_source : bfe765977dedf1949da4d5919032cadfb4675f1f
-Wc++14-compat warns about code whose meaning differs between C++11 and C++14. We want the new C++14 meanings, so we don't need to warn about these differences. We still want -Wc++1z-compat because we want to know about C++14 code whose meaning will change in C++17 or C++20.
MozReview-Commit-ID: 1CD11l2Fd86
--HG--
extra : rebase_source : 7bac029fd3e852fbb92f07e0358307c2c834ddc8
extra : source : 2d49767b136e420d39b88267f611fbe72ed0a3b8
We currently turn off the C++14 sized-deallocation facility on MSVC, and
we'd like to ensure we do the same thing for clang and gcc. To do so,
we add new functionality to moz.configure for checking and adding
compilation flags, similar to the facility for checking and adding
warning flags. The newly added facility is then used to add
-fno-sized-deallocation to the compilation flags, when the option is
supported.
Once we do this, we can't define the sized deallocation functions in
mozalloc.h; the compiler will complain that we are using
-fno-sized-deallocation, yet defining these special functions that we'll
never use. These functions were added for MinGW, where we needed to
compile with C++14 ahead of other platforms to be compatible with MSVC
headers. But they're no longer necessary, though they would be if we
removed -fno-sized-deallocation; the compiler will complain if we do
that and we'll add them back at that point.
We have code to test whether particular flags are supported for the
compiler we're using. Unfortunately, that code is tied up with checking
for warning flags. We're about to add a separate facility for generic
compilation flags, and we'd like to avoid cutting and pasting code if
possible. Let's split the core code out into a separate, reusable function.
Our toolchain detection logic checks whether we can reuse the target
C (resp. C++) compiler for the host compiler. This is generally only
applicable in the not-cross-compiling case, but we had special logic to
check for clang in the cross-compiling case and accept it, as clang is
able to generate code for multiple architectures from a single compiler
binary.
Our recent switch to clang on Android has exposed a problem in this
logic: we would never check whether the target clang, compiling for the
host, could actually find the host's headers. This was especially
problematic on OS X hosts, where the host clang contains special logic
to grovel inside the XCode installation to find C++ headers. The clang
from the NDK, however, was ignorant of the XCode installation.
Therefore, the NDK clang would happily compile code for the host, even
including C headers for the host, but would be hopelessly lost when it
came to compiling C++ headers during the actual build.
In hopes of mitigating this, we now include a check for a representative
header for C and C++ when checking compilers for each of those
languages. This check will detect such problems as the above, and will
also alert people to potentially misconfigured compilers in other
situations.
We need to modify our test framework to cope with headers being
included, since our mock environment isn't actually equipped with a full
set of compilers and headers.
Now that extra_toolchain_flags includes stlport_cppflags, there's no
reason bindgen_cflags_defaults should depend on them both. So remove
the stlport_cppflags dependency. (There's no harm to multiply-including
stlport_cppflags, but not repeating -I options is good practice.)
extra_toolchain_flags is used for compiling target-specific bits of code
elsewhere in configure. Very shortly, we'll need to compile bits of
code that depend on the C++ standard library, so we'll want to point the
compiler at the C++ standard library. Making extra_toolchain_flags
include stlport_cppflags is the best way to do that.
extra_toolchain_flags is going to depend on stlport_cppflags
momentarily, so this commit is just for moving code around. Note that
the diff shows other things moving *after* stlport_cppflags.
We compute the same set of data in extra_toolchain_flags as we were
computing in bindgen_cflags_defaults. We might as well reuse the former
to compute the latter.
The current code will fail if "RUSTC_OPT_LEVEL=" is passed. This can happen
if the value isn't present and that fact is injected into js' configure. We
only want to respect RUSTC_OPT_LEVEL if a value is passed, so we simply check
for the presence of a value rather than its origin.
MozReview-Commit-ID: 6GhLfprJEEn
--HG--
extra : rebase_source : 40f3e381a128e04d65cc0175df32cdcd8302e05e
The changesets in bug 1412932 changed the semantics for MOZ_PGO.
Before, it was effectively being set as an environment variable
by client.mk all the time. Afterwards - specifically after
2013c8dd1824 - the variable is set in mozconfigs via ac_add_options,
which means it is only exposed to configure, not the environment.
Investigation by dmajor revealed that -WX (warnings as errors) was
added to a js/src file's compiler invocation after the PGO code
refactor. (PGO and warnings as errors have a strange interaction
- bug 437002 - and should be disabled there.) Strangely, addition
of -WX was only present on Dev Edition PGO builds.
The reason for this is likely mozharness. Mozharness will export
the MOZ_PGO=1 environment variable for build configurations that
it knows are PGO. It appears to do this for all PGO build
configurations except Dev Edition. Since make and moz.configure
inherit environment variables, mozharness was basically papering
over the intended behavior change in 2013c8dd1824.
This commit fixes the problem by marking MOZ_PGO as a JS option
in moz.configure. This means `ac_add_options MOZ_PGO=1` (the new
convention for enabling PGO) will set MOZ_PGO for SpiderMonkey's
moz.configure.
Of course, MOZ_PGO=1 in an environment variable still works. And
mozharness's setting of this variable has the intended effect.
Eventually, I'd like to clean up the mozharness code so it is less
PGO aware and enables PGO via ac_add_options. But that's for another
day.
MozReview-Commit-ID: 1KYPJARI6SJ
--HG--
extra : amend_source : 5291cead9f1c1af9ed2a1f608af770bc8e4958c5
