While we're here, provide a reference to unique_list as defined in
moz.configure when executing config.data to avoid its redefinition
in m4.
MozReview-Commit-ID: AI6XhoYR0Ye
As the data in the bug shows, the current default of opt-level=2 is
several minutes slower at compiling than opt-level=1. This slows down
builds significantly and the added benefits of running opt-level=2
for local development can't be justified for the common/default case.
This commit changes the default for local builds from opt-level=2 to
opt-level=1.
--enable-release (what we use for builds shipped to users) will imply
opt-level=2. --enable-optimize (the default) will use opt-level=1,
and --disable-optimize will use opt-level=0.
The RUSTC_OPT_LEVEL environment variable in mozconfigs can be used
to set an explicit opt-level level, regardless of what other
configure options are set. This includes the other potential values,
"s" and "z."
A side-effect of this change is that -Copt-level is now *always*
specified by the build system. Before, it was only specified if
the value was adjusted to 0 for --disable-optimize builds.
MozReview-Commit-ID: 67KX5qScnFc
--HG--
extra : rebase_source : dac0134e952151992eee23e017e9a29f84b05172
extra : intermediate-source : c3a7cc11a987aedb81332f1a03cd082ab0ab0cb8
extra : source : 360827b8a5956d58f7f0200431d3a44c57ce8dc4
Before this commit, RUSTFLAGS was derived in rules.mk by consulting
various variables set by configure. It isn't clear to me why things
are implemented this way. We don't appear to have moz.build level
overrides for Rust compiler flags. So there doesn't appear to be a
compelling reason why we can't derive these values in configure.
So, this commit ports the code for deriving default RUSTFLAGS from
rules.mk to toolchain.configure.
The port is pretty straightforward as far as the logic goes.
MozReview-Commit-ID: JhAE9Qlo8SK
--HG--
extra : rebase_source : 6186cb81cd37c516b3d645419b9461bf501d6ba2
The Rust optimization logic is tied to --enable-optimize/MOZ_OPTIMIZE
and --enable-debug/MOZ_DEBUG. In order to more easily implement more
customization, let's move --enable-optimize/MOZ_OPTIMIZE to
moz.configure so its value can be consulted there.
The logic here is a bit wonky. The option behaves like a boolean
or a string. If a string, MOZ_OPTIMIZE is set to 2. Otherwise it
is 1 or unset depending on the boolean value.
The custom compiler flags string is passed to old-configure, where it
overwrites whatever old-configure derived as the default value.
We stop short of moving all references to MOZ_OPTIMIZE_FLAGS to
moz.configure because there are a handful of them and I don't want
to scope bloat.
MozReview-Commit-ID: 6iNDu2HwLGr
--HG--
extra : rebase_source : a64f1236012d13913f21253df1b9b5ff0ae8ea6e
We mix the added and modified variables from mozconfig and sort them.
We also print comments indicating where values come from.
MozReview-Commit-ID: 97x9iHxZe3m
--HG--
extra : rebase_source : 367bc410bc06532a91b488039e3cb0ec65850c09
Although there is emulator command in android_sdk_root/tools even if Android
Tools 26, it doesn't work well because emulator64-arm is missing in same
directory.
Valild emulator command moves from android_sdk_root/tools to
android_sdk_root/emulator from Android Tools 26 (updated by sdkmanager).
MozReview-Commit-ID: 4IGHK3YcZXD
--HG--
extra : rebase_source : cc0c67f24c32a55ac40e65a5a036d5072dff6ee4
extra : histedit_source : 9718746e56dd9ec16a540533acfe909c319c234b
For webrtc, the most important part of the code had already been removed in bug 1355048 and could no longer be called
MozReview-Commit-ID: Fx9XI0zR1gn
--HG--
extra : rebase_source : 360996760abab650684440fbeea258b43dccfd83
By the schedule we would have bumped the minimum-supported rust
version to 1.20.0 some weeks ago, just before Firefox 57 went to
beta. However, that was blocked on a couple of compatibility
issues with sccache and dsymutil. See bug 1396884.
Now that 1.20.0 is working in our automation builds, require
it for all builds to unblock servo and webrender upgrades.
Also update our integration test to verify building with
the new minimum.
MozReview-Commit-ID: 8otdix6f45f
--HG--
extra : rebase_source : 7d2550e8ddc772b45602bd488f174fc180e91484
Ideally, we'd backport it to the clang 4 toolchain too, but that results
in silently(!) missing crash symbols for libxul.
--HG--
extra : rebase_source : a3024abf720ea6e9dc0e3271a5aa2ebfae9bebee
When generating the pgo profile the timeout background budget depletes
for build/pgo/index.html, due to the profiling build actually being
slower than ordinary builds. This defeats the purpose of budget, so
turn off background throttling with budget when generating the profile.
Warn about duplicated conditions in if-else-if chains, which are unreachable code and likely copy/paste bugs. The -Wduplicated-cond flag is available in gcc 6 and later.
int example(int a)
{
if (a == 0)
a = 42;
else if (a == 0) // -Wduplicated-cond warning!
a = 43;
return a;
}
MozReview-Commit-ID: F9hqxMCct74
--HG--
extra : rebase_source : 71168f79ce8f03f650167cdedbbd2e5802846eab
Per schedule, bump the minimum-supported rust version to 1.21.0
two weeks after its stable release so we can use new code which
depends on it.
MozReview-Commit-ID: Bn8UjvTC7uw
Building Fennec/Android uses cross compiler toolchain. So we have to generate
clang options (include path for c++ headers and gcc headers, gcc-toolchain path and etc) from NDK path for bindgen.
The following options are required for android build.
(from stlport_cppflags)
-I$topsrcdir/android-ndk/sources/cxx-stl/llvm-libc++/libcxx/include
-I$topsrcdir/android-ndk/sources/android/support/include
-I$topsrcdir/android-ndk/sources/cxx-stl/llvm-libc++abi/libcxxabi/include"
(others for clang)
-isystem $topsrcdir/android-ndk/platforms/android-9/arch-arm/usr/include
-gcc-toolchain $topsrcdir/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64
-I$topsrcdir/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9/include
-I$topsrcdir/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9/include-fixed"
Also, since bindgen_cflags_defaults uses as default, some funcions requires '--help'.
MozReview-Commit-ID: 7zfhw3IxQ2W
--HG--
extra : rebase_source : 0ede4f0d1ec212cfe411f6abe0f55f4b0c70643d
To build sytlo, we have to set compiler flags via BINDGEN_CFLAGS. Since we
pass stlport flags to clang, I would like to move STLPORT_CPPFLAGS to
moz.configure.
MozReview-Commit-ID: 26jvUqUvwTY
--HG--
extra : rebase_source : 5568627368fbf2dce02904918e50a241713d0a85
In this patch, we add 3 tests:
1. test for blocked domain
2. test for blocked sub-domain
3. test for non-blocked domain
MozReview-Commit-ID: JzMImsbGoPr
--HG--
extra : rebase_source : 0eed42f6d2815bd8adbb9fb6be54b6feb3b1bcc8
Because Rust tests require panic=unwind crates and therefore recompiling
all crates we normally use (since most of our crates use panic=abort),
we've elected to enable Rust tests only if the user asks for it. Hence,
this configure option.
The configure option also enables build-time execution of the crates,
since it's not straightforward to determine at configure time the name
of the test binary Cargo will produce (Cargo produces test binaries of
the form ${NAME}-${HEX_FINGERPRINT}, but there's no way to determine
what ${HEX_FINGERPRINT} is without actually compiling the test binary).
At this point, we could tear out the `ignore-locales` attribute, since
l10n-bumper supplies that information. However, we may want to use flatfiles
for something; `ignore-locales` allows for that.
MozReview-Commit-ID: 8mD4iav3bKx
--HG--
extra : rebase_source : abe2075503838223a2c150676b9c72a1aa74df59
By using the PartialConfigEnvironment, the clients of buildconfig will
depend on config.statusd/ files instead of config.status directly.
Clients can access substs and defines using buildconfig.substs['FOO'] or
buildconfig.defines['BAR'], and then collect file-level dependencies for
make using buildconfig.get_dependencies(). All GENERATED_FILES rules
already make use of this because file_generate.py automatically includes
these dependencies (along with all python modules loaded).
As a result of this commit, re-running configure will no longer cause
the world to be rebuilt. Although config.status is updated, no build
steps use config.status directly and instead depend on values in
config.statusd/, which are written with FileAvoidWrite. Since those
files are not official targets according to the make backend, make won't
try to continually rebuild the backend when those files are out of date.
And since they are FileAvoidWrite, make will only re-run dependent steps
if the actual configure value has changed.
As a result of using JSON to load data from the config.statusd
directory, substs can be unicode (instead of a bare string type).
generate_certdata.py converts the subst manually to a string so the
value can be exported to the environment without issue on Windows.
Additionally, patching the buildconfig.substs dict no longer works, so
the unit-symbolstore.py test was modified to patch the underlying
buildconfig.substs._dict instead.
The other files that needed to be modified make use of all the defines
for the preprocessor. Those that are used during 'mach build' now use
buildconfig.defines['ALLDEFINES'], which maps to a special
FileAvoidWrite file generated for the PartialConfigEnvironment.
MozReview-Commit-ID: 2pJ4s3TVeS8
--HG--
extra : rebase_source : d6bb0208483f9f043e7be1b36907ca13243985f8
There is a known compilation bug that is fixed in Visual Studio 2017
15.3.
This commit establishes minimum version requirements for Visual
Studio 2017 to ensure modern and more supported toolchains are used.
We previously only had a minimum version requirement for Visual
Studio 2015.
MozReview-Commit-ID: WRrry8J5Yf
--HG--
extra : rebase_source : 1e60bb4277ae1ec679cfa103b0a730a7260b52c2
Technically this turns on gnu++14. I encountered a few errors when using c++14:
1) _USE_MATH_DEFINES needed to be defined for MinGW
2) MinGW did not define _finite under c++14
3) MinGW's float.h did not define Microsoft specific float functions under c++14
All of these were because c++14 defines _STRICT_ANSI_ which MinGW obeys and
avoids defining certain functions. The first two could be patched around, but
the third was a blocker, so we switched to gnu++14
MozReview-Commit-ID: 6Y7gEQgApYp
--HG--
extra : rebase_source : dabbd40c049c36e780b585e0bef0a8e25887d089
This is needed so we can import the WPT manifest parser.
Like for the reftest parser, this should only be temporary until we no
longer parse test manifests as part of Files() evaluation.
MozReview-Commit-ID: 2dmUfP8tLd2
--HG--
extra : rebase_source : 36624618f96fc89bae88172947b21108740d841d
Some moz.build files need to import in-tree Python modules in order to
parse test manifests. While bug 1402010 details a better solution
to remove this requirement, the easy fix is to expand the sparse
checkout to include the missing files.
MozReview-Commit-ID: FSx6u6r2XfE
--HG--
extra : rebase_source : 6e406dd76b4f78521cbbdc40110d0610c75e2b08
extra : amend_source : f68341a14f50c8ff6e7b62214fa455b75eabe908
Vendor new dependency for compare-locales, python-fluent.
This is the 0.4.2 release of python-fluent.
Also, make mach command a bare minimum wrapper.
The compare-locales mach command used to own a couple of defaults
and opinions.
Now that those opinions and defaults are in the Makefiles, this
command can be much simpler. As a side effect, this should
make the thunderbird port easier, where none of the mach defaults
worked.
Update l10n.mk for compare-locales 2.x:
The command line interface is different now, with positional arguments.
Also, the l10n config file isn't a default anymore (that never worked
for thunderbird).
And the merge dir is now the parent of the locale dir, much like
we already anticipated in the code.
MozReview-Commit-ID: DxVVKyVSt5y
--HG--
extra : rebase_source : 7c591fea8c7d9ecef615fb56f9359d4f1bd4b340
Strictly speaking we don't need all files in the directories listed
in the profile. But the checkout is still small enough and it is far
less effort than cherry-picking every file needed by every toolchain
task.
This brings the checkout down to ~3700 files, which only takes 1-2s.
MozReview-Commit-ID: 2BpKdZ2Pvx9
--HG--
extra : rebase_source : e9589b29f7e1b60ac7ee4e8050689dc3d5a8f418
extra : source : 2c0d6e566cc719823515ec2403e1d6b2ace955aa
toolchain.configure checks for information about the compilers we're
using and accumulates additional flags that we might need to
pass (e.g. switches for C/C++ versions, proper compiler targets, etc.),
and then rechecks the compilers with those additional flags to verify
that those flags are sufficient to configure the compiler properly.
Only after we've checked for the proper flags do we move on to verifying
the compiler versions are sufficient.
It's possible, however, that the flags we try to add might not be
supported by the compiler being tested, in which case we'd get an
unfriendly error about an "unknown compiler", rather than something like
"version X of Y not supported". In this case, we'd rather use the
information we gathered from the first run to provide information
messages. So we modify the second check to ignore any thrown exceptions
during the check itself.
This change results in another problem: the check for whether we had any
extraneous flags is done prior to version checks for the compilers we
support, which is also unhelpful. We choose to move this check after
the version checks themselves.
gcc 4.9 is the last version in Android NDK and our minimum requirement of gcc is 4.9+. --with-android-gnu-compiler-version is unnecessary option because gcc version of Android is always 4.9.
MozReview-Commit-ID: 1sutqlvbQU5
--HG--
extra : rebase_source : 77e42aba81da5276bcddd945ea41895ce2461afa
This adds some new optimization strategies. For tests, we use Either(SETA,
SkipUnlessSchedules), thereby giving both mechanisms a chance to skip tasks. On
try, SETA is omitted.
MozReview-Commit-ID: GL4tlwyeBa6
--HG--
extra : rebase_source : f208e4960cf76a9dfe77634b87f0058f676e9fa9
extra : source : 046d705929f7a41e977eec19c8503afccdec7592
Strictly speaking we don't need all files in the directories listed
in the profile. But the checkout is still small enough and it is far
less effort than cherry-picking every file needed by every toolchain
task.
This brings the checkout down to ~3700 files, which only takes 1-2s.
MozReview-Commit-ID: 2BpKdZ2Pvx9
--HG--
extra : rebase_source : 916c6722c6d495cdccea0076c673d75e4ee48f28
extra : source : 2c0d6e566cc719823515ec2403e1d6b2ace955aa
This adds some new optimization strategies. For tests, we use Either(SETA,
SkipUnlessSchedules), thereby giving both mechanisms a chance to skip tasks. On
try, SETA is omitted.
MozReview-Commit-ID: GL4tlwyeBa6
--HG--
extra : rebase_source : 4cf3efc9c57bb14d2f44147c8881d0a0a18703d6
extra : source : 046d705929f7a41e977eec19c8503afccdec7592
libstdc++ support is broken after moving to moz.configure. No one uses this option and NDK will remove GCC, so we should remove this and --with-android-cxx-stl option.
MozReview-Commit-ID: 3mqyHoRCE00
--HG--
extra : rebase_source : 35aa911a69e159e67f624ab5ab9aea8af4c5342f
LLVM can optimize code to a form `if B && A` when A is
always false but B is undefined. This triggers a valgrind
memcheck warning since the conditional depends on undefined
data but in practice it can never have side-effects.
Rust 1.20.0 seems to trigger this in the Option code. Since
we believe the transform is correct in these cases and
valgrind is incorrect to warn, we surpress the error.
Thanks to Julian Seward for the analysis and help
writing the suppression entries.
MozReview-Commit-ID: pF1Bmy5PRY
This was done in bug 736564 for the xulrunner SDK, which later became
the firefox SDK, which is now gone. So we don't actually need to keep it
separate anymore (except for logalloc/replay, which still needs to link
it directly, so we keep the library definition intact so it can be
referenced ; we just don't DIST_INSTALL it anymore, and always make it
linked into mozglue)
--HG--
extra : rebase_source : e4d0627ec907fe0139df5c0b2b9f7d04b43c7c78
The name `android-gradle-build` is an accident of history; let's rename it
before we attempt major surgery on it.
--HG--
rename : taskcluster/docker/android-gradle-build/Dockerfile => taskcluster/docker/android-build/Dockerfile
rename : taskcluster/docker/android-gradle-build/README.md => taskcluster/docker/android-build/README.md
rename : taskcluster/docker/android-gradle-build/REGISTRY => taskcluster/docker/android-build/REGISTRY
rename : taskcluster/docker/android-gradle-build/VERSION => taskcluster/docker/android-build/VERSION
rename : taskcluster/docker/android-gradle-build/buildprops.json => taskcluster/docker/android-build/buildprops.json
rename : taskcluster/docker/android-gradle-build/dot-config/pip/pip.conf => taskcluster/docker/android-build/dot-config/pip/pip.conf
rename : taskcluster/docker/android-gradle-build/oauth.txt => taskcluster/docker/android-build/oauth.txt
cpu_arch_dir isn't absolute path, we cannot always detect thumb library for c++ library. Our build tool still uses NDK r11c, so we should check thumb library correctly.
MozReview-Commit-ID: 912ZDNeUcKc
--HG--
extra : rebase_source : 8b69348ef3280f32b7a228d66462070413a25e73
The linker detection essentitally invokes `$CC -Wl,--version` to
determine which linker will be used. This invocation works OK for GCC
from the Android NDK, which can find the linker in the same directory;
it doesn't work so well for clang from the NDK, which needs additional
command-line options to point at the directory containing the linker.
We already have extra_toolchain_flags for communicating such options to
other compiler invocations for the target; we should be using
extra_toolchain_flags here as well.
This removes the double-include macro hackery that we use to define two
separate string types (nsAString and nsACString) in favor of a templated
solution.
Annotations for Valgrind and the JS hazard analysis are updated as well as
the rust binding generations for string code.
--HG--
extra : rebase_source : 63ab2c4620cfcd4b764d42d654c82f30f984d016
extra : source : 9115364cd4aa078c49bba7911069f8178e55166f
The Sphinx documentation only needs access to a relatively small number
of files in the repo in order to be generated. It is a good candidate
for using sparse profiles.
This commit defines and uses a "sphinx-docs" sparse profile containing
only the files relevant to Sphinx documentation generation.
There are some quirks with the profile:
* All moz.build files are included. This bloats the profile
by >1000 files. Worse, it realizes directories that have no business
being realized. This clutters the checkout and makes it harder to
find things. There is a moz.build reader that knows how to retrive
file data from Mercurial. We could use that. This feels like follow-up
fodder.
* All mach_commands.py files are included. `mach help` says you can do
things that you aren't able to do in the sparse checkout. There isn't
a good way to add all *.py files while excluding mach_commands.py
files. We /could/ do it with regular expressions. But those are slow.
Let's leave it as is for now and come up with a better solution later.
MozReview-Commit-ID: 7yiqGGE1nAh
--HG--
extra : rebase_source : c148040ea3618e8bfdd369b6f48fc60c6d179285
extra : source : b76e2f6204b20de137f2566dff8121ff3abe5760
To construct an empty set, we need to use the `set()` notation. In order
to do that, we need to expose `set` to the moz.build sandbox.
MozReview-Commit-ID: DMyKnF0FEx2
--HG--
extra : rebase_source : 5cfe8080ec333a1eca70cd3edba2aaaff6406820
Add a BuildFlag annotation, which when specified for classes, will wrap
generated code in `#ifdef` or `#ifndef` blocks. This functionality is
used for conditionally excluding generated code when NIghtly becomes
Beta, without the need to regenerate bindings.
MozReview-Commit-ID: L2NFM8CHKqF
--HG--
extra : rebase_source : 6ebc82d11fd1aa4aeb57a46262e678480d23de83
python-hglib is a Python client for Mercurial's command server. It
facilitates querying Mercurial efficiently (using a single process)
and without having to parse output in the common case.
Let's vendor it so we can make use of it for more advanced Mercurial
scenarios.
Content vendored from changeset 820d7c1e470a without modifications
(other than deleting unwanted files).
As part of vendoring, we add the package to the virtualenv and make
it available to mach.
MozReview-Commit-ID: F4KLbW1lAvk
--HG--
extra : rebase_source : ad81f3aa4843312d300d80ce563d40736af2c16a
This adds some new optimization strategies. For tests, we use Either(SETA,
SkipUnlessSchedules), thereby giving both mechanisms a chance to skip tasks. On
try, SETA is omitted.
MozReview-Commit-ID: GL4tlwyeBa6
--HG--
extra : rebase_source : 0c1ce762afc7a691788379d4f4206df106f6df63
The HTTP server can dynamically map URLs to local filesystem paths.
We employ this so Speedometer's files are available to the server.
MozReview-Commit-ID: EpF1aD6meZH
--HG--
extra : rebase_source : 70f80e073c2bec92c9429dcc35cd12e27018f3cb
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
In some cases, they are false positives. In other cases, the callsite is #ifdef-ed
out, so it's a true positive, but not one we would correct.
MozReview-Commit-ID: 6ThZH3wEXTp
--HG--
extra : rebase_source : aabdb93cb924e74cf9592a93c86062c4435ceaad
``print()`` has no business being in library code like this. It was
a holdover from this code being copied from bootstrap. So remove it.
While we're here, replace the generic exception with a specific one.
We don't want to be swallowing bugs via ``except Exception``.
MozReview-Commit-ID: 49goUstfPBz
--HG--
extra : rebase_source : a821159bd12a449ed1a0edf21a1f9eb29711ad95
An inline comment wanted it. The change is trivial.
MozReview-Commit-ID: CqyOYqNzwzr
--HG--
extra : rebase_source : d779d6bbca4aa57d9e95ead3a8403e0163106bdf
The file was using 2 space indent. I ran `autopep8` on it. Most of the
changes are whitespace only.
MozReview-Commit-ID: 1C7yDUQCfKl
--HG--
extra : rebase_source : 9f2b52ef6d0887fbaa7f8410b32348d371e884d8
Mercurial's sparse checkout support allows "profile" files defining
what's in sparse checkouts to be defined in-tree. This convenient
feature means you simply need to point a client at a path inside the
repository and it dynamically resolves what files to include in the
checkout. As you update revisions, the "profile" pulls in updates
to the underlying file.
We introduce 2 sparse profiles: 1 for mach and another for taskgraph.
The goal of the "mach" profile is to provide enough files to run
`mach`. If you activate this profile and run `mach`, it runs without
error. But there are practically no commands available. So it isn't
terribly useful.
The "taskgraph" profile allows us to run `mach taskgraph`. This
profile demonstrates a sparse profile feature: including other
profiles. The "taskgraph" profile is thus a union of "mach" and
its own entries.
There is definitely some fat in these profiles. I didn't feel like
chasing the long tail and getting overly granular with the profiles.
If we want to optimize later, we can do that.
For reference:
Full checkout: ~234,000 files
mach: ~2,000 files
taskgraph: ~3,600 files
MozReview-Commit-ID: 7pALt0MwHfE
--HG--
extra : rebase_source : 1a4ba4b8a63c522dab2841e2c0019501476da2fe
python-hglib is a Python client for Mercurial's command server. It
facilitates querying Mercurial efficiently (using a single process)
and without having to parse output in the common case.
Let's vendor it so we can make use of it for more advanced Mercurial
scenarios.
Content vendored from changeset 820d7c1e470a without modifications
(other than deleting unwanted files).
As part of vendoring, we add the package to the virtualenv and make
it available to mach.
MozReview-Commit-ID: F4KLbW1lAvk
--HG--
extra : rebase_source : 39321a880a13a0b0323a7217f538978b729e2afe
Sparse checkouts may not have all mach_commands.py files.
mach raises an error when a mach_commands.py file is missing.
So, we teach the mach driver to ignore missing file errors when
a sparse checkout is present.
The added code is optimized to avoid an import of mozversioncontrol
and some I/O as part of resolving the repo and VCS binaries because
this file is in the critical path of all mach commands and avoiding
I/O is worthwhile. Since we aren't using sparse checkouts in the
common case, this effectively makes the new code 0 cost.
MozReview-Commit-ID: C6itJga31t5
--HG--
extra : rebase_source : 4b2c18d30ff8b923a940c80ac81372a4076b8fdc
"mach" in this scope is both the mach module and the driver instance.
Let's change the latter's variable name so we can access the mach
module.
MozReview-Commit-ID: Db6sxDFl2oo
--HG--
extra : rebase_source : 0a51c7514cfdba40483b3b9b361cd03b8000c2c9
The Repository interface gains a new method to determine if a sparse
checkout is present. Mercurial's implementation is somewhat crude,
but it should work (Mercurial's sparse support is still experimental
and I only intend to support sparse checkout in Firefox CI until it
is less experimental). Git's always returns False (for now).
To prove it works and to expose the information more widely, we hook
it up to moz.configure. We do this by first implementing a function
that returns a Repository instance. Then we simply call a function on
it to resolve the sparse checkout flag.
MozReview-Commit-ID: AlsT5LdSPdZ
--HG--
extra : rebase_source : f1e9aaa7d15f11c7c5e8d268d4ad82468732103b
This change adds an upload-generated-sources task kind that runs after nightly
builds, fetches their `target.generated-files.tar.gz` artifact, and uploads
all the contained files to an S3 bucket. For actual nightly and release builds
on SCM level 3 trees, the S3 bucket is configured to be publicly accessible,
so that tools like Socorro will be able to fetch generated source files that
appear in crash reports, and debuggers will be able to fetch generated sources
when they show up while debugging Nightly or Release builds.
There are also level-2 and level-1 S3 buckets configured for builds happening
on trees of other levels such as try. They are not configured as publicly
accessible, but they exist so that these tasks can be tested in try.
MozReview-Commit-ID: Js1HRftbtep
--HG--
extra : rebase_source : b1172c9cc8b8be437d3b94a6bf0ff6b2f7d3508b
extra : source : 73bf88110b3821d62a3d393e85b56896a12f2930
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.
This will be used to provide links to generated source files when they appear
in stack traces in crash reports.
MozReview-Commit-ID: 6yQAdlZ5q3O
--HG--
extra : rebase_source : d92fb17ae737d1360e9724997f6688e29bedef12
extra : source : 14d18d7cf454c4c3d0f6d49d1d01660e06e4be4b
A followup change will be adding a new automation step that wants to be skipped
in artifact builds, and this will make that simpler.
MozReview-Commit-ID: 5xwRB9eCRQn
--HG--
extra : rebase_source : 2fccd9d128ab92c98515762a62a0a2e89bf9ca24
extra : source : a02695cbf5762eb0eb7087239319807eb447ca1e
shutil.copy2() will fail if the destination directory doesn't exist.
Switch to copy_tree() instead so we don't need to worry about the
error cases of copy2() and copytree().
MozReview-Commit-ID: 3kHfgL57KfX
--HG--
extra : rebase_source : c7335b0c2854d53699dda0f0d2bd9d17b57c4e5d
We define extra_toolchain_flags for passing extra flags to the target
compiler during configure. But the way things are currently set up, we
pass those flags to the host compiler during configure as well. This
behavior is incorrect, and we should only be passing the flags from
extra_toolchain_flags if we're compiling for the target.
This is similar to what we had until bug 1278456 removed them when we dropped
support for older libstdc++, but for a different symbol.
--HG--
extra : rebase_source : 641fc6c86c8f47e3dbd752bc20056f61646541a7
That the wrapper implementation works has been verified by creating a
dummy program such as:
$ cat test.cc
#include <thread>
int main() {
std::thread([]() {
printf("foo\n");
}).join();
return 0;
}
And compiling it with and without the hack:
$ g++ -fno-rtti -o test test.cc -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_State::~_State()
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_M_start_thread(std::unique_ptr<std:🧵:_State, std::default_delete<std:🧵:_State> >, void (*)())
$ ./test
foo
$ g++ -fno-rtti -o test test.cc $objdir/build/unix/stdc++compat/stdc++compat.o -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
$ ./test
foo
--HG--
extra : rebase_source : 53ca8e2d0424eaeb539d50510c441c8a3252c819
Various people want to start experimenting with Python 3 in the build
system and in related tools (like mach).
We want to make it easy to find and use an appropriate Python 3
binary.
This commit introduces a generic function for finding a Python 3
binary and resolving its version.
We use the new code in configure to set PYTHON3 and PYTHON3_VERSION
subst entries for later consultation.
We also expose a cached attribute on the base class used by many
mach and build system types to return a Python 3 executable's info.
By default, we only find Python 3.5+. From my experience, Python 3.5
was the first Python 3 where it was reasonable to write code that
supports both Python 2 and 3 (mainly due to the restoration of the
% operator on bytes types). We could probably support Python 3.4
in the build system. But for now I'd like to see if we can get
away with 3.5+.
MozReview-Commit-ID: BlwCJ3kkjY9
--HG--
extra : rebase_source : b00464972183ef1a97a0b5d888520be425717ae7
This patch:
* Adds a suppression for some leaks in libLLVM-3.6-mesa.so.
* Adds Valgrind flag --keep-debuginfo=yes so that the abovementioned leak
stacks can be symbolised and hence suppressed even after
libLLVM-3.6-mesa.so is unmapped from the process.
* Adds Valgrind flag --expensive-definedness-checks=yes as an attempt to
reduce Memcheck false positives from LLVM and rustc compiled code. This
change is aimed primarily at bug 1365915.
MozReview-Commit-ID: KiOZG2O8wzs
Bug 1338651 was backed out because when building a newer image, there
was a valgrind leak report that couldn't resolve symbols. Further
investigation showed the valgrind package installed had symbols stripped.
We upgrade valgrind version and build it from source with symbols.
We had to build inside the docker image because we need to run
"make install". Using "make dist" to generate a tar ball will also run
"make docs", and it is hard to make it work because of the outdated
texlive package present in CentOS 6.
We also apply a patch [1] to valgrind correctly generate symbols
for unloaded objects.
[1] https://bugs.kde.org/show_bug.cgi?id=79362#c62
MozReview-Commit-ID: 2IhuJY28Ke3
In bug 635961, elfhack was made to (ab)use the bss section as a
temporary space for a pointer. To find it, it scanned writable PT_LOAD
segments to find one that has a different file and memory size,
indicating the presence of .bss. This usually works fine, but when
the binary is linked with lld and relro is enabled, the end of the
file-backed part of the PT_LOAD segment containing the .bss section
ends up in the RELRO segment, making that location read-only and
subsequently making the elfhacked binary crash when it tries to restore
the .bss to a clean state, because it's not actually writing in the .bss
section: lld page aligns it after the RELRO segment.
So instead of scanning PT_LOAD segments, we scan for SHT_NOBITS
sections that are not SHF_TLS (i.e. not .tbss).
--HG--
extra : rebase_source : f18c43897fd0139aa8535f983e13eb785088cb18
Suppress leaks for allocations where any stack frame matches the string
style::gecko::global_style_data. This could be dangerous if someone were to
allocate non-global things with this on the stack, but hopefully that can be
avoided, given the name of the module.
MozReview-Commit-ID: 65HpUGsgPPM
--HG--
extra : rebase_source : 19ba5a5001de65628acdb0433341d14429e4af67
GYP of WebRTC should reference MOZ_SYSTEM_LIBEVENT values if available.
MozReview-Commit-ID: CshsPrRidM8
--HG--
extra : rebase_source : 9e619c2f49e7c2b3f680814b95b823996773fa6c
In some cases, we can end up linking some things with
--static-libstdc++. The notable (only?) example of that is for the
clang-plugin, and that happens because it gets some of its flags from
llvm-config, which contains --static-libstdc++ because clang itself is
built that way.
When that happens, the combination of --static-libstdc++ and
stdc++compat breaks the build because they have conflicting symbols,
which is very much by design.
There are two ways out of this:
- avoiding either -static-libstdc++ or stdc++compat
- work around the symbol conflicts
The former is not totally reliable ; we'd have to accurately determine
if we're in a potentially conflicting case, and remove one of the two in
that case, and while we can do that for the cases we explicitly know
about, that's not future-proof, and might fail just as much in the
future.
So we go with the latter. The way we do this is by defining all the
std++compat symbols weak, such that at link time, they're overridden by
any symbol with the same name. When building with -static-libstdc++,
libstdc++.a provides those symbols so the linker eliminates the weak
ones. When not building with -static-libstdc++, the linker keeps the
symbols from stdc++compat. That last assertion is validated by the
long-standing CHECK_STDCXX test that we run when linking shared
libraries and programs.
That still leaves the symbols weak in the final shared
libraries/programs, which is a change from the current setup, but
shouldn't cause problems because when using versions of libstdc++.so
that do provide those symbols, it's fine to use the libstdc++.so version
anyways.
Bump the minimum required version of the Rust toolchain to
the current stable release so we can take advantage of new
features.
Highlights of the 1.19.0 release:
* C-compatible `union` (untagged enums).
* Support for Visual Studio 2017.
* Non-capturing closures can be coerced to `fn` bindings.
* Numeric field names in tuple struct initializers.
* Higher macro recursion limit.
* `break` can return a value from `loop` expressions.
* Better error handling with mis-configured Visual Studio environments.
This change also enables 1.18.0 features. Some highlights:
* `pub(mod)` &c. for better control of symbol visibility.
* struct packing for better memory footprint in generated code.
* Faster build times.
MozReview-Commit-ID: 2OpUjAcytpE
--HG--
extra : rebase_source : 2ed0d7c4e7b78c26f7a7476e7b284bf1bdbe7c8b
Mostly removals, but also adding docs on how to create a
multilingual package right now.
I think I'd like to take another pass at those docs in a follow-up.
MozReview-Commit-ID: Dkw4MJ5DLyb
--HG--
extra : rebase_source : 4f79499e376cf6ddd23169a1c4525ed6b551a740
