In clang 9, the signature of `registerPPCallbacks` changed and it no longer provides us a `CompilerInstance` [0]. I couldn't find a way to reach the `CompilerInstance` from the new structures, so I've inserted a level into the class hierarchy that mimics the old functionality.
With this patch, clang-tidy compiles with either clang-8 or clang-9, so we can land this independently in preparation for bug 1573211.
[0] For reference: the new signature was introduced in bbc89dcb29, the old overload was marked deprecated in e9087fe75c and removed in ee737a84d7.
Differential Revision: https://phabricator.services.mozilla.com/D46702
--HG--
extra : moz-landing-system : lando
It doesn't really make sense to enforce this behavior for calls on stack
variables (which presumably we then want to do something with) and doesn't
necessarily make sense to enforce it for member variables either.
Differential Revision: https://phabricator.services.mozilla.com/D46105
--HG--
extra : moz-landing-system : lando
Setting this flag still attempts to build with 1-tier PGO, unless we
detect that 3-tier PGO is enabled. Since we should be using 3-tier PGO
everywhere, the flag should be redundant in our automation mozconfigs.
It can still be used locally in a mozconfig to do a 1-tier PGO build.
Depends on D46069
Differential Revision: https://phabricator.services.mozilla.com/D46070
--HG--
extra : moz-landing-system : lando
This file is no longer required, since taskcluster handles scheduling
PGO tasks.
Differential Revision: https://phabricator.services.mozilla.com/D46069
--HG--
extra : moz-landing-system : lando
The option was removed entirely several months ago, and this bit was missed.
Differential Revision: https://phabricator.services.mozilla.com/D46296
--HG--
extra : moz-landing-system : lando
Getting rid of unneeded code and having fewer global defines are both
worthy things.
Differential Revision: https://phabricator.services.mozilla.com/D45726
--HG--
extra : moz-landing-system : lando
A previous patch defined `_LIBCPP_HIDE_FROM_ABI` to the empty string.
This definition worked for normal (opt/debug) builds, but
PGO-instrumented builds fell over with some linker errors. A closer
look at the definition of `_LIBCPP_HIDE_FROM_ABI`:
```
#ifndef _LIBCPP_HIDE_FROM_ABI_PER_TU
# ifndef _LIBCPP_HIDE_FROM_ABI_PER_TU_BY_DEFAULT
# define _LIBCPP_HIDE_FROM_ABI_PER_TU 0
# else
# define _LIBCPP_HIDE_FROM_ABI_PER_TU 1
# endif
#endif
#ifndef _LIBCPP_HIDE_FROM_ABI
# if _LIBCPP_HIDE_FROM_ABI_PER_TU
# define _LIBCPP_HIDE_FROM_ABI _LIBCPP_HIDDEN _LIBCPP_INTERNAL_LINKAGE
# else
# define _LIBCPP_HIDE_FROM_ABI _LIBCPP_HIDDEN _LIBCPP_EXCLUDE_FROM_EXPLICIT_INSTANTIATION
# endif
#endif
```
says that there are two macros involved there: one to set hidden visibility
(which we don't want), and another to exclude the function definition
from explicit instantiation. The semantics of this are not obvious (the
clang documentation only documents this attribute in terms of C++
templates; the particular case we're interested in sets the attribute on
normal function definitions). But roughly, what this does is that it
forces the compiler to see that an equivalent function definition is
provided external to the compilation unit, so it doesn't have to keep
the annotated function definition around.
Differential Revision: https://phabricator.services.mozilla.com/D45073
--HG--
extra : moz-landing-system : lando
If the run task generates bad profile data, the merge step in the
profile-use task will fail. However, retrying the profile-use task
doesn't fix the problem, and there isn't a straightforward way to retry
the run task in this situation. Instead we can add a clang toolchain to
all the run tasks, and perform the merge there.
This means the output from the run task will always be a successfully
merged file called 'merged.profdata', and we no longer need to perform
the merge as part of the profile-use build as a GENERATED_FILES step.
Depends on D45262
Differential Revision: https://phabricator.services.mozilla.com/D45263
--HG--
extra : moz-landing-system : lando
This is needed in the following patch, but the diff is more clear with
this split out.
Differential Revision: https://phabricator.services.mozilla.com/D45262
--HG--
extra : moz-landing-system : lando
With the migration from AWS to GCP, we also need to migrate sccache
buckets from S3 to Google Storage.
The problem is how we deal with regions, since there isn't an exact
correspondence on the region names between the two cloud providers.
To make the transition smoother, docker-worker (and soon generic-worker)
provides a new environment variable called TASKCLUSTER_WORKER_LOCATION,
with information about the cloud provider the task is running on. Using
this new variable, we configure sccache to use the corresponding storage
service of the cloud provider where the task runs.
The bucket names in Google Storage are shorter because GCS imposes a
limit of 30 characteres for the names.
Ref: https://github.com/taskcluster/taskcluster-rfcs/pull/148/files
Differential Revision: https://phabricator.services.mozilla.com/D44845
--HG--
extra : moz-landing-system : lando
One less thing for configurations to vary is a good thing. This change
also means that when we do NDK bumps that introduce some kind of weird
incompatible change, configuration with a newly-pulled tree and an old
NDK halts at configure, rather than giving weird build errors.
Differential Revision: https://phabricator.services.mozilla.com/D44705
--HG--
extra : moz-landing-system : lando
We define some libc++-internal macros to avoid conflicts between the way
we do symbol visibility and the way libc++ expects visibility to work.
Newer NDK versions use a newer libc++ which changes the way the macro we
were using works. To enable building with newer libc++ versions, let's
also define macros appropriate to those versions as well.
Differential Revision: https://phabricator.services.mozilla.com/D44510
--HG--
extra : moz-landing-system : lando
We have an `ndk_version_major` and an `ndk_version_minor` that have a
lot of duplicated code in them. We could factor them into a single
function, but it seems better to just pull their logic into `ndk_version`
directly, rearranging that function to be IMHO more understandable.
Differential Revision: https://phabricator.services.mozilla.com/D44506
--HG--
extra : moz-landing-system : lando
Credit: mars for making the shell POSIX compliant
This embeds a blacklist of every mach command that needs to run with Python 2
directly in the mach driver itself. Initially this is every mach command. We
then use a bit of shell to determine whether the command being run needs Python
2 or 3.
While this approach may seem a bit hacky, it has several benefits:
1. No need to add complex machinery in mach's registration code.
2. No need to spawn two separate Python interpreters in the event a different
Python from the original interpreter is needed.
3. Perf impact is negligible.
4. New commands are Python 3 by default.
It is also only a temporary hack. Once all commands are running with Python 3,
we can revert back to the original mach driver.
Differential Revision: https://phabricator.services.mozilla.com/D36103
--HG--
extra : moz-landing-system : lando
common.gypi uses hard-coded absolute path for echo command, but this path
is invalid on Windows.
But this lines are unused, so we should remove it.
Differential Revision: https://phabricator.services.mozilla.com/D42779
--HG--
extra : moz-landing-system : lando
Also don't pass universal_newlines to check_cmd_output. That was added
to make python 3 happier when check_cmd_output still returned bytes, to
hint subprocess to return a unicode string, but now that
check_cmd_output does the decoding, that's not needed anymore.
Differential Revision: https://phabricator.services.mozilla.com/D43205
--HG--
extra : moz-landing-system : lando
The discrepancy of features used for syn between jsrust and gkrust
triggered syn and its reverse dependencies up to cranelift to be
built once for jsrust and once for gkrust, while they are time consuming
to build.
Differential Revision: https://phabricator.services.mozilla.com/D43209
--HG--
extra : moz-landing-system : lando
This change looks a lot more complicated than it actually is.
The motivating issue is that on x86-64 android, the NDK uses gold as its
default linker, and on this platform gold has some bugs in how it handles
`.note.*` sections that lead to `strip` and `objcopy` produce invalid
binaries...or at least invalid binaries according to `elfhack`. When
`elfhack` complains, the build stops, which is suboptimal. Instead, let's
use bfd `ld`, which doesn't have these problems.
Except that when we make the change to provide `--enable-linker` with a
platform-specific default, the lint configure tests complain that the
provided help string is not using the proper format. So we tack on the
sekret `{|}` string to satisfy the linter.
Differential Revision: https://phabricator.services.mozilla.com/D43098
--HG--
extra : moz-landing-system : lando
For some reason, cargo incremental doesn't work as well as it should,
and doesn't perform as well as sccache does. So when building with
sccache, disable cargo incremental. This brought a no-change clobber
build with 100% cache from 3:50 to 2:05 on a beefy AWS instance I was
testing this with.
Differential Revision: https://phabricator.services.mozilla.com/D43188
--HG--
extra : moz-landing-system : lando
Bug 1575135 changed check_cmd_output to return unicode strings, but a
couple places were already trying to do their own decoding, which now
can fail. Remove those.
Interesting the decoding was previously broken on Windows, this
actually fixes it (the output of hg config is not actually utf-8 on
Windows).
Differential Revision: https://phabricator.services.mozilla.com/D43044
--HG--
extra : moz-landing-system : lando
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.
OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.
Also while here, remove an unused import of encode().
Differential Revision: https://phabricator.services.mozilla.com/D42608
--HG--
extra : moz-landing-system : lando
There are a few problems with the strategy currently used to find the
rust target. For example, we don't find a target for arm freebsd, and we
pick the wrong target for armel linux. Both are related to how things
currently work when multiple targets have the same (cpu, endianness,
os).
So, to derive the rust target, we now use a more fine-grained approach.
Differential Revision: https://phabricator.services.mozilla.com/D41481
--HG--
extra : moz-landing-system : lando
Currently, we have no real visibility on the time spent after the build
finished, despite the fact that a large chunk is actually happening via
make check (although thankfully more and more of it is moving out to
separate tasks).
Also, the mozharness machinery for make check dates from when we were
running in buildbot and takes care of turning builds orange instead of
red in case of failure, etc. which doesn't do anything useful anymore.
Differential Revision: https://phabricator.services.mozilla.com/D42806
--HG--
extra : moz-landing-system : lando
The automation tiers all have a dependency:
automation/%: automation-start/%
The dependency runs the command for BUILDSTATUS, making the resource
monitor aware of what tiers start. Unfortunately, Make decides to run
all these automation-start/% targets way before starting the
automation/% targets, because of how the interdependencies are laid out.
Use a slightly better way, albeit still biased, making the
automation-start/% target for a tier depend on the dependencies, which
at least ensures that the resource monitor doesn't see a tier starting
before its dependencies are finished.
Differential Revision: https://phabricator.services.mozilla.com/D42805
--HG--
extra : moz-landing-system : lando
This warning, new in clang 9, is noisy about patterns like:
```
unsigned long x = ...
if (x > 4294967295)
```
The condition is always false on 32-bit and LLP64 systems, but on LP64 it's a valid test.
Differential Revision: https://phabricator.services.mozilla.com/D42346
--HG--
extra : moz-landing-system : lando
Disable -ftrivial-auto-var-init for DllBLocklistWin.cpp with clang-cl because the file's interceptions happen so early in the main process that the loader hasn't yet resolved the import of memset (used by -ftrivial-auto-var-init) from vcruntime140.dll.
Disable -ftrivial-auto-var-init on Linux32 because it causes some xpcshell test failures.
Differential Revision: https://phabricator.services.mozilla.com/D42273
--HG--
extra : moz-landing-system : lando
As a consequence, we can replace the encoded_open function that did the
same in an opt-in manner.
Differential Revision: https://phabricator.services.mozilla.com/D42605
--HG--
extra : moz-landing-system : lando
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.
Places that explicitly do want bytes can keep using subprocess directly.
Differential Revision: https://phabricator.services.mozilla.com/D42604
--HG--
extra : moz-landing-system : lando
This is not something that happens under normal circumstances, but it
can happen when you go fancy and run multiple configures in parallel
with different objdirs, and old-configure doesn't exist in the first
place ; then one configure may overwrite old-configure while another is
starting to execute it, resulting in the latter nor executing
old-configure completely.
Differential Revision: https://phabricator.services.mozilla.com/D42428
--HG--
extra : moz-landing-system : lando
This definition is now unnecessary, given that the source code is
fetched for us from elsewhere.
Depends on D41368
Differential Revision: https://phabricator.services.mozilla.com/D41369
--HG--
extra : moz-landing-system : lando
Otherwise we wind up with clownshoes like:
```
froydnj@hawkeye:/opt/build/froydnj/tmp/clang$ ls -l lib/libstdc++.so*
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6.0.24
```
and have duplicate copies of shared libraries in our toolchain packages,
which are not exactly small.
Differential Revision: https://phabricator.services.mozilla.com/D41368
--HG--
extra : moz-landing-system : lando
In some libstdc++ these appear in the inheritance hierarchy of __atomic_base, which is already whitelisted.
Differential Revision: https://phabricator.services.mozilla.com/D42348
--HG--
extra : moz-landing-system : lando
The current setup, where gtest/libxul uses the static library in
the same directory as the shared libxul, and somehow the backend ignores
gkrust for gtest/libxul, is fragile.
Differential Revision: https://phabricator.services.mozilla.com/D42246
--HG--
rename : toolkit/library/dependentlibs.py => toolkit/library/build/dependentlibs.py
extra : moz-landing-system : lando
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
The remaining uses all need adjustements to in-tree mozconfigs, so they
all need to be done at once.
However, to make things slightly more intelligible, we do this in two
steps. This is step 1: we modify the use_toolchain transform to take care of
the transformation, while keeping the task definitions intact, so that
we only deal with mozconfig and build script adjustements here.
Differential Revision: https://phabricator.services.mozilla.com/D41890
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
Just like C++14 sized deallocation support, we don't want to support
this. We shouldn't be using `new` on over-aligned types anyway.
Differential Revision: https://phabricator.services.mozilla.com/D41819
--HG--
extra : moz-landing-system : lando
As shown by python/mozbuild/mozbuild/test/test_preprocessor.py, whitespace is
allowed within expressions, and chained #if/#elif/#else sequences work as you'd
expect.
Differential Revision: https://phabricator.services.mozilla.com/D41494
--HG--
extra : moz-landing-system : lando
There are about the same number of occurrences of "ENABLE_INTL_API" and "EXPOSE_INTL_API"
in the tree, so preferring one over the other doesn't lead to fewer changes. Therefore
I went with "ENABLE_INTL_API", because "ENABLE_" (resp. "MOZ_ENABLE") is already used as
the prefix for other preprocessor ifdef's.
Differential Revision: https://phabricator.services.mozilla.com/D41188
--HG--
extra : moz-landing-system : lando
All .xul files have been loading as HTMLDocuments for a few weeks now, so
it should be safe to remove the XULDocument implementation.
Differential Revision: https://phabricator.services.mozilla.com/D41238
--HG--
extra : moz-landing-system : lando
Some recent changes to how we set cross-language LTO for Windows
resulted in compilation-time decreases and small performance regressions
on a few benchmarks. The changes intended to remove explicit enablement
of cross-language LTO for all builds, but rely on shippable builds being
built with PGO and moz.configure's clever defaulting of cross-language
LTO for PGO'd builds on Windows, which would then enable cross-language
LTO for only shippable builds.
Obviously something went wrong with those changes.
The problem was our defaulting wasn't visible to moz.configure's logic
for how to pass command-line options to the JS subconfigure. We set the
value (`cross`) after the value for `--enable-lto` has been determined,
and the default value is off (that is, `--disable-lto`). Since
moz.configure is very thorough in passing configure options down into
JS, it dutifully looked at what the default value of `--enable-lto` was
supposed to be, and passed `--disable-lto` to JS's configure.
There's some evidence that we knew our defaulting was a little sketchy:
we'd only attempt cross-language LTO when we were performing the PGO use
phase, and only if the value of `--enable-lto` wasn't explicitly passed.
Which was a fine idea--you don't want to override what the user was
trying to do--but in the case of JS backfired on us: the value *was*
coming from the explicitly-passed command-line option of
`--disable-lto`. So JS didn't enable any kind of LTO, with attendant
consequences.
This problem *didn't* happen before the aforementioned change because we
were explicitly specifying that `--enable-lto=cross` should be passed in
the mozconfig, which ensured that the correct setting was passed into
JS. We were just setting `--enable-lto=cross` for *all* builds, which
was less than desirable.
The easiest way to fix all this is simply to put the
`--enable-lto=cross` setting in the Windows mozconfigs, conditional on
`MOZ_PGO_PROFILE_USE`. That placement captures the intent of the
previous attempt at defaulting, but without the troubles described
above: the option explicitly appears on the command line, and
moz.configure will correctly pass it through to the JS subconfigure.
This also makes our Windows configuration closer to our Linux
configuration (the Linux configuration enables cross-language LTO for
both PGO phases, which is arguably a bug).
Differential Revision: https://phabricator.services.mozilla.com/D41080
--HG--
extra : moz-landing-system : lando
This patch makes BuildEnvironmentNotFoundException a subclass of AttributeError as well, because hasattr in
python3 no longer catches all tracebacks but only AttributeError, and we use both hasattr and
BuildEnvironmentNotFoundException to guard against a handful of buildconfig variables in a few places
where it is OK to not have a buildenvironment.
We also use universal_newlines in real_host in init.configure (since I found
that fix while working on the AttributeError one) so that we get the right string type back from the process call
Lastly this patch also uses BytesIO for calling into a ReducedConfigureSandbox as its stdout and stderr pipes,
This ensures that related code handling the sandbox doesn't complain about getbuffer() missing in StringIO in py3.
Differential Revision: https://phabricator.services.mozilla.com/D36605
--HG--
extra : moz-landing-system : lando
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
By default, the linker chooses a "generic" 32-bit CPU to optimize for,
and LLVM's "generic" 32-bit CPU model doesn't include some features that
are helpful for performance on microbenchmarks. We explicitly specify a
CPU model to ensure the model we want is selected.
On x86-64, we explicitly force a generically good processor model, even
though the automatically selected one didn't seem to hurt benchmarks.
Differential Revision: https://phabricator.services.mozilla.com/D40479
--HG--
extra : moz-landing-system : lando
Bug 1546136 wrapped subprocess execution output to capture cmake's, but
at the detriment of other output, and hiding everything unless an error
occurs.
So instead, we only capture the output when the called process is cmake,
and even when it is cmake, we don't pipe stderr at all (since we only
care about cmake's stdout) and we print out stdout as it comes in rather
than later. We then later check the output for hints at the more useful
cmake logs and dump them.
While here, add verbosity to ninja output (which gives the command
lines, rather than generic "Building foo.o" output).
Differential Revision: https://phabricator.services.mozilla.com/D40142
This change gives us the ability to selectively turn on cross-language
PGO, just like we have the ability to selectively turn on cross-language LTO.
There is room for things to go wrong here: it's not guaranteed that
`--enable-profile-generate=cross` will always be used with
`--enable-profile-use=cross`. Nothing bad will happen in the sense that
the build will succeed, but it's possible that we miss out on
optimizations on the Rust side. Either we fail to generate profile data
for the Rust code, or the Rust compiler fails to use the profile data.
In the future, we may want to default to cross-language PGO to avoid
these kind of mismatches.
Differential Revision: https://phabricator.services.mozilla.com/D39727
--HG--
extra : moz-landing-system : lando
To do properly checks on LLVM version correspondence between `clang` and
`rustc`, we need information about both of those compilers to be
available. The current placement of the LTO/PGO checks is after we know
something about `clang`, but before we know something about `rustc`.
Therefore we need to move those checks after we've gathered information
about `rustc`.
The PGO bits come along for this bug because the LTO bits depend on
them, and we're going to need the Rust information for cross-language
PGO checks in a different bug. So we might as well move everything all
at once.
Differential Revision: https://phabricator.services.mozilla.com/D39390
--HG--
rename : build/moz.configure/toolchain.configure => build/moz.configure/lto-pgo.configure
extra : moz-landing-system : lando
This change will eventually enable us to cross-check `rustc`'s version
with `clang`'s version when doing cross-language LTO/PGO and avoid
people running into peculiar errors at link time.
Differential Revision: https://phabricator.services.mozilla.com/D39388
--HG--
extra : moz-landing-system : lando
Plenty of places use `nproc`, and only a couple use `getconf
_NPROCESSORS_ONLN`. Use the former instead of the latter.
Differential Revision: https://phabricator.services.mozilla.com/D39999
--HG--
extra : moz-landing-system : lando
On Windows in Python 2, the subprocess module requires the use of bytes with
the 'env' argument. For that reason, we would sometimes use byte strings with
'os.environ' like so:
os.environ[b"FOO"] = b"bar"
However, this is a failure with Python 3 as 'os.environ' must only be used with
the text type. This patch creates a new 'setenv' helper that ensures we create
new environment with 'bytes' on Python 2, and 'text' on Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D38237
--HG--
extra : moz-landing-system : lando
These tests are based on `moz_global_class` and `moz_trivial_ctor_dtor` tests,
respectively, but adapted for the semantics of the new attributes.
Differential Revision: https://phabricator.services.mozilla.com/D39718
--HG--
rename : build/clang-plugin/tests/TestGlobalClass.cpp => build/clang-plugin/tests/TestStaticLocalClass.cpp
rename : build/clang-plugin/tests/TestTrivialCtorDtor.cpp => build/clang-plugin/tests/TestTrivialDtor.cpp
extra : moz-landing-system : lando
This patch is in support of adding a variant of Static{Auto,Ref}Ptr for use as
static locals, taking advantage of C++11 "magic statics" such that we can lazily
initialize those variables in a thread-safe way.
In support of those classes, this patch adds two new attributes:
* `moz_static_local_class` to ensure that any instantiations of that class only
occur as static local variables;
* `moz_trivial_dtor` to ensure that these classes do not implicitly call `atexit`
and add a whole bunch of shutdown crap.
`moz_static_local_class` works similarly to `moz_global_class`, except that its
object must only instantiate as static locals.
`TrivialDtorChecker` is based on `TrivialCtorDtorChecker`, with the ctor-specific
bits removed.
Differential Revision: https://phabricator.services.mozilla.com/D39717
--HG--
rename : build/clang-plugin/TrivialCtorDtorChecker.cpp => build/clang-plugin/TrivialDtorChecker.cpp
rename : build/clang-plugin/TrivialCtorDtorChecker.h => build/clang-plugin/TrivialDtorChecker.h
extra : moz-landing-system : lando
Toolchain path for Windows version is `<NDK ROOT>/toolchains/llvm/prebuilt/windows-x86_64` etc, so it isn't '`winnt`.
So we has to replace `host.kernel.lower()` with `windows`.
Differential Revision: https://phabricator.services.mozilla.com/D39474
--HG--
extra : moz-landing-system : lando
...rather than people running into peculiar crashes running their tests
because functions are pointing at the wrong thing.
It would be more robust to version-check `ld`, but I figure people
wanting to do local cross-language LTO builds is rare enough that
setting an environment variable and rerunning configure is not a huge
hardship.
Differential Revision: https://phabricator.services.mozilla.com/D36742
--HG--
extra : moz-landing-system : lando
...rather than people running into peculiar crashes running their tests
because functions are pointing at the wrong thing.
It would be more robust to version-check `ld`, but I figure people
wanting to do local cross-language LTO builds is rare enough that
setting an environment variable and rerunning configure is not a huge
hardship.
Differential Revision: https://phabricator.services.mozilla.com/D36742
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
This build re-uses the PGO profile from the win64 build in the
win64-aarch64-shippable-no-eme part of the aarch64 build. Even though
the profile isn't generated on the smae platform, we still get enough of
a performance win to make this worthwhile.
Note that the pgo_flags() in configure need to be tweaked slightly since
we don't supprt the -fprofile-generate flag for aarch64 (we don't build
the clang_rt.profile lib there). So we always want to return the flags
namespace to make sure we get the use_* versions of flags, which we do
need.
Differential Revision: https://phabricator.services.mozilla.com/D38928
--HG--
extra : moz-landing-system : lando
There is less incentive to keep things building with older versions of
clang for OSX builds, and we're going to require an objective-C feature
that was added in clang 5.
Differential Revision: https://phabricator.services.mozilla.com/D38581
--HG--
extra : moz-landing-system : lando
Actually we set _DEPEND_CFLAGS to both host and target compiler. But if host and target are different compiler type, we may pass invalid option.
Differential Revision: https://phabricator.services.mozilla.com/D38457
--HG--
extra : moz-landing-system : lando
This change is a little bit of a cheat, because of course MSVC doesn't
do cross-language LTO by default, but it seems consistent.
Depends on D33317
Differential Revision: https://phabricator.services.mozilla.com/D33318
--HG--
extra : moz-landing-system : lando
When we build mar, there is no reason not to build signmar as well. It
used to be optional because not all platforms were supported, but they
are now.
... except when building the newly added tools/update-packaging,
which builds the mar tool as a standalone thing, and building signmar
as well causes complications.
Differential Revision: https://phabricator.services.mozilla.com/D36992
--HG--
extra : moz-landing-system : lando
This was originally from bug 1528374 for Mac PGO, but that isn't able to
land yet and it should help Windows PGO runs in the meantime.
Differential Revision: https://phabricator.services.mozilla.com/D37807
--HG--
extra : moz-landing-system : lando
Credit: Callek for figuring out an issue in 'make check' making the binary
absolute in mozbuild.base.
Differential Revision: https://phabricator.services.mozilla.com/D37319
--HG--
extra : moz-landing-system : lando
I couldn't use 'mach vendor python' to get this module in-tree because that
puts a 'backports' module on the PYTHONPATH. This is a problem because many
(most?) compatibility backports also use a 'backports' library. So in the
likely event that a user happens to have one of these installed in their
system Python, we'll search that module before the vendored one and fail to
find it.
This gets around the issue by foregoing 'backports' and just putting
'shutil_which' on the path
Differential Revision: https://phabricator.services.mozilla.com/D36838
--HG--
extra : moz-landing-system : lando
When we build mar, there is no reason not to build signmar as well. It
used to be optional because not all platforms were supported, but they
are now.
Differential Revision: https://phabricator.services.mozilla.com/D36992
--HG--
extra : moz-landing-system : lando
When we build mar, there is no reason not to build signmar as well. It
used to be optional because not all platforms were supported, but they
are now.
--disable-verify-mar is kept to still allow to disable mar verification in
the updater for debugging purpose.
Differential Revision: https://phabricator.services.mozilla.com/D36992
--HG--
extra : moz-landing-system : lando
- Part 3 uses functions added in ICU 64, so we need to bump the version requirement.
- Also remove a version check which is now true be default.
Differential Revision: https://phabricator.services.mozilla.com/D26717
--HG--
extra : moz-landing-system : lando
- Part 3 uses functions added in ICU 64, so we need to bump the version requirement.
- Also remove a version check which is now true be default.
Differential Revision: https://phabricator.services.mozilla.com/D26717
--HG--
extra : moz-landing-system : lando
We need a fix from `cctools-port` master for cross-language LTO builds
to work properly on the Mac. Rather than cherry-picking yet another
commit, which would have to deal with a updated `ld64` upstream, we've
opted to go ahead and update directly to upstream.
This choice brings about some significant build changes, as TAPI support
has moved to a different library that is not easily buildable directly.
Differential Revision: https://phabricator.services.mozilla.com/D36636
--HG--
extra : moz-landing-system : lando
As it turns out, the version the minidump-stackwalk tasks use (1.31), is
the first version that actually builds the project: 1.30 and earlier
fail because for some reason, cargo wants to read all Cargo.toml files
in the workspace, including unrelated ones, and barfs on features that
weren't supported until 1.31.
Differential Revision: https://phabricator.services.mozilla.com/D37020
--HG--
extra : moz-landing-system : lando
When building Gecko/Android/aarch64 on Windows, `--target` parameter may not be incorrect value. Although `check_compiler`'s `info` is target compiler, clang on Windows is always detected as `clang-cl`, not `clang`.
```
c:/Users/mkato/.mozbuild/clang/bin/clang.exe -E -dM - < /dev/null
...
#define _MSC_VER 1916
```
So even if using clang on Windows, not clang-cl, we should detect as 'clang' correctly
Differential Revision: https://phabricator.services.mozilla.com/D36422
--HG--
extra : moz-landing-system : lando
When host and target are different compiler type, triple for rustc may be incorrect. If target is clang, host is always clang, not using host compiler type.
Example, when host is clang-cl for windows, and target is clang for Android, host's triple for ructc sets `windows-gnu`, not `windows-msvc`.
Differential Revision: https://phabricator.services.mozilla.com/D36421
--HG--
extra : moz-landing-system : lando
Bug 1451104 removed the GCC 4.9 toolchain, but left the build script and
the patch that only that toolchain build was using.
Differential Revision: https://phabricator.services.mozilla.com/D36886
--HG--
extra : moz-landing-system : lando
When building Gecko/Android/aarch64 on Windows, `--target` parameter may not be incorrect value. Although `check_compiler`'s `info` is target compiler, clang on Windows is always detected as `clang-cl`, not `clang`.
```
c:/Users/mkato/.mozbuild/clang/bin/clang.exe -E -dM - < /dev/null
...
#define _MSC_VER 1916
```
So even if using clang on Windows, not clang-cl, we should detect as 'clang' correctly
Differential Revision: https://phabricator.services.mozilla.com/D36422
--HG--
extra : moz-landing-system : lando
When host and target are different compiler type, triple for rustc may be incorrect. If target is clang, host is always clang, not using host compiler type.
Example, when host is clang-cl for windows, and target is clang for Android, host's triple for ructc sets `windows-gnu`, not `windows-msvc`.
Differential Revision: https://phabricator.services.mozilla.com/D36421
--HG--
extra : moz-landing-system : lando
The SDK headers may not be installed in /usr/include. The usual response
has been to have people run e.g. `open
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg`
which is not really sustainable.
This makes builds that happen on a macOS host try to detect their SDK
and use that as a default for --with-macos-sdk, which has the side
effect of enabling the SDK version check in that configuration.
Differential Revision: https://phabricator.services.mozilla.com/D36558
--HG--
extra : moz-landing-system : lando
Building host tools on macOS require a macOS SDK, but it's currently
not configurable when cross-compiling for e.g. Android.
Differential Revision: https://phabricator.services.mozilla.com/D36557
--HG--
extra : moz-landing-system : lando
The SDK headers may not be installed in /usr/include. The usual response
has been to have people run e.g. `open
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg`
which is not really sustainable.
This makes builds that happen on a macOS host try to detect their SDK
and use that as a default for --with-macos-sdk, which has the side
effect of enabling the SDK version check in that configuration.
Differential Revision: https://phabricator.services.mozilla.com/D36558
--HG--
extra : moz-landing-system : lando
Building host tools on macOS require a macOS SDK, but it's currently
not configurable when cross-compiling for e.g. Android.
Differential Revision: https://phabricator.services.mozilla.com/D36557
--HG--
extra : moz-landing-system : lando
LLVM r356397 added some new warnings to the compiler that in this file complain about `size argument is too large; destination buffer has size 100, but size argument is 101`.
The easiest path is to just remove those lines; doing so doesn't really weaken the test coverage of the SprintfLiteral checker.
Differential Revision: https://phabricator.services.mozilla.com/D36515
--HG--
extra : moz-landing-system : lando
LLVM r356397 added some new warnings to the compiler that in this file complain about `size argument is too large; destination buffer has size 100, but size argument is 101`.
The easiest path is to just remove those lines; doing so doesn't really weaken the test coverage of the SprintfLiteral checker.
Differential Revision: https://phabricator.services.mozilla.com/D36515
--HG--
extra : moz-landing-system : lando
I don't know if this covers all the things that use mozinfo (probably not)
but it covers all the suites that use mozinfo and have webrender conditions
in the test manifests (i.e. mochitest and wpt variants).
Differential Revision: https://phabricator.services.mozilla.com/D35869
--HG--
extra : moz-landing-system : lando
The condition that enabled ELOG to do something has been broken since
bug 279212 landed two years ago, because MAKEFLAGS doesn't contain the
dash for flags, so looking for "-s" never did anything.
Bug 1560527 changed the condition to !BUILD_VERBOSE_LOG, which as of the
previous commit, does work. This would restore the old behavior, at the
expense of build time, since it involves wrapping in a shell script,
which is not necessarily desirable (and hasn't been done for two years
now)
Differential Revision: https://phabricator.services.mozilla.com/D35967
--HG--
extra : moz-landing-system : lando
With clang-cl and PGO enabled, toolchain.configure automatically turns
on LTO for compatibility with MSVC. However, MOZ_PGO is set for both the
profile-generate and profile-use builds in a 3-tier PGO setup via
imply_option(), but we only want LTO enabled for the profile-use build
(see bug 1483778).
For 1-tier PGO builds, which are still used by local developers, MOZ_PGO
will be set and --enable-profile-generate will be unset, so LTO will be
automatically enabled. The profiledbuild target in make is responsible
for disabling MOZ_LTO on the profile-generate build.
For 3-tier PGO builds, MOZ_PGO will still be set, so we can skip setting
LTO in configure when --enable-profile-generate is set.
Differential Revision: https://phabricator.services.mozilla.com/D34800
--HG--
extra : moz-landing-system : lando
Windows finds llvm-profdata in the PATH, in contrast to Linux or Android
builds that set LLVM_PROFDATA as an environment variable in mozconfigs.
The pgo_profile_path() configure checks should still work in this case.
Differential Revision: https://phabricator.services.mozilla.com/D34799
--HG--
extra : moz-landing-system : lando
MOZ_PGO_PROFILE_USE is set when the use-pgo attribute is defined in the
task. This environment variable is used to enable --enable-profile-use
and related configure flags.
Differential Revision: https://phabricator.services.mozilla.com/D34798
--HG--
extra : moz-landing-system : lando
In the clang-cl 9 case where we have semicolon-separated paths, the `dirs` variable is never read, so the libclang detection fails.
Differential Revision: https://phabricator.services.mozilla.com/D35825
--HG--
extra : moz-landing-system : lando
This is a simple mach command that imports a PR from a whitelisted set
of github repositories into the local m-c clone. It works by downloading
the .patch file from github, splitting the different commits, and
applying those commits to the local repo via the `patch` tool and git/hg
commit. It optionally allows filing a bug or providing a bug number, and
specifying a reviewer.
This is one part of a larger workflow that facilitates landing
contributor patches into m-c when those patches are submitted as PRs.
Other components of the workflow (to be added in the future) will make
it easier to actually test and land the patch.
Differential Revision: https://phabricator.services.mozilla.com/D35206
--HG--
extra : moz-landing-system : lando
- `info.type in ('clang-cl', 'clang', 'gcc')` is always true since MSVC
support was removed.
- For some reason, we didn't enforce C++14 on GCC.
Differential Revision: https://phabricator.services.mozilla.com/D34990
--HG--
extra : moz-landing-system : lando
Improve support for Python 3 by running modernize over adb.py and a few other related changes.
Differential Revision: https://phabricator.services.mozilla.com/D19306
--HG--
extra : moz-landing-system : lando
We've been relying on frame pointers being indirectly enabled via things
like --enable-profiling for some time, but this doesn't scale because
some things may want frame pointers while wanting --disable-profiling.
So we move MOZ_FRAMEPTR_FLAGS to python configure and add a new option
to decide whether to enable frame pointers or not.
Differential Revision: https://phabricator.services.mozilla.com/D34117
--HG--
extra : moz-landing-system : lando
Bug 1394825 bumped the minimum version of clang we use on automation,
for the base toolchain jobs, and there's a libclang test for bindgen,
but we should reject the compiler in the first place.
The check to do that works in both C and C++, contrary to the original
3.6 test that was checking a C++ feature ; the test was later changed
for 3.9, using a check that would have worked in both C and C++, but
the C exception was left around. We remove it now.
Differential Revision: https://phabricator.services.mozilla.com/D34083
--HG--
extra : moz-landing-system : lando
Stand-alone JS builds now default to without-NSPR on all platforms.
Note that the JS shell builds we do in automation pass --enable-nspr-build so they shouldn't be affected by
the JS shell changes.
Differential Revision: https://phabricator.services.mozilla.com/D33933
--HG--
extra : moz-landing-system : lando
Use the same logic as for --with-android-sdk to use the NDK downloaded
by mach bootstrap by default.
Differential Revision: https://phabricator.services.mozilla.com/D34251
--HG--
extra : moz-landing-system : lando
We make it have a default value only if the corresponding directory exists,
and make it throw a more explicit error when the explicitly given directory
doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D34250
--HG--
extra : moz-landing-system : lando
The current logic is that if the compiler somehow uses gold rather than
BFD ld by default, we let it, but if it uses BFD ld, we default to lld.
When doing Android builds, the compiler finds the linker in the NDK, and
the default `ld` binary is gold. So we currently end up using fold for
Android builds.
Here, we change the logic such that we use lld when the default linker
the compiler uses is either BFD ld or gold. We can't go with "is not
lld" because the other possible kind, ld64, is what we actually want to
use on mac, since lld doesn't support mach-o fully just yet.
Differential Revision: https://phabricator.services.mozilla.com/D34040
--HG--
extra : moz-landing-system : lando
The current logic is that if the compiler somehow uses gold rather than
BFD ld by default, we let it, but if it uses BFD ld, we default to lld.
When doing Android builds, the compiler finds the linker in the NDK, and
the default `ld` binary is gold. So we currently end up using gold for
Android builds.
Here, we change the logic such that we use lld when the default linker
the compiler uses is either BFD ld or gold. We can't go with "is not
lld" because the other possible kind, ld64, is what we actually want to
use on mac, since lld doesn't support mach-o fully just yet.
Differential Revision: https://phabricator.services.mozilla.com/D34040
--HG--
extra : moz-landing-system : lando
https://reviews.llvm.org/rL360311 reworked the handling of `explicit` in `CXXConversionDecl`, and the `isExplicitSpecified()` method no longer exists. We can instead use `isExplicit()` which conveniently works on both old and new clangs. (Before 360311, `isExplicit()` just forwarded to `isExplicitSpecified()`.)
Differential Revision: https://phabricator.services.mozilla.com/D33958
--HG--
extra : moz-landing-system : lando
Valgrind report:
```
TEST-UNEXPECTED-FAIL | valgrind-test | 192 bytes in 3 blocks are definitely lost at malloc / dl_open_worker / _dl_catch_error / _dl_open
==2561== 192 bytes in 3 blocks are definitely lost in loss record 348 of 399
==2561== at 0x4C2B240: malloc+112 (vg_replace_malloc.c:298)
==2561== by 0x4012919: dl_open_worker+1977 (dl-open.c:457)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x4011CC9: _dl_open+185 (dl-open.c:633)
==2561== by 0x5051F65: dlopen_doit+101 (dlopen.c:67)
==2561== by 0x400DD55: _dl_catch_error+101 (dl-error.c:178)
==2561== by 0x50522EB: _dlerror_run+123 (dlerror.c:164)
==2561== by 0x5051EE0: dlopen@@GLIBC_2.2.5+48 (dlopen.c:88)
==2561== by 0x1148FC: GetLibHandle (xpcom/glue/standalone/nsXPCOMGlue.cpp:86)
==2561== by 0x1148FC: ReadDependentCB (xpcom/glue/standalone/nsXPCOMGlue.cpp:136)
==2561== by 0x1148FC: XPCOMGlueLoad (xpcom/glue/standalone/nsXPCOMGlue.cpp:306)
==2561== by 0x1148FC: mozilla::GetBootstrap(char const*, mozilla::LibLoadingStrategy)+572 (xpcom/glue/standalone/nsXPCOMGlue.cpp:374)
==2561== by 0x114213: InitXPCOMGlue(mozilla::LibLoadingStrategy)+131 (browser/app/nsBrowserApp.cpp:223)
==2561== by 0x113E6B: main+219 (browser/app/nsBrowserApp.cpp:284)
```
Nothing due directly to this patch, so it is likely a dlopen issue.
Differential Revision: https://phabricator.services.mozilla.com/D33904
--HG--
extra : moz-landing-system : lando
The current setup for bindgen relies on either finding clang/libclang
from the output of llvm-config, or from the paths given via the
configure flags --with-clang-path/--with-libclang-path.
One _very_ common problem is that the llvm-config we end up using does
not correspond to the clang used for compilation, which has some
undesirable side effect, like failing to build.
So instead of relying on llvm-config, we do the following:
- when the compiler is clang, we just use that
- when the compiler is clang-cl, we use clang from the same directory
- otherwise, we either try to find clang in PATH, or rely on
--with-clang-path.
Once clang is found, we try to deduce the location of the corresponding
libclang via the output of `clang -print-search-dirs`, or rely on
--with-libclang-path.
Differential Revision: https://phabricator.services.mozilla.com/D33241
--HG--
extra : moz-landing-system : lando
Inheriting constructors are implicitly introduced via a using-declaration.
Since the C++ grammar doesn't allow attributes on using-declarations, it
is currently impossible to add MOZ_IMPLICIT to implicit inheriting
constructors.
This commit changes the AST matcher such that it ignores inheriting
constructors altogether. If they are inheriting from an implicit inherited
constructor, that constructor's check should be enough to ensure that no
constructors are unintentionally implicit.
Differential Revision: https://phabricator.services.mozilla.com/D33281
--HG--
extra : moz-landing-system : lando
This code was already dead during the VS2017 era (since that compiler changed to the `Hostx64\x86` path scheme), let alone clang-cl.
Differential Revision: https://phabricator.services.mozilla.com/D33513
--HG--
extra : moz-landing-system : lando
The 3-tier PGO builds used a separate mozconfig called 'profile-use' for
the final tier. This created a problem when it rode to beta, since the
same mozconfig was used for all trees, which meant we ended up with
nightly branding on beta builds.
With the PGO-enabling logic in common mozconfigs, we can enable it by
setting the MOZ_PGO_PROFILE_USE environment variable from the task
definition. All of the final-tier PGO builds now use the nightly, beta,
etc mozconfigs like before, so branding should be intact.
Differential Revision: https://phabricator.services.mozilla.com/D33172
--HG--
extra : moz-landing-system : lando
This change is a little bit of a cheat, because of course MSVC doesn't
do cross-language LTO by default, but it seems consistent.
Depends on D33317
Differential Revision: https://phabricator.services.mozilla.com/D33318
--HG--
extra : moz-landing-system : lando
