Commit Graph

8401 Commits

Author SHA1 Message Date
Dustin J. Mitchell
be3fd44f98 Bug 1562686 - revert remaining unnecessary bit of bug 1187245; r=glandium
Depends on D41454

Differential Revision: https://phabricator.services.mozilla.com/D42644

--HG--
extra : moz-landing-system : lando
2019-08-21 03:32:05 +00:00
Dustin J. Mitchell
13f9bc563c Bug 1562686 - use AWS_IAM_CREDENTIALS_URL for all S3 sccache invocations r=chmanchester
Differential Revision: https://phabricator.services.mozilla.com/D41454

--HG--
extra : moz-landing-system : lando
2019-08-20 14:03:12 +00:00
Daniel Varga
b703653c18 Merge mozilla-central to autoland. On a CLOSED TREE
--HG--
extra : rebase_source : ae31c873768b4a95fc6574d1bfbe7d17125bb40e
2019-08-21 14:45:21 +03:00
Daniel Varga
8b04757a32 Backed out changeset e533d2907a31 (bug 1564252) for build bustage at /lib/gcc/x86_64-unknown-linux-gnu/6.4.0. a=backout 2019-08-21 14:35:03 +03:00
Daniel Varga
ae8445aebc Merge mozilla-central to autoland. On a CLOSED TREE 2019-08-21 13:06:00 +03:00
Sylvestre Ledru
16b33cea9a Bug 1564252 - Move to clang 8.0.1 r=glandium
Remove r355141-arm64-cfg.patch (merged upstream)

Differential Revision: https://phabricator.services.mozilla.com/D42325

--HG--
extra : rebase_source : f74202e4459c46d45a8b4f5a8c00da080904e8bf
2019-08-20 21:06:24 +02:00
Chris Peterson
41241f3418 Bug 1514965 - Part 2: Enable clang -ftrivial-auto-var-init to initialize local variables with 0xAA in debug builds. r=froydnj
Disable -ftrivial-auto-var-init for DllBLocklistWin.cpp with clang-cl because the file's interceptions happen so early in the main process that the loader hasn't yet resolved the import of memset (used by -ftrivial-auto-var-init) from vcruntime140.dll.

Disable -ftrivial-auto-var-init on Linux32 because it causes some xpcshell test failures.

Differential Revision: https://phabricator.services.mozilla.com/D42273

--HG--
extra : moz-landing-system : lando
2019-08-21 07:08:29 +00:00
Chris Peterson
539da64637 Bug 1514965 - Part 1: Refactor mingw_clang checks for reuse. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D27358

--HG--
extra : moz-landing-system : lando
2019-08-21 07:08:04 +00:00
Cameron Kaiser
efe5f0901c Bug 1571613 - basic support for Power ISA SIMD detection (v2). r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D42592

--HG--
extra : moz-landing-system : lando
2019-08-21 03:36:55 +00:00
Mike Hommey
e0be5605ec Bug 1575135 - Return an unicode path for nodejs. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42606

--HG--
extra : moz-landing-system : lando
2019-08-20 21:24:35 +00:00
Mike Hommey
00256d3837 Bug 1575135 - Make configure sandbox open() look more like python 3's. r=nalexander
As a consequence, we can replace the encoded_open function that did the
same in an opt-in manner.

Differential Revision: https://phabricator.services.mozilla.com/D42605

--HG--
extra : moz-landing-system : lando
2019-08-21 04:44:43 +00:00
Mike Hommey
16c5b7db29 Bug 1575135 - Change get_cmd_output to emit unicode strings. r=nalexander
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.

Places that explicitly do want bytes can keep using subprocess directly.

Differential Revision: https://phabricator.services.mozilla.com/D42604

--HG--
extra : moz-landing-system : lando
2019-08-20 16:43:15 +00:00
Mike Hommey
cce0006157 Bug 1575135 - Use functions from util.configure instead of subprocess.*. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42603

--HG--
extra : moz-landing-system : lando
2019-08-20 16:42:16 +00:00
Mike Hommey
cc52fc2e1a Bug 1575135 - Define whether the system encoding is mbcs or utf-8 once. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42601

--HG--
extra : moz-landing-system : lando
2019-08-20 16:40:24 +00:00
Mike Hommey
76e02a9e5d Bug 1575135 - Remove unused subprocess imports. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D42600

--HG--
extra : moz-landing-system : lando
2019-08-20 16:39:30 +00:00
Mike Hommey
2e9307d967 Bug 1574761 - Avoid race condition creating old-configure. r=nalexander
This is not something that happens under normal circumstances, but it
can happen when you go fancy and run multiple configures in parallel
with different objdirs, and old-configure doesn't exist in the first
place ; then one configure may overwrite old-configure while another is
starting to execute it, resulting in the latter nor executing
old-configure completely.

Differential Revision: https://phabricator.services.mozilla.com/D42428

--HG--
extra : moz-landing-system : lando
2019-08-19 23:21:27 +00:00
Nathan Froyd
e03796ae1a Bug 1572724 - remove unused URL_REPO definition from build-clang.py; r=glandium
This definition is now unnecessary, given that the source code is
fetched for us from elsewhere.

Depends on D41368

Differential Revision: https://phabricator.services.mozilla.com/D41369

--HG--
extra : moz-landing-system : lando
2019-08-09 21:12:31 +00:00
Nathan Froyd
10d28350d8 Bug 1572724 - preserve symlinks when installing libgcc; r=glandium
Otherwise we wind up with clownshoes like:

```
froydnj@hawkeye:/opt/build/froydnj/tmp/clang$ ls -l lib/libstdc++.so*
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug  6 20:44 lib/libstdc++.so
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug  6 20:44 lib/libstdc++.so.6
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug  6 20:44 lib/libstdc++.so.6.0.24
```

and have duplicate copies of shared libraries in our toolchain packages,
which are not exactly small.

Differential Revision: https://phabricator.services.mozilla.com/D41368

--HG--
extra : moz-landing-system : lando
2019-08-09 21:11:31 +00:00
David Major
0db5cad5ae Bug 1574565 - Whitelist __cxx_atomic for memmove static analysis r=Ehsan
In some libstdc++ these appear in the inheritance hierarchy of __atomic_base, which is already whitelisted.

Differential Revision: https://phabricator.services.mozilla.com/D42348

--HG--
extra : moz-landing-system : lando
2019-08-16 23:01:03 +00:00
Mike Hommey
312ec28e99 Bug 1573566 - Move the real libxul definition in a subdirectory. r=froydnj
The current setup, where gtest/libxul uses the static library in
the same directory as the shared libxul, and somehow the backend ignores
gkrust for gtest/libxul, is fragile.

Differential Revision: https://phabricator.services.mozilla.com/D42246

--HG--
rename : toolkit/library/dependentlibs.py => toolkit/library/build/dependentlibs.py
extra : moz-landing-system : lando
2019-08-16 21:44:10 +00:00
Razvan Maries
cff855aa9c Backed out changeset 91eca815c9fc (bug 1562686) for build bustages. CLOSED TREE 2019-08-16 22:38:19 +03:00
Dustin J. Mitchell
ad85abed68 Bug 1562686 - use AWS_IAM_CREDENTIALS_URL for all S3 sccache invocations r=chmanchester
Differential Revision: https://phabricator.services.mozilla.com/D41454

--HG--
extra : moz-landing-system : lando
2019-08-16 17:17:02 +00:00
Andreas Tolfsen
8350dc3134 bug 1540655: build, remote: add mach command for vendoring Puppeteer; r=firefox-build-system-reviewers,chmanchester
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.

The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.

The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/.  We also
wouldn't want other code in central to accidentally use a patched fork.

The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.

Differential Revision: https://phabricator.services.mozilla.com/D37007

--HG--
extra : moz-landing-system : lando
2019-08-16 12:58:06 +00:00
Andreas Tolfsen
1c7db08de2 bug 1540655: build: sort MACH_MODULES; r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D37006

--HG--
extra : moz-landing-system : lando
2019-08-16 12:58:05 +00:00
Mike Hommey
c173540215 Bug 1573435 - Use toolchain fetches for all remaining toolchain uses. r=nalexander
The remaining uses all need adjustements to in-tree mozconfigs, so they
all need to be done at once.

However, to make things slightly more intelligible, we do this in two
steps. This is step 1: we modify the use_toolchain transform to take care of
the transformation, while keeping the task definitions intact, so that
we only deal with mozconfig and build script adjustements here.

Differential Revision: https://phabricator.services.mozilla.com/D41890
2019-08-15 11:21:52 +09:00
Mike Hommey
639908c191 Bug 1573378 - Make build-clang independent of what MOZ_FETCHES_DIR resolves to. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D41711
2019-08-15 11:21:42 +09:00
Ciure Andrei
8a1785a6cc Backed out 11 changesets (bug 1540655) for test_resolve.py perma failures CLOSED TREE
Backed out changeset 1a23d770d8a1 (bug 1540655)
Backed out changeset 641a7cb25298 (bug 1540655)
Backed out changeset 73236f81da44 (bug 1540655)
Backed out changeset 8d7bad30be46 (bug 1540655)
Backed out changeset bb012df3018b (bug 1540655)
Backed out changeset 8c67b494e207 (bug 1540655)
Backed out changeset c0a80d37576d (bug 1540655)
Backed out changeset 939ce2afcf0b (bug 1540655)
Backed out changeset 3b3a2a9fbc8b (bug 1540655)
Backed out changeset b96dede008ad (bug 1540655)
Backed out changeset 997d1568d944 (bug 1540655)
2019-08-14 18:53:36 +03:00
Andreas Tolfsen
1c16637ab1 bug 1540655: build, remote: add mach command for vendoring Puppeteer; r=firefox-build-system-reviewers,chmanchester
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.

The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.

The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/.  We also
wouldn't want other code in central to accidentally use a patched fork.

The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.

Differential Revision: https://phabricator.services.mozilla.com/D37007

--HG--
extra : moz-landing-system : lando
2019-08-14 14:57:51 +00:00
Andreas Tolfsen
e24280acf0 bug 1540655: build: sort MACH_MODULES; r=firefox-build-system-reviewers,mshal
Differential Revision: https://phabricator.services.mozilla.com/D37006

--HG--
extra : moz-landing-system : lando
2019-08-14 14:57:49 +00:00
Nathan Froyd
08b51223a9 Bug 1560666 - turn off C++17 aligned allocation support; r=glandium
Just like C++14 sized deallocation support, we don't want to support
this.  We shouldn't be using `new` on over-aligned types anyway.

Differential Revision: https://phabricator.services.mozilla.com/D41819

--HG--
extra : moz-landing-system : lando
2019-08-14 01:37:34 +00:00
Sylvestre Ledru
0dc8a7e274 Bug 1573769 - Update of the build-clang doc after the move to git r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D41910

--HG--
extra : moz-landing-system : lando
2019-08-14 07:40:58 +00:00
Nicholas Nethercote
4e52c0f072 Bug 1573080 - Fix some incorrect preprocessor.py docs. r=glandium
As shown by python/mozbuild/mozbuild/test/test_preprocessor.py, whitespace is
allowed within expressions, and chained #if/#elif/#else sequences work as you'd
expect.

Differential Revision: https://phabricator.services.mozilla.com/D41494

--HG--
extra : moz-landing-system : lando
2019-08-13 22:20:04 +00:00
Nathan Froyd
9fc6f1be89 Bug 1573601 - remove tabs in toolchain.configure; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D41800

--HG--
extra : moz-landing-system : lando
2019-08-13 18:12:54 +00:00
André Bargull
268981bb20 Bug 1539780: Remove "--with-intl-api=build" build config option. r=jwalden
There are about the same number of occurrences of "ENABLE_INTL_API" and "EXPOSE_INTL_API"
in the tree, so preferring one over the other doesn't lead to fewer changes. Therefore
I went with "ENABLE_INTL_API", because "ENABLE_" (resp. "MOZ_ENABLE") is already used as
the prefix for other preprocessor ifdef's.

Differential Revision: https://phabricator.services.mozilla.com/D41188

--HG--
extra : moz-landing-system : lando
2019-08-09 19:43:19 +00:00
Brendan Dahl
b474db77c6 Bug 1551344 - Part 1: Remove XULDocument code. r=smaug,Jamie
All .xul files have been loading as HTMLDocuments for a few weeks now, so
it should be safe to remove the XULDocument implementation.

Differential Revision: https://phabricator.services.mozilla.com/D41238

--HG--
extra : moz-landing-system : lando
2019-08-09 19:57:50 +00:00
Nathan Froyd
3608ab49b8 Bug 1572216 - move LTO defaulting into mozconfig.win-common; r=glandium
Some recent changes to how we set cross-language LTO for Windows
resulted in compilation-time decreases and small performance regressions
on a few benchmarks.  The changes intended to remove explicit enablement
of cross-language LTO for all builds, but rely on shippable builds being
built with PGO and moz.configure's clever defaulting of cross-language
LTO for PGO'd builds on Windows, which would then enable cross-language
LTO for only shippable builds.

Obviously something went wrong with those changes.

The problem was our defaulting wasn't visible to moz.configure's logic
for how to pass command-line options to the JS subconfigure.  We set the
value (`cross`) after the value for `--enable-lto` has been determined,
and the default value is off (that is, `--disable-lto`).  Since
moz.configure is very thorough in passing configure options down into
JS, it dutifully looked at what the default value of `--enable-lto` was
supposed to be, and passed `--disable-lto` to JS's configure.

There's some evidence that we knew our defaulting was a little sketchy:
we'd only attempt cross-language LTO when we were performing the PGO use
phase, and only if the value of `--enable-lto` wasn't explicitly passed.
Which was a fine idea--you don't want to override what the user was
trying to do--but in the case of JS backfired on us: the value *was*
coming from the explicitly-passed command-line option of
`--disable-lto`.  So JS didn't enable any kind of LTO, with attendant
consequences.

This problem *didn't* happen before the aforementioned change because we
were explicitly specifying that `--enable-lto=cross` should be passed in
the mozconfig, which ensured that the correct setting was passed into
JS.  We were just setting `--enable-lto=cross` for *all* builds, which
was less than desirable.

The easiest way to fix all this is simply to put the
`--enable-lto=cross` setting in the Windows mozconfigs, conditional on
`MOZ_PGO_PROFILE_USE`.  That placement captures the intent of the
previous attempt at defaulting, but without the troubles described
above: the option explicitly appears on the command line, and
moz.configure will correctly pass it through to the JS subconfigure.
This also makes our Windows configuration closer to our Linux
configuration (the Linux configuration enables cross-language LTO for
both PGO phases, which is arguably a bug).

Differential Revision: https://phabricator.services.mozilla.com/D41080

--HG--
extra : moz-landing-system : lando
2019-08-09 13:22:08 +00:00
Mike Hommey
ee928b205d Bug 1572363 - Add a sparse profile for webrender tasks. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D41152

--HG--
extra : moz-landing-system : lando
2019-08-08 13:47:51 +00:00
Justin Wood
605aa0fa02 Bug 1473498 - More support for py3. r=firefox-build-system-reviewers,mshal
This patch makes BuildEnvironmentNotFoundException a subclass of AttributeError as well, because hasattr in
python3 no longer catches all tracebacks but only AttributeError, and we use both hasattr and
BuildEnvironmentNotFoundException to guard against a handful of buildconfig variables in a few places
where it is OK to not have a buildenvironment.

We also use universal_newlines in real_host in init.configure (since I found
that fix while working on the AttributeError one) so that we get the right string type back from the process call

Lastly this patch also uses BytesIO for calling into a ReducedConfigureSandbox as its stdout and stderr pipes,
This ensures that related code handling the sandbox doesn't complain about getbuffer() missing in StringIO in py3.

Differential Revision: https://phabricator.services.mozilla.com/D36605

--HG--
extra : moz-landing-system : lando
2019-08-06 21:26:54 +00:00
Mike Hommey
375094b924 Bug 1571596 - Repack GCC and related source tarballs. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D40748
2019-08-07 13:54:27 +09:00
Mike Hommey
392c0b5ec8 Bug 1571576 - Flush stderr before running subprocesses in build-clang. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D40728
2019-08-07 13:54:22 +09:00
Mike Hommey
9cfb69a1b0 Bug 1571566 - Fix cmake error handling in build-clang.py after python3 conversion. r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D40720
2019-08-07 13:54:21 +09:00
Mike Hommey
0d49eb3466 Bug 1571562 - Make tooltool-download.sh download and extract to MOZ_FETCHES_DIR. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D40712
2019-08-07 13:54:18 +09:00
Mike Hommey
034e9b6b7b Bug 1570541 - Use git fetch tasks for clang. r=froydnj
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).

In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.

This also means we don't need to download git for the windows toolchain
task.

Differential Revision: https://phabricator.services.mozilla.com/D40402
2019-08-07 13:54:15 +09:00
Eric Rahm
26bdfd6480 Bug 1571836 - Enable Rust PGO on Linux64. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D40858

--HG--
extra : moz-landing-system : lando
2019-08-06 22:26:49 +00:00
Nicholas Nethercote
28a0265183 Bug 1569864 - Enable PHC on Win64 Nightly builds. r=glandium
Depends on D39841

Differential Revision: https://phabricator.services.mozilla.com/D39842

--HG--
extra : moz-landing-system : lando
2019-08-03 00:20:09 +00:00
Bogdan Tara
0ffa9e372d Merge inbound to mozilla-central. a=merge 2019-08-03 12:47:05 +03:00
Mike Hommey
9b84c9014f Bug 1570598 - Pass the clang json file as an argument to the toolchain script. r=froydnj
Make the argument use the same format as resources, so move the
sub-script invocation accordingly.

Differential Revision: https://phabricator.services.mozilla.com/D40364
2019-08-03 07:08:49 +09:00
Nathan Froyd
909e9c6f30 Bug 1568450 - explicitly specify a cpu for LTO linking on Windows; r=dmajor
By default, the linker chooses a "generic" 32-bit CPU to optimize for,
and LLVM's "generic" 32-bit CPU model doesn't include some features that
are helpful for performance on microbenchmarks.  We explicitly specify a
CPU model to ensure the model we want is selected.

On x86-64, we explicitly force a generically good processor model, even
though the automatically selected one didn't seem to hurt benchmarks.

Differential Revision: https://phabricator.services.mozilla.com/D40479

--HG--
extra : moz-landing-system : lando
2019-08-02 20:43:52 +00:00
Mike Hommey
c638ab3c85 Bug 1570796 - Use a fetch task for hfsplus-tools source code. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D40329

MANUAL PUSH: avoid closing autoland while all docker images and
toolchains are rebuilt.
2019-08-02 19:07:06 +09:00
Mike Hommey
6e5fc78628 Bug 1570564 - Convert build-clang.py to python 3. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D40152
2019-08-02 19:06:20 +09:00