Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.
Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).
Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).
Differential Revision: https://phabricator.services.mozilla.com/D6117
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
CCTools-port needs to be rebuilt in some cases of clang changes, which
this one of, so touch the script so that it happens.
Differential Revision: https://phabricator.services.mozilla.com/D6117
Not waiting for rc3 or final from llvm because the issue is pretty severe
Differential Revision: https://phabricator.services.mozilla.com/D5124
--HG--
extra : moz-landing-system : lando
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
For some reason, clang 6 crashes with a stack overflow on PGO + LTO on
Linux 64 bits. Clang 7 doesn't, but has other problems.
After some bisecting, I found the following:
- r322684 is the first revision that is broken on the release_60 branch.
- that revision is a cherry pick of r322313 from trunk, which is
similarly broken.
- trunk was fixed by r322325, which, funnily enough, predates when
r322313 was cherry-picked.
While the change from r322325 is relatively large, mixing multiple
different changes in a single commit, there also haven't been
significant changes to the same file on trunk since (one macro name
change, one documentation change, and a change related to debug info),
which would tend to indicate the change is not going to break anything,
or at least not more than upgrading to clang 7 would.
The exact part that fixes the issue could probably be found in this
large commit, but I didn't feel like digging into it further considering
the above.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
https://github.com/rust-lang/rust/issues/52694 is a miscompilation I
found in rust when it uses system llvm 5 or 6, that was fixed 5 months
ago in the llvm rust bundles. This may or may not affect clang, but
considering it was also reported to upstream llvm independently of rust,
it's better to side with caution.
It doesn't affect 3.9, and bug 1478919 got rid of the last use of clang
5 (except for clang-tidy, but that's not used to compile).
The patches come from the llvm trunk from 5 months ago, so they're
already in our clang 7 snapshots.
Windows static analysis builds are still using an old trunk, but are
stuck on bug 1427808. They're "only" for static analysis, though.
--HG--
extra : rebase_source : f4fce69eb7c69b6245518a1bad37e04236c7075b
Instead of clang 4, which they were the last to use, so remove the
clang 4 toolchain.
--HG--
extra : rebase_source : d03a083e9217aeb6c1d2c91decb978426f0e8d1a
svn revert requires a path, and does not take a revision. This isn't an
issue on build machines because we do a fresh checkout every time.
But if you're trying to run build-clang locally, with existing checkouts,
it will:
1) successfully svn update
2) run svn revert, saying "Skipped <rev>"
(except you don't see it because of -q)
3) svn revert returns a successfull eror code
4) patch fails because the file was never reverted and it attempts
to re-apply the patch
Also I think the revert command needs to come first.
MozReview-Commit-ID: 4OfrJNZwJNU
--HG--
extra : rebase_source : b3474e8048b3110f3f5948c3351923c02735ca4d
Note that static analysis was the only remaining user of the 32-bit toolchain, so I've removed win32-clang-cl (or rather, renamed it to win32-clang-cl-st-an).
--HG--
rename : build/build-clang/clang-win32.json => build/build-clang/clang-win32-st-an.json
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-st-an.json
rename : taskcluster/scripts/misc/build-clang32-windows.sh => taskcluster/scripts/misc/build-clang32-st-an-windows.sh
rename : taskcluster/scripts/misc/build-clang64-windows.sh => taskcluster/scripts/misc/build-clang64-st-an-windows.sh
I was able to reproduce the failure to apply llvm-dsymutil on the last
mozilla-central revision before debug info was temporarily disabled for
rust, and validated that the problem was gone in clang 4.
After some bisection, r289565 was identified as having fixed the
problem, which, reading the commit message, makes sense.
It was however not possible to simply cherry-pick, because of multiple
code changes between 3.9 and 4. However, apart from the volume of
conflicting changes, it was more or less straightforward to backport.
Interestingly, the patch for r313872 was relying on changes from
r289565, which is why it required a variant specifically for 3.9, but
now we can use the same patch as for other versions. Well, except
there's a small difference in the context, and build-clang.py doesn't
allow fuzz, so we manually edit the patch to remove that line from the
context.
--HG--
extra : rebase_source : de0ab262d401c37c0e9300b0ef7923a07c009d87
I was able to reproduce the failure to apply llvm-dsymutil on the last
mozilla-central revision before debug info was temporarily disabled for
rust, and validated that the problem was gone in clang 4.
After some bisection, r289565 was identified as having fixed the
problem, which, reading the commit message, makes sense.
It was however not possible to simply cherry-pick, because of multiple
code changes between 3.9 and 4. However, apart from the volume of
conflicting changes, it was more or less straightforward to backport.
Interestingly, the patch for r313872 was relying on changes from
r289565, which is why it required a variant specifically for 3.9, but
now we can use the same patch as for other versions. Well, except
there's a small difference in the context, and build-clang.py doesn't
allow fuzz, so we manually edit the patch to remove that line from the
context.
--HG--
extra : rebase_source : cda077132ee499a8ffdc4c88a1160cfa5fd86a97
Before, the build root was not in a Docker cache or volume. With
current Docker works, that meant AUFS. We know AUFS is slow under
I/O load and can cause random failures due to missing data after
writes.
This commit changes the build root to a known Docker volume, which
will be backed by EXT4 and won't have the problems of AUFS.
MozReview-Commit-ID: 6WOH0yednAv
--HG--
extra : rebase_source : bbff0f00f55acdbe068fdf617a7903b8a303c397
32-bit clang-cl.exe was looking specifically for HostX86\x86\link.exe, which doesn't exist in our automation package. Make it look in HostX64\x86 instead.
This is an ugly hack and it would be preferable to just use a 64-bit compiler (bug 1414287).
In a proper VS install, the path to cl.exe looks like:
...\VC\Tools\MSVC\14.11.25503\bin\HostX64\x64\cl.exe
In our automation, the path is just:
...\VC\bin\HostX64\x64\cl.exe
Clang tries to do some sanity-checking to make sure that the cl.exe it finds is the Microsoft compiler and not some other program. But the checks are a little too strict for us, so just look for "bin\Host*\*\cl.exe".
Ideally, we'd backport it to the clang 4 toolchain too, but that results
in silently(!) missing crash symbols for libxul.
--HG--
extra : rebase_source : a3024abf720ea6e9dc0e3271a5aa2ebfae9bebee
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
shutil.copy2() will fail if the destination directory doesn't exist.
Switch to copy_tree() instead so we don't need to worry about the
error cases of copy2() and copytree().
MozReview-Commit-ID: 3kHfgL57KfX
--HG--
extra : rebase_source : c7335b0c2854d53699dda0f0d2bd9d17b57c4e5d
We're soon going to build multiple versions of clang and gcc for linux,
and we need to differentiate them. Furthermore, there is a need for the
base-toolchains builds to use a fixed version of clang and gcc. So
rename the clang and gcc toolchain jobs to include their version, add
aliases to satisfy all existing jobs, and adjust the base-toolchains
jobs to use the explicit version.
--HG--
rename : build/build-clang/clang-linux64.json => build/build-clang/clang-3.9-linux64.json
rename : taskcluster/scripts/misc/build-gcc-linux.sh => taskcluster/scripts/misc/build-gcc-4.9-linux.sh
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: zehcGJrUQX
--HG--
extra : source : feedcde68c2a54da210f03eb287ab5c862fc982b
extra : intermediate-source : 485d1af7805ad9fa0e701c3571fc1291fbfc6850
The return-empty-string-non-mangled.patch apparently landed in r249437
upstream, so we don't need to carry around our own version. The patch
is still nominally used for building clang on Mac, so I have not removed
it from version control.
Instead of relying on the assumption that a previous run of CMake was
using the same arguments, remove the CMake cache file and re-run it.
This way the script is robust no matter what kind of build directory
existed from before.
Since individual config files have different source repos declared,
it's better to deal with each individual source directory separately.
Also make sure to revert any of the existing changes in each directory
so that attempts to apply patches to the source directory or import
our static analysis checks into clang-tidy are guaranteed to always
succeed.
