Commit Graph

667 Commits

Author SHA1 Message Date
Mike Hommey
1d38c4309b Bug 1432398 - Remove the desktop-build docker image and related files. r=dustin
--HG--
extra : rebase_source : 158e14474ce049343105d2be95aabc03ec0b7854
2018-01-27 11:04:23 +09:00
Mike Hommey
e8b62fa5df Bug 1431251 - Remove CRT objects from the GCC build. r=rillian
They were added in bug 1427344 to reduce the differences between builds
on CentOS docker images and builds on Debian docker images. We're now
entirely on Debian docker images, so we can back this out.

--HG--
extra : rebase_source : 672e389cb8d6dbf7860a989024690de18e5c320e
2018-01-27 11:52:12 +09:00
Mike Hommey
e551e11340 Bug 1431251 - Remove PKG_CONFIG_LIBDIR from mozconfigs. r=rillian
As of bug 1430036 it was only set when building on CentOS, and as of bug
1432398, we don't have CentOS-based docker images anymore.

--HG--
extra : rebase_source : 5ade9bee773bca3283cfdb9d69209033fe82253f
2018-01-27 11:49:23 +09:00
Gregory Szorc
6f469dc037 Bug 1430908 - Use --progress=dot:mega with wget; r=glandium
By default, wget prints dots every 1k bytes. This can render a
lot of output for large files. We switch to the "mega" style, which
makes each dot represent 64k, thus reducing output by up to 64x.

We also force the use of dot display. By default, it uses "bar"
which attempts to use terminal formatting if possible. Since most
of this code executes in CI and terminal control characters can
interfere with logged output, we force the use of "dot." (Although
wget appears to automatically switch to dot in TC today. But
consistency is good.)

MozReview-Commit-ID: IpTWJdcauTV

--HG--
extra : rebase_source : 5c9aa1bbdcd78eaa0b31347ad026a2c1beaedc03
2018-01-16 14:24:31 -08:00
Mike Hommey
648d8e17e1 Bug 1430506 - Install pulseaudio 2.0 on CentOS build images. r=nalexander
Switching to Debian build images will force a version bump from
pulseaudio 0.9.something to pulseaudio 2.0. Practically speaking, as
long as bug 1427150 is fixed (which it is), this is strictly better,
because this enables all the PA_CHECK_VERSION(2,0,0) code in libcubeb,
which handles port availability (whether output is plugged or not), and
with bug 1427150, it does so in a backward compatible manner.

Now, since this is a behavior change from what we're currently shipping,
this has the potential of triggering unexpected test failures, or break
sound for users. The likelyhood of the latter happening is rather low,
though, because Linux distros have been building with pulseaudio >= 2.0
for a long time and we haven't heard about port availability breaking
sound for them. But it's still better to decouple this change from the
switch to Debian.

We abuse the build-gtk3.sh script which installs gtk3 in the CentOS
build image to install pulseaudio as well.

--HG--
extra : rebase_source : eb4e4033c50d59117b5199d1653d85f871503b2f
2018-01-13 06:24:51 +09:00
Csoregi Natalia
7476b71e00 Merge inbound to mozilla-central r=merge a=merge 2018-01-12 23:59:06 +02:00
Mike Hommey
f1dcc7b8db Bug 1427344 - Add Debian CRT objects to GCC. r=nfroyd
One of the last remaining differences when building Firefox on Debian
with all the changes we've done so far is that linking against the
system libc statically links some CRT objects. This causes massive
differences in the resulting binaries because of slight differences
in those objects (because they weren't compiled with the same compiler
and because they're not for the exact same glibc version)

In practice, their content difference don't cause any problem. If they
did, we wouldn't be able to run our builds on newer systems than those
we build them on. The only hypothetical risk would be to run on systems
with a glibc older than Debian 7's, but those already can't run Firefox
anyways (those systems don't have Gtk+3, which is a system requirement).

AFAICT, this is only an hypothetical problem anyways, even such systems
with Gtk+3 should be able to run those builds. Plus, this is a change
that will happen anyways when switching to Debian-based build images,
since they would be using the CRT objects from there. We're merely
making it happen earlier so that the differences from switching to
Debian-based build images are more tractable.

Note we only do this when building GCC on Debian, allowing to roll back
to CentOS-based toolchains by just switching back the toolchain jobs to
use the desktop-build docker image again.
2018-01-12 21:39:57 +09:00
Mike Hommey
e17b79e152 Bug 1427344 - Build GCC without --disable-initfini-array. r=nfroyd
When building on Debian (which we now are), this means we enable
.init_array/.fini_array.

When building on CentOS, this means no change. Which implies we could
roll back to CentOS-based toolchains by just switching back the
toolchain jobs to use the desktop-build docker image again.

This change causes massive differences in the resulting binaries because
of the offset differences, but practically speaking, there is no
difference. .init_array/.fini_array have been supported in glibc for 18
years.
2018-01-12 21:39:56 +09:00
Mike Hommey
c42e9d7d91 Bug 1429918 - Adjust the MPC download url. r=nalexander
--HG--
extra : rebase_source : 28fedbf2c08c179c2b2f1aeb15921d09e817f7af
2018-01-12 06:58:34 +09:00
shindli
f1c9a5da78 Backed out changeset 1af647b2f9e8 (bug 1429918) for a request made by the developer on a CLOSED TREE 2018-01-12 00:47:35 +02:00
Mike Hommey
e73fcbf697 Bug 1429918 - Adjust the MPC download url. r=nalexander
--HG--
extra : rebase_source : e457431f9aa04c58ec29eb0c6450d43232de12d2
2018-01-12 06:58:34 +09:00
Mike Hommey
0804de7e8e Bug 1430036 - Don't set PKG_CONFIG_LIBDIR when building on Debian. r=ted
--HG--
extra : rebase_source : 06f96fe0060a34f65f1126e3510580e0b6bc4969
2018-01-12 15:16:19 +09:00
Mike Hommey
87c31766de Bug 1429257 - Statically link newfs_hfs against libcrypto. r=gps
For the same reasons as bug 1427266.

--HG--
extra : rebase_source : f27c9d05b2ea74d323fde29c09c2faae9fe8ee19
2018-01-10 09:59:29 +09:00
Mike Hommey
e1983fc387 Bug 1429257 - Make build-hfsplus.sh fail when something goes wrong. r=gps
Currently, the build can finish succesfully even when one of the
programs fails to build and is not included in the final artifact. The
macosx build then fails because of that, which is the wrong place to
be failing.

--HG--
extra : rebase_source : 4a41b2f96eea45d3eefa2734900603b6e6ee0ea5
2018-01-10 08:37:44 +09:00
Jean-Luc Bonnafoux
5acf65c7fe Bug 1428629 - elfhack.cpp prefer prefix ++ operator for non primitive types r=froydnj
MozReview-Commit-ID: C0L2NUsbmc4

--HG--
extra : rebase_source : b4b3dfbbabbd610384448169b10c3f9b5c27e621
2018-01-08 09:30:32 +01:00
Jean-Luc Bonnafoux
83cf591ec7 Bug 1417215 - Prefer prefix ++ operator for non primitive types r=froydnj
MozReview-Commit-ID: Hjbj0PEjAnf

--HG--
extra : rebase_source : 659bfb57eba416e6105035e453d7366a9515ea3a
2017-12-30 21:09:58 +01:00
Mike Hommey
e34ef317f6 Bug 1427339 - Make mozconfig.stdcxx work with both CentOS and Debian-built GCCs. r=gps 2018-01-06 14:19:30 +09:00
Mike Hommey
0c1517dd8c Bug 1427339 - Configure binutils and gcc --with-sysroot=/. r=gps
The system binutils and gcc are built with that option on Debian, but
not on CentOS. That makes no practical difference, except for the fact
that when building GCC, we use our own-built binutils (as per bug
1427316), but use the system GCC. And a GCC built with --with-sysroot=/
doesn't work with a binutils built without. However, a GCC built without
--with-sysroot=/ works fine with a binutils built with it. So this
change is compatible with building our GCC on both CentOS and Debian.
2018-01-06 14:19:29 +09:00
Mike Hommey
511d112e2d Bug 1427316 - Use the binutils we just built to build GCC. r=gps
We're currently building GCC with the system binutils, which, at the
moment, is whatever version is available on the CentOS 6 build
environments. With the imminent switch to Debian 7, that will be a
different version.

It turns out the GCC configure script does enable some features
depending on the binutils it's built with. For the most notable
differences it makes when going from Centos 6 to Debian, it enables
.init_array/.fini_array depending on the binutils version, and enables
the use of CFI advances depending on gas and objdump respectively
supporting and displaying DW_CFA_advance_loc.

But we're already building a fixed version of binutils (which happens to
be more recent than the one in both CentOS 6 and Debian 7), and we're
using that version when using GCC to build, so we can just as much use
the version we built to build GCC.

In order to avoid any changes to the resulting builds, we explicitly
turn off .init_array/.fini_array (which currently happens implicitly
when building on CentOS 6). This will ensure that there is not other
change to the builds due to this binutils version bump
(.init_array/.fini_array being enabled shifts everything in the
binaries, so it makes the whole diff full of noise)
2018-01-04 19:16:19 +09:00
Mike Hommey
25c3f317c0 Bug 1427404 - Always export PATH when passing it through mk_add_options. r=nalexander
mk_add_options has this kind of awkward feature where
  mk_add_options VAR=value
would set VAR for the build through client.mk, but not when running
make -C objdir target. But
  mk_add_options "export VAR=value"
does.

We might want to change that on the long run, but the side effects would
have to be calculated first.

OTOH, we have automation jobs that run compilations during `make check`
(e.g. rusttests), which is not invoked through client.mk. So they
currently don't get the same PATH as the build part, meaning that
they're using system binutils instead of the one from the GCC toolchain
package.

--HG--
extra : rebase_source : aab7f221243c486cf70c7b0c91b9313231050ed8
2017-12-31 08:50:29 +09:00
Mike Hommey
4aa79652c5 Bug 1427232 - Remove .la files when building gtk3. r=gps
In bug 1426785, Gtk+3 build was moved to docker image creation time, and
at the same time, the removal of .la files was stripped, because it
seemed too broad to do that in /usr/local/lib*, and because I didn't
really remember why it was there in the first place.

I now do remember, and that's because libtool likes to add useless
dependencies on libraries just because direct dependencies transitively
depend on them. For example, this adds a dependency on libffi to
libpangocairo, which doesn't use it directly.

I also happens that /usr/local/lib* is empty at the moment we build
gtk3, so we can just do the cleanup there.

On its own, this change is useless, but:
- it restores the libraries in their state pre-bug 1426785,
- it helps reduce some differences when building on Debian (for bug
1399679), easing the comparison of those builds.

--HG--
extra : rebase_source : 3fa644d8ae5eb4ccac5940c7d3605201f589936d
2017-12-28 11:15:00 +09:00
Mike Hommey
6a21d1f58e Bug 1426785 - Remove mozconfig.gtk. r=gps
It now only does something trivial, which also happens to be a no-op
because it's the default. It does have a commented entry for possible
gtk+2 builds, but we're soon going to remove that possibility anyways in
bug 1278282.

--HG--
extra : rebase_source : 9ac927bb7bd8c057264c8f6f9ca5cbf79a839c4e
2017-12-22 07:57:05 +09:00
Mike Hommey
79b5314945 Bug 1426785 - Use gtk+3 from /usr/local on automation. r=gps
Now that build environment docker images have gtk+3 installed in
/usr/local, adjust mozconfigs to point pkg-config there, and remove
all the glue that was required to build using the tooltool package.

Also remove the --x-libraries=/usr/lib on 32-bits builds, which only
confuses the linker.

--HG--
extra : rebase_source : c7de7b3959a3c6b77ea202d9609c891b5b7ec442
2017-12-22 07:53:33 +09:00
Mike Hommey
a6e25e84a3 Bug 1426785 - Install gtk+3 in the Centos images used for desktop builds. r=gps
Back when we started needing gtk+3 to build Firefox, we were using mock
to setup the build environment, and a tooltool package was the most
sensible way to handle this.

Fast forward to today, and we're close to moving the build environment
to Debian, which comes with gtk+3 packages. But in order to simplify
the various checks for the transition, it is desirable to stop using the
tooltool package. Which we can actually do in a reasonable way now that
we use docker images instead of mock, by building and installing gtk+3
in the build environment images.

So we modify the script that was producing the gtk+3 tooltool packages
such that it installs gtk+3 in the docker images, both 32 and 64 bits.
And invoke it when creating the desktop build environment docker images.

--HG--
extra : rebase_source : 75e987d6de7f3ae8a3d9b478fc173e191d28aace
2017-12-22 07:41:56 +09:00
Coroiu Cristina
dbb27acb6d Backed out 5 changesets (bug 1426785) for failing repackage the nightly build on Linux a=backout.
Backed out changeset 08b5850633de (bug 1426785)
Backed out changeset 61453b6473f1 (bug 1426785)
Backed out changeset 851ce8944b41 (bug 1426785)
Backed out changeset 386cd0532519 (bug 1426785)
Backed out changeset 2a52bf9e0898 (bug 1426785)
2017-12-24 14:03:02 +02:00
Mike Hommey
2dca68b63f Bug 1426785 - Remove mozconfig.gtk. r=gps
It now only does something trivial, which also happens to be a no-op
because it's the default. It does have a commented entry for possible
gtk+2 builds, but we're soon going to remove that possibility anyways in
bug 1278282.

--HG--
extra : rebase_source : 0de751e523ee002bbe6638d223eb384364edd22b
2017-12-22 07:57:05 +09:00
Mike Hommey
2e32acf893 Bug 1426785 - Use gtk+3 from /usr/local on automation. r=gps
Now that build environment docker images have gtk+3 installed in
/usr/local, adjust mozconfigs to point pkg-config there, and remove
all the glue that was required to build using the tooltool package.

Also remove the --x-libraries=/usr/lib on 32-bits builds, which only
confuses the linker.

--HG--
extra : rebase_source : 22b1273ae4b78807b355d33ed5895bdfe83a141d
2017-12-22 07:53:33 +09:00
Mike Hommey
089e3a8dc2 Bug 1426785 - Install gtk+3 in the Centos images used for desktop builds. r=gps
Back when we started needing gtk+3 to build Firefox, we were using mock
to setup the build environment, and a tooltool package was the most
sensible way to handle this.

Fast forward to today, and we're close to moving the build environment
to Debian, which comes with gtk+3 packages. But in order to simplify
the various checks for the transition, it is desirable to stop using the
tooltool package. Which we can actually do in a reasonable way now that
we use docker images instead of mock, by building and installing gtk+3
in the build environment images.

So we modify the script that was producing the gtk+3 tooltool packages
such that it installs gtk+3 in the docker images, both 32 and 64 bits.
And invoke it when creating the desktop build environment docker images.

--HG--
extra : rebase_source : fe18bfb2ec8db183c44838d5a7a0051322b2a9c0
2017-12-22 07:41:56 +09:00
Mike Hommey
cffd7e8396 Bug 1426555 - Move --enable-stdcxx-compat to python configure. r=chmanchester
At the same time, we make it actually do something on spidermonkey
builds. We also add an environment variable alternative, that we use
in mozconfig.stdcxx, allowing for mozconfig.no-compile to override it
and avoid configure failures on e.g. artifact builds.

--HG--
extra : rebase_source : b68d362025e0c99f9184a03391c652ec2c9357ad
2017-12-21 11:13:08 +09:00
Mike Hommey
e45c2bc4bc Bug 1426283 - Use nproc for the number of parallel jobs to run when building gcc. r=gps
While in the vicinity.

--HG--
extra : rebase_source : 8d885ff5fd4fa124a08326c0ca002a49ef559e57
2017-12-20 14:39:26 +09:00
Mike Hommey
d61456f499 Bug 1426283 - Work around bug 1409276. r=gps
Both the cc crate and the rust compiler may want to use "cc", which,
on automation, points to the system GCC compiler instead of ours.

As a workaround, we add a cc symbolic link in the GCC toolchain artifact
so that, as long as the GCC toolchain artifact's bin directory is in
$PATH early enough, it's picked over /usr/bin/cc.

--HG--
extra : rebase_source : 53cacf8a750539706a484218e168c8c9e0ba49a6
2017-12-20 10:32:36 +09:00
Mike Hommey
fbaf337e4b Bug 1426322 - Separate gcc and mingw32-gcc. r=gps
The "contract" for toolchains is that extracting foo.tar.xz creates a
directory named foo/. That is however not true for mingw32.tar.xz, which
extracts into gcc/, possibly overwriting files from the gcc.tar.xz
archive (which is also used for mingw builds, for the host part).

This is also not true for nsis.tar.xz, but it reportedly has problems
when it's not in the same directory as mingw32.

But mingw32 doesn't actually need to be mixed with gcc, so it's better
to separate them as they are supposed to be.

--HG--
extra : rebase_source : 30d90af64459bbb31bc076e48f3c661fa9cd4a79
2017-12-20 13:46:53 +09:00
Mike Hommey
8bb6a1a03e Bug 1423821 - Add a consistency check for section offsets to elfhack. r=froydnj
lld is being too smart for its own good, and places non-relocatable data
right after the program headers, which prevents the program headers from
growing. But elfhack wasn't checking for that, so happily placed the
non-relocatable data at its non-relocated location, overwriting the last
item of the program headers.

--HG--
extra : rebase_source : 6f26d475f0a19d88ddf21399dbce8ceac62b492d
2017-12-07 15:34:58 +09:00
Mike Hommey
ded54a5e92 Bug 1423813 - Properly handle elfhack -r after bug 1385783. r=froydnj
Bug 1385783 changed things such that the two elfhack sections are not
adjacent anymore. They can even be in different segments in some cases,
but the undo code doesn't know how to actually handle that case.

So for now, allow non adjacent sections, but still verify that they are
in the same segment.

--HG--
extra : rebase_source : da95ef7df19eeea8dfd07b24f22e7bee18939b69
2017-12-07 15:22:22 +09:00
Tom Prince
1d74db87ce Bug 1424651: Remove unused SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE mozconfig variable; r=ted.mielczarek
MozReview-Commit-ID: CkIg3fiwp1z

--HG--
extra : rebase_source : 5a4a50c8feb477a9b50c30e35b72a316b1f1bc8c
2017-12-10 23:05:05 -07:00
Dorel Luca
2f271a1136 Backed out 4 changesets (bug 1424651) as requested by tomprice r=backout on a CLOSED TREE
Backed out changeset 10ebf78f32bb (bug 1424651)
Backed out changeset 746d96792d18 (bug 1424651)
Backed out changeset 6038fb7b458c (bug 1424651)
Backed out changeset 189fd4f1df41 (bug 1424651)
2017-12-12 06:33:18 +02:00
Tom Prince
4dfc8f7a46 Bug 1424651: Remove unused SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE mozconfig variable; r=ted.mielczarek
MozReview-Commit-ID: CkIg3fiwp1z

--HG--
extra : rebase_source : 475e2d8888ff4b93efa9886581de9d145b51c51c
2017-12-10 23:05:05 -07:00
Ted Mielczarek
8b7140ce04 bug 1424323 - remove MOZ_AUTOMATION_UPLOAD_SYMBOLS from in-tree mozconfigs. r=rillian
With all of our builds in Taskcluster now, we should never be uploading
symbols from build tasks. Unfortunately Windows builds were still doing so.
This patch removes MOZ_AUTOMATION_UPLOAD_SYMBOLS from all the in-tree
mozconfigs and a few other places so that it should always default off
(per moz-automation.mk). The rest of the uploadsymbols bits will be
removed once Thunderbird fixes their automation.

This patch was mostly autogenerated by running:
rg --files-with-matches UPLOAD_SYMBOLS browser/config/mozconfigs/ mobile/android/config/mozconfigs/ | xargs sed -ri '/.*UPLOAD_SYMBOLS.*/d'
sed -ri '/.*UPLOAD_SYMBOLS.*/d' build/unix/mozconfig.linux build/mozconfig.win-common build/macosx/local-mozconfig.common build/mozconfig.automation

Then mobile/android/config/mozconfigs/common and
taskcluster/scripts/builder/build-linux.sh were hand-edited.

MozReview-Commit-ID: Cy8kSEodSg4

--HG--
extra : rebase_source : 01caf1651b4eb428313e1f371aa585f8f34c4151
2017-12-08 13:50:17 -05:00
Mike Hommey
9ec14dddcb Bug 1417689 - Remove explicit --enable-elf-hack in mozconfigs. r=nalexander
--enable-elf-hack is the default on all platforms where it's supported,
and is completely ignored on platforms where it's not supported.
While moving the flag to moz.configure, we're going to make it only
work on platforms where elfhack is supported, so we at least need to
remove it from mozconfigs for those platforms where it's not supported.
But generally speaking, we want less things in mozconfigs, so just
remove it from there, since it's the default anyways.
2017-11-16 09:37:17 +09:00
Chris Manchester
fb88a7e614 Bug 1407388 - Remove build/unix/elfhack/inject/Makefile.in and replace with generated files. r=mshal
MozReview-Commit-ID: Cr2RUlksKBJ

--HG--
extra : rebase_source : 03f32e52d754d29a23e774707b6f92e265bf3ce0
2017-11-07 16:54:22 -08:00
Nathan Froyd
bce27af988 Bug 1163171 - part 2 - switch to using -isystem rather than -idirafter for Android; r=glandium
This command-line flag is a little more evocative and works correctly
with both GCC and clang.
2017-10-28 17:38:59 -04:00
Chris Manchester
e361318990 Bug 1403346 - Remove clang plugin flags from stdc++compat through moz.build rather than Makefile.in r=glandium
MozReview-Commit-ID: DuFMdNru2i4
2017-10-25 15:12:10 -07:00
Chris Manchester
3aac6ce692 Bug 1403346 - Implement cflags filtering for elfhack in mozbuild COMPILE_FLAGS r=glandium
MozReview-Commit-ID: GO2mqMjHuHd
2017-10-25 15:12:10 -07:00
Tom Ritter
302cef9ace Bug 1407359 Set up a framework for patching the MinGW toolchain r=glandium
MozReview-Commit-ID: 8HtjLXAIXTP

--HG--
extra : rebase_source : b32cb4ac931c9dc599572bc5e726e4d68982c8a4
2017-10-16 20:52:47 -05:00
Robert-André Mauchin
3d620068f1 Bug 1407211 - Fix typo in build/unix/run-mozilla.sh from bug 1403366. r=glandium 2017-10-12 07:28:20 +09:00
Mike Hommey
37d01456dc Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : e0b1dcdabe798af478e054cde0df65facf25ea21
2017-09-28 11:00:09 +09:00
Sebastian Hengst
2e58d81866 Backed out changeset ff0705eda4bd (bug 1403366) 2017-10-04 01:26:56 +02:00
Mike Hommey
5f2f5b4e64 Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : e0b1dcdabe798af478e054cde0df65facf25ea21
2017-09-28 11:00:09 +09:00
Sebastian Hengst
6a0c7a5682 Backed out changeset 28b00bdf83a3 (bug 1403366) 2017-09-29 17:19:35 +02:00
Mike Hommey
8142d74974 Bug 1403366 - Don't set MOZILLA_FIVE_HOME from multiple scripts. r=froydnj
It was seldom used before previous commit and now does nothing.

--HG--
extra : rebase_source : 59ba89dbd8de9c0b9361872f3f45504a46f454a2
2017-09-28 11:00:09 +09:00