There are e.g. some build infrastructure changes that we want to have a
controlled impact on the Firefox builds we produce. We have, in multiple
occasions, gone through manual work to compare Firefox builds, most of
the time using the diffoscope tool (https://diffoscope.org/).
This change introduces a new task kind that takes two Firefox builds as
input, either by name (reference to a build from the current task graph)
or by index (reference to a build from a previous push), and compares
them.
In order to get a Firefox build by index, we rely on dummy tasks with
an optimization we expect to always hit, so we add the necessary bits
to ensure those dummy tasks can go through up to the optimization phase
and be optimized out there.
--HG--
extra : rebase_source : 37482f67652dab2fcef2db4e6b8efe653999bae5
The spidermonkey mozjs and rust-bindings builds run sed on
$topsrcdir/.cargo/config.in to generate the cargo config they use, but
they previously only replaced the @top_srcdir@ substitution. This patch
makes them replace any other substitutions with an empty value to add
a bit of future-proofing.
MozReview-Commit-ID: 1DzP9vXxHMD
--HG--
extra : rebase_source : e8c0268a2a6e91ca2000b340beee2dcff0636591
We currently use a 32-bit Rust toolchain for win32 builds, but this can lead
to OOM situations. This patch makes win32 builds use a 64-bit Rust toolchain,
which requires a little bit of extra configuration because rustc needs to
be able to find a link.exe that produces 64-bit binaries for building
things like build scripts, which are host binaries.
We will now generate a batch file that sets LIB to the paths to 64-bit
libraries and invokes the x64-targeting link.exe, and add a section to the
.cargo/config file to instruct cargo to use that batch file as the linker
when producing 64-bit binaries.
MozReview-Commit-ID: 9vKBbm7Gvra
--HG--
extra : rebase_source : 599b3b661c7a8a5db1f32a2a9732fc202fb55e1e
Add a cross-compilation copy of rust's standard library targeting
i686-pc-windows-msvc to the win64-rust toolchain package so it
can be used to build for win32 as well.
MozReview-Commit-ID: 3598VZrDjIH
--HG--
extra : rebase_source : f1b25a68a67ae7f9c505a42d17f29dbedf59a49d
Instead of duplicating Dockerfiles between taskcluster/docker/*
directories, which can be error prone for very close images, it can be
desirable to use the same file. This change allows to set the
`definition` keyword on a docker image definition in kind.yml that
will make the task use the files from taskcluster/docker/<definition>
instead of taskcluster/docker/<image_name>.
--HG--
extra : rebase_source : 11ae231f66ca6a77896c1cff6c1580d04210f052
Ideally, we'd simply use the --build-arg docker argument along with ARG
in the Dockerfile, but that's only supported from Docker API 1.21, and
we're stuck on 1.18 for the moment.
So we add another hack to how we handle the Dockerfile, by adding a
commented syntax that allows to declare arguments to the Dockerfile.
The arguments can be defined in the docker images kind.yml file through
the `args` keyword. Under the hood, they are passed down to the docker
image task through the environment. The mach taskcluster-build-image
command then uses the corresponding values from the environment to
generate a "preprocessed" Dockerfile for its context.
--HG--
extra : rebase_source : 26a43dd680c1ab97b1a4689a23c55594a3b21b67
Landings of e.g. bug 1427336 triggered new toolchain jobs. One of those
jobs, because of wrong changes in bug 1421100, downloaded a new rust
compiler beta instead of the intended fixed beta version. In turn, that
new rust compiler beta fails to compile the slog crate.
Now, because of how toolchain cache indexes work, every new win32 job
picks that new unintended rust compiler beta version, even on branches
where 1427336 hasn't landed.
I couldn't find a way to force the right beta version, so we're pretty
much stuck with that toolchain index pointing to the wrong version of
rust beta.
By backing out bug 1426324, we return to a toolchain index that is known
to have an artifact for the right rust compiler beta.
Unfortunately, if something triggers a new TW32(rust) job after this,
that toolchain index will be busted as well.
Giving a directory to %include would copy all leaf files under one
single directory in the context image. The only image affected is
valgrind-build, which ended up having a dot-config/pip.conf file instead
of dot-config/pip/pip.conf, meaning valgrind jobs weren't using the
pip config.
--HG--
extra : rebase_source : 518c8ca1617b57ae4b4bb83a85340de5515c26c5
Android cppunit, test-verify, and mochitest-gpu have been running on lower end
aws instances. It is probably better to run all emulator tasks on xlarge.
libcrypto, part of OpenSSL, and that dmg links against, has a varying
ABI, and something built against libcrypto on Centos won't run on Debian
and vice versa. It might not even work between versions of the same OS
(e.g. Debian 7 vs. Debian 9).
Because of that, it is desirable to statically link it.
This incorporates https://github.com/mozilla/libdmg-hfsplus/pull/1
and sets OPENSSL_USE_STATIC_LIBS when building libdmg-hfsplus.
--HG--
extra : rebase_source : 21a46f707494f388a899c08d0923f8b393d12cd1
fontconfig uses expat by default to read its xml configuration, but when
expat is not there at build time, it falls back to libxml2. We ended up
in this situation, while on Debian, fontconfig is built against expat.
This makes no practical difference, since we're not actually using
fontconfig, but for some reason, the difference in dependencies has an
impact on how ld chooses to arrange the .plt and .got.plt sections,
meaning that even though the code and data is originally identical, in
the .o files, the resulting linked machine code is largely different
because of all the applied relocations changing the offsets of e.g. call
instructions for function calls through the .plt. This results in large
differences in .plt, .init, .text, .data.rel.ro, as well as symbols list
when building on Debian.
This thus is meant to help make the differences more tractable.
--HG--
extra : rebase_source : 7a731c34074a50e84412f73ab9499248345fb14a
OSX (cross) repackages are currently using a tooltool manifest to get
libdmg and hfsplus. Change those jobs to use the toolchain artifacts
instead.
At the same time, modify the repackage mozharness script's _run_tooltool
so that it doesn't fail with MOZ_TOOLCHAINS being set but without a
tooltool_manifest_src, matching the similar function in buildbase.py.
--HG--
extra : rebase_source : d128d4709c5d1d28d1a6b9c585fde82e99f725c7
I suppose it was setup through ~worker/.hgrc before we started
installing a /etc/mercurial/hgrc that enables a few other extensions
and sets some preferences.
There is no reason to now have two places where mercurial is being set
up, and it feels natural that we set it up at the system level.
Ideally, we'd also clean up the centos6-based images, but they require
an update of the centos6-build and centos6-build-upd images on the
docker hub, which is not really convenient, and those images are going
to be obsoleted soon anyways (bug 1399679).
--HG--
extra : rebase_source : 32c9cdf5d0fe8ac2c60a1c5a38e572c83a4783b2
Bug 1389715 removed the image definition in taskcluster/ci/docker-image
as well as the files associated with it in
taskcluster/docker/desktop-test, but the Dockerfile in there was the
only use of the Ubuntu 12.04 setup script, so it is currently unused.
--HG--
extra : rebase_source : 7d8018e7c94e2625ff9822a2d66231722a030394
The debian-build-system-setup.sh script doesn't use the install-make and
install-cmake scripts, so it's unnecessary to install them in /setup.
--HG--
extra : rebase_source : 4ba24b9827e67b9c7ad203e789e00e19d37786da
The script was added in bug 1179893 but looks like it has actually never
been used. A duplicate of the file was used for the upload-symbols
image, but that was removed in bug 1422740.
Since it was the only file in desktop-build/bin, we stop copying the
directory in docker images.
--HG--
extra : rebase_source : 4bcdb5ba0118e87455c6f596bf54e4528fe1b1ef
While we're here, add a missing prepare_vcs_checkout for the
comm-central checkout.
--HG--
extra : rebase_source : 788a288330e34b5551ec2b12726a755e268566c2
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : d8447b5422e63e88444008fddb76d658829694de
We're about to remove some tooltool manifests, so we need those calls to
work properly when TOOLTOOL_MANIFEST is not set.
--HG--
extra : rebase_source : 89d41021a87915dc9133e61543352e3bda1dace4
Back when we started needing gtk+3 to build Firefox, we were using mock
to setup the build environment, and a tooltool package was the most
sensible way to handle this.
Fast forward to today, and we're close to moving the build environment
to Debian, which comes with gtk+3 packages. But in order to simplify
the various checks for the transition, it is desirable to stop using the
tooltool package. Which we can actually do in a reasonable way now that
we use docker images instead of mock, by building and installing gtk+3
in the build environment images.
So we modify the script that was producing the gtk+3 tooltool packages
such that it installs gtk+3 in the docker images, both 32 and 64 bits.
And invoke it when creating the desktop build environment docker images.
--HG--
extra : rebase_source : 75e987d6de7f3ae8a3d9b478fc173e191d28aace
It turns out that in all cases it was the last tooltool manifest entry,
so we can remove the tooltool manifests entirely, and remove all
references to them.
--HG--
extra : rebase_source : 0aa9ef8151c2fccf62507dfecc0bc57b157772e1
