This was released a few minutes ago. It contains some fixes necessary to support
partial clones.
Differential Revision: https://phabricator.services.mozilla.com/D13768
--HG--
extra : moz-landing-system : lando
We need to run Mercurial 4.8 so we can start using SQLite storage
and wire protocol version 2 for partial clones.
--auto-deconfigure was added because desktop1604-test now inherits
from a base image with Mercurial installed.
Differential Revision: https://phabricator.services.mozilla.com/D11399
--HG--
extra : moz-landing-system : lando
We need to run Mercurial 4.8 so we can start using SQLite storage
and wire protocol version 2 for partial clones.
Differential Revision: https://phabricator.services.mozilla.com/D11399
--HG--
extra : rebase_source : 712fa08226f74ef9cfe306b7c68ff3dd3f54d9d3
extra : source : a438eff7d106f25a951c690c22f30b37d40d54a6
We need to run Mercurial 4.8 so we can start using SQLite storage
and wire protocol version 2 for partial clones.
Differential Revision: https://phabricator.services.mozilla.com/D11399
--HG--
extra : moz-landing-system : lando
This extracts the current logic for finding nodejs into its own module in mozbuild. Configure and ESLint then use it.
For ESLint, this will change the first location it looks for nodejs to be the .mozbuild directory.
Differential Revision: https://phabricator.services.mozilla.com/D6430
--HG--
extra : moz-landing-system : lando
Bumping the Mercurial version to 4.7.1 to apply a fix for a
performance regression introduced in 4.7.
Differential Revision: https://phabricator.services.mozilla.com/D5193
--HG--
extra : rebase_source : b06eae5268fe7f38f85f8644da7f08516840a827
This commit points the `install_mercurial.sh` script at the
newest Mercurial distributions uploaded to tooltool.
Differential Revision: https://phabricator.services.mozilla.com/D2626
--HG--
extra : amend_source : a9cd500cc86106750aa5b5472e624a00128cac68
extra : histedit_source : 7a06d9c77b6f7a7b2110dc6bc15858c7aa3b7242
Our normal ubuntu 16.04 test image is suitable for hosting an Android x86
emulator with these minor updates: Install kvm and make sure /dev/kvm
rw permissions are open for everyone. Note that /dev/kvm is generally
only visible when running docker with --privileged; its permissions
cannot be modified in the Dockerfile, only at run-time: run-task is the
first opportunity.
In preparation for making it usable on Windows, after which point
having it in a directory with "docker" in it doesn't make much sense.
MozReview-Commit-ID: Hgu0buFyJwF
--HG--
rename : taskcluster/docker/recipes/run-task => taskcluster/scripts/run-task
extra : rebase_source : 3c0b502d28b5aad54bd04069efbfda88e25bbb20
Previously, we installed the latest version of pip and virtualenv.
This commit pins the pip and virtualenv version so we install known
working versions (pip 10 breaks the image build for some reason).
MozReview-Commit-ID: hOAMencdcr
--HG--
extra : rebase_source : 2cb44c2ef55e29c55cf3d1b354e90d6fb5414cce
We bump the version of Mercurial for Debian packages and for Ubuntu
Docker images.
MozReview-Commit-ID: KYmG4rOm3TQ
--HG--
extra : rebase_source : cbb817a9ee4c27f0bc59f4fa1e2a708fac1cb093
All in-tree Docker images should be installing Mercurial via
install-mercurial.sh so that the Mercurial install is consistent
across all Docker images.
I noticed this image wasn't using install-mercurial.sh because
attempting to rebuild the image currently fails due to
mercurial-4.3.1-2 not being available in the upstream package repo.
install-mercurial.sh has been taught to handle Ubuntu 18.04 and the
Docker image building process has been taught to use
install-mercurial.sh. install-mercurial.sh uses tooltool and behavior
should work and be deterministic over all of time.
As part of this, we had to establish a standalone shell script for
building the image. That's because install-mercurial.sh requires a
"tooltool_fetch" alias. Meaningful image building code has been
moved into the new setup.sh. This also means things run as a single
RUN statement. So we don't need to hack around minimizing RUN
invocations.
I also discovered that the pinned curl version is no longer available.
So I removed the version pinning. FWIW we can't rely on version
pinning unless the Apt repository is snapshotted. Packages do get
yanked from time to time. Unless we absolutely require a specific
version of a specific package, we can probably get away without pinning
- at least for this Docker image. But that can be a follow-up.
MozReview-Commit-ID: As7Hq470QcK
--HG--
extra : rebase_source : e5868ff1dfd6a14a12d3df08aca2954d1d9ea99f
In automation for both Linux and Linux64 the same docker image is used,
which is based on Ubuntu 16.04 64bit. To run the 32bit Firefox binary
the necessary libraries for the i386 architecture get installed.
What's missing is the creation of the font cache for i386 binaries. As
such it will be created at the first start of the appropriate application.
It means there is an additional ~30s delay before Firefox opens, which
causes several test failures especially for wdspec tests.
To prevent this delay the i386 version of fontconfig will be installed
and apt automatically creates the font cache for the i386 architecture
by leaving the x86_64 cache in-place.
So the cache creation will happen one time when the docker image is getting
build, and not for each and every test job, which saves us 30s for each.
MozReview-Commit-ID: 58xdOI4xYkP
--HG--
extra : rebase_source : f246fc47caa1acc8657c4963c9450c1ca7653fec
We start from the image used for Firefox builds, and add the debug
packages for all the system libraries.
--HG--
extra : rebase_source : 2c759975d9837beabdc08a15fd926a99fd1cecf8
By default, wget prints dots every 1k bytes. This can render a
lot of output for large files. We switch to the "mega" style, which
makes each dot represent 64k, thus reducing output by up to 64x.
We also force the use of dot display. By default, it uses "bar"
which attempts to use terminal formatting if possible. Since most
of this code executes in CI and terminal control characters can
interfere with logged output, we force the use of "dot." (Although
wget appears to automatically switch to dot in TC today. But
consistency is good.)
MozReview-Commit-ID: IpTWJdcauTV
--HG--
extra : rebase_source : 5c9aa1bbdcd78eaa0b31347ad026a2c1beaedc03
Because Ubuntu 16.04 changed in a way that busts gl3 tests and we can't
update the desktop1604-test image anymore.
--HG--
extra : amend_source : bfa07f9f77990dd6915b8c92d218227436bc6fc4
The install-mercurial.sh script currently installs a global mercurial
configuration after installing mercurial manually. In order to share
that configuration with docker images installing a mercurial package
through packages tasks, we move it to a separate file.
We however keep the part setting web.cacerts in install-mercurial.sh,
since it uses a path depending on what kind of environment the script is
run. Moreover, the instructions that come with mercurial to build
Debian packages come with web.cacerts set to the right path already, so
it's not needed in that case anyways.
At the same time, use multiple files in /etc/mercurial/hgrc.d/ instead
of a single /etc/mercurial/hgrc file.
--HG--
extra : rebase_source : 8140d8243cf012489025afe058f467c72224c891
I suppose it was setup through ~worker/.hgrc before we started
installing a /etc/mercurial/hgrc that enables a few other extensions
and sets some preferences.
There is no reason to now have two places where mercurial is being set
up, and it feels natural that we set it up at the system level.
Ideally, we'd also clean up the centos6-based images, but they require
an update of the centos6-build and centos6-build-upd images on the
docker hub, which is not really convenient, and those images are going
to be obsoleted soon anyways (bug 1399679).
--HG--
extra : rebase_source : 32c9cdf5d0fe8ac2c60a1c5a38e572c83a4783b2
Bug 1389715 removed the image definition in taskcluster/ci/docker-image
as well as the files associated with it in
taskcluster/docker/desktop-test, but the Dockerfile in there was the
only use of the Ubuntu 12.04 setup script, so it is currently unused.
--HG--
extra : rebase_source : 7d8018e7c94e2625ff9822a2d66231722a030394
While we're here, add a missing prepare_vcs_checkout for the
comm-central checkout.
--HG--
extra : rebase_source : 788a288330e34b5551ec2b12726a755e268566c2
A lot of effort has been spent optimizing VCS operations for peak
performance. But not utilizing caches or volumes for the VCS store
or checkouts can undermine that work.
Let's print a warning when VCS is configured sub-optimally.
I'm pretty sure we still have some rogue tasks not using caches
or volumes. We can convert this to a fatal error once those are
fixed.
MozReview-Commit-ID: C6CT1zViy75
--HG--
extra : rebase_source : 91760250bed263c789b95d16cc0542a53ca2bfbf
This seems like a reasonable thing to enforce.
MozReview-Commit-ID: 3BZQSkwRYeN
--HG--
extra : rebase_source : 8dae62edb35202da0f0e90ddec3eacb212ada371
We introduce a per-cache .cachelog file containing important events
in the cache's history such as creation, requirements adjusting,
and utilization. If cache validation fails, we print the cache log.
If a previous task was responsible for getting the cache in a bad
state, its TASK_ID should be printed, allowing us to more easily
identify mis-configured tasks.
MozReview-Commit-ID: BJun5Hi5w0s
--HG--
extra : rebase_source : f4758741ee294a0de53882b6891b473c01463e28
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
CLOSED TREE
--HG--
extra : amend_source : 84120d6bacb5d72a9fbe41e4c3b405d63825da7c
extra : histedit_source : 8320c2193761b745f10850055ee74a3c9ac73615%2Cfbc2a28d8c5004a53305ef858ca5aea4245691e0
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
