Commit Graph

18 Commits

Author SHA1 Message Date
Mike Hommey
bd810a1c0d Bug 1391408 - Build minidump_stackwalk as a toolchain. r=froydnj
This produces the same executables (produced for the same platforms) as
those currently pulled from tooltool (modulo timestamps, maybe changes
since last manifest change, etc.). Unfortunately, as of currently, the
Windows variant needs to be cross-built with mingw because it doesn't
compile without some POSIX APIs that MSVC/Windows SDK don't provide.

One thing that is left out of this change is whether to be completely
accurate with the toolchain cache hash (requiring a large list of files
as resources, and making those built very frequently), whether we'd
rely on manual updates, or if we should go with completely uncached
tasks. This can be left for a followup, the tasks not being hooked up
to be actually used by other tasks yet.

Differential Revision: https://phabricator.services.mozilla.com/D16302

--HG--
extra : moz-landing-system : lando
2019-01-11 22:29:35 +00:00
Dustin J. Mitchell
1516eb0435 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere in production, and
TASKCLUSTER_PROXY_URL is set wherever the proxy is active.

The taskgraph Taskcluster utils module gets a `get_root_url()` that gets the
root URL for the current run, either from an environment variable in production
or, on the command line, defaulting to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

Other changes to use this function are reserved for later commits.

This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.

--HG--
extra : rebase_source : 4f50e9d066da62a1887baabd8603844c85a32ee6
extra : source : 5ea6f03f845e49d503f5d0283557f54561c41654
2018-09-25 20:18:19 +00:00
Sebastian Hengst
767c971623 Backed out 21 changesets (bug 1492664) for breaking cron task for nightlies. a=backout
Backed out changeset a7d50dbb2c8e (bug 1492664)
Backed out changeset 2d876c4ece8b (bug 1492664)
Backed out changeset c82285d253de (bug 1492664)
Backed out changeset bf6d089640eb (bug 1492664)
Backed out changeset d9a7f2ce49c3 (bug 1492664)
Backed out changeset 06c466ab4323 (bug 1492664)
Backed out changeset c1ea4a10cc8d (bug 1492664)
Backed out changeset 4c63a04fdd47 (bug 1492664)
Backed out changeset 742b038bb1dd (bug 1492664)
Backed out changeset 911b4b0fb683 (bug 1492664)
Backed out changeset 870c8cec99e5 (bug 1492664)
Backed out changeset 77699b51336b (bug 1492664)
Backed out changeset 29f33f22fd8b (bug 1492664)
Backed out changeset e7f305408708 (bug 1492664)
Backed out changeset 335a92b1f424 (bug 1492664)
Backed out changeset c566f1c8dcdf (bug 1492664)
Backed out changeset c77ae59aba41 (bug 1492664)
Backed out changeset 9c35dd209c6b (bug 1492664)
Backed out changeset a972d6b4434e (bug 1492664)
Backed out changeset 5ea6f03f845e (bug 1492664)
Backed out changeset 0699d3873e44 (bug 1492664)

--HG--
extra : histedit_source : 5cb1f7e50f25d4a875c1a58c86b7dce902e1a89c%2C20f1ab1a843b612cfcc67cf5c6ff745d65abf076
2018-12-20 12:43:22 +02:00
Dustin J. Mitchell
4a0a6eae05 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.

--HG--
extra : rebase_source : 0626ebdb66a9f4078cb75ab71be256c334297363
2018-09-25 20:18:19 +00:00
Margareta Eliza Balazs
2e5e28f518 Backed out 16 changesets (bug 1492664) for breaking developer artifact builds, requested by standard8 a=backout
Backed out changeset 31e500489665 (bug 1492664)
Backed out changeset f4945658d45f (bug 1492664)
Backed out changeset 6d17291b8b92 (bug 1492664)
Backed out changeset 90f3faa36137 (bug 1492664)
Backed out changeset 0b229b00818a (bug 1492664)
Backed out changeset 5eb2c77d70a9 (bug 1492664)
Backed out changeset e1ebad5d89c5 (bug 1492664)
Backed out changeset 3017e5890739 (bug 1492664)
Backed out changeset c8b7e620eabf (bug 1492664)
Backed out changeset d3dfbd848236 (bug 1492664)
Backed out changeset 5c92bb5ac895 (bug 1492664)
Backed out changeset fb7cfca6ebc3 (bug 1492664)
Backed out changeset 0c4101230d4d (bug 1492664)
Backed out changeset b93a0fcc86f3 (bug 1492664)
Backed out changeset 6dc9522ee0bf (bug 1492664)
Backed out changeset 85d7f8b330eb (bug 1492664)
2018-12-19 11:45:29 +02:00
Dustin J. Mitchell
211a6cf5e3 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process to propagate TASKCLUSTER_ROOT_URL into
the docker images, and for good measure includes some code to use that value to
generate debian repo paths.

Differential Revision: https://phabricator.services.mozilla.com/D14196

--HG--
extra : moz-landing-system : lando
2018-12-18 05:37:41 +00:00
Mike Hommey
db4394b257 Bug 1433033 - Create a base Debian docker image. r=dustin
That image is used to derive all the debian7-* images, and its
definition is parametrized, which will allow to create other images
based on other versions of Debian, from the same definition.

XZ_OPT is kept in each of those because we don't want to automatically
set it in all further derived images.

--HG--
extra : rebase_source : 7f4597c1ea4af83627a9373dbdc7945d20b7d996
2018-01-25 22:15:19 +09:00
Mike Hommey
3972032498 Bug 1433033 - Don't apt-get clean from Debian-based docker images. r=dustin
The base images from docker hub actually contain a
/etc/apt/apt.conf.d/docker-clean that does the equivalent of an apt-get
clean after installing packages.

--HG--
extra : rebase_source : 190de9e3b10a0309cf9cfb3260a91477a5a93ba3
2018-01-25 21:57:13 +09:00
Mike Hommey
04b424481d Bug 1433033 - Set apt defaults to avoid passing -q, -y and --no-install-recommends to it. r=dustin
--HG--
extra : rebase_source : 53c8f00ef24dc8757d8eb45d1a9e08da48539a7a
2018-01-25 22:09:56 +09:00
Mike Hommey
ce90eb437b Bug 1433033 - Set Acquire::Check-Valid-Until in apt config file rather than pass it on the command line. r=dustin
Also set it in the android-build image, which is Debian-snapshot-based too.

--HG--
extra : rebase_source : a64a0d6833c98025bfe287e197a9ef255b12a9bc
2018-01-25 15:24:05 +09:00
Mike Hommey
fd6008421c Bug 1433033 - Remove a few packages from the toolchain-build image. r=dustin
python-dev was required to build mercurial, but the need for that was
removed in bug 1429669.
The others were required for mingw32 toolchains, but they are using a
different docker image and will switch to another different docker image.

--HG--
extra : rebase_source : b65c586a325f220c565e79afb3d3c9acc9f922bc
2018-01-25 15:13:20 +09:00
Mike Hommey
1e499e6b27 Bug 1392370 - Enable xz parallel compression on Debian-based docker images. r=gps
This makes the XZ_OPT environment variable set for every task that runs
on those docker images, effectively enabling parallel compression for
all of them.

For e.g. the gcc 4.9 toolchain build task, this makes the final
compression step take 23s instead of > 3 minutes.

--HG--
extra : rebase_source : 83809f1f72007cc0829573f1def29a9afdc300c9
2018-01-18 15:22:30 +09:00
Mike Hommey
d6510c894f Bug 1431297 - Remove the pxz package from the toolchain-build image. r=dustin
Now that we have a version of xz-utils that supports parallel compression,
we're not going to use it.

--HG--
extra : rebase_source : 7ac7840e979176b8d2042a4ef35f5868acd6f224
2018-01-18 14:58:33 +09:00
Mike Hommey
4de239b64f Bug 1429669 - Install the mercurial package in the toolchain-build docker image. r=dustin
--HG--
extra : rebase_source : 2ee1fc5816d1269a2fb189a33f5f2b0efb7e35d6
2018-01-11 14:07:57 +09:00
Mike Hommey
5f8431c3b1 Bug 1429669 - Separate mercurial conf from install-mercurial.sh. r=dustin
The install-mercurial.sh script currently installs a global mercurial
configuration after installing mercurial manually. In order to share
that configuration with docker images installing a mercurial package
through packages tasks, we move it to a separate file.

We however keep the part setting web.cacerts in install-mercurial.sh,
since it uses a path depending on what kind of environment the script is
run. Moreover, the instructions that come with mercurial to build
Debian packages come with web.cacerts set to the right path already, so
it's not needed in that case anyways.

At the same time, use multiple files in /etc/mercurial/hgrc.d/ instead
of a single /etc/mercurial/hgrc file.

--HG--
extra : rebase_source : 8140d8243cf012489025afe058f467c72224c891
2018-01-11 14:01:31 +09:00
Mike Hommey
d45f599f7a Bug 1429285 - Add other packages required to build toolchains. r=gps
--HG--
extra : rebase_source : 0446a2164492b34cecb608c76080b526901fc406
2018-01-10 13:12:22 +09:00
Mike Hommey
48af0db891 Bug 1429285 - Add cmake and ninja packages to the toolchain-build docker image. r=gps
We build packages of the same versions that were installed by
taskcluster/docker/recipes/install-cmake.sh and
taskcluster/docker/centos6-build/system-setup.sh in the desktop-build
image.

--HG--
extra : rebase_source : 843b89065daabd450f54ebf7a2cf55d00977e23a
2017-12-29 15:43:43 +09:00
Mike Hommey
2b4b15ad7e Bug 1427326 - Add a Debian 7-based docker image for toolchain builds. r=dustin
--HG--
extra : rebase_source : 97992266ffabc0db2d3e9eea567c3bd841aec936
2017-12-29 14:56:52 +09:00