The only tricky piece here is that the resulting toolchain archive is
private, and uses a newly allocated Task Cluster scope
(queue:get-artifact:project/gecko/android-sdk/*) to restrict access to
the archive. All SCM levels (1, 2, 3) have been given the new scope:
see https://tools.taskcluster.net/auth/roles/moz-tree:level:1 and
friends.
MozReview-Commit-ID: CcDqDOHODpe
--HG--
extra : rebase_source : 062bca8c65556f0f46e9c9cc6cd81eb04cf2b522
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
I also snuck in some last-minute assertions and minor fixes into this patch:
- don't stop reporting for a callee if we've seen it already (or rather, make the reachable set local to a root rather than global to all roots). This slows down runs with hundreds of hazards, but results in every problematic root being reported, for a more accurate count.
- annotate away some thread assertions
- special-case annotation for bug 1400435 since it's a whole family of hazards
--HG--
extra : rebase_source : ac7335d45e3e0772d34cb42cc6a3f628564fd3d1
CLOSED TREE
--HG--
extra : amend_source : 84120d6bacb5d72a9fbe41e4c3b405d63825da7c
extra : histedit_source : 8320c2193761b745f10850055ee74a3c9ac73615%2Cfbc2a28d8c5004a53305ef858ca5aea4245691e0
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
We're about to ban files in Docker volumes so they behave almost
identically to caches (which start empty).
We move the install of nexus.xml from Docker image time to
task time. This also means that changes to nexus.xml don't result
in having to rebuild the Docker image.
MozReview-Commit-ID: JIjeJN4mt2
--HG--
rename : taskcluster/docker/android-gradle-build/nexus.xml => taskcluster/scripts/builder/build-android-dependencies/nexus.xml
extra : rebase_source : 53848f06820bda7979b2ae15456e07f8aed2363d
The old process ran "before" and "after" steps as root. The
mozharness script doesn't run as root, which required some small
changes to not run Sonatype Nexus as root. Everything else is a
straight-forward move of the scripts out of the `android-gradle-build`
image and into `taskcluster/scripts`.
MozReview-Commit-ID: CqnNI33OKmb
--HG--
rename : taskcluster/docker/android-gradle-build/bin/after.sh => taskcluster/scripts/builder/build-android-dependencies/after.sh
rename : taskcluster/docker/android-gradle-build/bin/before.sh => taskcluster/scripts/builder/build-android-dependencies/before.sh
rename : taskcluster/docker/android-gradle-build/bin/repackage-jdk-centos.sh => taskcluster/scripts/builder/build-android-dependencies/repackage-jdk-centos.sh
extra : rebase_source : f94e6b9b780f96038c60d3825039a0f94add0404
Allow an extra heap write hazard introduced by enabling stylo
in default builds until it can be addressed. See bug 1384625.
MozReview-Commit-ID: 2N3z6FVHa0G
--HG--
extra : rebase_source : d4c9c18c34ab829b59b9e16ec3cd8e736c418fd3
Bug 1374940 adds a MOZ_TOOLCHAINS environment variable with a list of
path@task-id strings, where task-id is corresponding to the (possibly
optimized) toolchain job, and path corresponding to the
toolchain-artifact defined for that toolchain job.
We want to use that to pull artifacts instead of tooltool packages.
--HG--
extra : rebase_source : 277daa2c83d6d197975cb4ef36ee131176afa992
This is a quick work around for not blocking the progress of Bug 1380133.
We should definely investigate the real root cause sooner than later.
MozReview-Commit-ID: 8X1FH6f2GyN
--HG--
extra : rebase_source : 9edaec6a50f7251e3c05d9fbc252e1385d82a1bb
This is a quick work around for not blocking the progress of Bug 1380133.
We should definely investigate the real root cause sooner than later.
MozReview-Commit-ID: 8X1FH6f2GyN
We're about to make MOZ_AUTOMATION more strict about things like having
a source checkout.
The whole point of build-sm-package.sh is to verify that SpiderMonkey
can be built outside of Mozilla's source repo and automation from a
standalone package. Since the presence of MOZ_AUTOMATION can influence
so much behavior in the build system, unset it so that the job
tests a !Mozilla environment more accurately.
MozReview-Commit-ID: EMfyLKfY0uU
--HG--
extra : rebase_source : 3632a9abf9fac3f916ed9043f30d4b6aa4abb390
We're about to make MOZ_AUTOMATION more strict about things like having
a source checkout.
The whole point of build-sm-package.sh is to verify that SpiderMonkey
can be built outside of Mozilla's source repo and automation from a
standalone package. Since the presence of MOZ_AUTOMATION can influence
so much behavior in the build system, unset it so that the job
tests a !Mozilla environment more accurately.
MozReview-Commit-ID: EMfyLKfY0uU
--HG--
extra : rebase_source : 3632a9abf9fac3f916ed9043f30d4b6aa4abb390
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: zehcGJrUQX
--HG--
extra : source : feedcde68c2a54da210f03eb287ab5c862fc982b
extra : intermediate-source : 485d1af7805ad9fa0e701c3571fc1291fbfc6850
Hazard jobs use a specific tooltool-manifest field in their definition.
Since there is no post-processing happening on those definitions, and
since generalizing it would require adding the field to a bunch of
validation schemas, and the same code to various transforms, it's just
simpler to move to use environment variable definitions here too.
Eventually, tooltool manifests won't be necessary anyways, and those
environment variables will go away.
--HG--
extra : rebase_source : bd619024bde051424a2d811a1996caaf0f853247
A quick fix for hazard bustage by increase the NUM_ALLOWED_WRITE_HAZARDS
from 3 to 7 is pushed in bug 1348173 comment 37. In this bug, we shall do
the actual fix and restore the NUM_ALLOWED_WRITE_HAZARDS.
The -moz-border-*-colors bindings trigger errors because they're using
outparams (nsStyleBorder) which further manipulate its member (mBorderColors)
which is a double raw pointers. Since we don't have the ability to
whitelist the indirect access to mBorderColors[x] list, we can only add
them to the ignoreContents for now.
We might be able to move these bindings to the whitelist of the above
treatAsSafeArgument function, if we could refactor mBorderColors to use
nsTArray directly.
MozReview-Commit-ID: 2cQz58K2A10
--HG--
extra : rebase_source : af2b5b944fb9d19fe28f57eaa37f77174d48bfa4
The TASKCLUSTER_WORKER_GROUP environment variable used to contain the full
AWS availability zone, but a recent docker-worker change changed it to
be simply the AWS region, which broke sccache in taskcluster because we
were using it as part of the S3 bucket name.
MozReview-Commit-ID: 1KsfWpB4PoY
--HG--
extra : rebase_source : bdc61f180bf079eb0ad2cdbbd25e3e3a0deb62e6
(For "Integrate and fully support OSX Signing in taskcluster")
Written as a mozharness script rather than using bare ./mach command because we need to download the upstream artifact
and because we need to download artifacts from tooltool to do the packing back into a .dmg. Future ideal would be to get
rid of the mozharness script and use JUST ./mach.
This is using the ./mach repackage code being created in Bug 1347576. Taking a signed tarball from a dmg supported with
Bug 1346015, and the taskgraph work to schedule this is in Bug 1318505.
MozReview-Commit-ID: rv9l285HKC
--HG--
extra : rebase_source : 054219511419b8bf44b1f57a8d834a12c13710e3
extra : intermediate-source : a52bc37e08efbf4d6c68cc0f4e2d4b76f79b192a
extra : source : 6ad7468a590f5a2779ffdc3713c1f6f74ce23731
Tasks calling these generally use tooltool and the hazard
manifest to provide toolchains, but the setup job doesn't
they don't use a mozconfig to configure paths, and the analysis
job uses a different TOOLTOOL_DIR.
The build calls configure, which defaults to --enable-rust,
so we need to add the correct rust toolchain path to the
environment like we do for C++.
MozReview-Commit-ID: gFnZ0SK1f7
--HG--
extra : rebase_source : f7cabb5b15551f5b00ff8271ceddeb4b47146c03
