This was preventing us from running ./mach try fuzzy from a subdirectory of the
topsrcdir. This also fixes taskgraph/utils/verify.py to find the docs directory
based off of topsrcdir instead of cwd. This was needed as cwd was being set at
import time.
MozReview-Commit-ID: CgQqD6bQ5q4
--HG--
extra : rebase_source : 4d863d442b0e715b3fe386c43b4077054c10dc70
Now that we have a Docker image with newer library versions on it, we
can move our builds over. The new images differ from the old
CentOS-based images in two important ways, though:
1) The system compilers in the new image are new enough to be used as
host compilers; additionally, our CentOS-built GCC compilers will not
work. We need to change the Android mozconfigs to reflect that. We
also need to change the Android tasks to not depend on the GCC
toolchain builds.
2) In a similar fashion, we can use the system JDK; we no longer need to
use the JDK from the Android NDK, which we had packaged up via the
Android dependencies task.
Both of these changes come with caveats: our l10n repack jobs continue
to run on the CentOS-based images; l10n repacks have not been completely
converted to Taskcluster. So we need to:
1) Retain the use of our custom GCC toolchain for HOST_CC/HOST_CXX on
the CentOS-based images.
2) Retain the JDK packages in the tooltool manifests, and referencing
them when we build on the CentOS-based images.
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
I also snuck in some last-minute assertions and minor fixes into this patch:
- don't stop reporting for a callee if we've seen it already (or rather, make the reachable set local to a root rather than global to all roots). This slows down runs with hundreds of hazards, but results in every problematic root being reported, for a more accurate count.
- annotate away some thread assertions
- special-case annotation for bug 1400435 since it's a whole family of hazards
--HG--
extra : rebase_source : ac7335d45e3e0772d34cb42cc6a3f628564fd3d1
CLOSED TREE
--HG--
extra : amend_source : 84120d6bacb5d72a9fbe41e4c3b405d63825da7c
extra : histedit_source : 8320c2193761b745f10850055ee74a3c9ac73615%2Cfbc2a28d8c5004a53305ef858ca5aea4245691e0
Now that we have a Docker image with newer library versions on it, we
can move our builds over. The new images differ from the old
CentOS-based images in two important ways, though:
1) The system compilers in the new image are new enough to be used as
host compilers; additionally, our CentOS-built GCC compilers will not
work. We need to change the Android mozconfigs to reflect that. We
also need to change the Android tasks to not depend on the GCC
toolchain builds.
2) In a similar fashion, we can use the system JDK; we no longer need to
use the JDK from the Android NDK, which we had packaged up via the
Android dependencies task.
Both of these changes come with caveats: our l10n repack jobs continue
to run on the CentOS-based images; l10n repacks have not been completely
converted to Taskcluster. So we need to:
1) Retain the use of our custom GCC toolchain for HOST_CC/HOST_CXX on
the CentOS-based images.
2) Retain the JDK packages in the tooltool manifests, and referencing
them when we build on the CentOS-based images.
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
The name `android-gradle-build` is an accident of history; let's rename it
before we attempt major surgery on it.
--HG--
rename : taskcluster/docker/android-gradle-build/Dockerfile => taskcluster/docker/android-build/Dockerfile
rename : taskcluster/docker/android-gradle-build/README.md => taskcluster/docker/android-build/README.md
rename : taskcluster/docker/android-gradle-build/REGISTRY => taskcluster/docker/android-build/REGISTRY
rename : taskcluster/docker/android-gradle-build/VERSION => taskcluster/docker/android-build/VERSION
rename : taskcluster/docker/android-gradle-build/buildprops.json => taskcluster/docker/android-build/buildprops.json
rename : taskcluster/docker/android-gradle-build/dot-config/pip/pip.conf => taskcluster/docker/android-build/dot-config/pip/pip.conf
rename : taskcluster/docker/android-gradle-build/oauth.txt => taskcluster/docker/android-build/oauth.txt
This just adds two basic tests, one for a passing test and another for a
failing one. In mochitest, we use privileged APIs to also tests crashes,
assertions, asan and leaks. But these APIs aren't available to reftests
so I'm not sure how we can test these things.
I figure it's not worth holding the framework up on this though, I'll file
a follow-up to figure out something to do for that.
MozReview-Commit-ID: 59TSbsugT5T
--HG--
extra : rebase_source : 72ecd817017c8b7d55eab879db4f6ad5fecc54c0
This includes code for downloading a Firefox binary, downloading + setting up a tests.zip and
running output through mozharness' output parsers. This is all stuff that will also be required
for the reftest selftests.
I couldn't think of a better location to put this stuff, suggestions welcome.
MozReview-Commit-ID: 59TSbsugT5T
--HG--
extra : rebase_source : a328f6bc90e73fe23f9054933cd01a30065419f6
