Volumes are a docker-worker concept. They shouldn't be encountered on
Windows, which uses generic-worker.
MozReview-Commit-ID: KUdSxVHVJQ
--HG--
extra : rebase_source : ff54131d471ae2ae2465b11ca5ba6e787f5d11de
main() is quite long. And the control flow will become more complicated
as we support Windows.
Let's move the bulk of the cache configuration code into a standalone
function so main() is less cluttered.
MozReview-Commit-ID: xredCubr1E
--HG--
extra : rebase_source : 385fe6fe9e083cf585edc922b1fa26355d580c02
The code for cache and volume normalization makes assumptions that uid
and gid are defined. They are not defined if not running as root and
Python would crash in these code paths.
So, presumably this means that all tasks using run-task are running as
root. Let's codify that requirement.
This requirement is arbitrary. But let's not scope bloat run-task to
support scenarios until we need them.
MozReview-Commit-ID: 2uW4OSovzWi
--HG--
extra : rebase_source : 56ced8ecbd0eef3a55dc68413f4eab7601756cc0
I want to make run-task work on Windows. The script is currently very
POSIX oriented, as it assumes the existence of the grp and pwd modules,
that user IDs are numeric, and that system calls like setresgid() and
setresuid() are available, etc.
This commit starts to make some of the POSIX-centric code conditional
on running on POSIX.
Code for uid/gid extraction has been moved to its own function. Some
error messages were tweaked slightly as part of the move. Otherwise,
the changes should be pretty straightforward.
There are still other parts of this file that won't work on e.g.
Windows. But this gets us a big step closer.
MozReview-Commit-ID: HNyytKcBbBo
--HG--
extra : rebase_source : 01d653c801865e36e562a8de4c9e6d6962498f19
In preparation for making it usable on Windows, after which point
having it in a directory with "docker" in it doesn't make much sense.
MozReview-Commit-ID: Hgu0buFyJwF
--HG--
rename : taskcluster/docker/recipes/run-task => taskcluster/scripts/run-task
extra : rebase_source : 3c0b502d28b5aad54bd04069efbfda88e25bbb20
This option shouldn't be used for local builds (see bug 1294157). Set
the option from the crate's taskcluster script instead, so that it's
used only for automated builds.
--HG--
extra : rebase_source : 94b398a0f1fac60094269d755735c426ecc63e17
This will allow mozharness configs to be specified exclusively in the taskgraph.
Differential Revision: https://phabricator.services.mozilla.com/D1017
--HG--
extra : rebase_source : a3a9b6cc9d1004c4bd396fccc3e4354a7316651d
extra : source : 10acd193df92b7c495789dc24157b85f116ade5e
And adapt the build-gcc.sh script for the changes to
contrib/download_prerequisites.
--HG--
rename : taskcluster/scripts/misc/build-gcc-6-linux.sh => taskcluster/scripts/misc/build-gcc-7-linux.sh
extra : rebase_source : b1d785777b8c141c0eb0f52a73734abd2db21b94
The URL is now being redirected to
https://www.openssl.org/source/old/1.1.0/openssl-1.1.0g.tar.gz. Let's
add a -L so we follow redirects automatically.
MozReview-Commit-ID: AuZ98jGidzl
--HG--
extra : rebase_source : 07e61558024e789df45d8e2ab67ab5ad9d3d355b
Build the latest tup master branch with the LD_PRELOAD dependency
checker.
MozReview-Commit-ID: ALfnnmOZrky
--HG--
extra : rebase_source : 529d4392ef73e03f66fb76f089f8b88f45b44972
Note that static analysis was the only remaining user of the 32-bit toolchain, so I've removed win32-clang-cl (or rather, renamed it to win32-clang-cl-st-an).
--HG--
rename : build/build-clang/clang-win32.json => build/build-clang/clang-win32-st-an.json
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-st-an.json
rename : taskcluster/scripts/misc/build-clang32-windows.sh => taskcluster/scripts/misc/build-clang32-st-an-windows.sh
rename : taskcluster/scripts/misc/build-clang64-windows.sh => taskcluster/scripts/misc/build-clang64-st-an-windows.sh
Ensure better determinism when creating rust toolchain packages
by rejecting generic channels like 'stable' or 'nightly'. Instead,
insist on a specific version or date.
The current valid dates for beta and nightly can be obtained with:
curl -s https://static.rust-lang.org/dist/channel-rust-beta.toml | grep ^date
curl -s https://static.rust-lang.org/dist/channel-rust-nightly.toml | grep ^date
MozReview-Commit-ID: I0DXw1KJGZz
--HG--
extra : rebase_source : 92e158193072582b8568d9c9f00ffdefa0af1a9c
The Proguard dependency is now managed by Gradle.
MozReview-Commit-ID: EOvKSE5z28P
--HG--
extra : rebase_source : 760b117f500cc639cc8c24e9c02933990f358dd7
We'd like to install the NDK through the Android SDK manager. But we
can't pin versions of the NDK with the SDK manager, and so Google
can silently upgrade the NDK on us. Since that is undesirable, this is
the next best thing.
With the toolchain task in hand, we can make all the relevant tasks
depend on the toolchain task and remove the download of the NDK from
tooltool as well.
I tested this on automation and the build went on, though I couldn't test
the bindgen output because the build right now is busted on one dependent crate
with rust beta, which is the first toolchain that has this package, and will go
to release shortly.
This should work though! If I need more changes I'll adjust them in bug 1432153.
You can test the repackage manually with repack_rust.py --toolchain beta, for
example.
MozReview-Commit-ID: GI2f6vGVqTe
Don't build ucl when building upx, Debian stretch has a recent enough
version. In fact, the last upstream version doesn't build with GCC in
Debian stretch (http://bugs.debian.org/811707)
--HG--
extra : rebase_source : aae67773b9dd3b99f6ddf9ab7f59a628037e6925
New Android-Gradle plugins pin the build-tools version, and we want to
be consistent between Gradle and moz.build.
MozReview-Commit-ID: ApWS4rHzPuH
--HG--
extra : rebase_source : 22008e9333b15c594ce26c2a52f67396d6e3ab84
extra : source : f918500d9cf5112b70bc8e0a120df435b02252b7
Turns out Google's maven repository doesn't publish checksums. I
can't imagine why not, but there it is. We have to think more about
whether to trust the artifacts downloaded from maven.google.com.
MozReview-Commit-ID: CdWijorq1IV
--HG--
extra : rebase_source : 6c66cf1444876624f10409ea6437863e2c2ea9b0
extra : source : 0850b319efd43ac8f4d61485451722975da55ca1
Bump mingw version to get the newest commit and do not include the
un-needed dw-extras.h on MinGW (thanks Jacek!)
MozReview-Commit-ID: OjO93XHCxs
--HG--
extra : rebase_source : 933bbb385004988a23d1069c9cd3241b3a3b336e
llvm-symbolizer is necessary to get symbols in llvm-dsymutil crash
dumps. While we could use the one from clang during the build, it's
better if the llvm-dsymutil toolchain is standalone for local testing.
--HG--
extra : rebase_source : 5cd234a3e14ab52a4ce759821e0e756e68167797
When I originally wrote the llvm-dsymutil build script in bug 1430315,
I wasn't setting CMAKE_BUILD_TYPE to Release, and was ending up with
a very large binary (> 300MB), so I stripped it.
When I later set CMAKE_BUILD_TYPE to Release, I left the manual
stripping on, but that removes symbols that are useful for stacktraces
when dsymutil crashes (the Release type still leaves out debug info).
--HG--
extra : rebase_source : 802daadc24c0090574b1a44ea8b4e6c25735f703
By default, wget prints dots every 1k bytes. This can render a
lot of output for large files. We switch to the "mega" style, which
makes each dot represent 64k, thus reducing output by up to 64x.
We also force the use of dot display. By default, it uses "bar"
which attempts to use terminal formatting if possible. Since most
of this code executes in CI and terminal control characters can
interfere with logged output, we force the use of "dot." (Although
wget appears to automatically switch to dot in TC today. But
consistency is good.)
MozReview-Commit-ID: IpTWJdcauTV
--HG--
extra : rebase_source : 5c9aa1bbdcd78eaa0b31347ad026a2c1beaedc03
We've had problems with crashes in llvm-dsymutil for a while, and while
they are, in essence, due to the fact that rustc produces bad debug
info, they are a hurdle to our builds. The tool comes along clang, and
updating clang is not necessarily easy (witness bug 1409265), so, so
far, we've relied on backporting fixes, which can be time confusing
(witness bug 1410148).
OTOH, llvm-dsymutil is a rather specific tool, that doesn't strictly
need to be tied to clang. It's only tied to it because it uses the llvm
code to do some of the things it does, and it's part of the llvm source
tree. But it could just as well be a separate tool, like it was(is?) on
OSX.
So, we add a toolchain job to build it from the llvm source,
independently from clang, so that we can update it separately, if we
hit new crashes that happen to already be fixed on llvm trunk. It will
also allow to more easily update after upstream fixes crashes after we
report them.
--HG--
extra : rebase_source : b814353b4b4632e46646a24b8f54c5300618ff49
New Android-Gradle plugins pin the build-tools version, and we want to
be consistent between Gradle and moz.build.
MozReview-Commit-ID: ApWS4rHzPuH
--HG--
extra : rebase_source : 38a9781c472d858f3300cbbcbdc6d2311c465713
Turns out Google's maven repository doesn't publish checksums. I
can't imagine why not, but there it is. We have to think more about
whether to trust the artifacts downloaded from maven.google.com.
MozReview-Commit-ID: CdWijorq1IV
--HG--
extra : rebase_source : a884971e51ce7b1ff993754b130f462c476646ab
