Sign language packs via Autograph instead of AMO.
Differential Revision: https://phabricator.services.mozilla.com/D38148
--HG--
rename : taskcluster/ci/release-sign-and-push-langpacks/kind.yml => taskcluster/ci/release-push-langpacks/kind.yml
extra : moz-landing-system : lando
MANUAL PUSH: to allow docker images to build without closing autoland
Differential Revision: https://phabricator.services.mozilla.com/D41038
--HG--
extra : rebase_source : 60ae00549917411d1839b6e3f8e6ae962d217470
extra : amend_source : a2531b115f5732345f8c34c88669428510d100a4
Changed the required yml, ini, json, js and html files to add Jetstream2 benchmark test to Raptor.
The changes on the javascript files are on github.
Differential Revision: https://phabricator.services.mozilla.com/D35418
--HG--
extra : moz-landing-system : lando
There are now only two left:
- one for the OSX 10.11 SDK
- one for Visual Studio 2017
Differential Revision: https://phabricator.services.mozilla.com/D40751
MANUAL PUSH: avoid closing autoland while all docker images and
toolchains are rebuilt.
--HG--
rename : browser/config/tooltool-manifests/win32/build-clang-cl.manifest => browser/config/tooltool-manifests/win64/vs2017.manifest
Now that all GCC and related source tarballs extract to paths
independent of their version number, the scripts are all very
look-alike, so they can be consolidated.
Differential Revision: https://phabricator.services.mozilla.com/D40749
Bug 1479533 was proposing to add a similar functionality, but this
iteration avoids actually unpacking anything, and ensures
reproducibility by relying on the reproducible bits from the original
archives: file ordering, flags, etc. (since they are checksummed, those
are never going to change for a given archive).
Another notable difference is that this applies the repack on the fetch
task itself, rather than create a separate task to apply the repack. The
latter has advantages, in that it allows to change the repacking without
redownloading the original file from a third-party server, but in
practice, most changes to the repacking would trigger the download tasks
anyways.
This patch only takes care of changing the archive type (zip->tar), and
the compression type (anything->zstandard).
Differential Revision: https://phabricator.services.mozilla.com/D40740
This also allows toolchain tasks to use aliases via fetches, which they
currently aren't allowed via use_toolchain. There are more toolchains
now than there were when the restriction was added, and it might be
useful to be able to use aliases. The flip side is that there are some
risks involved with aliases, which is why the restriction was there in
the first place. Let's see how things play out.
Differential Revision: https://phabricator.services.mozilla.com/D40713
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
The build docker images need python-dev installed to build psutil, used
by the build resources monitor.
Differential Revision: https://phabricator.services.mozilla.com/D40781
--HG--
rename : python/mozbuild/mozbuild/resources/html-build-viewer/index.html => python/mozbuild/mozbuild/resources/html-build-viewer/build_resources.html
extra : moz-landing-system : lando
They are now all very similar, and there are only a few variations.
Differential Revision: https://phabricator.services.mozilla.com/D40365
MANUAL PUSH: avoid closing autoland while toolchains are rebuilt.
This removes the need for the hacks in wr-macos-cross-build-setup.sh and
wrench-windows-tests.sh, while keeping things working in other scripts.
Differential Revision: https://phabricator.services.mozilla.com/D40184
Bug 1563711 changed MOZ_FETCHES_DIR to make the MOZ_ANDROID_FAT_AAR_*
environment variables absolute paths. Unfortunately, the replacement
relies on non-osx/windows tasks to run in docker-worker, which is not
necessarily true, and that makes the MOZ_FETCHES_DIR wrong in
non-osx/windows generic-worker tasks. Apparently, that currently works,
but that's not guaranteed to stay this way.
The MOZ_ANDROID_FAT_AAR_* environment variables don't need to be
absolute paths, though. MOZ_FETCHES_DIR is normalized by run-task, and
MOZ_ANDROID_FAT_AAR_* can be set relative to that, which we do here.
Differential Revision: https://phabricator.services.mozilla.com/D40334
API lint is arguably the most valuable lint of all, but it's also hard
to fit into the Phab ecosystem, since there's no place to hang the
"API hash not correct" message in the case when the hash hasn't been
updated at all. Therefore, this commit doesn't convert it. See also
https://github.com/mozilla-mobile/gradle-apilint/issues/61 for adding
file/line information to API lint.
Differential Revision: https://phabricator.services.mozilla.com/D35277
--HG--
rename : mobile/android/config/mozconfigs/android-api-16-frontend/nightly => mobile/android/config/mozconfigs/android-api-16/nightly-android-lints
extra : moz-landing-system : lando
This was Gradle-only and then !Gradle-only. Now Gradle is required
and this is unused.
Differential Revision: https://phabricator.services.mozilla.com/D31381
--HG--
extra : moz-landing-system : lando
This bug fixes up some minor issues in the raptor taskcluster configuration by removing the comm-central repository, adding a comment about requiring --full when run-on-projects is [], removing the run-on-projects entry for raptor-tp6-3-firefox, and moving the android/opt run-on-projects definition closer to the android/pgo definition in the defaults section.
Differential Revision: https://phabricator.services.mozilla.com/D40282
--HG--
extra : moz-landing-system : lando
This was originally (I assume) used to control checking out build-tools, in a previous iteration of
our automation for checking out source. Our current tool doesn't look at this variable, and doesn't
support checking out build-tools.
Differential Revision: https://phabricator.services.mozilla.com/D40234
This turns on mochitests (except for gpu and remote) on Linux 64 (opt+debug) and
Win64 (opt only).
Differential Revision: https://phabricator.services.mozilla.com/D39917
--HG--
extra : moz-landing-system : lando
Plenty of places use `nproc`, and only a couple use `getconf
_NPROCESSORS_ONLN`. Use the former instead of the latter.
Differential Revision: https://phabricator.services.mozilla.com/D39999
--HG--
extra : moz-landing-system : lando
Add functionality for being able to send extra parameters for the test_url query of a test, directly from a taskcluster config.
Also, the PR adds logic in the `setup-raptor` taskgraph transform for dynamically changing the subset of youtube-playback tests based on the platform and project
Differential Revision: https://phabricator.services.mozilla.com/D39006
--HG--
extra : moz-landing-system : lando
Run fenix and refbrow raptor cold page-load tests once a day, through cron, at 3 AM.
Differential Revision: https://phabricator.services.mozilla.com/D39485
--HG--
extra : moz-landing-system : lando
This is loosely based on what was in bug 1467359, but simplified to
handle git only, and simply using git-archive because, at least now,
it's deterministic (it uses the commit date as timestamp in tar
archives).
This also adds 4 tasks for some of the things we use for toolchains, but
doesn't hook them up yet.
This also upgrades the fetch docker image to Debian buster, and installs
the required packages in it.
Differential Revision: https://phabricator.services.mozilla.com/D39480
Bug 1525373 had been waiting for a while for aarch64 windows fixes that
are still not there, and landed with a workaround for those. But while
waiting, bug 1555479 changed the run-task transform, making bug 1525373
double-wrap bitbar commands with /builds/taskcluster/script.py.
So remove the now redundant wrapping.
Differential Revision: https://phabricator.services.mozilla.com/D39821
--HG--
extra : moz-landing-system : lando
For some tasks, the workdir known to the decision task doesn't actually match
the workdir used in the task, which makes MOZ_FETCHES_DIR wrong when the
decision task derives it from the workdir.
On other tasks, MOZ_FETCHES_DIR is set to a relative directory, which
may work in some places where MOZ_FETCHES_DIR is used, but not in
others, that happen to be executed from a different directory.
To solve both problems, we set MOZ_FETCHES_DIR as a relative directory
everywhere, and we make run-task normalize it to an absolute path
before executing the task.
Differential Revision: https://phabricator.services.mozilla.com/D39152
--HG--
extra : moz-landing-system : lando
Avoid intermittently exceeding the already very long max-run-time in some chunks.
Differential Revision: https://phabricator.services.mozilla.com/D39679
--HG--
extra : moz-landing-system : lando
For some tasks, the workdir known to the decision task doesn't actually match
the workdir used in the task, which makes MOZ_FETCHES_DIR wrong when the
decision task derives it from the workdir.
On other tasks, MOZ_FETCHES_DIR is set to a relative directory, which
may work in some places where MOZ_FETCHES_DIR is used, but not in
others, that happen to be executed from a different directory.
To solve both problems, we set MOZ_FETCHES_DIR as a relative directory
everywhere, and we make run-task normalize it to an absolute path
before executing the task.
Differential Revision: https://phabricator.services.mozilla.com/D39152
--HG--
extra : moz-landing-system : lando
It took me several searchs after having looked here for how to create windows
interactive tasks. Add a link to the documentation here as a hint.
Differential Revision: https://phabricator.services.mozilla.com/D39568
--HG--
extra : moz-landing-system : lando
This adds the mobile project name, like "geckoview" or "fennec" to the Android
task labels. This makes the firefox/geckoview/fennec distinction more obvious
in 'mach try fuzzy' and 'mach try chooser'; it does not appear to affect
try syntax. This also adds "geckoview"/"fennec" to the job names displayed in
the lower-left treeherder pane. Hopefully this will help to clarify which
Android application is under test for each task.
Differential Revision: https://phabricator.services.mozilla.com/D39093
--HG--
extra : moz-landing-system : lando
Switch android-hw jsreftest to geckoview. Technically this runs all
android-hw reftests on geckoview -- I don't imagine we'll ever want to
run any android-hw fennec reftests.
Differential Revision: https://phabricator.services.mozilla.com/D39350
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
Increase test chunks to avoid intermittent task timeouts. At 18 chunks,
wpt-4 is the longest running, at about 90 minutes; 18 also aligns with
several /debug platforms.
Differential Revision: https://phabricator.services.mozilla.com/D39129
--HG--
extra : moz-landing-system : lando
This build re-uses the PGO profile from the win64 build in the
win64-aarch64-shippable-no-eme part of the aarch64 build. Even though
the profile isn't generated on the smae platform, we still get enough of
a performance win to make this worthwhile.
Note that the pgo_flags() in configure need to be tweaked slightly since
we don't supprt the -fprofile-generate flag for aarch64 (we don't build
the clang_rt.profile lib there). So we always want to return the flags
namespace to make sure we get the use_* versions of flags, which we do
need.
Differential Revision: https://phabricator.services.mozilla.com/D38928
--HG--
extra : moz-landing-system : lando
mozharness and mozharness test transforms for generic-worker currently
don't wrap the commands with run-task. This changes things such that the
commands are wrapped with run-task, by piggy-backing on the run_task
transform.
Some things then become redundant with what the run_task transform does,
and some others need to happen later than they currently do in order to
work.
Depends on D28026
Differential Revision: https://phabricator.services.mozilla.com/D28027
--HG--
extra : moz-landing-system : lando
This gets the wrench android builds working again after
07168db23c565e4506690612a7be50738844ddb2.
Differential Revision: https://phabricator.services.mozilla.com/D38882
--HG--
extra : moz-landing-system : lando
Our datadog contract is coming to an end. Soon we'll add influxdb data collection
Differential Revision: https://phabricator.services.mozilla.com/D38086
--HG--
extra : moz-landing-system : lando
We should test what we ship, and having talos benchmark the shipped
platform will help verify any PGO performance wins.
Differential Revision: https://phabricator.services.mozilla.com/D37896
--HG--
extra : moz-landing-system : lando
In order to run tests against the win64-aarch64-shippable build, we need
the test packages from the aarch64-no-eme build since the tests aren't
compiled during the -shippable stage. Since packages like cppunittest
and the target.test_packages.json file won't be generated correctly in
the -shippable stage, we disable the package-step tests there by setting
MOZ_AUTOMATION_PACKAGE_TESTS to 0 in the environment.
Differential Revision: https://phabricator.services.mozilla.com/D37895
--HG--
extra : moz-landing-system : lando
Not that it's causing any problem, but I noticed the build-base image
was being rebuilt when updating valgrind, and that seemed odd.
Differential Revision: https://phabricator.services.mozilla.com/D38297
--HG--
extra : moz-landing-system : lando
This fixes issues from bug 1565644 which caused speedometer to run on android-hw-p2-8-0-arm7-api-16/pgo on integration branches and android-hw-p2-8-0-arm7-api-16/pgo raptor tasks to run on mozilla-beta. This patch disables the aforementioned tests.
Differential Revision: https://phabricator.services.mozilla.com/D38111
--HG--
extra : moz-landing-system : lando
We've been lucky that non-sanitizer cross-builds for macosx have not
required the clang runtime so far, but they soon will. And it's only
available in the mac-cross clang toolchain, so we need to use that on
all macosx builds.
Differential Revision: https://phabricator.services.mozilla.com/D38260
--HG--
extra : moz-landing-system : lando
Changed the required yml, ini, json, js and html files to migrate ARES6 benchmark test to Raptor.
Differential Revision: https://phabricator.services.mozilla.com/D34178
--HG--
extra : moz-landing-system : lando
We should test what we ship, and having talos benchmark the shipped
platform will help verify any PGO performance wins.
Differential Revision: https://phabricator.services.mozilla.com/D37896
--HG--
extra : moz-landing-system : lando
In order to run tests against the win64-aarch64-shippable build, we need
the test packages from the aarch64-no-eme build since the tests aren't
compiled during the -shippable stage. Since packages like cppunittest
and the target.test_packages.json file won't be generated correctly in
the -shippable stage, we disable the package-step tests there by setting
MOZ_AUTOMATION_PACKAGE_TESTS to 0 in the environment.
Differential Revision: https://phabricator.services.mozilla.com/D37895
--HG--
extra : moz-landing-system : lando
Also add the `mac_geckodriver` behavior, and remove the unused `mac_pkg` behavior.
Differential Revision: https://phabricator.services.mozilla.com/D36690
--HG--
extra : moz-landing-system : lando
This patch disables raptor tests running on android opt-builds and as a side, prevents raptor tests running on android from running on try by default. With these changes, --full in mach try fuzzy will be required to schedule raptor mobile tests on try.
Differential Revision: https://phabricator.services.mozilla.com/D37905
--HG--
extra : moz-landing-system : lando
Only one place should need to know how to generate VCS URLs. This
makes it easier to support other VCSs.
Differential Revision: https://phabricator.services.mozilla.com/D37772
--HG--
extra : moz-landing-system : lando
We need a fix from `cctools-port` master for cross-language LTO builds
to work properly on the Mac. Rather than cherry-picking yet another
commit, which would have to deal with a updated `ld64` upstream, we've
opted to go ahead and update directly to upstream.
This choice brings about some significant build changes, as TAPI support
has moved to a different library that is not easily buildable directly.
Differential Revision: https://phabricator.services.mozilla.com/D36636
--HG--
extra : moz-landing-system : lando
This only uses cross-platform tools, so switch to running these on linux, which
cuts the runtime down from ~20m to ~3m.
Differential Revision: https://phabricator.services.mozilla.com/D1080
--HG--
extra : moz-landing-system : lando
This also switches it to use the generic toolchain build image, as
it is no longer being used exclusively by mingw builds.
Differential Revision: https://phabricator.services.mozilla.com/D24230
--HG--
extra : moz-landing-system : lando
Since the taskgraph knows the mar-channel-id to use, pass it explicitly to
the partial tasks, rather than extracting the channel from a MAR.
Differential Revision: https://phabricator.services.mozilla.com/D37482
--HG--
extra : moz-landing-system : lando
Rather than relying on the mar-channel-id set in the `mar` binary, set the channel
explicitly from taskcluster. This allows us to re-use the `mar` binary between
builds/channels.
Differential Revision: https://phabricator.services.mozilla.com/D37481
--HG--
extra : moz-landing-system : lando
Taskgraph needs to know the correct mar-channel, so allow it to pass it into the build,
rather than keying off the update-channel in configure. This will allow using a `mar`
binary that doesn't have the mar-channel configured in.
Differential Revision: https://phabricator.services.mozilla.com/D37480
--HG--
extra : moz-landing-system : lando
This patch prevents raptor-youtube-playback-geckoview-live from running on Pixel 2 arm7 builds on mozilla-central.
Differential Revision: https://phabricator.services.mozilla.com/D36935
--HG--
extra : moz-landing-system : lando
geckoview-junit now runs on x86_64 opt so we don't need to run the tests on arm
too.
Differential Revision: https://phabricator.services.mozilla.com/D37464
--HG--
extra : moz-landing-system : lando
In newer docker versions the LeakSanitizer will fail unless it runs in a
privileged container. During the gecko build process in asan builds, the
support tools built as part of the build process will also run with
LeakSanitizer by default, but there is no need for such.
We then disable runtime leak sanitizer in asan builds to avoid erros in
builds inside a docker container.
Differential Revision: https://phabricator.services.mozilla.com/D37295
--HG--
extra : moz-landing-system : lando
We need a fix from `cctools-port` master for cross-language LTO builds
to work properly on the Mac. Rather than cherry-picking yet another
commit, which would have to deal with a updated `ld64` upstream, we've
opted to go ahead and update directly to upstream.
This choice brings about some significant build changes, as TAPI support
has moved to a different library that is not easily buildable directly.
Differential Revision: https://phabricator.services.mozilla.com/D36636
--HG--
extra : moz-landing-system : lando
Reusing sessions doesn't actually save that much time and avoids a lot of
hard-to-debug intermittents.
Differential Revision: https://phabricator.services.mozilla.com/D32578
--HG--
extra : moz-landing-system : lando
Debug mochitests are running as well as opt now, and sufficient for tier 1.
Differential Revision: https://phabricator.services.mozilla.com/D37016
--HG--
extra : moz-landing-system : lando
There are some intermittent failures in xpcshell tests (eg bug 1562344)
in the new emulator; let's revert to the older emulator until those can
be figured out.
Differential Revision: https://phabricator.services.mozilla.com/D37110
--HG--
extra : moz-landing-system : lando
`mochitest-media` now runs on macosx1014 reliably. Turn off this test suite for macosx1010.
Differential Revision: https://phabricator.services.mozilla.com/D37094
--HG--
extra : moz-landing-system : lando
The periodic file update job pulls firefox from the index. The index path used
is no longer built on release branches, so is using a outdated builds
(mozilla-beta is using tasks from 2019-03). Switch to using a task that is built
on all branches.
Differential Revision: https://phabricator.services.mozilla.com/D36979
--HG--
extra : moz-landing-system : lando
As far as I can tell, the intermittent suite_start failures are due to
stdout/stderr output interleaving -- a known issue for reftest logging,
without a clear way forward. Let's work around it the same way we did
on Windows.
Differential Revision: https://phabricator.services.mozilla.com/D37023
--HG--
extra : moz-landing-system : lando
Only enabled on try/m-c as tier-2 for now, per email discussion, to minimize
load on bitbar Pixel 2 devices.
Depends on D36799
Differential Revision: https://phabricator.services.mozilla.com/D36800
--HG--
extra : moz-landing-system : lando
Bug 1451104 removed the GCC 4.9 toolchain, but left the build script and
the patch that only that toolchain build was using.
Differential Revision: https://phabricator.services.mozilla.com/D36886
--HG--
extra : moz-landing-system : lando
Changes:
- disable some tests in `toolkit/content/tests/chrome/chrome.ini` with approval from enndeakin
- migrate tests to run on macosx1014
Differential Revision: https://phabricator.services.mozilla.com/D36985
--HG--
extra : moz-landing-system : lando
Changes:
- remove `fail-if` conditions for tests that now pass on macosx1014
- add one `fail-if` condition for macosx1014 (`test_2_conformance2__textures__misc__tex-base-level-bug.html`) with approval from jgilbert
- migrate mochitest-webgl2-core test to run on macosx1014
Differential Revision: https://phabricator.services.mozilla.com/D36982
--HG--
extra : moz-landing-system : lando
Changes:
- tying up loose ends from D34589 where I forgot to remove mochitest-plain from macosx1010
Differential Revision: https://phabricator.services.mozilla.com/D36983
--HG--
extra : moz-landing-system : lando
Often times, PGO builds aren't required for testing things (in particular,
testing release automation). However, at least when testing release automation,
we do need to use the shippable build type.
Add an option to mach try that will disable using the 3-tier PGO jobs.
Differential Revision: https://phabricator.services.mozilla.com/D36365
--HG--
extra : moz-landing-system : lando
Only enabled on try/m-c as tier-2 for now, per email discussion, to minimize
load on bitbar Pixel 2 devices.
Depends on D36799
Differential Revision: https://phabricator.services.mozilla.com/D36800
--HG--
extra : moz-landing-system : lando
With this patch, all android power tests (idle, idle-bg, and speedometer) will only run once a week, through cron, on saturday.
Differential Revision: https://phabricator.services.mozilla.com/D36827
--HG--
extra : moz-landing-system : lando
