This includes a fix for the dist server people will need to efficiently
distribute builds. The version required by configure is not bumped with this
change, as this difference is crucial to the server but not relevant to the
client.
Differential Revision: https://phabricator.services.mozilla.com/D45230
--HG--
extra : moz-landing-system : lando
With the migration from AWS to GCP, we also need to migrate sccache
buckets from S3 to Google Storage.
The problem is how we deal with regions, since there isn't an exact
correspondence on the region names between the two cloud providers.
To make the transition smoother, docker-worker (and soon generic-worker)
provides a new environment variable called TASKCLUSTER_WORKER_LOCATION,
with information about the cloud provider the task is running on. Using
this new variable, we configure sccache to use the corresponding storage
service of the cloud provider where the task runs.
The bucket names in Google Storage are shorter because GCS imposes a
limit of 30 characteres for the names.
Ref: https://github.com/taskcluster/taskcluster-rfcs/pull/148/files
Differential Revision: https://phabricator.services.mozilla.com/D44845
--HG--
extra : moz-landing-system : lando
I found that I did not restore the `set -e` flag after temporarily disabling it in the debian10-specific piece of experimental code in `test-linux.sh`, and this caused a bunch of my try pushes to register as successful despite having multiple unexpected failures.
Differential Revision: https://phabricator.services.mozilla.com/D44774
--HG--
extra : moz-landing-system : lando
Increase max-run-time to avoid intermittent failures due to variance in robustcheckout
performance.
Differential Revision: https://phabricator.services.mozilla.com/D45200
--HG--
extra : moz-landing-system : lando
Adds command line option for developers to run tests against experimental debian 10 image (from D42597).
This is an experimental flag and will be removed once debian 10 image is used for production CI tests.
Differential Revision: https://phabricator.services.mozilla.com/D44236
--HG--
extra : moz-landing-system : lando
Without these scopes, we can't build Android configs from interactive
tasks, because we can't fetch the NDK and SDK from their toolchain tasks.
Differential Revision: https://phabricator.services.mozilla.com/D45048
--HG--
extra : moz-landing-system : lando
This commit prepares the decks for turning specific Raptor tasks into
Raptor + browsertime tasks. The `--browsertime` flag to `mach try
...` flips the switch; eventually, the Raptor harness will recognize
the `--browsertime` flag and use browsertime to perform the pageload
measurements.
To run browsertime, we need:
1) Node.js
2) the browsertime `node_modules` (provided by the
`toolchain-browsertime` task)
3) ffmpeg (for producing videos from captured frames)
4) chromedriver (in the future, when targeting Chrome/Chromium)
5) geckodriver (provided by the `toolchain-*-geckodriver` tasks)
6) `PATH` configured
This commit arranges those things.
Since the configuration varies by test platform, and eventually we
expect the changes implemented by the flag to be moved into YAML task
definitions, we elect to use `by-test-platform` conditionals as much
as possible. The end expression is pleasant, thanks to
`evaluate_keyed_by`.
Handling PATH, however, is a rabbit hole. At this time, it's not
possible to use `fetch` task repackaging, because `releng-hardware`
doesn't support `zstandard` (Bug 1576244) and there's no appetite to
avoid `zstandard` entirely (Bug 1576698). Generally PATH is
configured using `mozharness` configuration files, which can execute
arbitrary Python and configure the PATH only for browsertime jobs.
However, the Raptor mozharness script itself runs the Raptor harness
in a stripped down environment, throwing away modifications to PATH.
It's not clear what impacts changing that has, so we leave it alone,
and add a `--browsertime-ffmpeg` flag and custom handling in the
Raptor harness. This can transition smoothly into a browsertime flag
(so that the PATH doesn't need to be set at all) and into a unified
interface for Raptor and `mach browsertime` to configure the
browsertime execution environment.
Differential Revision: https://phabricator.services.mozilla.com/D38781
--HG--
extra : moz-landing-system : lando
Browsertime needs these to produce videos, and to invoke Chrome, respectively.
Differential Revision: https://phabricator.services.mozilla.com/D43698
--HG--
extra : moz-landing-system : lando
This patch prevents the job-defaults run-on-projects setting from overriding the task-specific run-on-projects setting. It also fixes a minor error in raptor-fennec.yml.
Differential Revision: https://phabricator.services.mozilla.com/D44877
--HG--
extra : moz-landing-system : lando
Stop running all Fennec functional (non-performance) tests:
- stop running all Android 4.3 tests
- switch android-em-7 cppunit and android-hw jittest from the Fennec apk to the
geckoview apk (no difference in behavior expected)
- stop running Android 7.0 marionette tests, since they also run against Fennec
- remove android-em-4.* references from taskcluster configs
- remove android instance: extra-large references from taskcluster configs,
since they only affect aws, which is no longer used for Android
Android-hw raptor tests running against Fennec remain; I will prepare a separate
patch for those.
Differential Revision: https://phabricator.services.mozilla.com/D43684
--HG--
extra : moz-landing-system : lando
The `./mach try {fuzzy,chooser}` commands now support a `--visual-metrics-jobs`
option which can be used to pass the job descriptions to the visual-metrics
task.
Differential Revision: https://phabricator.services.mozilla.com/D41878
--HG--
extra : moz-landing-system : lando
This new task fetches the visualmetrics.py script from the
github.com/mozilla/browsertime repository and runs it in parallel for the
specified jobs. Jobs are specified in a JSON blob passed through to the task in
an environment variable. A follow up patch specifies a command line argument to
make this configuration available to `./mach try {fuzzy|chooser}`
Differential Revision: https://phabricator.services.mozilla.com/D41052
--HG--
extra : moz-landing-system : lando
The `./mach try {fuzzy,chooser}` commands now support a `--visual-metrics-jobs`
option which can be used to pass the job descriptions to the visual-metrics
task.
Differential Revision: https://phabricator.services.mozilla.com/D41878
--HG--
extra : moz-landing-system : lando
This new task fetches the visualmetrics.py script from the
github.com/mozilla/browsertime repository and runs it in parallel for the
specified jobs. Jobs are specified in a JSON blob passed through to the task in
an environment variable. A follow up patch specifies a command line argument to
make this configuration available to `./mach try {fuzzy|chooser}`
Differential Revision: https://phabricator.services.mozilla.com/D41052
--HG--
extra : moz-landing-system : lando
This picks up 0a1d495478d8ed1a94fc77b9dbb428b7e0372588 needed by
Tor for the updater.
Differential Revision: https://phabricator.services.mozilla.com/D44535
--HG--
extra : moz-landing-system : lando
We need this change so that the newly-built clang will have
C++17-compatible libstdc++ headers installed. I believe this change
also means that the newly-built clang (and associated tools) links
against GCC 7's libstdc++, but we set RPATH or similar appropriately, so
there shouldn't be issues stemming from that.
Differential Revision: https://phabricator.services.mozilla.com/D41251
--HG--
extra : moz-landing-system : lando
Major changes:
- add new directory under `taskcluster/docker` named `linux-test` that will supersede `desktop1604-test` when the appropriate time comes
- add new job to support building this docker image in `kind.yml`
- create a setup script similar to `ubuntu1604-test-system-setup.sh` tailored to debian 10
- modify `test-linux.sh` to support switching based on detected distribution; this is important due to differences in how pulseaudio is handled
Minor changes:
- modify logic in the mochitest harness in order to support gstreamer0.1, 0.10 and 1.0 notations (varies slightly from ubuntu1604)
Goal:
- this patch will place all the necessary piping to enable debian 10 when needed
- with the required debian 10 files and scripts recorded in tree, it is easier to have other developers quickly enable debian 10 to run tests on try
Differential Revision: https://phabricator.services.mozilla.com/D42597
--HG--
extra : moz-landing-system : lando
The mar utility stores MOZ_APP_VERSION as the default Product Version for mar creation. So mar should be rebuilt whenever the milestone changes.
Differential Revision: https://phabricator.services.mozilla.com/D44089
--HG--
extra : moz-landing-system : lando
The mar utility stores MOZ_APP_VERSION as the default Product Version for mar creation. So mar should be rebuilt whenever the milestone changes.
Differential Revision: https://phabricator.services.mozilla.com/D44089
--HG--
extra : moz-landing-system : lando
This causes the unit tests in browser/components/newtab to be run by Phabricator each time someone uploads a patch that touches those directories. Once they've run (it can often take an hour or so to do the required Linux PGO build first), they will be visible as node(newtab) in the page linked to by "Treeherder Jobs" on the main review page.
Differential Revision: https://phabricator.services.mozilla.com/D43983
--HG--
extra : moz-landing-system : lando
