This patches fixes several problems found on Raptor and the condprof:
Raptor:
- Make sure the conditioned profile dir is removed after
it's been used, not before.
- Adds the --project option to raptor so we know if we're on try
autoland or mozilla-central.
- Both Fennec and Fenix are deactivated for now
- Use the allow-downgrade flag to be flexible on build ids (the next step will be bug 1628666)
Conditioned profiles, curation of the profile prefs:
- Fully deactivates Normandy during Raptor tests (app.normandy.enabled)
- Removes any GFX blacklisting (gfx.blacklist.*)
- Removes any marionette pref
- Enforce extensions sideloading (extensions.startupScanScopes)
Differential Revision: https://phabricator.services.mozilla.com/D70518
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
We'll want some kind of backstop no matter what optimization algorithm we use.
We don't want to go too long without running any given task so we can find
regressions quickly and have a good merge candidate.
This pulls the logic that handles this out of the SETA strategy and into its
own strategy.
This will also make the SETA shadow scheduler more representative of what the
algorithm is doing.
Note in the future we may find ways to make this backstop more efficient (i.e
only run tasks that didn't run in the last 9 pushes for example).
Depends on D68621
Differential Revision: https://phabricator.services.mozilla.com/D68622
--HG--
extra : moz-landing-system : lando
This allows to nest strategies without having to register ever intermediate
composite strategy first. For example:
All(Any("skip-unless-schedules", "seta"), "backstop")
Prior to this patch, we'd need to register that 'Any' one first and then use it
in the 'All'.
Depends on D68620
Differential Revision: https://phabricator.services.mozilla.com/D68621
--HG--
extra : moz-landing-system : lando
I'd like to implement a 'backstop' strategy, such that it will prevent all other
optimizers from removing tasks under certain conditions (e.g every 10th push).
The nicest way to implement this seems to be an 'All' composite strategy
(similar to 'Either' which this patch renames to 'Any'). This means we could
do something like:
All("seta", "backstop")
which means we would only remove tasks if *all* substrategies say to remove
tasks.
Differential Revision: https://phabricator.services.mozilla.com/D68620
--HG--
extra : moz-landing-system : lando
Activates the conditioned profile by doing the following changes:
- make sure the conditioned profile dir is removed after
it's been used, not before.
- add the --project option to raptor so we know if we're on try
or mozilla-central.
- Both Fennec and Fenix are deactivated for now.
- Remove any gfx.blacklist.* prefs when using a conditioned profile
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
Make sure the conditioned profile dir is removed after
it's been used, not before. This patch also adds the
--project option to raptor so we know if we're on try
or mozilla-central. Both Fennec and Fenix are deactivated
for now.
Differential Revision: https://phabricator.services.mozilla.com/D70518
--HG--
extra : moz-landing-system : lando
Since we don't look at the changesets, there is no need for hgmo to generate
them for us.
Differential Revision: https://phabricator.services.mozilla.com/D70567
--HG--
extra : moz-landing-system : lando
Previously source-tests requiring a build had a "global" mapping of
platform to build type in kind.yml. But this made it confusing to
figure out how to add task-specific configuration. To simplify things,
make the configuration for the dependent platforms also go in the task
definition.
Differential Revision: https://phabricator.services.mozilla.com/D68636
--HG--
extra : moz-landing-system : lando
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.
Differential Revision: https://phabricator.services.mozilla.com/D68466
--HG--
extra : moz-landing-system : lando
This adds an optional routes key to the task_task_config schema,
which is a list of strings. Anything in this key is added to the list
of routes for tasks scheduled by the decision task.
Differential Revision: https://phabricator.services.mozilla.com/D67827
--HG--
extra : moz-landing-system : lando
The definition for reftest/crashtest variants of web-platform-tests in web-platform.yml has a trailing `s` which is technically incorrect.
This causes problems when querying ActiveData for the runtimes for these subsuites since the suite name recorded in ActiveData has the trailing s.
Changes:
- remove the trailing `s` from the definitions
Differential Revision: https://phabricator.services.mozilla.com/D68069
--HG--
extra : moz-landing-system : lando
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.
Differential Revision: https://phabricator.services.mozilla.com/D68466
--HG--
extra : moz-landing-system : lando
This patch adds the capability to run Google Chrome for Android tests through Raptor-Browsertime.
Chrome must be available on the device being tested, and there are no installation steps being added in this patch since CI already has the latest Chrome version available to test with. That said, this patch adds the ability to find the version of Chrome that is being tested and stores this in the Perfherder artifact. Getting this version is also necessary to be able to run Chrome with the correct chromedriver.
Two tests are initially be enabled for Chrome in this patch: Amazon, and YouTube. They will only run through a cron task three days a week. The other changes done in this patch are required for Chrome to work with Raptor-Browsertime.
Differential Revision: https://phabricator.services.mozilla.com/D65610
--HG--
extra : moz-landing-system : lando
Changes:
Migrate all of the currently running tsan test suites to run on linux1804 instead.
All future test suites should also be added under the linux1804 label.
Depends on D67125.
Differential Revision: https://phabricator.services.mozilla.com/D68087
--HG--
extra : moz-landing-system : lando
Ensure a minimum of 1 chunk for "per-file" tests like test-verify when run on try, even when
no local file changes are detected. --full is still required.
Differential Revision: https://phabricator.services.mozilla.com/D67694
--HG--
extra : moz-landing-system : lando
Use finer granularity for some reftest/mochitest SCHEDULES.exclusive entries,
so that reftest-plain does not run when only crashtests are modified, and
vice versa; similarly, break up mochitest into mochitest/browser-chrome/chrome/
a11y. Use schedules-component instead of category.
Differential Revision: https://phabricator.services.mozilla.com/D60085
--HG--
extra : moz-landing-system : lando
The former two are not used anymore, and the latter can't be generated
on cross-builds, and it was agreed in bug 1563564 (and reconfirmed on
Matrix) that we can go without it until it's generated from separate
tasks.
More formally removing the former two will be the subject of a followup.
Differential Revision: https://phabricator.services.mozilla.com/D67580
--HG--
extra : moz-landing-system : lando
Add test package mach support for gtest and hook into the custom retrigger
action. Some existing custom retrigger features, like setting gecko prefs,
are not (easily) applicable to gtest, which doesn't use mozprofile; for
this reason, use a separate action context with items suitable for gtest.
Differential Revision: https://phabricator.services.mozilla.com/D67384
--HG--
extra : moz-landing-system : lando
Changes:
While stepping through the debugger for Bug 1608837, it was found that for `platform = windows` the conditional was always evaluating to `False` thereby loading the fallback `unix` runtimes.
Differential Revision: https://phabricator.services.mozilla.com/D67390
--HG--
extra : moz-landing-system : lando
The diff jobs currently take a Firefox package and compares against the
one from another build. When that fails, it also makes the diff of the
generated files, which can be useful when there are differences, but
can also be useful on its own. Also, because sometimes, like right now,
there are differences in generated files that have no impact on Firefox
itself, when differences do show up for Firefox, the differences in
generated files are added noise that sheriffs can't work around.
Differential Revision: https://phabricator.services.mozilla.com/D67113
--HG--
extra : moz-landing-system : lando
Remove the remainder of automation.py.in and the build support for generating
automation.py.
Some of this functionality was in use, especially for android tests.
Some code was moved or re-implemented in remoteautomation.py or in the affected
harness(es). Some features were removed: There are some minor changes in behavior.
For instance, instead of using a different server startup timeout for debug builds,
one value is used for all builds (due to performance improvements over time,
the longer timeout is no longer needed).
Differential Revision: https://phabricator.services.mozilla.com/D66839
--HG--
extra : moz-landing-system : lando
Update the default values to avoid common pitfalls, such as trying to repeat
a 30-minute long tasks 30x times with extra logging!
The new defaults allow a simple re-run of most tasks with no changes.
While we are here, tweak the parameter descriptions.
Differential Revision: https://phabricator.services.mozilla.com/D66842
--HG--
extra : moz-landing-system : lando
This patch removes the chromedriver for Chrome 78, and adds a chromedriver for Chrome 81.
Differential Revision: https://phabricator.services.mozilla.com/D66987
--HG--
extra : moz-landing-system : lando
