Commit Graph

315 Commits

Author SHA1 Message Date
Mitchell Hentges
6e1e037cd3 Bug 1569115 - Don't throw TypeError exception when using |./mach try chooser| r=rstewart
This was an issue around "six" and "click", this thread sheds some light:
https://github.com/pallets/click/issues/564.

This issue was reproducible by having a local python 2 flask app with Flask==1.0.2, Werkzeug==0.14.1 and Click==7.0. Upon self-shutdown, the TypeError was raised.

Upgrading Click to 7.1.2 resolves this issue both in that separate "reproduce app" and here in-tree.

Differential Revision: https://phabricator.services.mozilla.com/D73351
2020-05-01 14:45:40 +00:00
Matthew Gaudet
fd2c42d1ce Bug 1628343 - Remove Windows and OSX builds from sm-shell preset, and add 'sm-shell-all' preset r=tcampbell
After this patch:

```
$ ./mach try --no-push --preset=sm-shell
Task configuration changed, generating target task set
estimates: Runs 17 tasks (17 selected, 0 dependencies)
estimates: Total task duration 7:51:00
estimates: In the 61% percentile
estimates: Should take about 1:26:32 (Finished around 2020-05-01 08:30)
Commit message:
Fuzzy query=!win !osx 'spidermonkey | 'shell-haz

Pushed via `mach try fuzzy`
Calculated try_task_config.json:
{
    "env": {
        "TRY_SELECTOR": "fuzzy"
    },
    "tasks": [
        "hazard-linux64-shell-haz/debug",
        "spidermonkey-sm-arm-sim-linux32/debug",
        "spidermonkey-sm-arm64-sim-linux64/debug",
        "spidermonkey-sm-asan-linux64/opt",
        "spidermonkey-sm-compacting-linux64/debug",
        "spidermonkey-sm-fuzzing-linux64/opt",
        "spidermonkey-sm-gdb-linux64/debug",
        "spidermonkey-sm-mozjs-sys-linux64/debug",
        "spidermonkey-sm-nojit-linux64/opt",
        "spidermonkey-sm-nonunified-linux64/debug",
        "spidermonkey-sm-package-linux64/opt",
        "spidermonkey-sm-plain-linux64/debug",
        "spidermonkey-sm-plain-linux64/opt",
        "spidermonkey-sm-rootanalysis-linux64/debug",
        "spidermonkey-sm-rust-bindings-linux64/debug",
        "spidermonkey-sm-tsan-linux64/opt",
        "spidermonkey-sm-warp-linux64/debug"
    ],
    "version": 1
}
```

Differential Revision: https://phabricator.services.mozilla.com/D73439
2020-05-01 13:17:55 +00:00
Mitchell Hentges
26afa282b0 Bug 1634116 - Removes "mach try again" history migration r=rstewart
The history migration has been around since February 2019 and is ready for cleanup.

Differential Revision: https://phabricator.services.mozilla.com/D73099
2020-04-29 19:40:51 +00:00
Bob Clary
ce5d5ab1aa Bug 1629039 - Limit --rebuild to 3 for mach try --full, r=jmaher.
Differential Revision: https://phabricator.services.mozilla.com/D73050
2020-04-29 15:15:26 +00:00
Joel Maher
18ac0dd2f1 Bug 1632822 - create webrender list of crashtests to have focused tests on android phones. r=bc
create webrender list of crashtests to have focused tests on android phones

Differential Revision: https://phabricator.services.mozilla.com/D72570
2020-04-27 18:06:47 +00:00
Joel Maher
f8875234b3 Bug 1621095 - add reftest-qr to run focused webrender reftests on android phones. r=bc
add reftest-qr to run focused webrender reftests on android phones.

Differential Revision: https://phabricator.services.mozilla.com/D72312
2020-04-24 18:54:51 +00:00
Bogdan Tara
a5112d03db Backed out changeset 97280448a042 (bug 1621095) for test_ext_test.html failures CLOSED TREE 2020-04-24 17:09:37 +03:00
Joel Maher
b9a6e9d31c Bug 1621095 - add reftest-qr to run focused webrender reftests on android phones. r=bc
add reftest-qr to run focused webrender reftests on android phones.

Differential Revision: https://phabricator.services.mozilla.com/D72312
2020-04-24 12:35:12 +00:00
Andrew Halberstadt
5454fdd8bb Bug 1624859 - [taskgraph] Always optimize build tasks with ./mach try auto, r=tomprince
This ensures we don't run every build with every push via ./mach try auto. It
introduces a new 'optimization-overrides' try_config that can be used to
replace optimizations. For now, there is no user interface to pass this in via
the 'mach try' command line.

Differential Revision: https://phabricator.services.mozilla.com/D68207
2020-04-22 15:51:35 +00:00
Andrew Halberstadt
a0bcf43891 Bug 1624859 - [taskgraph] Remove 'bugbug' internal class in experimental optimization strategies, r=tomprince
It was making things more complicated.

Differential Revision: https://phabricator.services.mozilla.com/D71821
2020-04-22 15:50:13 +00:00
Edwin Takahashi
e2f3473016 Bug 1630350 - apply mach try filter to try syntax selector r=jmaher,gbrown
Changes:

Applies the `filter_tasks_by_blacklist` method to try syntax pushes as well.

  - moved `TARGET_TASK_BLACKLIST`and `filter_tasks_by_blacklist` method to live in `taskcluster/taskgraph/target_tasks.py`.
  - removed existing filters against `ccov, windows10-aarch64` and `android-hw` filters against try syntax pushes.
  - update imports for `fuzzy` and `chooser` selectors to refer to the new location of `filter_tasks_by_blacklist` method.

The reason for moving the logic (again) from `tools/tryselect` to `taskcluster/` is due to the placement of `try_option_syntax` and `target_tasks` files and both of those files handle the processing of `mach try syntax` pushes.

Differential Revision: https://phabricator.services.mozilla.com/D71698
2020-04-21 19:51:25 +00:00
Tom Prince
01409a49a6 Bug 1630047 - Allow mozversioncontrol to add or remove multiple files at once; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D70938

--HG--
extra : moz-landing-system : lando
2020-04-15 22:16:49 +00:00
Edwin Takahashi
66ea42cb18 Bug 1627340 - separate filter for try selectors and apply them to try chooser r=bc,ahal,jmaher
Changes:

  - implement filtering of taskgraph items against a list of known task filters in `tasks.py`.
  - apply these filters to both `fuzzy` and `chooser` selectors, unless `--full` is specified.

Differential Revision: https://phabricator.services.mozilla.com/D70097

--HG--
extra : moz-landing-system : lando
2020-04-14 13:43:17 +00:00
Mihai Alexandru Michis
9a16882603 Backed out changeset c5907d661bb7 (bug 1627340) for causing lint failures in tryselect/selectors/chooser/__init__.py
CLOSED TREE
2020-04-10 04:48:08 +03:00
Edwin Takahashi
ae9fc8a3c6 Bug 1627340 - separate filter for try selectors and apply them to try chooser r=ahal,jmaher
Changes:

  - implement filtering of taskgraph items against a list of known task filters in `tasks.py`.
  - apply these filters to both `fuzzy` and `chooser` selectors, unless `--full` is specified.

Differential Revision: https://phabricator.services.mozilla.com/D70097

--HG--
extra : moz-landing-system : lando
2020-04-09 22:01:41 +00:00
Andrew Halberstadt
7349bc85c9 Bug 1628810 - [tryselect] Use bugbug.debug optimization strategy by default in ./mach try auto, DONTBUILD, r=marco
Differential Revision: https://phabricator.services.mozilla.com/D70419

--HG--
extra : moz-landing-system : lando
2020-04-09 18:31:24 +00:00
Bob Clary
770a924bf7 Bug 1627344 - Remove leading and trailing .* from mach try fuzzy.py's TARGET_TASK_FILTERS, r=ahal.
Differential Revision: https://phabricator.services.mozilla.com/D69622

--HG--
extra : moz-landing-system : lando
2020-04-06 14:48:46 +00:00
Andrew Halberstadt
367ad2118d Bug 1627074 - [tryselect] Make it more clear that 'mach try auto' is experimental, DONTBUILD, r=marco,sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D69445

--HG--
extra : moz-landing-system : lando
2020-04-02 20:43:13 +00:00
Andrew Halberstadt
ba40f078a0 Bug 1626393 - [tryselect] Catch OSError to check for file existence while invalidating task caches, r=gbrown
This prevents a race condition where the watchman hook can potentially
invalidate the cache in-between the call to 'isfile' and 'getmtime'.

Depends on D69189

Differential Revision: https://phabricator.services.mozilla.com/D69190

--HG--
extra : moz-landing-system : lando
2020-04-01 15:12:33 +00:00
Andrew Halberstadt
be64ef9701 Bug 1626393 - [tryselect] Remove logic to clean up old cache dirs, r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D69189

--HG--
extra : moz-landing-system : lando
2020-04-01 15:20:47 +00:00
Andrew Halberstadt
d45b3cf7a4 Bug 1624902 - [taskgraph] Implement substrategies in the bugbug scheduler for dealing with platforms, r=marco
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.

Differential Revision: https://phabricator.services.mozilla.com/D68466

--HG--
extra : moz-landing-system : lando
2020-04-01 13:43:48 +00:00
Andrew Halberstadt
1ff013f0ca Bug 1624902 - [tryselect] Validate '--strategy' argument to ./mach try auto, r=marco
This ensures we fail with invalid module paths early. Otherwise users wouldn't
find out until the decision task fails.

Differential Revision: https://phabricator.services.mozilla.com/D68464

--HG--
extra : moz-landing-system : lando
2020-04-01 13:43:30 +00:00
Andrew Halberstadt
9359f4cba8 Bug 1624902 - [taskgraph] Move experimental strategies into a common namespace, r=marco
Small cleanup to help compartmentalize these strategies.

Differential Revision: https://phabricator.services.mozilla.com/D68463

--HG--
extra : moz-landing-system : lando
2020-04-01 13:43:17 +00:00
James Graham
5eba193cb7 Bug 1624280 - Allow specifying routes in mach try fuzzy, r=ahal
These routes are applied to all tasks except the decision task. There
is currently no validity checking in the frontend, so if the provided
routes aren't valid the decision task will fail.

Differential Revision: https://phabricator.services.mozilla.com/D67828

--HG--
extra : moz-landing-system : lando
2020-03-27 19:02:07 +00:00
James Graham
28b1ea3d10 Bug 1624280 - Add support for routes in try_task_config.json, r=ahal
This adds an optional routes key to the task_task_config schema,
 which is a list of strings. Anything in this key is added to the list
of routes for tasks scheduled by the decision task.

Differential Revision: https://phabricator.services.mozilla.com/D67827

--HG--
extra : moz-landing-system : lando
2020-04-01 09:18:37 +00:00
Csoregi Natalia
e1af17b6c1 Backed out 4 changesets (bug 1624902) for failures on test_optimize_strategies.py. CLOSED TREE
Backed out changeset b500403daa0c (bug 1624902)
Backed out changeset bf817363344e (bug 1624902)
Backed out changeset c7a62f2d9034 (bug 1624902)
Backed out changeset 08356d478d16 (bug 1624902)
2020-04-01 00:09:34 +03:00
Andrew Halberstadt
8787bbc73a Bug 1624902 - [taskgraph] Implement substrategies in the bugbug scheduler for dealing with platforms, r=marco
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.

Differential Revision: https://phabricator.services.mozilla.com/D68466

--HG--
extra : moz-landing-system : lando
2020-03-31 17:57:50 +00:00
Andrew Halberstadt
9d99e8d17f Bug 1624902 - [tryselect] Validate '--strategy' argument to ./mach try auto, r=marco
This ensures we fail with invalid module paths early. Otherwise users wouldn't
find out until the decision task fails.

Differential Revision: https://phabricator.services.mozilla.com/D68464

--HG--
extra : moz-landing-system : lando
2020-03-31 17:57:42 +00:00
Andrew Halberstadt
9a971df100 Bug 1624902 - [taskgraph] Move experimental strategies into a common namespace, r=marco
Small cleanup to help compartmentalize these strategies.

Differential Revision: https://phabricator.services.mozilla.com/D68463

--HG--
extra : moz-landing-system : lando
2020-03-31 18:06:58 +00:00
Simon Fraser
5736cab8ce Bug 1625933 Only complain about unmanaged fzf if it was found r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D68792

--HG--
extra : moz-landing-system : lando
2020-03-30 13:49:57 +00:00
Simon Fraser
ba6f2afbaa Bug 1623197 Change fzf management check to avoid side effects r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D67830

--HG--
extra : moz-landing-system : lando
2020-03-25 10:07:26 +00:00
Simon Fraser
f02a5bdd6c Bug 1623328 Handle missing target_task_set in try estimates r=ahal
I'm unsure of the root cause as the file should exist if generate_tasks has worked, but this should avoid errors in the meantime while we investigate.

Differential Revision: https://phabricator.services.mozilla.com/D67833

--HG--
extra : moz-landing-system : lando
2020-03-24 16:39:37 +00:00
Kartikaya Gupta
5867f01980 Bug 1624307 - Add a try preset for a Linux/Android F-push. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D67858

--HG--
extra : moz-landing-system : lando
2020-03-24 17:05:57 +00:00
Sylvestre Ledru
be18fd4dba Bug 1622328 - add license info to all eslintrc files r=Standard8,webcompat-reviewers,miketaylr
Differential Revision: https://phabricator.services.mozilla.com/D67319

--HG--
extra : moz-landing-system : lando
2020-03-19 13:47:51 +00:00
Simon Fraser
77fba7508b Bug 1616354 Change priority order for |mach try| push estimates target set r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D66575

--HG--
extra : moz-landing-system : lando
2020-03-16 18:32:11 +00:00
Andrew Halberstadt
41e1cf03ee Bug 1603463 - [tryselect] Ignore 'auto' and 'empty' selectors in estimates and history, r=gbrown
Neither of these selectors involve the user choosing tasks, so showing
estimates or saving them in the 'mach try again' history doesn't make much
sense.

Differential Revision: https://phabricator.services.mozilla.com/D66628

--HG--
extra : moz-landing-system : lando
2020-03-13 20:33:51 +00:00
Andrew Halberstadt
2cf0c4fbc5 Bug 1603463 - [tryselect] Add the ability to run 'mach try auto' with alternative scheduling algorithms, r=tomprince
This allows us to change the default optimization strategy used in try pushes.
While probably not super useful to developers, it can help us easily test
changes to new and experimental optimizations on try.

This also changes the default to the 'bugbug_push_schedules' strategy, since
SETA is more or less random and shouldn't be used by 'mach try auto'. In the
future, we'll switch this back to simply using the default optimization as the
default will ideally be the best one that we have.

Differential Revision: https://phabricator.services.mozilla.com/D65746

--HG--
extra : moz-landing-system : lando
2020-03-13 20:32:14 +00:00
Andrew Halberstadt
56ba8a9ddf Bug 1603463 - Implement |mach try auto| which selects tasks on try automatically, r=tomprince
The 'auto' in 'mach try auto' stands for two things:

1. It automatically picks tasks for you.
2. It runs the same scheduling algorithms as autoland.

It accomplishes this by creating a new target_tasks method that spoofs the
'project' parameter to autoland.

Differential Revision: https://phabricator.services.mozilla.com/D60184

--HG--
extra : moz-landing-system : lando
2020-03-13 20:35:43 +00:00
Simon Fraser
dfdd97e607 Bug 1621658 Update |mach try fuzzy| fzf version check r=ahal
Add filename support for plain display

Update fzf version in tests

Update tooltool url for fzf

Differential Revision: https://phabricator.services.mozilla.com/D66571

--HG--
extra : moz-landing-system : lando
2020-03-12 12:56:35 +00:00
Simon Fraser
54f21ea31e Bug 1619554 Improve performance of |mach try fuzzy| preview r=ahal
Remove use of requests module in preview pane

Reformat task duration data to avoid reprocessing in preview pane

Avoid loading task durations json more than once.

Increase required fzf version, use temporary file instead of arglist

Differential Revision: https://phabricator.services.mozilla.com/D65094

--HG--
extra : moz-landing-system : lando
2020-03-05 19:58:43 +00:00
Mark Banner
15cfe23b88 Bug 1620542 - Automatically fix ESLint errors in .eslintrc.js files. r=mossop
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D65703

--HG--
extra : moz-landing-system : lando
2020-03-07 10:09:44 +00:00
Tom Prince
5a52cec97c No bug: Fix missing space in mach try syntax help text.
Differential Revision: https://phabricator.services.mozilla.com/D65840

--HG--
extra : moz-landing-system : lando
2020-03-07 01:27:13 +00:00
Edwin Takahashi
6e6db2f667 Bug 1620044 - remove unnecessary debian and ubuntu-bionic codepaths r=jmaher
Changes:

Remove `ubuntu-bionic` flag that was used during development to enable use of ubuntu1804-test docker image.

Remove unnecessary conditional and in the process rewrite how `runtests.py` checks the environment for `pactl` prior to running mochitests.

Differential Revision: https://phabricator.services.mozilla.com/D65389

--HG--
extra : moz-landing-system : lando
2020-03-04 20:47:43 +00:00
Andrew Halberstadt
8649491808 Bug 1618633 - Add --disable-target-task-filter to |mach try fuzzy| to bypass task filtering, r=jgraham
In |mach try fuzzy| there's a TARGET_TASK_FILTERS variable that we use to make
it more difficult to run certain kinds of expensive / resource constrained
tasks. The problem is that the only way to run these is to use '--full' at
which point there's no way to distinguish which tasks are "valid" (i.e run on
mozilla-central) and which tasks don't.

This adds a flag that will ensure the default set truly matches what we run
on central.

Differential Revision: https://phabricator.services.mozilla.com/D64648

--HG--
extra : moz-landing-system : lando
2020-02-28 20:56:10 +00:00
Tom Prince
e797b75c9a Bug 1618079: [tryselect] Use public ship-it api from scriptworker selector; r=rail
Differential Revision: https://phabricator.services.mozilla.com/D64663

--HG--
extra : moz-landing-system : lando
2020-02-27 21:30:34 +00:00
Andrew McCreight
4babb2b5ab Bug 1609815 - Remove Web Replay C++ implementation. r=jgilbert,jandem,gbrown
Patch by bhackett and jlaster. Also reviewed by mccr8.

Differential Revision: https://phabricator.services.mozilla.com/D60197

--HG--
extra : moz-landing-system : lando
2020-02-27 17:39:15 +00:00
Ciure Andrei
00dd87f6f4 Backed out changeset d407a28318e6 (bug 1609815) for causing windows ming bustages CLOSED TREE
--HG--
extra : histedit_source : b2c748e31e0f6ba8fcf9960a336e0bbd361b07e6
2020-02-27 07:05:19 +02:00
Andrew McCreight
b197e1f783 Bug 1609815 - Remove Web Replay C++ implementation. r=jgilbert,jandem,gbrown
Patch by bhackett and jlaster. Also reviewed by mccr8.

Differential Revision: https://phabricator.services.mozilla.com/D60197

--HG--
extra : moz-landing-system : lando
2020-02-27 04:43:48 +00:00
octavian.negru
5348ca01b5 Bug 1617877 - TypeError: environment can only contain strings with mach try fuzzy on Windows r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D64062

--HG--
extra : moz-landing-system : lando
2020-02-26 09:07:31 +00:00
Bob Clary
a3aec07b06 Bug 1610879 - restrict Tier 2 fission web-platform-tests on Linux to try with --full, r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D64050

--HG--
extra : moz-landing-system : lando
2020-02-25 11:43:33 +00:00