Commit Graph

66 Commits

Author SHA1 Message Date
Tom Prince
acd19bf66a Bug 1477706: [try-select] Let taskgraph know that it is being called from try-select; r=ahal
Currently, `mach try fuzzy` generates a taskgraph that is configured exactly
like the most recent push to mozilla-central. This isn't always desirabe, so
pass some configuration down, to allow the taskgraph to behave differently.

Differential Revision: https://phabricator.services.mozilla.com/D2729

--HG--
extra : rebase_source : 99d6958b33211697227e65df17edc1eb337f63a4
extra : histedit_source : 69b5ff6805bc8409340eb71323a1f6fc637259d7
2018-08-03 10:25:42 -06:00
Andrew Halberstadt
0e4eda282f Bug 1471990 - Implement |mach try again| to repeat your last 'try_task_config.json' try push r=gbrown
One of the big downsides to |mach try fuzzy| is that if you use the interactive
finder, there's no way to repeat your last action. If you want to run the same
push again, you have to manually re-select all the same tasks a second time.

It is possible to save presets, but this is fairly heavy-weight and (more)
permanent. Sometimes you just want to re-run a push a few times and forget
about it. It's also possible to craft the query on the command line with -q,
but then you don't get the immediate visual feedback, so you can't be sure that
you typed out the right things without actually pushing.

With |mach try again|, everytime you generate a try_task_config.json via
'fuzzy', 'empty' or any other subcommands that may exist in the future, it'll
get stored in a history file under ~/.mozbuild. Then running |mach try again|
will simply re-run the most recent try_task_config.json.

You'll also be able to view the whole history via |mach try again --list| and
select a specific try_task_config.json (i.e not the most recent one) via
|mach try again --index <index>|.

Example usage will be:
$ ./mach try fuzzy
<select a bunch of tasks>
$ ./mach try again
<re-pushes exact same set of tasks>

MozReview-Commit-ID: 3EZjVCy08uq

Depends on D1808.

Differential Revision: https://phabricator.services.mozilla.com/D1867

--HG--
extra : moz-landing-system : lando
2018-07-16 13:35:30 +00:00
Andrew Halberstadt
0ad2e9342d Bug 1413922 - [tryselect] Merge vcs.py into mozversioncontrol r=gps
Differential Revision: https://phabricator.services.mozilla.com/D1808

--HG--
rename : tools/tryselect/vcs.py => tools/tryselect/push.py
extra : moz-landing-system : lando
2018-07-10 19:22:00 +00:00
Coroiu Cristina
07726be245 Backed out 2 changesets (bug 1413922) for breaking Nightly L10n a=backout
Backed out changeset e7a1d749ff9a (bug 1413922)
Backed out changeset b3bead1f5729 (bug 1413922)

--HG--
rename : tools/tryselect/push.py => tools/tryselect/vcs.py
2018-07-10 03:11:28 +03:00
Andrew Halberstadt
23c2416671 Bug 1413922 - [tryselect] Merge vcs.py into mozversioncontrol r=gps
Differential Revision: https://phabricator.services.mozilla.com/D1808

--HG--
rename : tools/tryselect/vcs.py => tools/tryselect/push.py
extra : moz-landing-system : lando
2018-07-07 00:32:11 +00:00
Andrew Halberstadt
9435736ade Bug 1471620 - Skip python-tests locally that don't run with python 3 in CI r=davehunt
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).

MozReview-Commit-ID: 3OBr9yLSlSq

--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
2018-06-27 11:10:02 -04:00
Andrew Halberstadt
4984a86e83 Bug 1464419 - [tryselect] Ability to specify --query multiple times with |mach try fuzzy| r=jmaher
Currently it's possible to specify a single query and take the union of terms with the '|'
symbol. However if you want to craft anything more complicated (i.e linux mochitest and
xpcshell, but windows reftest), it becomes really difficult. This allows developers to union
the result of multiple queries.

For example:
./mach try fuzzy -q "'linux 'mochitest | 'xpschell" -q "'windows 'reftest"

Differential Revision: https://phabricator.services.mozilla.com/D1838
2018-06-26 22:08:07 +00:00
Joel Maher
28e3660b54 Bug 1400895 - Better try support for test-verify. r=ahal 2018-05-23 10:00:03 -04:00
Geoff Brown
5aaf4e3257 Bug 1459728 - Include more Android test suites in try list of non-compiled suites; r=maja_zf 2018-05-09 07:36:53 -06:00
Andrew Halberstadt
135003aa80 Bug 1414921 - Add --geckoProfile to capture profiles from talos tasks to |mach try fuzzy|, r=dustin
Enables |./mach try fuzzy --talos-profile|. This template only applies to talos
tasks. It also provides --geckoProfile for consistency with |mach try syntax|,
but I don't like this name so it's hidden from the help.

The 'talos-profile.yml' template is also very specific (only applies to Talos
tasks).  Ideally I'd like a general 'command.yml' template that just appends
arguments to the command for any arbitrary tasks. But then we'd need to invent
an expression syntax in try_task_config.json so we could make sure it only
applies to Talos. Then I thought rather than implement it for a specific
template, we should have a general way of doing this which could apply to any
and all of the templates.

Needless to say, it's a rabbit hole and something that's best left to a
follow-up so we don't delay this bug.

MozReview-Commit-ID: GhllZ7sr0ar

--HG--
extra : rebase_source : 1de4deecc2f73130904d7c95d4ff12f85883cd91
2017-12-13 16:16:09 -05:00
Dustin J. Mitchell
d0f0657464 Bug 1443974: add a template to run try jobs at 'low' priority; r=ahal
This is gated by the `--chemspill-prio` flag, which should at least make anyone
abusing it to get faster results feel sorry for what they've done.

MozReview-Commit-ID: J4EwH45IkMX

--HG--
extra : rebase_source : 1bfbfafd7de914aaab52f48f0e37c09c0df05dd7
2018-04-02 16:53:05 +00:00
Emilio Cobos Álvarez
e9676561ce Bug 1446954: Remove support for running tests on automation without stylo configurations. r=froydnj
You can still run them on a --disable-stylo build, as long as that works
(presumably not for long).

I think I haven't missed anything, but please double-check.

MozReview-Commit-ID: 3BIAEjgTLo5
2018-03-20 11:29:08 +01:00
Andrew Halberstadt
f249b70e43 Bug 1437912 - [moztest] Map flavors and subsuites to a suite definition, r=gbrown
The end goal here is to be able to use |mach try fuzzy <path>| with tests that
belong to a subsuite. To do this, we need a unique 'task_regex' value for each
subsuite so that we can map a test path back to a set of tasks.

This removes the TEST_FLAVORS dict (which was mostly just a redefinition of the
data in TEST_SUITES), and instead provides two new private mappings:

<flavor> -> suite definition
(<flavor>, <subsuite>) -> suite definition

To retrieve a suite definition given a flavor/subsuite, consumers can now call
get_suite_definition.

MozReview-Commit-ID: 2pe1v1IHUVy

--HG--
extra : rebase_source : 6fff947ba214112ccf16c894174a6a0e2487111a
2018-02-13 14:16:46 -05:00
Andrew Halberstadt
3087b17a69 Bug 1413928 - [tryselect] Implement paths for |mach try fuzzy| r=maja_zf
This enables the syntax like:
./mach try fuzzy dom/indexedDB

This will open up the fzf interface like normal, except only tasks
that have tests under dom/indexedDB will be selectable (and there
will only be one chunk per configuration).

This can be combined with -q/--query like normal:
./mach try fuzzy dom/indexedDB -q "!pgo !cov !asan"

When the tasks get scheduled, only the tests under the specified
path(s) will run within the harness.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 8a89f255591e6dfa31b1420196c4698f2015d10c
2017-11-15 16:36:07 -05:00
Andrew Halberstadt
564efd2a35 Bug 1413928 - [tryselect] Create a new argument group for 'task' arguments r=maja_zf
This simply groups arguments related to generating the list of
tasks from taskgraph into their own argument group.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 6c746f0cb8d93f957f50ba12f77d63edb04b6d1e
2017-11-02 16:03:38 -04:00
Andrew Halberstadt
9b0698e9b6 Bug 1413928 - [tryselect] Add a new 'path' template r=maja_zf
This sets the MOZHARNESS_TEST_PATHS environment variables for all tasks.
When specifying paths, this will cause many test tasks to only run the
tests under that directory as opposed to the normal default.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : e7132311641f4d36a5bff56424988fbb3ae87238
2018-01-15 16:05:37 -05:00
Andrew Halberstadt
28b0ebdf63 Bug 1413928 - [tryselect] Change templates to return an entire context dict instead of a single value r=maja_zf
This changes templates so they return an entire context dict instead of
only returning context based on their name. For example, now the 'path'
template can set context for 'env'.

A side effect of this is that there is no longer a 1-to-1 mapping of templates
in tryselect and taskgraph.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 4d7e398e60598a5de7961fb126f1d05a0b983681
2018-01-15 16:05:17 -05:00
Andrew Halberstadt
24922a65ba Bug 1413928 - [tryselect] Add python unittest for templates r=davehunt
This makes use of pytest's generation feature. To add a new
template test, just add a new entry containing the input and
expected output to the dict in test_templates.py

MozReview-Commit-ID: 4qMefYHMjAp

--HG--
extra : rebase_source : ba3049885d1a2485048e1ff9913be43317559376
2017-11-21 10:11:00 -05:00
Andrew Halberstadt
d8a05f2775 Bug 1397433 - [tryselect] Create in-tree documentation for |mach try|, r=jmaher
This adds some basic documentation for |mach try| and its various subcommands.
This was a bit hastily made for the Austin all-hands, but at least provides a
place to link to and can be improved upon in the future.

MozReview-Commit-ID: 8N6LZO5kTlL

--HG--
extra : rebase_source : c7e215703426f6cfb03b044d188a53d1ba878a75
2017-11-07 16:26:14 -05:00
Andrew Halberstadt
ab5fe4707a Bug 1423887 - [tryselect] Enable taskgraph.fast when generating tasks from |mach try|, r=catlee
This speeds up taskgraph generation by ~6 seconds on my machine. Future
improvements are also planned for 'fast' mode.

MozReview-Commit-ID: CLORvLXuV8y

--HG--
extra : rebase_source : a59dc5f166eff15e5031f5736eadac41dcd46ffe
2017-12-07 09:01:35 -05:00
Andrew Halberstadt
0e697ce235 Bug 1422302 - Create python/mozterm for sharing terminal blessings across modules r=gps
This is a new module that will provide a place to store some common
abstractions around the 'blessings' module. The main entrypoint is:

    from mozterm import Terminal
    term = Terminal()

If blessings is available, this will return a blessings.Terminal()
object. If it isn't available, or something went wrong on import,
this will return a NullTerminal() object, which is a drop-in
replacement that does no formatting.

MozReview-Commit-ID: 6c63svm4tM5

--HG--
extra : rebase_source : 9ab221774d92a418d9b098d79bb2c88f75d937f8
2017-12-04 09:38:24 -05:00
Tom Prince
98988ab6fc Bug 1421002: Get tasks for mach try fuzzy from the root of the repository; r=ahal
`mach try` pushes the repository containing the current directory. When this is
a comm-central checkout, the taskcluster configuration should also come from that
repository.

MozReview-Commit-ID: KWbNAe4jrHT

--HG--
extra : rebase_source : e40f5038bdd190fb4cb801ba817c31a3e4031354
extra : source : 7164b051c965280aeb3083e47a93c6d4ac44e2ed
2017-11-21 13:39:21 -07:00
Andrew Halberstadt
e3109289c8 Bug 1419512 - [tryselect] Consolidate subcommand parser retrieval in mach_commands.py, r=armenzg
This is a minor cleanup around finding and importing subcommand parser in |mach try|.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 6b05b6f3fafed607992b9427225fd43165c4102f
2017-11-02 16:02:35 -04:00
Andrew Halberstadt
28d07aeea3 Bug 1414919 - [tryselect] Add --rebuild support to |mach try fuzzy|, r=jmaher
This allows rebuilding all selected tasks. This defines an upper limit of
20 rebuilds per push. If more are needed, developers can either change it
in code or push multiple times.

MozReview-Commit-ID: I0XtMP5yEEq

--HG--
extra : rebase_source : 2583bed5dd33df72a4d7f1cd0ce012e412418c5e
2017-11-07 10:27:44 -05:00
Andrew Halberstadt
52dad304bf Bug 1414399 - [moztest] Refactor |mach test|'s resolving logic into moztest.resolve r=gps
The code in |mach test| for test resolving, should get merged with the TestResolver
class in moztest.resolve. This way it can be shared with other modules and we'll
have a single canonical place for all our test resolving logic.

MozReview-Commit-ID: IHRXXi5mB4G

--HG--
extra : rebase_source : 6f96d06412ab8fa152ac5d9bdd15acbcdc9695c4
2017-11-08 09:59:51 -05:00
Andrew Halberstadt
a69182174e Bug 1414399 - [mozbuild/moztest] Move mozbuild.testing.TestResolver to moztest.resolve r=gps
The TestMetadata and TestResolver classes aren't technically part of the build
system. The only connection is that they consume some build system output.

The next patch in this series is going to be merging in a bunch of other test
resolving logic from other parts of the tree. Moving this out first allows us
to keep that extra logic out of mozbuild.

MozReview-Commit-ID: 1eq4SjFVCyW

--HG--
rename : python/mozbuild/mozbuild/test/test_testing.py => testing/mozbase/moztest/tests/test_resolve.py
extra : rebase_source : 7ff11f9ec455547533082d20cb5371845f7a4f21
2017-11-06 08:41:42 -05:00
Andrew Halberstadt
8f49c97d69 Bug 1414894 - [tryselect] Don't install shell extensions when bootstrapping fzf for |mach try fuzzy|, r=armenzg
Currently the prompts don't make it clear enough that running fzf will mess with your
shell settings. This means users could install it without realizing, forget and get
confused later on.

Rather than try to address this, it's simpler to always skip the shell extensions as
|mach try fuzzy| doesn't need them anyway. The extensions are useful, but are better
installed via something like |mach bootstrap| which can be tackled in a separate bug.

MozReview-Commit-ID: 2kx7UGO5LJ0

--HG--
extra : rebase_source : 64474626aeab9f2f04f491afc65ca7cadd717296
2017-11-06 20:29:25 -05:00
Andrew Halberstadt
22eb565843 Bug 1413374 - [tryselect] Fix |mach try empty| regression and add a test, r=armenzg
MozReview-Commit-ID: Iv2RDqqhL57

--HG--
extra : rebase_source : c609fea4ff27766027fb072bd7c778f3884a1d46
2017-11-01 08:12:42 -04:00
Andrew Halberstadt
25bfeaa7a6 Bug 1412134 - [tryselect] Redirect stderr to a separate pipe, r=chmanchester
We parse the output of several version control commands in |mach try|, yet
redirect stderr to stdout. This is causing issues for people whenever the
vcs outputs some kind of warning.

MozReview-Commit-ID: F0L56at0MYS

--HG--
extra : rebase_source : 40be54cd1a9e2b87c4d2b3ed7adeeec40f937a39
2017-10-30 16:35:45 -04:00
Andrew Halberstadt
fed6622b29 Bug 1400503 - [tryselect] Add an option to open the presets file in an editor, r=armenzg
MozReview-Commit-ID: 2jwdAaNWHH7

--HG--
extra : rebase_source : 8b3d054ce7b6deae439ba657c9c7607043898b3d
2017-10-27 14:58:34 -04:00
Andrew Halberstadt
d801a6a681 Bug 1400503 - [tryselect] Separate 'common_arguments' into groups sub-parsers can opt-in to, r=armenzg
This allows subparsers more control over which sets of arguments they need to
implement.  For example, it doesn't make sense for the 'empty' selector to
accept the preset arguments.  Now it can opt-out of those and only implement
the 'push' arguments.

MozReview-Commit-ID: GOfjcFtlfDD

--HG--
extra : rebase_source : 4542adc49e38ff8c450a16363706aba669ad6594
2017-10-27 11:06:24 -04:00
Gregory Szorc
a56fef0fd4 Bug 1410969 - Consolidate code for resolving path metadata; r=ahal
find_paths_and_tags() can instantiate a BuildReader to read moz.build
metadata. We switch `mach test` to call this to avoid some redundant
code.

To facilitate `mach try`'s use case, we add test flavors to the
result of find_paths_and_metadata() and change the function name to
reflect its more generic purpose.

MozReview-Commit-ID: Lcz05W8g2dt

--HG--
extra : rebase_source : 17d6f23275f4eed1bd544a9b2cd991214625dabc
2017-10-23 10:42:41 -07:00
Gregory Szorc
9a4d779385 Bug 1410969 - Return dict from find_paths_and_tags(); r=ahal
This will make the return value more easily extensible and will help
consumers know what the data structures represent.

MozReview-Commit-ID: DaeYsqfMW37

--HG--
extra : rebase_source : 440d16f0604b21f0ec885e1bca5066c32b18f641
2017-10-23 10:36:38 -07:00
James Graham
f8e091e39a Bug 1401150 - Support try pushes on a closed tree with mach try, r=ahal
This adds the magic CLOSED TREE string to the commit message for both
try syntax and the fuzzy frontend.

MozReview-Commit-ID: 492UME3VcJN

--HG--
extra : rebase_source : 425a83d6370609e1e162379213d547458c8e8cf2
2017-10-16 18:59:45 +01:00
Andrew Halberstadt
7532e51ef9 Bug 1406777 - Support --message and --no-push with |mach try empty|, r=armenzg
This adds all the common arguments to |mach try empty|, including preset
arguments which will just be ignored if specified. The commit message can be
changed with:
./mach try empty -m DONTBUILD

MozReview-Commit-ID: 5RYfLBOIX1g

--HG--
extra : rebase_source : 412cbc50c2901a97893779a73a7f70d5ee6e2d72
2017-10-11 15:45:08 -04:00
Andrew Halberstadt
cb9156b339 Bug 1401199 - [tryselect] Pass in strict=False when generating tasks, r=dustin
MozReview-Commit-ID: 9XWlLeGcPeQ

--HG--
extra : rebase_source : 05dbf742089e857eb697736ee6a6619192c07365
2017-09-29 11:36:42 -04:00
Andrew Halberstadt
776ed09b77 Bug 1404067 - [tryselect] Improve error message on parameter mismatch, r=dustin
MozReview-Commit-ID: GMiGuNApoUF

--HG--
extra : rebase_source : 326df920793b54806cd4423ec0bf0e20df6cd42d
2017-09-28 15:25:34 -04:00
Wes Kocher
3042f50d29 Bug 1400425 - Add a mach try empty command to push to try with no prompts r=ahal
MozReview-Commit-ID: 6F5vEpTTDS

--HG--
extra : histedit_source : be02948a579b5dfc274f96a0d9ea07a80bc150a7%2C9e820e3058b87aea959a2fa764e3ef0789dc0652
2017-09-25 16:06:50 -07:00
Wes Kocher
31bba17090 Backed out changeset 7fc37806848f (bug 1400425) for flake8 issues a=backout
MozReview-Commit-ID: 7EI7PgX4YoS
2017-09-25 16:06:09 -07:00
Wes Kocher
b4daee707a Bug 1400425 - Add a mach try empty command to push to try with no prompts r=ahal
MozReview-Commit-ID: 6F5vEpTTDS

--HG--
extra : rebase_source : b173ca390dad7bbdc9a4c0516ed06331f4b1367a
2017-09-25 15:17:50 -07:00
Andrew Halberstadt
5e9a048a4b Bug 1400469 - Add ability to specify commit message to |mach try|, r=armenzg
MozReview-Commit-ID: LWkAEDWn8NC

--HG--
extra : rebase_source : 24fd977d23f9f09859aa9f9a44c53ffd6c1f4673
2017-09-18 12:43:03 -04:00
Andrew Halberstadt
b9233c3ea8 Bug 1400424 - [tryselect] Make sure taskgraph generation is run from topsrcdir, r=dustin
This was preventing us from running ./mach try fuzzy from a subdirectory of the
topsrcdir. This also fixes taskgraph/utils/verify.py to find the docs directory
based off of topsrcdir instead of cwd. This was needed as cwd was being set at
import time.

MozReview-Commit-ID: CgQqD6bQ5q4

--HG--
extra : rebase_source : 4d863d442b0e715b3fe386c43b4077054c10dc70
2017-09-16 00:31:45 -04:00
Chris Manchester
83c39d5afe Bug 1401017 - Make mach try work with -j in the presence of modified local files. r=ahal
Running |./mach try -j <job>| may fail if there are changes in the local
working copy because the command will attempt to provide test paths based
on those changes and subsequently require platforms to be specified on the
basis of those paths. This commit makes this auto-detection only run when
a particular option is passed so this doesn't interfere with the common
case of simply running a selected job on try.

MozReview-Commit-ID: F3RBgDAYi27

--HG--
extra : rebase_source : 81e7774d7b1af31ee188b76afdac5062db1ab811
2017-09-18 14:21:18 -07:00
Gregory Szorc
8f958c2424 Bug 1400429 - Advertise use of mach try in commit message; r=ahal
This will allow us to inspect Try pushes to see how many are using
`mach try`.

The commit message for `mach try fuzzy` already advertised its usage.
We tweak the syntax a little so it isn't redundant.

MozReview-Commit-ID: 6kmYRKp0ffF

--HG--
extra : rebase_source : 9dd7caa5404ba235e6a4071802000b8bb740a6b7
2017-09-15 16:17:37 -07:00
Wes Kocher
44fefedf8c Bug 1391123 - Add a newline to the end of try_task_config.json r=gps
MozReview-Commit-ID: BtQntIHOo2G

--HG--
extra : rebase_source : b8e6d6a32ea728590bf4aec85c7605a8829d47c1
2017-09-15 19:02:13 -07:00
ruby
af79c59e2a Bug 1397427 - Enable py2 linter on tools/tryselect, r=ahal
MozReview-Commit-ID: K12XisbVQGq

--HG--
extra : rebase_source : 6a97dda2be81fa45601c2f09486c4a187f177cf5
2017-09-13 11:57:47 +05:30
Andrew Halberstadt
c451dcfe84 Bug 1391075 - Add template to modify task env from |mach try fuzzy|, r=dustin
This adds a new morph template for modifying a task's env and the corresponding glue
to specify it from |mach try fuzzy|. It can be used like:

./mach try fuzzy --env FOO=1 --env BAR=baz

This will simply set those environment variables in *all* tasks. We could add the
ability to only specify it for a subset of tasks in the future, but that seems like
a hard problem that probably isn't worth it.

MozReview-Commit-ID: C4sokv886PU

--HG--
extra : rebase_source : 64c9105848e6becd33fae63aeabc3423cfda3bdf
2017-08-25 15:24:22 -04:00
Andrew Halberstadt
e4dbc67d72 Bug 1395267 - [tryselect] Make default try selector used when running |mach try| without subcommands configurable, r=armenzg
To use this, create a ~/.mozbuild/machrc file and add:
[try]
default = fuzzy

Now, running |mach try| without a subcommand will default to the fuzzy selector. The syntax selector can still be
run with |mach try syntax|.

MozReview-Commit-ID: Ai6f8cW3Swq

--HG--
extra : rebase_source : 6a107478f7fbd7d8deb4882f0892963fed755b33
2017-08-30 14:45:46 -04:00
Andrew Halberstadt
6cd4e21e4b Bug 1394391 - [tryselect] Split hg implementation of files_changed to a list, r=jhford
This fixes a regression from bug 1384593. I was going to add a test for this, but it would be
a better use of time to fix bug 1185599 and add a test there. We already have vcs tests in other
parts of the tree so consolidating them into a single module is likely easier than standing up
another one-off vcs tester.

MozReview-Commit-ID: E51Tb1qC9Wb

--HG--
extra : rebase_source : f957b1857d162fd229dcef0e87668dd270512cf7
2017-08-28 09:13:46 -04:00
Andrew Halberstadt
f3a31869bb Bug 1393509 - Fix artifact builds on windows with |mach try fuzzy|, r=dustin
MozReview-Commit-ID: Fo3WzFEdLUF

--HG--
extra : rebase_source : f52c2e19bdad6e28ded106cfcdc64f3464281d54
2017-08-24 12:30:40 -04:00