Commit Graph

126 Commits

Author SHA1 Message Date
Andrew Halberstadt
6ee8586201 Bug 1531360 - [fuzzy] Add ability to intersect queries, r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D22028

--HG--
extra : moz-landing-system : lando
2019-03-06 16:14:00 +00:00
Andrew Halberstadt
029b7b915f Bug 1531886 - [tryselect] Stop allowing **kwargs in subcommand run functions, r=gbrown
This forces us to be more strict with what gets passed into the try selector.
This change means test_preset.py will error if someone makes a typo in an
in-tree preset.

Differential Revision: https://phabricator.services.mozilla.com/D22024

--HG--
extra : moz-landing-system : lando
2019-03-06 16:13:01 +00:00
Andrew Halberstadt
db5e67fdae Bug 1531886 - [tryselect] Delete template context from kwargs, r=gbrown
When we parse template arguments, we stuff them all in kwargs['templates'],
however we don't delete the old argument. This results in all kinds of unused
variables lying around in kwargs. E.g we would have both
kwargs['templates']['env'] and kwargs['env'] (the latter being unused). This is
the main reason why all the selector's run functions need to have a **kwargs at
the end of them.

Depends on D22022

Differential Revision: https://phabricator.services.mozilla.com/D22023

--HG--
extra : moz-landing-system : lando
2019-03-05 23:49:22 +00:00
Andrew Halberstadt
5e06797460 Bug 1531886 - [tryselect] Handle templates in mach_commands.py, r=gbrown
This was previously only in the cli parser because that was the only shared
place that ran for all selectors. Now that we have the 'self.run' function in
mach_commands.py, we can move it there. This move is also needed to allow us to
remove 'templates' from kwargs (which happens in the next commit).

Depends on D22021

Differential Revision: https://phabricator.services.mozilla.com/D22022

--HG--
extra : moz-landing-system : lando
2019-03-05 23:48:31 +00:00
Andrew Halberstadt
fae875eee3 Bug 1531886 - [tryselect] Use consistent 'run' method to kickstart all selectors, r=gbrown
This allows us to refactor mach_commands.py so we can call self.handle_presets
implicitly. This in turn gives us a future place to add shared code and makes
adding new selectors easier.

Differential Revision: https://phabricator.services.mozilla.com/D22021

--HG--
extra : moz-landing-system : lando
2019-03-06 17:34:44 +00:00
Andrew Halberstadt
bbce8d7a3b Bug 1513951 - [tryselect] Add some documentation for presets r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D21431

--HG--
extra : moz-landing-system : lando
2019-02-28 20:15:24 +00:00
Andrew Halberstadt
30fa8f8a4c Bug 1513951 - [tryselect] Implement in-tree try_presets.yml file r=gbrown
This creates a global preset file at:
tools/tryselect/try_presets.yml

Any presets defined here will be available for everyone to use.

Differential Revision: https://phabricator.services.mozilla.com/D21435

--HG--
extra : moz-landing-system : lando
2019-02-28 21:38:26 +00:00
Andrew Halberstadt
b1f7c90d52 Bug 1513951 - [tryselect] Pass preset path into PresetHandler's constructor r=gbrown
This will make it possible to have multiple instances of PresetHandler to
support multiple preset files.

Differential Revision: https://phabricator.services.mozilla.com/D21429

--HG--
extra : moz-landing-system : lando
2019-02-28 18:26:03 +00:00
Andrew Halberstadt
26d3e5b30a Bug 1530775 - [tryselect] Fix bug when migrating try syntax presets, r=kats,a=NPOTB
Differential Revision: https://phabricator.services.mozilla.com/D21250

--HG--
extra : rebase_source : 1688a24623f92134cd91de087d52cf8fc607476c
2019-02-26 14:33:03 -05:00
Andrew Halberstadt
91456de750 Bug 1518572 - [tryselect] Migrate old preset files to the new format r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D20524

--HG--
extra : moz-landing-system : lando
2019-02-22 23:25:54 +00:00
Andrew Halberstadt
5caae33416 Bug 1518572 - [tryselect] Store all arguments when saving a preset r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D20523

--HG--
rename : taskcluster/taskgraph/util/templates.py => tools/tryselect/util/dicttools.py
extra : moz-landing-system : lando
2019-02-25 19:47:29 +00:00
Andrew Halberstadt
b9afd5605b Bug 1518572 - [tryselect] Remove unused 'flask-wtf' dependency from |mach try chooser| r=gbrown
I forgot to remove this after re-implementing without this dependency.

Depends on D20521

Differential Revision: https://phabricator.services.mozilla.com/D20522

--HG--
extra : moz-landing-system : lando
2019-02-22 19:43:56 +00:00
Andrew Halberstadt
80facb6f27 Bug 1528980 - [tryselect] Print host:port before running the trychooser flask app, r=tcampbell
I'm not really sure why Flask isn't printing this for us, but it's not worth
investigating when the alternative is so easy.

Differential Revision: https://phabricator.services.mozilla.com/D20712

--HG--
extra : moz-landing-system : lando
2019-02-21 19:45:57 +00:00
Tom Prince
4ecf218463 Bug 1525736: [taskgraph] Allow looking up parameters by project for trust-domains other than gecko; r=dustin
Previously, this code looked parameters under `gecko.v2`, but that doesn't work
for projects using the out-of-tree taskgraph code, or Thunderbird. This moves
the parameter loading slightly later to vary the index used based on trust domain.

Differential Revision: https://phabricator.services.mozilla.com/D19028

--HG--
extra : moz-landing-system : lando
2019-02-14 21:34:49 +00:00
Andrew Halberstadt
66888b2e2c Bug 1526129 - Fix test failure in tryselect cram tests, r=jmaher
This fixes a regression from bug 1483228. It started printing a message to
stdout whenever the local state dir was generated. This caused a failure in
these cramtests since they depend on the stdout of the shell process being
identical.

To fix, make sure we trigger state dir creation in the setup script and
redirect to /dev/null.

Differential Revision: https://phabricator.services.mozilla.com/D19155

--HG--
extra : moz-landing-system : lando
2019-02-08 15:17:26 +00:00
Tom Prince
7c4e3eb5a8 No bug: [tryselect] Slightly improve the accuracy of the error message on parameter mismatch; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D19024

--HG--
extra : moz-landing-system : lando
2019-02-07 19:38:13 +00:00
Andrew Halberstadt
4cdb2c3165 Bug 1483228 - [tryselect] Use get_state_dir(srcdir=True) to determine directory to store history state r=nalexander
This was previously implemented by creating the local state dir within the
global state dir (using a hash of the path in the name). This moves the
|mach try again| history state to the new local state dir.

It also updates the migration code to be a little more robust, ensuring we
don't accidentally lose people's history. This migration should be removed
at some point in the future (2020 seemed like a safe enough window).

Differential Revision: https://phabricator.services.mozilla.com/D15725

--HG--
extra : moz-landing-system : lando
2019-02-04 21:07:10 +00:00
Andrew Halberstadt
8bc7850637 Bug 1483228 - [mozboot] Simplify get_state_dir()'s return value r=nalexander
mozboot.util.get_state_dir() returns a tuple of (<path>, <bool). The bool
denotes whether or not the state dir came from an environment variable.

But this value is only used in a single place, and is very easy to test for
anyway. It's not worth the added complexity it imposes on all other consumers
of this function. Let's just make this function return the path.

Differential Revision: https://phabricator.services.mozilla.com/D15723

--HG--
extra : moz-landing-system : lando
2019-02-04 20:52:31 +00:00
Andrew Halberstadt
4222e67eac Bug 1400426 - [try] Add --exact argument to |mach try fuzzy|, r=jgraham
Passing in --exact reverses the behaviour of the ' operator. For example,
take the query "foo 'bar".

By default: foo is a fuzzy match and bar is an exact match.
With --exact: foo is an exact match and bar is a fuzzy match

Differential Revision: https://phabricator.services.mozilla.com/D16734

--HG--
extra : moz-landing-system : lando
2019-01-30 17:09:34 +00:00
Andrew Halberstadt
414d48031e Bug 1499822 - [tryselect] Implement |mach try chooser| r=sclements
Usage:

  $ ./mach try chooser

Will start a local flask server and server a "trychooser-like" page
that is dynamically generated from the taskgraph.

Differential Revision: https://phabricator.services.mozilla.com/D14903

--HG--
extra : moz-landing-system : lando
2019-01-09 17:04:39 +00:00
Andrew Halberstadt
b578a9b3ce Bug 1499822 - [lint] Enable codespell on tools/tryselect, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D15588

--HG--
extra : moz-landing-system : lando
2019-01-09 17:06:44 +00:00
Andreea Pavel
a7eee8304c Backed out 3 changesets (bug 1499822) for breaking gecko decision task on a CLOSED TREE
Backed out changeset beb286aeae33 (bug 1499822)
Backed out changeset 9a330195eca6 (bug 1499822)
Backed out changeset f4bd2ef9b729 (bug 1499822)
2019-01-09 00:40:16 +02:00
Andrew Halberstadt
0c56961d92 Bug 1499822 - [tryselect] Implement |mach try chooser| r=sclements
Usage:

  $ ./mach try chooser

Will start a local flask server and server a "trychooser-like" page
that is dynamically generated from the taskgraph.

Differential Revision: https://phabricator.services.mozilla.com/D14903

--HG--
extra : moz-landing-system : lando
2019-01-08 16:46:57 +00:00
Andrew Halberstadt
6ac5500593 Bug 1499822 - [lint] Enable codespell on tools/tryselect, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D15588

--HG--
extra : moz-landing-system : lando
2019-01-08 16:45:10 +00:00
Andrew Halberstadt
c8b2301a9f Bug 1517621 - [tryselect] Mock out entire vcs object in python tests, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D15695

--HG--
extra : moz-landing-system : lando
2019-01-04 10:40:35 +00:00
Sebastian Hengst
767c971623 Backed out 21 changesets (bug 1492664) for breaking cron task for nightlies. a=backout
Backed out changeset a7d50dbb2c8e (bug 1492664)
Backed out changeset 2d876c4ece8b (bug 1492664)
Backed out changeset c82285d253de (bug 1492664)
Backed out changeset bf6d089640eb (bug 1492664)
Backed out changeset d9a7f2ce49c3 (bug 1492664)
Backed out changeset 06c466ab4323 (bug 1492664)
Backed out changeset c1ea4a10cc8d (bug 1492664)
Backed out changeset 4c63a04fdd47 (bug 1492664)
Backed out changeset 742b038bb1dd (bug 1492664)
Backed out changeset 911b4b0fb683 (bug 1492664)
Backed out changeset 870c8cec99e5 (bug 1492664)
Backed out changeset 77699b51336b (bug 1492664)
Backed out changeset 29f33f22fd8b (bug 1492664)
Backed out changeset e7f305408708 (bug 1492664)
Backed out changeset 335a92b1f424 (bug 1492664)
Backed out changeset c566f1c8dcdf (bug 1492664)
Backed out changeset c77ae59aba41 (bug 1492664)
Backed out changeset 9c35dd209c6b (bug 1492664)
Backed out changeset a972d6b4434e (bug 1492664)
Backed out changeset 5ea6f03f845e (bug 1492664)
Backed out changeset 0699d3873e44 (bug 1492664)

--HG--
extra : histedit_source : 5cb1f7e50f25d4a875c1a58c86b7dce902e1a89c%2C20f1ab1a843b612cfcc67cf5c6ff745d65abf076
2018-12-20 12:43:22 +02:00
Dustin J. Mitchell
4a0a6eae05 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.

--HG--
extra : rebase_source : 0626ebdb66a9f4078cb75ab71be256c334297363
2018-09-25 20:18:19 +00:00
Margareta Eliza Balazs
2e5e28f518 Backed out 16 changesets (bug 1492664) for breaking developer artifact builds, requested by standard8 a=backout
Backed out changeset 31e500489665 (bug 1492664)
Backed out changeset f4945658d45f (bug 1492664)
Backed out changeset 6d17291b8b92 (bug 1492664)
Backed out changeset 90f3faa36137 (bug 1492664)
Backed out changeset 0b229b00818a (bug 1492664)
Backed out changeset 5eb2c77d70a9 (bug 1492664)
Backed out changeset e1ebad5d89c5 (bug 1492664)
Backed out changeset 3017e5890739 (bug 1492664)
Backed out changeset c8b7e620eabf (bug 1492664)
Backed out changeset d3dfbd848236 (bug 1492664)
Backed out changeset 5c92bb5ac895 (bug 1492664)
Backed out changeset fb7cfca6ebc3 (bug 1492664)
Backed out changeset 0c4101230d4d (bug 1492664)
Backed out changeset b93a0fcc86f3 (bug 1492664)
Backed out changeset 6dc9522ee0bf (bug 1492664)
Backed out changeset 85d7f8b330eb (bug 1492664)
2018-12-19 11:45:29 +02:00
Dustin J. Mitchell
211a6cf5e3 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process to propagate TASKCLUSTER_ROOT_URL into
the docker images, and for good measure includes some code to use that value to
generate debian repo paths.

Differential Revision: https://phabricator.services.mozilla.com/D14196

--HG--
extra : moz-landing-system : lando
2018-12-18 05:37:41 +00:00
Marco Castelluccio
a2850b812e Bug 1512146 - Update expected results for some tests using mach try fuzzy. r=ahal
--HG--
extra : rebase_source : 4cf706b219126f90b11c2e17f0401d82fe228aa5
2018-12-04 22:54:36 -05:00
Marco Castelluccio
a1348d373a Bug 1512082 - Add TRY_SELECTOR env to templates expected value. r=me
--HG--
extra : rebase_source : 498a3f1162ab15157ce553db9b38b4218a81ef48
2018-12-04 17:42:41 -05:00
Marco Castelluccio
8eff89e376 Bug 1488849 - Don't fail when a chunk is not running any test when the try selector is 'coverage'. r=ahal
--HG--
extra : rebase_source : 8099420cf47593ea7b38fadc8b4c5f87f791f250
2018-11-23 17:59:04 +01:00
Marco Castelluccio
33976b9bd2 Bug 1510391 - When a task is not found, don't add None to the list of tasks to run. r=ahal
--HG--
extra : rebase_source : ef53880797df740bbbd289099cb602539b208599
2018-11-27 17:29:34 +01:00
Marco Castelluccio
04087c53c1 Bug 1510452 - Use srcdir_relpath instead of file_relpath for tests and test support files. r=ahal
--HG--
extra : rebase_source : ec726b7b2a409e6e30dcdee2e5795eabacd56fb6
2018-11-27 23:53:33 +01:00
Marco Castelluccio
65ca4461a5 Bug 1489100 - Specify tests per suite in MOZHARNESS_TEST_PATHS. r=ahal
--HG--
extra : rebase_source : 09d05ca38ad25ce6178172e607ce0207525ea484
2018-11-15 17:54:13 +01:00
Andrew Halberstadt
78d1bf89f0 Bug 1509048 - [tryselect] Pass full taskgraph into selectors, r=marco
Currently selectors that generate the taskgraph receive a list of task labels
back. While this works, it isn't very robust and limits the selectors in what
they can accomplish.

For example, it means we need to use regexes to map test suites to tasks. If
we had the full taskgraph, we could use filter functions instead.

There are also new selectors in the works which will need this (like ./mach
try chooser). Finally the try syntax selector would also need this if we
ever migrate it to use the 'try_task_config.json' format.

Differential Revision: https://phabricator.services.mozilla.com/D12570

--HG--
extra : moz-landing-system : lando
2018-11-23 18:17:33 +00:00
Tom Prince
e92faba0fc No bug: [try-staging] Rename beta-sim to release-sim to reflect flexibility; r=aryx
Now that `mach try release` also supports simulating release and esr, rename
the option to `release-sim`.

Differential Revision: https://phabricator.services.mozilla.com/D11516

--HG--
extra : moz-landing-system : lando
2018-11-10 00:50:27 +00:00
Marco Castelluccio
4c1e631f97 Bug 1507111 - Fallback to latest mozilla-central pushes if the local base revision is not known by hgmo. r=ahal
DONTBUILD

--HG--
extra : rebase_source : 232ae71e580e253b8d8271f7184c84b91c6f13f6
extra : histedit_source : 6715707e1d92a182db89021023c1a84286342ff8
2018-11-14 16:32:01 +01:00
Marco Castelluccio
37092104ae Bug 1485741 - More precisely select chunks to run and assert when there are unexpected conditions. r=ahal
--HG--
extra : rebase_source : d9d1d5b2d132bd6af7011bc8231c5bfde2cd6194
2018-11-14 12:11:13 +01:00
Marco Castelluccio
d9538dfb46 Bug 1429463 - Use 'chunk' variable name instead of 'suite'. r=me
--HG--
extra : histedit_source : b75d43c6e7ea9212bca305dc3430fc210c70467c%2C259186696138c2c63db64db1bcd9d1501d1aadb9
2018-11-13 20:54:31 +01:00
Marco Castelluccio
70d9f1eb85 Bug 1429463 - Use new Taskcluster index URL for the code coverage bot. r=me
--HG--
extra : histedit_source : 776ac2b5517f3bbf79847d019abd16b6c53cf729
2018-11-13 11:35:08 +01:00
Tudor-Gabriel Vîjială
3546d45bf6 Bug 1429463 - Prototype ./mach try coverage tool. r=ahal,marco
--HG--
rename : tools/tryselect/test/test_fuzzy.py => tools/tryselect/test/test_tasks.py
2018-11-13 11:17:01 +01:00
Tom Prince
6285866fdb No bug: [try-staging] Geneate the correct release_type for esr60 staging runs; r=nthomas
Differential Revision: https://phabricator.services.mozilla.com/D11391

--HG--
extra : moz-landing-system : lando
2018-11-09 02:02:25 +00:00
Tom Prince
078e06296f No bug: [try-staging] Don't generate versions with esr in them; r=nthomas
Balrog accepts them, but then fails to serve them.

Differential Revision: https://phabricator.services.mozilla.com/D11390

--HG--
extra : moz-landing-system : lando
2018-11-09 19:39:52 +00:00
Tom Prince
134fb09296 No bug: [try-staging] Allow specifying no migrations for staging releases; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D11388

--HG--
extra : moz-landing-system : lando
2018-11-09 02:48:35 +00:00
Tom Prince
8802c67020 No bug: [try-staging] Add support for early-to-late-beta migration; r=aryx
Differential Revision: https://phabricator.services.mozilla.com/D11393

--HG--
extra : moz-landing-system : lando
2018-11-08 23:10:15 +00:00
Rob Wood
1dfcab1a30 Bug 1502922 - Add --gecko-profile flag support to mach try fuzzy for raptor and talos jobs; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D10114

--HG--
rename : taskcluster/taskgraph/templates/talos-profile.yml => taskcluster/taskgraph/templates/gecko-profile.yml
extra : moz-landing-system : lando
2018-10-30 15:15:32 +00:00
Tom Prince
fa2fc2145a Bug 1501759: [tryselect/release] Update milestone to new version too; r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D9703

--HG--
extra : moz-landing-system : lando
2018-10-24 20:44:11 +00:00
Tom Prince
5c9bcda5ed Bug 1501759: [taskgraph] Add a beta-sim target; r=aki,RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D9690

--HG--
extra : moz-landing-system : lando
2018-10-24 20:44:11 +00:00
Tom Prince
088aac0cdb Bug 1501759: [taskgraph] Remove include_nightly parameter; r=aki
We only use `include_nigthly` where we are also using
`filter_beta_release_tasks`, so just change the later to include nightly.

Differential Revision: https://phabricator.services.mozilla.com/D9687

--HG--
extra : moz-landing-system : lando
2018-10-24 20:44:03 +00:00