Commit Graph

130 Commits

Author SHA1 Message Date
Andrew Halberstadt
95449daa6d Bug 1732723 - Rename "taskgraph" Python module to "gecko_taskgraph". r=jmaher
For a long time two copies of the 'taskgraph' module have existed in parallel.
We've attempted to keep them in sync, but over time they have diverged and the
maintenance burden has increased.

In order to reduce this burden, we'd like to re-join the two code bases. The
canonical repo will be the one that lives outside of mozilla-central, and this
module will depend on it. Since they both have the same module name (taskgraph)
we need to rename the version in mozilla-central to avoid collisions.

Other consumers of 'taskgraph' (like mobile repos) have standardized on
'<project>_taskgraph' as their module names. So replicating that here as well.

Differential Revision: https://phabricator.services.mozilla.com/D127118
2021-09-30 09:50:08 -04:00
Andrew Halberstadt
7b9414647d Bug 1725404 - [taskgraph] Re-use 'decision' args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122524
2021-08-16 14:56:26 +00:00
Andrew Halberstadt
cb434f3943 Bug 1721729 - Increase decision task timeout to 60 minutes, r?#taskgraph-reviewers! CLOSED TREE
Decision task durations have been creeping over the 20 minute mark, which means
we no longer have a lot of leeway when things timeout (like waiting for
bugbug). Longer term we should focus on reducing Decision task duration, but
for now this is needed to re-open autoland.

Differential Revision: https://phabricator.services.mozilla.com/D120644
2021-07-22 17:27:36 +00:00
Justin Wood
88d5d90dae Bug 1696944 - Part 3 - Add new image to taskcluster.yml. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D108423
2021-03-17 11:22:02 +00:00
Andrew Halberstadt
c288f18538 Bug 1666809 - Insert decision task indexes directly via taskluster API rather than index-task, r=taskgraph-reviewers,aki
We started using the "backstop" index added by bug 1660506 to determine whether
a push should be a backstop based on a time interval. The problem is that this
index gets added by an index-task that runs after the decision task has
completed.  Therefore, if two pushes land at roughly the same time (i.e, the
second decision task starts before the first completes), then they can both
determine themselves as backstops.

This patch gets around the problem by inserting the "backstop" index as early
as possible (immediately after resolving parameters), so the chances of this
happening become very low. It's still theoretically possible that it could
happen again, but we don't need this to be 100% perfect. As long as it is rare,
it's good enough.

Depends on D91191

Differential Revision: https://phabricator.services.mozilla.com/D91192
2020-09-24 19:19:36 +00:00
Tom Prince
1691c9892d Bug 1652184: Report to perherder stats about bugbug timings; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D83212
2020-07-13 19:53:00 +00:00
Tom Prince
fc3ae614bc No bug: Update links to hg.mozilla.org to use default instead of tip in taskcluster/; r=Callek
The latter sometimes bites us if we have a legacy support branch that gets
pushed more recently than default.

Differential Revision: https://phabricator.services.mozilla.com/D82628
2020-07-08 18:41:17 +00:00
Tom Prince
4eacbe9962 Bug 1630710: Add link to specfic task in treeherder for decision task; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D82627
2020-07-08 18:27:52 +00:00
Tom Prince
a22772e2a2 Bug 1626058: Build docker images with kaniko, instead of dind; r=glandium,hwine
Differential Revision: https://phabricator.services.mozilla.com/D77864
2020-07-07 19:53:32 +00:00
Tom Prince
b89d332bae No bug: Don't include push-id in treeherder routes; r=aki
Treeherder ignores this.

Differential Revision: https://phabricator.services.mozilla.com/D82251
2020-07-03 23:37:49 +00:00
Tom Prince
d8143d554a No bug: Rename s/trust_domain/trustDomain in .taskcluster.yml; r=aki
This bring mozilla-central in line with other projects.

Differential Revision: https://phabricator.services.mozilla.com/D82248
2020-07-03 23:33:06 +00:00
Tom Prince
dd9ee7a86e No bug: Update name of run-task to be accurate; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D82247
2020-07-03 23:32:39 +00:00
Tom Prince
76678c0e16 No bug: update root url in cron description in .taskcluster.yml; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D82246
2020-07-03 23:32:22 +00:00
Tom Prince
5fd0b0cf5f No bug: Fix a typo in .taskcluster.yml; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D82245
2020-07-03 23:32:02 +00:00
Tom Prince
2371948e17 No bug: Remove unused TASKCLUSTER_PROXY_URL and TASKCLUSTER_ROOT_URL env variables; r=aki
These are provied by the worker directly, now.

Differential Revision: https://phabricator.services.mozilla.com/D82244
2020-07-03 23:31:43 +00:00
Tom Prince
9ef67b9fdd No bug: Make .taskcluster.yml yamllint clean; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D82243
2020-07-03 23:30:59 +00:00
Tom Prince
909e7275e8 Bug 1648591: Remove taskgraph based cron implementation; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D81270
2020-07-07 16:04:00 +00:00
Tom Prince
2324b3daf4 Bug 1628832: [KaiOS] Factor out trust-domain in .taskcluster.yml; r=Callek
To help support KaiOS, we are running the kaios project branch in automation, with a
different trust domain. Factor out the usages of `trust-domain` in `.taskcluster.yml`
to make it easy for the kaios branch to merge changes from mozilla-central.

Differential Revision: https://phabricator.services.mozilla.com/D75242
2020-05-14 18:47:47 +00:00
Tom Prince
288031e45e Bug 1628832: Remove some obsolete routes from the decision task; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D75241
2020-05-14 18:47:29 +00:00
Tom Prince
4fd158fabf Bug 1626795: Use revision rather than push-id to look up existing tasks; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D70568

--HG--
extra : moz-landing-system : lando
2020-04-10 22:29:39 +00:00
Aki Sasaki
a83870e1ee Bug 1624887 - set decision task retries to 0. r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D68250

--HG--
extra : moz-landing-system : lando
2020-03-27 17:57:48 +00:00
Tom Prince
b6a284f66b Bug 1606869: Switch decision task docker caches to match the rest the in-tree caches; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D58638

--HG--
extra : moz-landing-system : lando
2020-01-03 22:18:29 +00:00
Tom Prince
f69f47ec54 Bug 1589706: [firefox-ci] Use AWS Provider-based decision worker; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D50357

--HG--
extra : moz-landing-system : lando
2019-10-24 00:54:12 +00:00
Tom Prince
7cc0f0e89c Bug 1580622: Disable python bytecode generation in the decision task; r=nalexander on a CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D48457

--HG--
extra : source : 1e29bf90b3ea69e9e7c026f4626ac11bba762689
extra : amend_source : bf551282203dbf6553e9eba534a4eca7eff93696
2019-10-07 22:48:48 +00:00
Daniel Varga
52729f9f2b Backed out changeset 1e29bf90b3ea (bug 1580622) for breaking retriggers, backfill and reruns on autoland 2019-10-08 06:14:27 +03:00
Tom Prince
6b5e340266 Bug 1580622: Disable python bytecode generation in the decision task; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D48457

--HG--
extra : moz-landing-system : lando
2019-10-07 22:48:48 +00:00
Tom Prince
c9b692824c Bug 1459355: Don't pass parameters as part of actions; r=dustin
This causes actions to fail on graphs with large parameters, such as
release graphs and try pushes.

Differential Revision: https://phabricator.services.mozilla.com/D37499

--HG--
extra : moz-landing-system : lando
2019-08-09 17:02:15 +00:00
Brian Stack
f0b92f7367 Bug 1542233 - Add link to treeherder for all notification emails r=tomprince
This should make all emails from pushes link to treeherder as requested in the bug. I've made  a try push and confirmed that the "thanks for your submission" email still happens with proper links.  I am pretty sure this will set the link in non-try pushes correctly but if it doesn't, nothing will actually break afaict.

Differential Revision: https://phabricator.services.mozilla.com/D37116

--HG--
extra : moz-landing-system : lando
2019-07-08 18:47:27 +00:00
Dustin J. Mitchell
f3654593cc Bug 1547724 - send cron failure notifications to sheriffs / ciduty r=aryx
Differential Revision: https://phabricator.services.mozilla.com/D30598

--HG--
extra : moz-landing-system : lando
2019-05-16 15:37:05 +00:00
Dustin J. Mitchell
a4256b1185 Bug 1547781 - expire try decision artifacts after 28 days r=Callek
This ensures that the artifacts an action task might consult will be gone at
the same time that the push's tasks expire.

Differential Revision: https://phabricator.services.mozilla.com/D29270

--HG--
extra : moz-landing-system : lando
2019-04-30 13:48:25 +00:00
Dustin J. Mitchell
850305dbcc Bug 1547781 - test that .taskcluster.yml renders; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D29269

--HG--
extra : moz-landing-system : lando
2019-04-30 13:48:05 +00:00
Tom Prince
b4849975d4 Bug 1488313: [taskgraph] Don't notify sherrifs on cron decision task failures on try; r=dustin
Differential Revision: https://phabricator.services.mozilla.com/D25090

--HG--
extra : moz-landing-system : lando
2019-03-27 19:38:05 +00:00
Dustin J. Mitchell
ed9ceeeeb7 Bug 1491371, 1512188 - use new decision task image, with new options r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D24401

--HG--
extra : moz-landing-system : lando
2019-03-21 18:12:23 +00:00
Dustin J. Mitchell
4b406f0a1e Bug 1534283 - add a 'tasks_for' property; r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D23569

--HG--
extra : moz-landing-system : lando
2019-03-14 20:01:32 +00:00
Dustin J. Mitchell
c8fe0990f8 Bug 1524358 - get commit message locally, rather than from task parameters r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D18288

--HG--
extra : moz-landing-system : lando
2019-02-01 23:27:48 +00:00
Dustin J. Mitchell
0a44f01f62 Bug 1524358 - use ownTaskId, instead of as_slugid('decision'), in .taskcluster.yml r=Callek,tomprince
This is a forward-looking change to allow evaluation of .taskcluster.yml in
situations where as_slugid is not defined -- such as in the Taskcluster hooks
service.

Differential Revision: https://phabricator.services.mozilla.com/D18269

--HG--
extra : moz-landing-system : lando
2019-02-01 23:27:47 +00:00
Dustin J. Mitchell
da6fa3a08b Bug 1524358 - document inputs to .taskcluster.yml more deeply r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D18287

--HG--
extra : moz-landing-system : lando
2019-02-01 23:27:48 +00:00
Aki Sasaki
99c00fea5e Bug 1521264 - add clientId to action context + description. r=dustin
Differential Revision: https://phabricator.services.mozilla.com/D17976

--HG--
extra : moz-landing-system : lando
2019-01-29 20:46:13 +00:00
Cosmin Sabou
cdfad80704 Backed out changeset aee9f213f3c7 (bug 1496890) for causing Bug 1521264. a=backout 2019-01-19 11:22:32 +02:00
Dustin J. Mitchell
5e4c79d90d Bug 1496890 - include client calling triggerHook in action description r=aki
Differential Revision: https://phabricator.services.mozilla.com/D16875

--HG--
extra : moz-landing-system : lando
2019-01-17 18:22:20 +00:00
Dustin J. Mitchell
1516eb0435 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere in production, and
TASKCLUSTER_PROXY_URL is set wherever the proxy is active.

The taskgraph Taskcluster utils module gets a `get_root_url()` that gets the
root URL for the current run, either from an environment variable in production
or, on the command line, defaulting to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

Other changes to use this function are reserved for later commits.

This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.

--HG--
extra : rebase_source : 4f50e9d066da62a1887baabd8603844c85a32ee6
extra : source : 5ea6f03f845e49d503f5d0283557f54561c41654
2018-09-25 20:18:19 +00:00
Danut Labici
9027f6e4c6 BUG 1500166 - notify ciduty by email if a nightly hook fails + fix identation issue. r=tomprice
--HG--
extra : rebase_source : 3c816fde539f2ac3ba0292737f8d00e7cc95af5e
2018-12-20 18:39:43 +02:00
Sebastian Hengst
767c971623 Backed out 21 changesets (bug 1492664) for breaking cron task for nightlies. a=backout
Backed out changeset a7d50dbb2c8e (bug 1492664)
Backed out changeset 2d876c4ece8b (bug 1492664)
Backed out changeset c82285d253de (bug 1492664)
Backed out changeset bf6d089640eb (bug 1492664)
Backed out changeset d9a7f2ce49c3 (bug 1492664)
Backed out changeset 06c466ab4323 (bug 1492664)
Backed out changeset c1ea4a10cc8d (bug 1492664)
Backed out changeset 4c63a04fdd47 (bug 1492664)
Backed out changeset 742b038bb1dd (bug 1492664)
Backed out changeset 911b4b0fb683 (bug 1492664)
Backed out changeset 870c8cec99e5 (bug 1492664)
Backed out changeset 77699b51336b (bug 1492664)
Backed out changeset 29f33f22fd8b (bug 1492664)
Backed out changeset e7f305408708 (bug 1492664)
Backed out changeset 335a92b1f424 (bug 1492664)
Backed out changeset c566f1c8dcdf (bug 1492664)
Backed out changeset c77ae59aba41 (bug 1492664)
Backed out changeset 9c35dd209c6b (bug 1492664)
Backed out changeset a972d6b4434e (bug 1492664)
Backed out changeset 5ea6f03f845e (bug 1492664)
Backed out changeset 0699d3873e44 (bug 1492664)

--HG--
extra : histedit_source : 5cb1f7e50f25d4a875c1a58c86b7dce902e1a89c%2C20f1ab1a843b612cfcc67cf5c6ff745d65abf076
2018-12-20 12:43:22 +02:00
Dustin J. Mitchell
4a0a6eae05 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.

--HG--
extra : rebase_source : 0626ebdb66a9f4078cb75ab71be256c334297363
2018-09-25 20:18:19 +00:00
Margareta Eliza Balazs
2e5e28f518 Backed out 16 changesets (bug 1492664) for breaking developer artifact builds, requested by standard8 a=backout
Backed out changeset 31e500489665 (bug 1492664)
Backed out changeset f4945658d45f (bug 1492664)
Backed out changeset 6d17291b8b92 (bug 1492664)
Backed out changeset 90f3faa36137 (bug 1492664)
Backed out changeset 0b229b00818a (bug 1492664)
Backed out changeset 5eb2c77d70a9 (bug 1492664)
Backed out changeset e1ebad5d89c5 (bug 1492664)
Backed out changeset 3017e5890739 (bug 1492664)
Backed out changeset c8b7e620eabf (bug 1492664)
Backed out changeset d3dfbd848236 (bug 1492664)
Backed out changeset 5c92bb5ac895 (bug 1492664)
Backed out changeset fb7cfca6ebc3 (bug 1492664)
Backed out changeset 0c4101230d4d (bug 1492664)
Backed out changeset b93a0fcc86f3 (bug 1492664)
Backed out changeset 6dc9522ee0bf (bug 1492664)
Backed out changeset 85d7f8b330eb (bug 1492664)
2018-12-19 11:45:29 +02:00
Dustin J. Mitchell
211a6cf5e3 Bug 1492664 - set TASKCLUSTER_ROOT_URL and TASKCLUSTER_PROXY_URL; r=tomprince,glandium
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015).  But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.

The setup for the mach commands defaults to https://taskcluster.net for user
convenience.  When the production instance's URL changes, we can simply change
that default.

This changes the docker build process to propagate TASKCLUSTER_ROOT_URL into
the docker images, and for good measure includes some code to use that value to
generate debian repo paths.

Differential Revision: https://phabricator.services.mozilla.com/D14196

--HG--
extra : moz-landing-system : lando
2018-12-18 05:37:41 +00:00
Tom Prince
fe977ab920 Bug 1486970 - Create revision and pushlog-id index routes for cron decision tasks. r=dustin,aki
This changeset actually creates the new routes we want for decision tasks, and adds the supporting code for passing push info
in to the taskgraph.

This must land after scriptworker is updated and deployed. This has the affect of also un-hardcoding the .taskcluster.yml
fields applied in the first part of this bug.

This changeset in particular will ride trains and not be uplifted (outside of compelling reasons).

Differential Revision: https://phabricator.services.mozilla.com/D5195

--HG--
extra : moz-landing-system : lando
2018-11-27 22:33:09 +00:00
Dustin J. Mitchell
d5fbd16a25 Bug 1509997 - set priorities cron > action > decision r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D12991

--HG--
extra : moz-landing-system : lando
2018-11-26 22:13:41 +00:00
J.C. Jones
329de1ce5e Bug 1505930 - Promote try-push analysis result r=catlee
"It's the best!" used to be front-and-center on try push email notifications,
within the content area of the automated happiness-intensifier email.

Bug 1490515 regressed the happiness to the far-right side of the subject line.
In keeping with "It's the best!" deserving a top-level treatment, it should be
duplicated (by value) in the message body.

Differential Revision: https://phabricator.services.mozilla.com/D11374

--HG--
extra : moz-landing-system : lando
2018-11-08 21:40:53 +00:00
Sebastian Hengst
823c8a37c2 Bug 1500166 - notify ciduty by email if a nightly hook fails + fix identation issue: fix indentation. a=bustage-fix 2018-10-23 18:27:00 +03:00